An Overview of Internet Email

The Basic Internet Email System

Key Elements of the Internet Mail System

The most important logical elements of the Internet mail system are:

N.B. Commercial products typically hide the distinction between these logical elements from the user, e.g. Microsoft Exchange contains at least one MTA, plus several MDAs and MRAs.

Basic Internet Mail System Diagram

Gateways Between Non-Standard (Alien) Mail Systems and the Internet

After buying and configuring the software, the main requirement for using non-standard mail (e.g. MS mail) across the internet, is a `Gateway' MTA, i.e. a `server' which re-writes messages to and from the Internet format then passes them, respectively, to an Internet MTA or to the proprietary mail system. (see figure below)

Unfortunately, if you want to receive mail in a different proprietary format from the one it was composed in, you'll usually have to settle for lowest common denominator translation.

Obviously, those who stick to Internet standards throughout do not lose functionality in this way.

Sending non-standard mail via the Internet

Mail Transfer Agents (MTAs)

MTA Basic Tasks

Despite their name MTAs do not physically transfer mail. They simply ensure that it is sent to its next destination and that it is properly formatted.

When a message arrives, the MTA identifies its sender and recipient from SMTP envelope information and re-writes the message and/or the envelope, as required for it to pass through the part of the network for which the MTA is responsible. The MTA then passes the message to an MDA for delivery to a local mailbox or another MTA.

N.B. Envelope information is added to a message as part of the SMTP protocol. It is usually added and modified automatically by mail system software. It should not be confused with the To: and From: message headers which users write for themselves.

Messages may pass through many MTAs and several re-writes, especially where translation to and from proprietary formats is required.

Part or all of a message may have to be re-written by MTAs on route.

Passing messages on requires each MTA to make routing decisions based on the recipient address found in the envelope:

Just because DNS MX records show an MTA will accept mail for a particular domain, doesn't mean that the recipient is local to that server. The accepting MTA may simply relay mesages to another MTA, or it may even route the message to another address as part of a virtual domain service, e.g. changing the recipient details in the envelope before passing the message on.

MTAs as Spam Busters

Modern MTAs can use the senders' address in a message envelope or headers to block spam (unsolicited junk mail), by checking them against various `blacklists'. N.B. These lists also include the fake addresses which spammers often use.

It is good policy to prevent people from using an MTA to relay mail from arbitrary adresses.

Varieties of MTA

Simply dialing-up and retrieving mail from a mailbox on an ISP's server doesn't require you to have a local MTA. Mail can be sent by a MUA, via SMTP, to the ISP's MTA for relay to the Internet. See the sections below on `Getting Mail from Remote Mailboxes' and `Mail User Agents'.

If you have a moderate number of users with relatively simple mail needs (i.e. just to send and receive mail locally and via the Internet), you can probably get by with the smail, or qmail MTAs

If you have lots of people who need to access complex mail and fax facilities from many locations, you probably need a heavy-duty MTA like sendmail or exim. Until the recent emergence of point-and-click configuration tools (e.g. Liunxconf), sendmail was notoriously difficult to set up. This led a few serious mail users (e.g. ISPs and Universities) to switch to exim, because it offers very similar services with much simpler configuration.

N.B. Sendmail still processes over 80% of Internet mail.

MDA Mail Delivery

Each MTA relys on multiple MDAs; a different one for each type of delivery required.

An MDA does not need to know anything about a message's content or its headers and will not change them or the envelope. It simply assumes that the MTA which passed on the message has ensured that it is correctly formatted and addressed.

A local-delivery MDA just writes messages to recipient mailboxes.

Some MDAs, like /bin/mail will filter mail into different mailboxes, according to the recipient's instructions. Some prefer to call an external MDA-level filtering program, like procmail to do this job.

Remote-delivery MDAs just transfer messages to a remote MTA.

Obviously, every MDA has to speak the protocol used by the mailbox, MTA or gateway MTA it is delivering to or from.

Getting Mail from Remote Mailboxes (MRAs)

MRAs simply retrieve messages and/or their headers from remote mailboxes and deliver them to the MUA on the user's machine.

Internet MRAs work using one of two available protocols:

  1. The Internet Message Access Protocol (IMAP) — a complex and powerful protocol which allows all sorts of devices (e.g. NCs, PDAs, PCs, etc) to access and manipulate server-based mail from any location, from multiple mailboxes, etc, etc. For example, a device with limited local storage capacity could simply view message headers, to avoid downloading huge binary attachments that it is unable to display anyway. IMAP offers huge opportunities to exploit email as an ecommerce tool. Unfortunately, the dominance of POP amongst ISPs and commercial MUAs (see below) is hindering such development.
  2. The Post Office Protocol (POP3) — A tiny subset of IMAP functions which allows server-based mail to be downloaded, but has extremely limited remote mailbox management functions (basically, limited to retaining or deleting the messages held on the remote server). POP is favoured by many ISPs because it is simpler to configure and administer than IMAP.

The MRAs which are used to transfer mail from remote mailboxes to MUAs are usually called daemons; the unix term for programs that permanently run in the background, waiting for other aplications to request their services.

Mail User Agents (MUAs)

True Internet Mail User Agents (MUAs can directly read and write to and from mailboxes on the same machine. If they are not local, they have to employ a POP or IMAP-based MRA to retreive mail for them, but they can usually send mail directly to a remote MTA via SMTP.

Special purpose MUAs exist to carry out other interactions with local or remote mailboxes, e.g.

Beyond these `transport issues' the main function of an MUA is to provide the interface through which users interact with Internet mail, including:

Getting Mail From Remote Mailboxes

Email Standards

Internet Standards

Internet mail is composed and delivered according to Internet standards, defined in up to 150 RFC documents. Only 5 of these are fully adopted and `recommended', but it is advisable to use the `elective' and even the `experimental' Internet standards for any of the email functions they cover.

We (GBdirect) endeavour to implement all RFC standards for the pragmatic reason that they provide the only sure means of securing true inter-operability between ecommerce applications and data.

The 5 essential standards are defined in RFCs 821, 822, 1049, 1869 and 1870. They cover the format of mail messages, their headers, the Simple Mail Transfer Protocol (SMTP) and its extensions (ESMTP), i.e. the protocols used to transfer mail from one system to another.

The 25 or so most important `elective' standards refer to:

  1. Retrieving mail from remote servers
    • Internet Message Access Protocol (IMAP)
    • Post Office Protocol Version 3 (POP3)
  2. Encoding binary data inside text-based email messages, i.e. Multi-part Internet Mail Extensions (MIME)
  3. Encryption, Decryption and Authentication, e.g. PGP, vCard, etc

Non-Standard Email

Most of the non-Internet email systems implement proprietary standards, provided by the likes of Microsoft, IBM and Lotus.

Proprietary Lock-in

Whilst these systems occasionally add functions for which only `proposed' or `experimental' Internet standards may exist, their main effect is to lock data into structures which can only be directly accessed by one company's software.

Proprietary Systems Inter-operate via Internet Standards

People have to communicate with users of different proprietary systems. The Internet is invariably used to transport messages between these systems. To do this, the proprietary mail has to be transformed into a format which obeys the minimum Internet standards, i.e. so that it has an RFC-compliant structure which can be posted by E/SMTP

In most cases, these minimum standards allow basic data to be exchanged between fundamentally incompatible systems. Sadly, one company has a track record of idiosyncratically implementing the elective internet standards, i.e. in ways which make complex data more difficult to access using other companies' software. You know who we mean.

ISO X.400 and X.500 Standards

Although theoretically more comprehensive than the Internet standards, the ISO's X.400 email standard has largely been superseded by them; mainly because the big software houses obstructed or side-stepped it. A subset of the matching X.500 directory services standard has recently been resurrected via the Internet as the Lightweight Directory Access Protocol (LDAP).