Accepting Mail with SMTP

As described in Chapter 7, SMTP is the Simple Mail Transfer Protocol, the basic means by which email messages are delivered on the Internet. SMTP is the most popular messaging protocol in use today: everyone uses email, and email uses SMTP.

This lab demonstrates how to write a simple SMTP server using Twisted. It shows how to accept SMTP connections, decide what to do with the incoming message based on the email address, and then process the message data.

8.1.1. How Do I Do That?

Write classes to implement the smtp.IMessage and smtp.IMessageDelivery interfaces. Then create a Factory that uses your implementation of IMessageDelivery to initialize the smtp.SMTP protocol. Example 8-1 accepts email for all addresses in a given domain and stores the messages to maildir directories.

Example 8-1. smtpserver.py

from twisted.mail import smtp, maildir from zope.interface import implements from twisted.internet import protocol, reactor, defer import os from email.Header import Header class MaildirMessageWriter(object): implements(smtp.IMessage) def _ _init_ _(self, userDir): if not os.path.exists(userDir): os.mkdir(userDir) inboxDir = os.path.join(userDir, 'Inbox') self.mailbox = maildir.MaildirMailbox(inboxDir) self.lines = [] def lineReceived(self, line): self.lines.append(line) def eomReceived(self): # message is complete, store it print "Message data complete." self.lines.append('') # add a trailing newline messageData = ' '.join(self.lines) return self.mailbox.appendMessage(messageData) def connectionLost(self): print "Connection lost unexpectedly!" # unexpected loss of connection; don't save del(self.lines) class LocalDelivery(object): implements(smtp.IMessageDelivery) def _ _init_ _(self, baseDir, validDomains): if not os.path.isdir(baseDir): raise ValueError, "'%s' is not a directory" % baseDir self.baseDir = baseDir self.validDomains = validDomains def receivedHeader(self, helo, origin, recipients): myHostname, clientIP = helo headerValue = "by %s from %s with ESMTP ; %s" % ( myHostname, clientIP, smtp.rfc822date( )) # email.Header.Header used for automatic wrapping of long lines return "Received: %s" % Header(headerValue) def validateTo(self, user): if not user.dest.domain in self.validDomains: raise smtp.SMTPBadRcpt(user) print "Accepting mail for %s..." % user.dest return lambda: MaildirMessageWriter( self._getAddressDir(str(user.dest))) def _getAddressDir(self, address): return os.path.join(self.baseDir, "%s" % address) def validateFrom(self, helo, originAddress): # accept mail from anywhere. To reject an address, raise # smtp.SMTPBadSender here. return originAddress class SMTPFactory(protocol.ServerFactory): def _ _init_ _(self, baseDir, validDomains): self.baseDir = baseDir self.validDomains = validDomains def buildProtocol(self, addr): delivery = LocalDelivery(self.baseDir, self.validDomains) smtpProtocol = smtp.SMTP(delivery) smtpProtocol.factory = self return smtpProtocol if __name__ == "_ _main_ _": import sys mailboxDir = sys.argv[1] domains = sys.argv[2].split(",") reactor.listenTCP(25, SMTPFactory(mailboxDir, domains)) from twisted.internet import ssl # SSL stuff here... and certificates... reactor.run( )

This example uses the SMTP standard TCP port 25, so it can receive mail from other servers. If you're already running another SMTP server, you'll need to stop it before you run smtpserver.py, to make port 25 available. Also, most operating systems don't give ordinary users the right to run services on TCP ports below 1024, which are reserved for system services. So you'll have to run smtpserver.py as root or from an administrator account.

Run smtpserver.py with two arguments: the directory to use for storing messages and a comma-delimited list of domains for which to accept mail. You can tell it to handle mail for as many domains as you like, but for the purposes of this example, make one of them localhost:

$ python smtpserver.py mail_storage localhost,example.com

Now that the server is running, you can try sending some mail. In the real world, you'd run your SMTP server on an Internet-connected computer with a public IP address, where all the the other SMTP servers in the world (including your ISP's SMTP server, which your regular email client is configured to use for outgoing mail) could connect to it. However, let's assume that you're running smtpserver.py, and the other examples in this chapter, on a computer that's behind the firewall on your local network. In this case, it's best to use an email client configured to send mail through localhost.

Try sending a message to test@localhost, as shown in Figure 8-1.

Figure 8-1. Sending a test email to a local SMTP server

Your server should print a couple of lines showing that it received the message:

Accepting mail for test@localhost... Message data complete.

You can then take a look at the message file that was created. smtpserver.py writes messages in maildir format, where each mailbox is a directory with the subdirectories new, cur, and tmp, and each message is a file with a unique name. Message files in a maildir directory are initially stored in new, and then moved to cur to indicate that they've been read. If you told smtpserver.py to use mail_storage as the base directory for storing email, the message you sent should have a unique filename in the directory mail_storage/test@localhost/Inbox/new. The contents of that file will be the message you sent:

$ cd mail_storage $ ls test@localhost $ cd test@localhost $ ls Inbox/new 1115584078.M4515569P19924Q4.sparky $ cat Inbox/new/1115584078.M4515569P19924Q4.sparky Received: by [127.0.0.1] from 127.0.0.1 with ESMTP ; Sun, 08 May 2005 16:27:58 -0400 Message-ID: <427E764E.1000900@localhost> Date: Sun, 08 May 2005 16:27:58 -0400 From: Test User User-Agent: Mozilla Thunderbird 1.0.2 (X11/20050404) To: test@localhost Subject: My Test Message Testing my SMTP server!

8.1.2. How Does That Work?

The smtp.SMTP Protocol class provides a very clean, high-level interface. Instead of asking you respond directly to the commands coming from the client, smtp.SMTP asks you to give it an object implementing the smtp.IMessageDelivery interface. smtp.IMessageDelivery requires three methods: receivedHeader, validateTo, and validateFrom.

The receivedHeader method is used to generate a Received: header that will be added to an incoming email message. SMTP servers are responsible for adding a Received header when they accept a message; these headers can be used later to see the path the message took en route to being delivered. receivedHeader takes three arguments. The first, helo, is a tuple containing two strings: the server name by which the the client addressed the server when it said HELO, and the client's IP address. The second argument, origin, is an smtp.Address object identifying who the message is coming from. The third argument, recipients, is a list of smtp.Address objects identifying who the message is being delivered to. From all that information, receivedHeader is asked to return a simple response: a string containing a valid SMTP Received header.

According to RFC 821, the Received header should be in the following form:

Received: FROM domain BY domain [VIA link] [WITH protocol] [ID id] [FOR path] ; timestamp

The VIA, WITH, ID, and FOR parts are optional. The LocalDelivery object used in Example 8-1 has a basic implementation of receivedHeader that generates a valid header string.

The next method required by smtp.IMessageDelivery is validateFrom. This method gives your server a chance to accept or reject messages based on the address it's coming from. For example, if you had an application that allowed users to post to their weblogs through email, you might use validateFrom to make sure the message was coming from the email address of a user who had permission to post. (Keep in mind, though, that this isn't a bulletproof security mechanism; it's trivially easy to forge the sending address on an email message.) validateFrom takes two arguments: a tuple with the hostname used in by the client when it said HELO and the client's IP address, and an smtp.Address object identifying the sender. In Example 8-1, the LocalDelivery object has a validateFrom method that always returns the sender's address, indicating that it's willing to accept mail from that sender. To reject a sender, you'd raise an smtp.SMTPBadSender exception instead.

The third method of smtp.IMessageDelivery is the most important. validateTo takes a single smtp.User object (which contains the address of the recipient and information about where the message came from) as an argument. It then either raises an smtp.SMTPBadRecipient exception, or returns a function that returns an object implementing smtp.IMessage .

That's a little confusing, and worth repeating: validateTo should return a function that will return an object implementing smtp.IMessage when it is called (with no arguments). The easy way to handle this quirk of the IMessageDelivery API is to return lambda: myObject instead of myObject, as demonstrated in Example 8-1.

The SMTP server in Example 8-1 is a little different from the typical email server. It doesn't have a fixed list of users, but will accept mail for any address within one of its domains. As long as the domain is valid, it assigns that user a directory in the form baseDir/user@domain, and passes the directory to a MaildirMessageWriter object, which implements smtp.IMessage.

smtp.IMessage defines an interface for receiving an email message. There are three functions in smtp.IMessage. The first, lineReceived, will be called once for each line of the incoming email message, with the line data (which does not include the trailing newline) as an argument. The second method, eomReceived, is called after the entire message has been received. Return a Deferred result from eomReceived to let the client know when the message has been processed successfully. The third method, connectionLost, is called only if the connection is broken while message data is being received, and indicates that your class should discard whatever incomplete data has been received so far.

In Example 8-1, MaildirMessageWriter uses the maildir.MaildirMailbox class provided by twisted.mail to write to write to the Inbox maildir in each user's directory. It isn't necessary to check whether the Inbox directory exists; maildir.MaildirMailbox will create it automatically.