Providing POP3 Access to Mailboxes

The POP3 protocol allows mail clients to periodically download new mail from a server. POP3 support is a requirement for most applications that store mail. It's also the easiest way to let people pull their messages out of your application and store them on their computer.

8.3.1. How Do I Do That?

To make a Twisted POP3 server , first write a class to represent a user's Inbox. This class should implement twisted.mail.pop3.IMailbox, which defines methods for reading and deleting messages from a mailbox. Then set up a Portal that checks each user's login credentials and gives them access to their Inbox. Example 8-3 offers POP access to the maildir mailboxes created by the SMTP server in Example 8-1.

Example 8-3. pop3server.py

from twisted.mail import pop3, maildir from twisted.cred import portal, checkers, credentials, error as credError from twisted.internet import protocol, reactor, defer from zope.interface import implements import os class UserInbox(maildir.MaildirMailbox): """ maildir.MaildirMailbox already implements the pop3.IMailbox interface, so methods need to be defined only to override the default behavior. For non-maildir mailboxes, you'd have to implement all of pop3.IMailbox: """ def _ _init_ _(self, userdir): inboxDir = os.path.join(userdir, 'Inbox') maildir.MaildirMailbox._ _init_ _(self, inboxDir) class POP3Protocol(pop3.POP3): debug = True def sendLine(self, line): if self.debug: print "POP3 SERVER:", line pop3.POP3.sendLine(self, line) def lineReceived(self, line): if self.debug: print "POP3 CLIENT:", line pop3.POP3.lineReceived(self, line) class POP3Factory(protocol.Factory): protocol = POP3Protocol portal = None def buildProtocol(self, address): p = self.protocol( ) p.portal = self.portal p.factory = self return p class MailUserRealm(object): implements(portal.IRealm) avatarInterfaces = { pop3.IMailbox: UserInbox, } def _ _init_ _(self, baseDir): self.baseDir = baseDir def requestAvatar(self, avatarId, mind, *interfaces): for requestedInterface in interfaces: if self.avatarInterfaces.has_key(requestedInterface): # make sure the user dir exists userDir = os.path.join(self.baseDir, username) if not os.path.exists(userDir): os.mkdir(userDir) # return an instance of the correct class avatarClass = self.avatarInterfaces[requestedInterface] avatar = avatarClass(userDir) # null logout function (FIXME: explain why) logout = lambda: None return defer.succeed(requestedInterface, avatar, logout) # none of the requested interfaces was supported raise KeyError("None of the requested interfaces is supported") class CredentialsChecker(object): implements(checkers.ICredentialsChecker) credentialInterfaces = (credentials.IUsernamePassword, credentials.IUsernameHashedPassword) def _ _init_ _(self, passwords): "passwords: a dict-like object mapping usernames to passwords" self.passwords = passwords def requestAvatarId(self, credentials): username = credentials.username if self.passwords.has_key(username): realPassword = self.passwords[username] checking = defer.maybeDeferred( credentials.checkPassword, realPassword) # pass result of checkPassword, and the username that was # being authenticated, to self._checkedPassword checking.addCallback(self._checkedPassword, username) return checking else: raise credError.UnauthorizedLogin("No such user") def _checkedPassword(self, matched, username): if matched: # password was correct return username else: raise credError.UnauthorizedLogin("Bad password") def passwordFileToDict(filename): passwords = {} for line in file(filename): if line and line.count(':'): username, password = line.strip( ).split(':') passwords[username] = password return passwords if __name__ == "_ _main_ _": import sys dataDir = sys.argv[1] factory = POP3Factory( ) factory.portal = portal.Portal(MailUserRealm(dataDir)) passwordFile = os.path.join(dataDir, 'passwords.txt') passwords = passwordFileToDict(passwordFile) passwordChecker = CredentialsChecker(passwords) factory.portal.registerChecker(passwordChecker) reactor.listenTCP(110, factory) reactor.run( )

This example, pop3server.py, uses the directory structure created by smtpserver.py in Example 8-1, and illustrated in Figure 8-4. A base directory called mail_storage has directories for each email address, each of which contains a maildir directory called Inbox.

Figure 8-4. The mail directory structure

Before you run pop3server.py, create a file called passwords.txt in your base mail_storage directory. This file should contain a list of usernames and passwords, separated by colons:

testuser@localhost:password bob@localhost:password

Run pop3server.py with the name of the base mail directory as the only argument:

$ python pop3server.py mail_storage

Now configure your mail client to download mail from localhost using POP3. Use a full email address as the username, and the corresponding password. Then you should be able to download all the messages you sent to that email address while running smtpserver.py from Example 8-1. Figure 8-5 illustrates the process.

8.3.2. How Does That Work?

There are two major pieces that pop3.POP3Protocol requires to create a working POP3 server . First, you need to have a class implementing pop3.IMailbox, which gives the server methods for reading and deleting messages in a user's mailbox. Second, you need to have a twisted.cred.portal.Portal object that will take a username and password, make sure they are valid, and then return the mailbox object for that user.

Figure 8-5. Downloading messages from the POP3 server

In Example 8-3, each user's mail is kept in a maildir directory. The maildir.MaildirMailbox class helpfully implements pop3.IMailbox already, so it isn't necessary to write a special class. If you were serving mail that wasn't in a maildir directory, you'd have to implement pop3.IMailbox yourself, by providing the following methods:

def listMessages(self, index=None):

Return a list of integers representing the sizes of all the messages in this mailbox, in bytes. If index is not None, return an integer representing the size of that message.

def getMessage(self, index):

Return a file-like object containing the message at index.

def getUidl(self, index):

Return a unique identifier string for the message at index. This value will be used by POP3 clients to determine whether they've seen a message before. getUidl should never return the same value for different messages, and it should consistently return the same value for a specific message.

def deleteMessage(self, index):

Mark the message at index for deletion. The number of messages in the mailbox should not change, and the message shouldn't actually be deleted until the sync method is called.

def undeleteMessages(self):

Cancel pending deletions. Any messages that have been marked for deletion using deleteMessage should be restored to their previous state.

def sync(self):

Delete any messages that have been marked for deletion, and perform any other housecleaning tasks. This method will be called when a POP3 client sends the QUIT command.

Once you've set up a pop3.IMailbox interface for your mailboxes, you need to give the server a way to load the appropriate mailbox for each user based on their username and password. This is done using the Twisted authentication framework, twisted.cred. (For a complete discussion of twisted.cred, see Chapter 6.)

The MailUserRealm class in Example 8-3 needs to work with pop3.POP3 only, so it offers just one interface: pop3.IMailbox. Still, it's designed in a fairly generic way. The class defines a dictionary called avatarInterfaces, which maps the pop3.IMailbox interface to UserInbox, a class that implements that interface. When requestAvatar is called, MailUserRealm loops through list of requested interfaces and tries to find a matching interface in avatarInterfaces. If it finds a matching interface, it constructs an avatar based on the interface and the requested avatarID. In this case, the avatarID is the email address whose mailbox should be opened. The MailUserRealm checks to make sure that mailbox exists, creates it if needed, and then creates the avatar by passing the user's mailbox directory to the class whose interface matched the requested interface, which in this case will always be UserInbox. Finally, it returns a tuple containing three values: the interface that matched, the avatar (which implements that interface), and a function to clean up the avatar when the user is done with his session. In this case, there isn't anything to do at logout, so MailUserRealm uses an anonymous function that doesn't actually do anything.

The requestAvatarId method takes a credentials object that implements one of the interfaces listed in credentialsInterfaces; in this case, either credentials.IUsernamePassword or credentials.IUsernameHashedPassword . Both of these interfaces have a checkPassword method that returns a Deferred Boolean value saying whether the password matched. Using checkPassword lets your code work with both plain-text passwords and encrypted password hashes. In the specific case of a POP3 server , accepting credentials that implement credentials.IUsernameHashedPassword allows the server to accept MD5 passwords.

Once you've defined classes for your implementations of pop3.IMailbox, portal.IRealm and credentials.ICredentialsChecker, it doesn't take much code to put them together into a working POP3 server. In Example 8-3, POP3Factory has a custom buildProtocol method that creates a POP3Protocol object and sets its portal attribute. POP3Protocol is a simple class that inherits from pop3.POP3 and adds an optional debugging mode: when debug is true, it will print all the commands received from the client and the replies sent back from the server. This technique can be very useful when you want to see exactly what a protocol is doing.