Listing the Newsgroups on a Server

This lab demonstrates how to use the twisted.news.nntp module to perform one of the most common tasks in Usenet: retrieving a list of the available newsgroups on a server.

9.1.1. How Do I Do That?

Use a subclass of twisted.news.nntp.NNTPClient as your Protocol. NNTPClient is one of the older protocol implementations in Twisted, and it's starting to show its age. It was developed before the best practices for writing Twisted client protocols had really been understood. In particular, you'll notice that it doesn't make use of Deferreds. Instead, NNTPClient has an API that provides methods for sending commands to the server, and you write handler methods for receiving the server's reply.

To get the list of newsgroups from the server, call the fetchGroups method of NNTPClient. Use the gotAllGroups method as shown in Example 9-1 to handle the result, and getAllGroupsFailed to handle an error.

Example 9-1. nntplist.py

from twisted.news import nntp from twisted.internet import protocol, defer class NNTPGroupListerProtocol(nntp.NNTPClient): def connectionMade(self): nntp.NNTPClient.connectionMade(self) self.fetchGroups( ) def gotAllGroups(self, groups): groupnames = [groupInfo[0] for groupInfo in groups] self.factory.deferred.callback(groupnames) self.quit( ) def getAllGroupsFailed(self, error): self.factory.deferred.errback(error) def connectionLost(self, error): if not self.factory.deferred.called: self.factory.deferred.errback(error) class NNTPGroupListerFactory(protocol.ClientFactory): protocol = NNTPGroupListerProtocol def _ _init_ _(self): self.deferred = defer.Deferred( ) if __name__ == "_ _main_ _": from twisted.internet import reactor def printGroups(groups): for group in groups: print group reactor.stop( ) def handleError(error): print >> sys.stderr, error.getErrorMessage( ) reactor.stop( ) factory = NNTPGroupListerFactory( ) factory.deferred.addCallback(printGroups).addErrback(handleError) reactor.connectTCP('news-server.maine.rr.com', 119, factory) reactor.run( )

Run nntplist.py with the hostname of an NNTP server as the only argument. It will download and print the list of available newsgroups:

$ python nntplist.py freetext.usenetserver.com aaaustin.jobs aacc.general aapg.general ab.agriculture ... zipworld.help zoom.cheesedog zoomnet.general zz.unity.chat zz.unity.netlink zzz.3

9.1.2. How Does That Work?

NNTP doesn't require authentication, so NNTPGroupListerProtocol only has to call one method: self.fetchGroups( ). When the server responds with a list of newsgroups, the gotAllGroups method is called, with the newsgroup list as an argument. This list is a set of tuples in the form (groupName, articleCount). NNTPGroupListerProtocol's gotAllGroups method extracts the group names and uses them to call back the Factory's Deferred object.

The getAllGroupsFailed method is the error handler for fetchGroups. If an error occurs, it will be passed to the factory's Deferred.