Running a Twisted Application as a Daemon

Just as servers are different from regular applications, server processes are different from regular processes. A server needs to be available all the time, to all users, so it can't be tied to one user's session. Server processes run in the background and are usually started and stopped automatically by the operating system. In Unix and Unix-like operating systems, such long-running background processes are traditionally knows as daemons.

Twisted comes with a program called twistd that you can use to run your application as a background process. twistd can also help your program to do other things you'd expect from a well-behaved server: write log messages to the system log, run in a chrooted environment where it has limited directory access, and run with the permissions of an unprivileged user.

11.1.1. How Do I Do That?

First, encapsulate each logical service in your application in a class implementing twisted.application.service.IService. Example 11-1 shows how easy this is to do for a typical server based on a Protocol and Factory.

Example 11-1. reverse.py

from twisted.application import service, internet from twisted.internet import protocol, reactor from twisted.protocols import basic def reverse(string): return string[::-1] class ReverserProtocol(basic.LineReceiver): def lineReceived(self, line): if hasattr(self, 'handle_' + line): getattr(self, 'handle_' + line)( ) else: self.sendLine(reverse(line)) def handle_quit(self): self.transport.loseConnection( ) class ReverserFactory(protocol.ServerFactory): protocol = ReverserProtocol class ReverserService(internet.TCPServer): def _ _init_ _(self): internet.TCPServer._ _init_ _(self, 2323, ReverserFactory( ))

Next, write a Python script that defines a variable named application pointing to an object created by calling twisted.application.service.Application . Add your service to the application, and run the script using the command twistd -y yourappname. Example 11-2 does this for the service from Example 11-1.

Example 11-2. reverse_app.py

from twisted.application import service import reverse application = service.Application("Reverser") reverserService = reverse.ReverserService( ) reverserService.setServiceParent(application)

Run this application using twistd. It will start the process in the background, and immediately return:

$ twistd -y reverse_app.py $

At this point, the server is running as a background process. It runs a tiny protocol that sends back a reversed copy of every line it receives, unless that line is quit, in which case it closes the connection. You can connect to the server on port 2323 using Telnet:

$ telnet localhost 2323 Trying 127.0.0.1... Connected to sparky. Escape character is '^]'. hello world! !dlrow olleh quit Connection closed by foreign host.

twistd creates in the current directory a file called twistd.pid, which contains the process ID of the daemon process. To stop the application, kill this process. You can do this easily on Unix-like operating systems by running the command kill 'cat twistd.pid'. twistd also creates a logfile called twistd.log, which you can use to monitor the activity of your server while it's running.

11.1.2. How Does That Work?

The twisted.application.service module provides the interface IService, which represents part of an application that provides a specific service. IService objects have startService and stopService methods that can be used to control the service. Services can be grouped together into a hierarchy, so that each service manages a set of child services. An application is made up of one or more services.

Example 11-1 defines a protocol and factory, ReverserProtocol and ReverserFactory. Instead of starting the factory directly by calling reactor.listenTCP, it creates a subclass of twisted.application.internet.TCPServer . The TCPServer class lets you define a service that sets up a factory to listen on a port. Initialize a TCPServer with the same arguments you'd pass to reactor.connectTCP. The TCPServer won't start the factory immediately, however; it will wait until its startService method is called. TCPServer also provides a stopService method that stops listening on the port.

Example 11-2 takes the ReverserService from Example 11-1 and makes it into an application suitable for running with twistd. To create an application object, call twisted.application.service.Application with a string representing the name of your application. service.Application is actually not a class, but a function that creates a twisted.python.components.Componentized object. Componentized objects are multifaceted: internally, they contain multiple objects, which they can expose in order to implement different interfaces. The application object returned by service.Application is three objects in one. First, it's a twisted.application.service.MultiService object, which is a service that contains other services. When this service is started or stopped, all its child services will be started or stopped along with it. Second, the application is a twisted.application.service.Process object, meaning that it will be the top-level object in a process whose permissions can be managed by the operating system. And third, it's a twisted.persisted.sob.Persistent object, which means that its state can be preserved between sessions by writing it to disk.

Attach a service to an application by calling service.setServiceParent(application). This step registers the service as a child of the application, so it will be started and stopped automatically when the application is run with twistd.

The -y flag tells twistd that you're passing it a Python file with your application in the application variable. twistd is capable of loading applications from other formats, including Python pickle files and XML files, but using -y and a Python script is the most straightforward. You can think of the script you pass to twistd as being the configuration file for your application. It's best to define your application's services elsewhere, and then use a minimal script to tie it together into an application object for twistd.

An interesting thing that twistd does by default is save a pickled version of your application object at shutdown in a file called YourAppName-shutdown.tap. Although it's in a different format than your Python script, this file can also be used to run your application with twistd. After running Example 11-2, you should have a file named Reverser-shutdown.tap in the directory where you started twistd. You could run your application from this file by passing it to twistd with the -f flag:

$ twistd -f Reverser-shutdown.tap

The potentially useful thing about this feature is that the .tap file contains the state of the application object just before shutdown, including any configuration changes made while the program was running. This information could be useful if you want your program to include its own configuration interface, perhaps allowing the administrator to change the port your server will listen on. In practice, however, this knowledge is rarely needed. If you're not planning to use this feature, and don't want .tap files cluttering up your filesystem, disable it by passing the -o flag to twistd.

Another useful twistd flag is -n, which tells the application not to run as a background process. This option is useful for testing purposes. It will run in the foreground and print log messages to stdout.