Uploading a File

From a user's perspective, there's no easier way to upload a file than going to a web page, using an HTML form to select the file, and pressing the submit button. Because of this, many web sites have adopted HTTP as a means of allowing file uploads. There are times, however, when you might need to perform a file upload without using a browser. Perhaps you want to develop an application that can upload image files to a photo-sharing service, or HTML documents to a web-based content management system. This lab shows how to use Twisted's HTTP client support to perform a file upload.

3.3.1. How Do I Do That?

First, encode the field/value pairs and file data that you wish to upload as a multipart/form-data MIME document . Neither the Python standard library nor Twisted provides an easy way to do this, but you can do it yourself without too much effort. Then pass the encoded form data as the formdata keyword argument to client.getPage or client.downloadPage, along with POST as the HTTP method. You can then work with the results of getPage or downloadPage as you would any other HTTP response. Example 3-4 shows a script named validate.py that uploads a file to the W3C validation service, saves the response to a local file, and then displays it in the user's browser.

Example 3-4. validate.py

from twisted.web import client import os, tempfile, webbrowser, random def encodeForm(inputs): """ Takes a dict of inputs and returns a multipart/form-data string containing the utf-8 encoded data. Keys must be strings, values can be either strings or file-like objects. """ getRandomChar = lambda: chr(random.choice(range(97, 123))) randomChars = [getRandomChar( ) for x in range(20)] boundary = "---%s---" % ''.join(randomChars) lines = [boundary] for key, val in inputs.items( ): header = 'Content-Disposition: form-data; name="%s"' % key if hasattr(val, 'name'): header += '; filename="%s"' % os.path.split(val.name)[1] lines.append(header) if hasattr(val, 'read'): lines.append(val.read( )) else: lines.append(val.encode('utf-8')) lines.append('') lines.append(boundary) return " ".join(lines) def showPage(pageData): # write data to temp .html file, show file in browser tmpfd, tmp = tempfile.mkstemp('.html') os.close(tmpfd) file(tmp, 'w+b').write(pageData) webbrowser.open('file://' + tmp) reactor.stop( ) def handleError(failure): print "Error:", failure.getErrorMessage( ) reactor.stop( ) if __name__ == "_ _main_ _": import sys from twisted.internet import reactor filename = sys.argv[1] fileToCheck = file(filename) form = encodeForm({'uploaded_file': fileToCheck}) postRequest = client.getPage( 'http://validator.w3.org/check', method='POST', headers={'Content-Type': 'multipart/form-data; charset=utf-8', 'Content-Length': str(len(form))}, postdata=form) postRequest.addCallback(showPage).addErrback(handleError) reactor.run( )

Run validate.py with the name of an HTML file as the first argument:

$ python validate.py test.html

As long as there are no errors, you should get a validation report in your default browser, similar to the report shown in Figure 3-1.

Figure 3-1. Result of running validate.py

3.3.2. How Does That Work?

The W3C validate page, http://validator.w3.org, contains a form that looks like this:

Local File:

You can build an HTTP client that sends the same data a browser would use when submitting that form. The encodeForm function takes a dictionary that maps keys to string values or file-like objects, and returns a string containing the data as a multipart/form-encoded MIME document. When validate.py runs, it opens the file specified as its first argument and passes it to encodeForm as the 'uploaded_file' value of a dictionary. This returns the data that should be posted to the validator service.

validate.py then uses client.getPage to post the form data, passing headers to advise the server of the Content-Length and Content-Type of the data being posted. The showPage callback handler takes the data returned from the validator service and writes it to a temporary file. Then it uses Python's webbrowser module to open the file in the user's default browser.