The copy_reg Module

The copy_reg module provides a registry that you can use to register your own extension types. The pickle and copy modules use this registry to figure out how to process non-standard types.

For example, the standard pickle implementation cannot deal with Python code objects, as shown in the following example:

File: copy-reg-example-1.py import pickle CODE = """ print 'good evening' """ code = compile(CODE, "", "exec") exec code exec pickle.loads(pickle.dumps(code)) good evening Traceback (innermost last): ... pickle.PicklingError: can't pickle 'code' objects

We can work around this by registering a code object handler. Such a handler consists of two parts: a pickler, which takes the code object and returns a tuple that can only contain simple datatypes, and an unpickler, which takes the contents of such a tuple as its arguments. Example 4-14 demonstrates this.

Example 4-14. Using the copy_reg Module to Enable Pickling of Code Objects

File: copy-reg-example-2.py import copy_reg import pickle, marshal, types # # register a pickle handler for code objects def code_unpickler(data): return marshal.loads(data) def code_pickler(code): return code_unpickler, (marshal.dumps(code),) copy_reg.pickle(types.CodeType, code_pickler, code_unpickler) # # try it out CODE = """ print "suppose he's got a pointed stick" """ code = compile(CODE, "", "exec") exec code exec pickle.loads(pickle.dumps(code)) suppose he's got a pointed stick suppose he's got a pointed stick

If you're transferring the pickled data across a network or to another program, the custom unpickler must be available at the receiving end as well.

For the really adventurous, Example 4-15 shows a version that allows you to pickle open file objects.

Example 4-15. Using the copy_reg Module to Enable Pickling of File Objects

File: copy-reg-example-3.py import copy_reg import pickle, types import StringIO # # register a pickle handler for file objects def file_unpickler(position, data): file = StringIO.StringIO(data) file.seek(position) return file def file_pickler(code): position = file.tell() file.seek(0) data = file.read() file.seek(position) return file_unpickler, (position, data) copy_reg.pickle(types.FileType, file_pickler, file_unpickler) # # try it out file = open("samples/sample.txt", "rb") print file.read(120), print "", print pickle.loads(pickle.dumps(file)).read() We will perhaps eventually be writing only small modules, which are identified by name as they are used to build larger ones, so that devices like indentation, rather than delimiters, might become feasible for expressing local structure in the source language. -- Donald E. Knuth, December 1974