Transforming a Local Function Call into a Remote Procedure

We begin our exploration of RPC programming by converting a simple program with a single local function call into a clientserver configuration with a single RPC. Once generated, this RPC-based program can be run in a distributed setting whereby the server process, which will contain the function to be executed, can reside on a host different from the client process. The program that we will convert (Program 9.2) is a C program [4] that invokes a single local function, print_hello , which generates the message Hello, world . As written, the print_hello function will display its message and return to the function main the value returned from printf . The returned value indicates whether printf was successful in carrying out its action. [5]

[4] Up to this point, our examples have been primarily C++-based. Due to the inability of the compiler to handle full blown C++ code in conjunction with rpcgen -generated output, we will stick to C program examples in this section. Think of this as an opportunity to brush up on your C programming skills!

[5] Many programmers are not aware that printf returns a value. However, a pass of any C program with a printf function through the lint utility will normally return a message indicating that the value returned by printf is not being used.

Program 9.2 A simple C program to display a message.

File : hello.c /* A C program with a local function */ #include + int print_hello( ); int main( ){ printf("main : Calling function. "); if (print_hello()) 10 printf("main : Mission accomplished. "); else printf("main : Unable to display message."); return 0; } + int print_hello( ) { return printf("funct: Hello, world. "); }

In its current configuration, the print_hello function and its invocation reside in a single source file. The output of Program 9.2 when compiled and run is shown in Figure 9.5.

Figure 9.5 Output of Program 9.2.

linux$ hello main : Calling function. funct: Hello, world. main : Mission accomplished

The first step in converting a program with a local function call to an RPC is for the programmer to create a protocol definition file. This file will help the system keep track of what procedures are to be associated with the server program. The definition file is also used to define the data type returned by the remote procedure and the data types of its arguments. When using RPC, the remote procedure is part of a remote program that runs as the server process. The RPC language is used to define the remote program and its component procedures. The RPC language is actually XDR with the inclusion of two extensionsthe program and version types. Appendix C addresses the syntax of the RPC language. For the diligent, the manual pages on xdr provide a good overview of XDR data type definitions and syntax.

Figure 9.6 contains the protocol definition file for the print_hello function. Syntactically, the RPC language is a mix of C and Pascal. By custom, the extension for protocol definition files is .x .

The keyword program marks the user -defined identifier DISPLAY_PRG as the name of the remote procedure program. [6] The program name, like the program name in a Pascal program, does not need to be the same as the name of the executable file. The program block encloses a group of related remote procedures. Nested within the program definition block is the keyword version followed by a second user-generated identifier, DISPLAY_VER , which is used to identify the version of the remote procedure. It is permissible to have several versions of the same procedure, each indicated by a different integer value. The ability to have different versions of the same procedure eases the upgrade process when updating software by facilitating backward compatibility. If the number of arguments, the data type of an argument, or the data type returned by the function change, the version number should be changed.

[6] Most often, the identifiers placed in the protocol definition file are in capitals. Note that this is a convention, not a requirement.

Figure 9.6 Protocol definition file hello.x .

File : hello.x /* This is the protocol definition file. The programmer writes this file using the RPC language. This file is passed to the protocol generator rpcgen. Every remote procedure is part of + a remote program. Each procedure has a name and number. A version number is also supplied so different versions of the same procedure may be generated. */ program DISPLAY_PRG { 10 version DISPLAY_VER { int print_hello( void ) = 1; } = 1; } = 0x20000001;

As this is our first pass at generating a remote procedure, the version number is set to 1 after the closing brace for the version block. Inside the version block is the declaration for the remote procedure (line 11). [7] A procedure number follows the remote procedure declaration. As there is only one procedure defined, the value is set to 1. An eight-digit hexadecimal program number follows the closing brace for the program block. The program, version, and procedure numbers form a triplet that uniquely identifies a specific remote procedure. To prevent conflicts, the numbering scheme shown in Table 9.2 should be used in assigning version numbers .

[7] If the procedure name is placed in capitals, the RPC compiler, rpcgen , will automatically convert it to lowercase during compilation.

Protocol specifications can be registered with Sun by sending a request (including the protocol definition file) to rpc@sun.com . Accepted specifications will receive a unique program number from Sun (in the range 000000001FFFFFFF).

Table 9.2. RPC Program Numbers.

A check of the file /etc/rpc on your system will display a list of some of the RPC programs (and their program numbers) known to the system.

As shown below, the name of the protocol definition file is passed to the RPC protocol compiler, rpcgen , on the command line

$ rpcgen -C hello.x

The rpcgen compiler produces the requisite C code to implement the defined RPCs. There are a number of command-line options for rpcgen , of which we will explore only a limited subset. A summary of the command-line options and syntax for rpcgen is given in Figure 9.7.

Figure 9.7 Command-line options for rpcgen .

usage: rpcgen infile rpcgen [-abkCLNTM][-Dname[=value]] [-i size] [-I [-K seconds]] [-Y path] infile rpcgen [-c -h -l -m -t -Sc -Ss -Sm] [-o outfile] [infile] rpcgen [-s nettype]* [-o outfile] [infile] rpcgen [-n netid]* [-o outfile] [infile] options: -a generate all files, including samples -b backward compatibility mode (generates code for SunOS 4.1) -c generate XDR routines -C ANSI C mode -Dname[=value] define a symbol (same as #define) -h generate header file -i size size at which to start generating inline code -I generate code for inetd support in server (for SunOS 4.1) -K seconds server exits after K seconds of inactivity -l generate client side stubs -L server errors will be printed to syslog -m generate server side stubs -M generate MT-safe code -n netid generate server code that supports named netid -N supports multiple arguments and call-by-value -o outfile name of the output file -s nettype generate server code that supports named nettype -Sc generate sample client code that uses remote procedures -Ss generate sample server code that defines remote procedures -Sm generate makefile template -t generate RPC dispatch table -T generate code to support RPC dispatch tables -Y path directory name to find C preprocessor (cpp)

In our invocation, we have specified the -C option requesting rpcgen output conform to the standards for ANSI C. While some versions of rpcgen generate ANSI C output by default, the extra keystrokes ensure rpcgen generates the type of output you want. When processing the hello.x file, rpcgen creates three output filesa header file, a client stub, and a server stub file. Again, by default rpcgen gives the same name to the header file as the protocol definition file, replacing the .x extension with .h . [8] In addition, the client stub file is named hello_clnt.c (the rpcgen source file name with _clnt.c appended), and the server stub file is named hello_svc.c (using a similar algorithm). Should the default naming convention be too restrictive , the header file as well as the client and server stub files can be generated independently and their names uniquely specified. For example, to generate the header file with a uniquely specified name, rpcgen would be passed the following options and file names:

[8] This can be a troublesome default if, per chance, you have also generated your own local header file with the same name and extension.

linux$ rpcgen -C -h -o unique_file_name hello.x

With this invocation, rpcgen will generate a header file called unique_file_name.h . Using a similar technique, unique names for the client and server stub files can be specified with the -Sc and -Ss options (see Figure 9.7 for syntax details).

The contents of the header file, hello.h , generated by rpcgen is shown in Figure 9.8.

Figure 9.8 File hello.h generated by rpcgen from the protocol definition file hello.x .

File : hello.h /* * Please do not edit this file. * It was generated using rpcgen. */ + #ifndef _HELLO_H_RPCGEN #define _HELLO_H_RPCGEN #include 10 #ifdef __cplusplus extern "C" { #endif + #define DISPLAY_PRG 0x20000001 #define DISPLAY_VER 1 20 #if defined(__STDC__) defined(__cplusplus) #define print_hello 1 extern int * print_hello_1(void *, CLIENT *); extern int * print_hello_1_svc(void *, struct svc_req *); extern int display_prg_1_freeresult (SVCXPRT *, xdrproc_t, caddr_t); + #else /* K&R C */ #define print_hello 1 extern int * print_hello_1(); extern int * print_hello_1_svc(); 30 extern int display_prg_1_freeresult (); #endif /* K&R C */ #ifdef __cplusplus } + #endif #endif /* !_HELLO_H_RPCGEN */

The hello.h file created by rpcgen is referenced as an include file in both the client and server stub files. The #ifndef _HELLO_H_RPCGEN , #define _HELLO_H_RPCGEN , and #endif preprocessor directives prevent the hello.h file from being included multiple times. Within the file hello.h , the inclusion of the file , as noted in its internal comments, ". . . just includes the billions of rpc header files necessary to do remote procedure calling ." [9] The variable __cplusplus (see line 20) is used to determine if a C++ programming environment is present. In a C++ environment, the compiler internally adds a series of suffixes to function names to encode the data types of its parameters. These new "mangled" function names allow C++ to check functions to ensure parameters match correctly when the function is invoked. The C compiler does not provide the mangled function names that the C++ compiler needs. The C++ compiler has to be warned that standard C linking conventions and non-mangled function names are to be used. This is accomplished by the lines following the #ifdef __cplusplus compiler directive.

[9] While this comment is somewhat tongue-in-cheek, it is not all that farfetched (check it out)!

The program and version identifiers specified in the protocol definition file are found in the hello.h file, as defined constants (lines 17 and 18). These constants are assigned the value specified in the protocol definition file. Since we indicated the -C option to rpcgen (standard ANSI C), the if branch of the preprocessor directive (i.e., #if defined (__STDC__) ) contains the statements we are interested in. If the remote procedure name in the protocol definition file was specified in uppercase, it is mapped to lowercase in the header file. The procedure name is defined as an integer and assigned the value previously given as its procedure number. Note that we will find this defined constant used again in a switch statement in the server stub to select the code to be executed when calling the remote procedure.

Following this definition are two print_hello function prototypes . The first prototype, print_hello_1 , is used by the client stub file. The second, print_hello_1_svc , is used by the server stub file. The naming convention used by rpcgen is to use the name of the remote procedure as the root and append an underscore (_), version number (1), for the client stub, and underscore, version number, underscore , and svc for the server. The else branch of the preprocessor directive contains a similar set of statements that are used in environments that do not support standard C prototyping.

Before we explore the contents of the client and server stub files created by rpcgen , we should look at how to split our initial program into client and server components . Once the initial program (for example hello.c ) is split, and we have run rpcgen , we will have the six files shown in Figure 9.9 available to us.

Figure 9.9. Client-server files and relationships.

We begin with writing the client component. As in the initial program, the client invokes the print_hello function. However, in our new configuration, the code for the print_hello function, which used to be a local function, resides in a separate program that is run by the server process. The code for the client component program, which has been placed in a file named hello_client.c , is shown in Program 9.3.

Program 9.3 The client program hello_client.c.

File : hello_client.c /* The CLIENT program: hello_client.c This will be the client code executed by the local client process. */ + #include #include "hello.h" /* Generated by rpcgen from hello.x */ int main(int argc, char *argv[]) { CLIENT *client; 10 int *return_value, filler; char *server; /* We must specify a host on which to run. We will get the host name from the command line as argument 1. + */ if (argc != 2) { fprintf(stderr, "Usage: %s host_name ", *argv); exit(1); } 20 server = argv[1]; /* Generate the client handle to call the server */ if ((client=clnt_create(server, DISPLAY_PRG, + DISPLAY_VER, "tcp")) == (CLIENT *) NULL) { clnt_pcreateerror(server); exit(2); } printf("client : Calling function. "); 30 return_value = print_hello_1((void *) &filler, client); if (*return_value) printf("client : Mission accomplished. "); else printf("client : Unable to display message. "); + return 0; }

While much of the code is similar to the original hello.c program, some changes have been made to accommodate the RPC. Let's examine these changes point by point. At line 6 the file hello.h is included. This file, generated by rpcgen and whose contents were discussed previously, is assumed to reside locally.

In this example, we pass information from the command line to the function main in the client program. Therefore, the empty parameter list for main has been replaced with standard C syntax to reference the argc and argv parameters. Following this, in the declaration section of the client program, a pointer to the data type CLIENT is allocated. A description of the CLIENT data type is shown in Figure 9.10.

The CLIENT typedef is found in the include file . The reference to the CLIENT data structure will be used when the client handle is generated. Following the declarations in Program 9.3 is a section of code to obtain the host name on which the server process will be running. In the previous invocation, this was not a concern, as all code was executed locally. However, in this new configuration, the client process must know the name of the host where the server process is located; it cannot assume the server program is running on the local host. The name of the host is passed via the command line as the first argument to hello_client . As written, there is no checking to determine if a valid, reachable host name has been passed. The client handle is created next (line 24). This is done with a call to the clnt_create library function. The clnt_create library function, which is part of a suite of remote procedure functions, is summarized in Table 9.3.

Figure 9.10 The CLIENT data structure.

struct CLIENT { AUTH *cl_auth; /* authenticator */ struct clnt_ops { enum clnt_stat (*cl_call) (CLIENT *, u_long, xdrproc_t, caddr_t, xdrproc_t, caddr_t, struct timeval); /* call remote procedure */ void (*cl_abort) (void); /* abort a call */ void (*cl_geterr) (CLIENT *, struct rpc_err *); /* get specific error code */ bool_t (*cl_freeres) (CLIENT *, xdrproc_t, caddr_t); /* frees results */ void (*cl_destroy) (CLIENT *); /* destroy this structure */ bool_t (*cl_control) (CLIENT *, int, char *); /* the ioctl() of rpc */ } *cl_ops; caddr_t cl_private; /* private stuff */ };

Table 9.3. Summary of the clnt_create Library Call.

The clnt_create library call requires four arguments. The first, host , a character string reference, is the name of the remote host where the server process is located. The next two arguments, prog and vers , are, respectively, the program and version number. These values are used to indicate the specific remote procedure. Notice the defined constants generated by rpcgen are used for these two arguments. The proto argument is used to designate the class of transport protocol. In Linux, this argument may be set to either tcp or udp . Keep in mind that UDP (Unreliable Datagram Protocol) encoded messages are limited to 8KB of data. Additionally, UDP is, by definition, less reliable than TCP (Transmission Control Protocol). However, UPD does require less system overhead.

Table 9.4. Summary of the clnt_pcreateerror Library Call.

If the clnt_create library call fails, it returns a NULL value. If this occurs, as shown in the example, the library routine clnt_pcreateerror can be invoked to display a message that indicates the reason for failure. See Table 9.4.

The error message generated by clnt_pcreateerror , which indicates why the creation of the client handle failed, are appended to the string passed as clnt_pcreateerror 's single argument (see Table 9.5 for details). The argument string and the error message are separated with a colon , and the entire message is followed by a newline. If you want more control over the error messaging process, there is another library call, clnt_spcreateerror (char *s ) , that will return an error message string that can be incorporated in a personalized error message. In addition, the cf_stat member of the external structure rpc_createerr may be examined directly to determine the source of the error.

Table 9.5. clnt_creat Error Messages.

Returning to the client program, the prototype for the print_hello function has been eliminated. The function prototype is now in the hello.h header file. The invocation of the print_hello function uses its new name, print_hello_1 . The function now returns not an integer value but a pointer to an integer, and has two arguments (versus none). By design, all RPCs return a pointer reference. In general, all arguments passed to the RPC are passed by reference, not by value. As this function originally did not have any parameters, the identifier filler is used as a placeholder. The second argument to print_hello_1 , client , is the reference to the client structure returned by the clnt_create call. The server component, which now resides in the file hello_server.c , is shown in Program 9.4.

Program 9.4 The hello_server.c component.

File : hello_server.c /* The SERVER program: hello_server.c This will be the server code executed by the "remote" process */ + #include #include "hello.h" /* is generated by rpcgen from hello.x */ int * print_hello_1_svc(void * filler, struct svc_req * req) { static int ok; 10 ok = printf("server : Hello, world. "); return (&ok); }

The server component contains the code for the print_hello function. Notice that to accommodate the RPC, several things have been added and/or modified. First, as noted in the discussion of the client program, the print_hello function now returns an integer pointer, not an integer value (line 7). In this example, the address that is to be returned is associated with the identifier ok . This identifier is declared to be of storage class static (line 9). It is imperative that the return identifier referenced be of type static , as opposed to local. Local identifiers are allocated on the stack, and a reference to their contents would be invalid once the function returns. The name of the function has had an additional _1 appended to it (the version number). As the -C option was used with rpcgen , the auxiliary suffix _svc has also been added to the function name. Do not be concerned by the apparent mismatch of function names. The mapping of the function invocation as print_hello_1 in the client program to print_hello_1_svc in the server program is done by the code found in the stub file hello_svc.c produced by rpcgen .

The first argument passed to the print_hello function is a pointer reference. If needed, multiple items (representing multiple parameters) can be placed in a structure and the reference to the structure passed. In newer versions of rpcgen , the -N flag can be used to write multiple argument RPCs when a parameter is to be passed by value, not reference, or when a value, not a pointer reference, is to be returned by the RPC. A second argument, struct svc_req *req , has also been added. This argument will be used to communicate invocation information.

The client component (program) is compiled first. When only a few files are involved, a straight command-line compilation sequence is adequate. Later we will discuss how to generate a make file to automate the compilation process. The compiler is passed the names of the two client files, hello_client.c (which we wrote) and hello_clnt.c (which was generated by rpcgen ). We specify the executable to be placed in the file client . Figure 9.11 shows details of the compilation command.

Figure 9.11 Compiling the client component.

linux$ gcc hello_client.c hello_clnt.c -o client

The server component (program) is compiled in a similar manner (Figure 9.12).

Figure 9.12 Compiling the server component.

linux$ gcc hello_server.c hello_svc.c -o server

Initially, we test the program by running both the client and server programs on the same workstation. We begin by invoking the server by typing its name on the command line. The server process is not automatically placed in the background, and thus a trailing & is needed. [10] A check of the ps command will verify the server process is running (see Figure 9.13).

[10] This is just the opposite of what happens in a Sun Solaris environment where no trailing & is needed, as the process is automatically placed in the background.

Figure 9.13 Running the server program and checking for its presence with ps.

linux$ server & [1] 21149 [linux$ ps -ef grep server . . . gray 21149 15854 0 08:09 pts/5 00:00:00 server gray 21154 15854 0 08:10 pts/5 00:00:00 grep server

The ps command reports that the server process, in this case process ID 21149, is in memory. Its parent process ID is 15854 (in this case the login shell) and its associated controlling terminal device is listed as pts/5 . The server process will remain in memory even after the user who initiated it has logged out. When generating and testing RPC programs, it is important the user remember to remove extraneous RPC-based server type processes before they log out.

When the process is run locally, the client program is invoked by name and passed the name of the current workstation. When this is done, the output will be as shown in Figure 9.14. Notice that since our system has an existing program called client that resides in the /usr/sbin directory, the call to our client program is made with a relative reference (i.e., ./client ).

Figure 9.14 Running the client program on the same host as the server program.

linux$ ./client linux client : Calling function. server : Hello, world. client : Mission accomplished.

While our clientserver application still needs some polishing, we can test it in a setting whereby the server runs on one host and the client on another. Say we have the setting shown in Figure 9.15, where one host is called medusa and the other linux .

Figure 9.15. Running the client program on a remote host.

On the host linux the server program is run in the background. On the host medusa the client program is passed the name of the host running the server program. Interestingly, on the host medusa the messages Calling function. and Mission accomplished . are displayed, but the message Hello, world . is displayed on the host linux . This is not surprising, as each program writes to its standard output, which in turn is associated with a controlling terminal (in our example this is the same terminal that is associated with the user's login shell). However, it is just as likely that the server program will write to its standard output, but what it has written will not be seen. This happens when there is no controlling terminal device associated with the server process. Remember that the server process remains in memory until removed. It is not removed when the user logs out. However, when the user does log out, the operating system drops the controlling terminal device for the process (a call to ps will list the controlling terminal device for the process as ? ). If, in a standard setting, there is no controlling terminal device associated with a process, anything the process sends to standard output goes into the bit bucket!

There are several ways of correcting this problem. First, the output from the server could be hardcoded to be displayed on the console. In this scenario, the server would, upon invocation, execute an fopen on the /dev/console device. The FILE pointer returned by the fopen call could then be used with the fprintf function to display the output on the console. Unfortunately, there is a potential problem with this solution: The user may not have access to the console device. If this is so, the fopen will fail. A second approach is to pass the console device of the client process to the server as the first parameter of the RPC. This is a somewhat better solution, but will still fail when the client and server processes are on different workstations with different output devices. A third approach is to have the server process return its message to the client and have the client display it locally.

We should also examine the two RPC stub files generated by rpcgen . The hello_clnt.c file is quite small (Figure 9.16). This file contains the actual call to the print_hello_1 function.

Figure 9.16 The hello_clnt.c file.

File : hello_clnt.c /* * Please do not edit this file. * It was generated using rpcgen. */ + #include /* for memset */ #include "hello.h" /* Default timeout can be changed using clnt_control() */ 10 static struct timeval TIMEOUT = { 25, 0 }; int * print_hello_1(void *argp, CLIENT *clnt) { static int clnt_res; + memset((char *)&clnt_res, 0, sizeof(clnt_res)); if (clnt_call (clnt, print_hello, (xdrproc_t) xdr_void, (caddr_t) argp, (xdrproc_t) xdr_int, (caddr_t) &clnt_res, TIMEOUT) != RPC_SUCCESS) { 20 return (NULL); } return (&clnt_res); }

As we are using rpcgen to reduce the complexity of the RPC, we will not formally present the clnt_call . However, in passing, we note that the clnt_call function (which actually does the RPC) is passed, as its first argument, the client handle that was generated from the previous call to clnt_creat . The second argument for clnt_call is obtained from the hello.h include file and is actually the print _ hello constant therein. The third and fifth arguments are references to the XDR data encoding/ decoding routines. Sandwiched between these arguments is a reference, argp , to the initial argument that will be passed to the remote procedure by the server process. The sixth argument for clnt_creat is a reference to the location where the return data will be stored. The seventh and final argument is the TIMEOUT value. While the cautionary comments indicate you should not edit this file, and in general you should not, the TIMEOUT value can be changed from the default of 25 to some other reasonable user-imposed maximum.

The code in the hello_svc.c file is much more complex and, in the interest of space, not presented here. Interested readers are encouraged to enter the protocol definition in hello.x and to generate and view the hello_svc.c file. At this juncture it is sufficient to note that the hello_svc.c file contains the code for the server process. Once invoked, the server process will remain in memory. When notified by a client process, it will execute the print_hello_1_svc function.