C++ Network Programming, Volume I: Mastering Complexity with ACE and Patterns

Motivation

Section 2.3 on page 37 described the complexities associated with direct use of I/O handles. The fd_set represents another source of accidental complexity in the following areas:

• The macros or functions that platforms supply to manipulate and scan an fd_set must be used carefully to avoid processing handles that aren't active and to avoid corrupting an fd_set inadvertently.

• The code to scan for active handles is often a hot spot because it executes continually in a tight loop. It's therefore an obvious candidate for optimization to improve performance.

• The fd_set is defined in system-supplied header files, so its representation is exposed to programmers. Naturally, it's tempting to make use of this platform-internal knowledge in "creative" ways. However, doing so causes portability problems because representations vary widely across platforms.

• There are subtle nonportable aspects of fd_set when used in conjunction with the select() function.

To illustrate the latter point, let's revisit the function signature for the select() function:

int select (int width, // Maximum handle plus 1 fd_set *read_fds, // Set of "read" handles fd_set *write_fds, // Set of "write" handles fd_set *except_fds, // Set of "exception" handles struct timeval *timeout);// Time to wait for events

The three fd_set arguments specify the handles to use when selecting for each event type. The timeout argument is used to specify a time limit to wait for events to occur. The particular details concerning these parameters vary in only minor ways across platforms. Due to the changing nature of I/O handles (and, thus, fd_set ) across platforms, however, the meaning of the first argument ( width ) varies widely across operating systems:

• On UNIX, I/O handles start at the value 0 and increase to an OS-defined maximum limit. The meaning of the width parameter on UNIX is therefore "how far into the fd_set to scan for handles"; that is, it's defined as the highest numbered handle in any of the three fd_set parameters, plus 1.

• On Win32, socket handles are implemented as pointers rather than as low-numbered integers. The width argument to select() is therefore ignored completely. Instead, each fd_set contains its own handle count.

• On other platforms, where handles may be integers that don't start at 0, the definition may differ yet again.

The size - related values in an fd_set are also computed differently depending on the characteristics of the platform's representation and its select() semantics. Any applications written directly to native OS APIs must therefore adapt to all these subtle differences, and each project may end up redesigning and reimplementing code that manages the proper value to supply for width . Addressing these portability differences in each application is tedious and error prone, which is why ACE provides the ACE_Handle_Set class.

Class Capabilities

The ACE_Handle_Set class uses the Wrapper Facade pattern [SSRB00] to guide the encapsulation of fd_set s. This class provides the following capabilities:

• It enhances the portability, ease of use, and type safety of event-driven applications by simplifying the use of fd_set and select() across OS platforms

• It tracks and adjusts the fd_set size-related values automatically as handles are added and removed

Since differences in fd_set representations are hidden in the implementation of ACE_Handle_Set , application code can be written once and then simply recompiled when it's ported to a new platform.

The interface for ACE_Handle_Set is shown in Figure 7.1 and its key methods are shown in the following table:

Example

Sections 4.4.3 and 5.1 outlined some of the problems associated with iterative servers. Using synchronous event demultiplexing to implement the reactive server model is one strategy to avoid these problems. We now show an example that enhances the iterative logging server implementation from Chapter 4 by using select() together with the ACE_Handle_Set class. This server demultiplexes the following two types of events:

1. Arrival of new connections from clients

2. Arrival of log records on client connections

We put our implementation in a header file named Reactive_Logging_Server.h , which starts by including ACE's Handle_Set.h file and the Iterative_Logging_Server.h file. We derive a new class, Reactive_Logging_Server , from the Iterative_Logging_Server class defined on page 92. We reuse its data members and define two more that are used to manage multiple clients.

#include "ace/Handle_Set.h" #include "Iterative_Logging_Server.h" class Reactive_Logging_Server : public Iterative_Logging_Server { protected: // Keeps track of the acceptor socket handle and all the // connected stream socket handles. ACE_Handle_Set master_handle_set_; // Keep track of handles marked as active by <select>. ACE_Handle_Set active_handles_; // Other methods shown below... };

To implement reactive server semantics, we override the hook methods in Iterative_Logging_Server , starting with open () : ^[1]

^[1] To save space, we omit more of the error handling than in our earlier iterative logging server implementation.

virtual int open (u_short logger_port) { Iterative_Logging_Server::open (logger_port); master_handle_set_.set_bit (acceptor () .get_handle ()); acceptor ().enable (ACE_NONBLOCK); return 0; }

After calling down to its parent class's open() method to initialize the acceptor, we call the set_bit() method on the master_handle_set_ to keep track of the acceptor's socket handle. We also set the acceptor into non-blocking mode for the reasons described in Sidebar 14.

Next , we implement the wait_for_multiple_events() method for the reactive server. We copy master_handle_set_ into active_handles_ , which will be modified by select() . We need to calculate the width argument for select() using a cast to compile on Win32 (the actual value passed to select() is ignored on that platform). The second parameter to select() is a pointer to the underlying fd_set in active_handles_ accessed via the ACE_Handle_Set: : fdset() method.

virtual int wait_for_multiple_events () { active_handles_ = master_handle_set_; int width = (int) active_handles_.max_set () + 1; if (select (width, active_handles_.fdset (), 0, // no write_fds 0, // no except_fds 0)) // no timeout return -1; active_handles_.sync ((ACE_HANDLE) active_handles_.max_set () + 1); return 0; }

If an error occurs, select() returns “1 , which we return as the value of the method. If it succeeds, the number of active handles is returned and the active_handles_fd_set is modified to indicate each handle that's now active. The call to ACE_Handle_Set::sync() resets the handle count and size-related values in active_handles_ to reflect the changes made by select() .

We now show the implementation of the handle_connections() hook method. If the acceptor's handle is active, we accept all the connections that have arrived and add them to the master_handle_set_ . Since we set the acceptor into non-blocking mode, it returns -1 and sets errno to EWOULDBLOCK when all pending connections have been accepted, as described in Sidebar 14. We therefore needn't worry about the process hanging indefinitely.

virtual int handle_connections() { if (active_handles_.is_set (acceptor ().get_handle ())) { while (acceptor ().accept (logging_handler ().peer ()) == 0) master_handle_set_.set_bit (logging_handler ().peer ().get_handle ()); // Remove acceptor handle from further consideration. active_handles_.clr_bit (acceptor().get_handle()); } return 0; }

Note our use of the get_handle() method here, as well as the use of set_handle() in the handle_data() method below. This direct use of a low-level socket handle is permitted by the Allow Controlled Violations of Type Safety design principle explained in Section A.2.2 on page 237.

The handle_data() hook method loops over the set of handles that may be active; for each active handle it calls the log_record() method defined on page 91. This method processes one log record from each active connection.

// This method has problems. Do not copy this example! virtual int handle_data (ACE_SOCK_Stream *) { for (ACE_HANDLE handle = acceptor ().get_handle () + 1; handle < active_handles_.max_set () + 1; handle++) { if (active_handles_.is_set (handle)) { logging_handler ().peer ().set_handle (handle); if (logging_handler ().log_record () == -1) { // Handle connection shutdown or comm failure. master_handle_set_.clr_bit (handle); logging_handler ().close (); } } } return 0; }

In handle_data() we use ACE_Handle_Set methods to

• Check if a handle is active

• Clear the handle in the master_handle_set_ when a client closes its connection

Although this method is fairly concise , there are several drawbacks with using ACE_Handle_Set in the manner shown above. The next section describes how the ACE_Handle_Set_Iterator addresses these problems, so make sure to read it before applying the solution above in your own code!

Our main() function is almost identical to the iterative server in Section 4.4.3. The only difference is we define an instance of Reactive_ Logging_Server rather than Iterative_Logging_Server .

#include "ace/Log_Msg.h" #include "Reactive_Logging_Server.h" int main (int argc, char *argv[]) { Reactive_Logging_Server server; if (server.run (argc, argv) == -1) ACE_ERROR_RETURN ((LM_ERROR, "%p\n", "server.run()"), 1); return 0; }