Reactor and Proactor: two I/O multiplexing approaches

In general,I/O multiplexing mechanisms rely on an event demultiplexor [1,3],an object that dispatches I/O events from a limited number of sources to the appropriate read/write event handlers. The developer registers interest in specific events and provides event handlers,or callbacks. The event demultiplexor delivers the requested events to the event handlers.

Two patterns that involve event demultiplexors are called Reactor and Proactor [1]. The Reactor patterns involve synchronous I/O,whereas the Proactor pattern involves asynchronous I/O. In Reactor,the event demultiplexor waits for events that indicate when a file descriptor or socket is ready for a read or write operation. The demultiplexor passes this event to the appropriate handler,which is responsible for performing the actual read or write.

In the Proactor pattern,by contrast,the handler—or the event demultiplexor on behalf of the handler—initiates asynchronous read and write operations. The I/O operation itself is performed by the operating system (OS). The parameters passed to the OS include the addresses of user-defined data buffers from which the OS gets data to write,or to which the OS puts data read. The event demultiplexor waits for events that indicate the completion of the I/O operation,and forwards those events to the appropriate handlers. For example,on Windows a handler could initiate async I/O (overlapped in Microsoft terminology) operations,and the event demultiplexor could wait for IOCompletion events [1]. The implementation of this classic asynchronous pattern is based on an asynchronous OS-level API,and we will call this implementation the "system-level" or "true" async,because the application fully relies on the OS to execute actual I/O.

An example will help you understand the difference between Reactor and Proactor. We will focus on the read operation here,as the write implementation is similar. Here's a read in Reactor:

• An event handler declares interest in I/O events that indicate readiness for read on a particular socket
• The event demultiplexor waits for events
• An event comes in and wakes-up the demultiplexor,and the demultiplexor calls the appropriate handler
• The event handler performs the actual read operation,handles the data read,declares renewed interest in I/O events,and returns control to the dispatcher

By comparison,here is a read operation in Proactor (true async):

• A handler initiates an asynchronous read operation (note: the OS must support asynchronous I/O). In this case,the handler does not care about I/O readiness events,but is instead registers interest in receiving completion events.
• The event demultiplexor waits until the operation is completed
• While the event demultiplexor waits,the OS executes the read operation in a parallel kernel thread,puts data into a user-defined buffer,and notifies the event demultiplexor that the read is complete
• The event demultiplexor calls the appropriate handler;
• The event handler handles the data from user defined buffer,starts a new asynchronous operation,and returns control to the event demultiplexor.

Current practice

The open-source C++ development framework ACE [3] developed by Douglas Schmidt,et al.,offers a wide range of platform-independent,low-level concurrency support classes (threading,mutexes,etc). On the top level it provides two separate groups of classes: implementations of the ACE Reactor and ACE Proactor. Although both of them are based on platform-independent primitives,these tools offer different interfaces.

The ACE Proactor gives much better performance and robustness on MS-Windows,as Windows provides a very efficient async API,based on operating-system-level support [4,128); text-decoration:none" rel="nofollow">5].

Unfortunately,not all operating systems provide full robust async OS-level support. For instance,many Unix systems do not. Therefore,ACE Reactor is a preferable solution in UNIX (currently UNIX does not have robust async facilities for sockets). As a result,to achieve the best performance on each system,developers of networked applications need to maintain two separate code-bases: an ACE Proactor based solution on Windows and an ACE Reactor based solution for Unix-based systems.

As we mentioned,the true async Proactor pattern requires operating-system-level support. Due to the differing nature of event handler and operating-system interaction,it is difficult to create common,unified external interfaces for both Reactor and Proactor patterns. That,in turn,makes it hard to create a fully portable development framework and encapsulate the interface and OS- related differences.

Proposed solution

In this section,we will propose a solution to the challenge of designing a portable framework for the Proactor and Reactor I/O patterns. To demonstrate this solution,we will transform a Reactor demultiplexor I/O solution to an emulated async I/O by moving read/write operations from event handlers inside the demultiplexor (this is "emulated async" approach). The following example illustrates that conversion for a read operation:

• An event handler declares interest in I/O events (readiness for read) and provides the demultiplexor with information such as the address of a data buffer,or the number of bytes to read.
• Dispatcher waits for events (for example,onselect());
• When an event arrives,it awakes up the dispatcher. The dispatcher performs a non- blocking read operation (it has all necessary information to perform this operation) and on completion calls the appropriate handler.
• The event handler handles data from the user-defined buffer,declares new interest,along with information about where to put the data buffer and the number bytes to read in I/O events. The event handler then returns control to the dispatcher.

As we can see,by adding functionality to the demultiplexor I/O pattern,we were able to convert the Reactor pattern to a Proactor pattern. In terms of the amount of work performed,this approach is exactly the same as the Reactor pattern. We simply shifted responsibilities between different actors. There is no performance degradation because the amount of work performed is still the same. The work was simply performed by different actors. The following lists of steps demonstrate that each approach performs an equal amount of work:

Standard/classic Reactor:

• Step 1) wait for event (Reactor job)
• Step 2) dispatch "Ready-to-Read" event to user handler ( Reactor job)
• Step 3) read data (user handler job)
• Step 4) process data ( user handler job)

Proposed emulated Proactor:

• Step 1) wait for event (Proactor job)
• Step 2) read data (now Proactor job)
• Step 3) dispatch "Read-Completed" event to user handler (Proactor job)
• Step 4) process data (user handler job)

With an operating system that does not provide an async I/O API,this approach allows us to hide the reactive nature of available socket APIs and to expose a fully proactive async interface. This allows us to create a fully portable platform-independent solution with a common external interface.