TProactor

The proposed solution (TProactor) was developed and implemented at Terabit P/L [6 ]. The solution has two alternative implementations,one in C++ and one in Java. The C++ version was built using ACE cross-platform low-level primitives and has a common unified async proactive interface on all platforms.

The main TProactor components are the Engine and WaitStrategy interfaces. Engine manages the async operations lifecycle. WaitStrategy manages concurrency strategies. WaitStrategy depends on Engine and the two always work in pairs. Interfaces between Engine and WaitStrategy are strongly defined.

Engines and waiting strategies are implemented as pluggable class-drivers (for the full list of all implemented Engines and corresponding WaitStrategies,see Appendix 1). TProactor is a highly configurable solution. It internally implements three engines (POSIX AIO,SUN AIO and Emulated AIO) and hides six different waiting strategies,based on an asynchronous kernel API (for POSIX- this is not efficient right now due to internal POSIX AIO API problems) and synchronous Unix select(),poll(),/dev/poll (Solaris 5.8+),port_get (Solaris 5.10),RealTime (RT) signals (Linux 2.4+),epoll (Linux 2.6),k-queue (FreeBSD) APIs. TProactor conforms to the standard ACE Proactor implementation interface. That makes it possible to develop a single cross-platform solution (POSIX/MS-WINDOWS) with a common (ACE Proactor) interface.

With a set of mutually interchangeable "lego-style" Engines and WaitStrategies,a developer can choose the appropriate internal mechanism (engine and waiting strategy) at run time by setting appropriate configuration parameters. These settings may be specified according to specific requirements,such as the number of connections,scalability,and the targeted OS. If the operating system supports async API,a developer may use the true async approach,otherwise the user can opt for an emulated async solutions built on different sync waiting strategies. All of those strategies are hidden behind an emulated async façade.

For an HTTP server running on Sun Solaris,for example,the /dev/poll or port_get() -based engines is the most suitable choice,able to serve huge number of connections,but for another UNIX solution with a limited number of connections but high throughput requirements,a select() -based engine may be a better approach. Such flexibility cannot be achieved with a standard ACE Reactor/Proactor,due to inherent algorithmic problems of different wait strategies (see Appendix 2).

In terms of performance,our tests show that emulating from reactive to proactive does not impose any overhead—it can be faster,but not slower. According to our test results,the TProactor gives on average of up to 10-35 % better performance (measured in terms of both throughput and response times) than the reactive model in the standard ACE Reactor implementation on varIoUs UNIX/Linux platforms. On Windows it gives the same performance as standard ACE Proactor.

Performance comparison (JAVA versus C++ versus C#).

In addition to C++,as we also implemented TProactor in Java. As for JDK version 1.4,Java provides only the sync-based approach that is logically similar to C select() [7,8 ]. Java TProactor is based on Java's non-blocking facilities (java.nio packages) logically similar to C++ TProactor with waiting strategy based on select() .

Figures 1 and 2 chart the transfer rate in bits/sec versus the number of connections. These charts represent comparison results for a simple echo-server built on standard ACE Reactor,using RedHat Linux 9.0,TProactor C++ and Java (IBM 1.4JVM) on Microsoft's Windows and RedHat Linux9.0,and a C# echo-server running on the Windows operating system. Performance of native AIO APIs is represented by "Async"-marked curves; by emulated AIO (TProactor)—AsyncE curves; and by TP_Reactor—Synch curves. All implementations were bombarded by the same client application—a continuous stream of arbitrary fixed sized messages via N connections.

The full set of tests was performed on the same hardware. Tests on different machines proved that relative results are consistent.

User code example

The following is the skeleton of a simple TProactor-based Java echo-server. In a nutshell,the developer only has to implement the two interfaces: OpRead with buffer where TProactor puts its read results,and OpWrite with a buffer from which TProactor takes data. The developer will also need to implement protocol-specific logic via providing callbacks onReadCompleted() and onWriteCompleted() in the AsynchHandler interface implementation. Those callbacks will be asynchronously called by TProactor on completion of read/write operations and executed on a thread pool space provided by TProactor (the developer doesn't need to write his own pool).

class EchoServerProtocol implements AsynchHandler
{

    AsynchChannel achannel = null;

    EchoServerProtocol( Demultiplexor m,SelectableChannel channel ) throws Exception
    {
        this.achannel = new AsynchChannel( m,this,channel );
    }

    public void start() throws Exception
    {
        // called after construction
        System.out.println( Thread.currentThread().getName() + ": EchoServer protocol started" );
        achannel.read( buffer);
    }

    public void onReadCompleted( OpRead opRead ) throws Exception
    {
        if ( opRead.getError() != null )
        {
            // handle error,do clean-up if needed
 System.out.println( "EchoServer::readCompleted: " + opRead.getError().toString());
            achannel.close();
            return;
        }

        if ( opRead.getBytesCompleted () <= 0)
        {
            System.out.println( "EchoServer::readCompleted: Peer closed " + opRead.getBytesCompleted();
            achannel.close();
            return;
        }

        ByteBuffer buffer = opRead.getBuffer();

        achannel.write(buffer);
    }

    public void onWriteCompleted(OpWrite opWrite) throws Exception
    {
        // logically similar to onReadCompleted
        ...
    }
}

IOHandler is a TProactor base class. AsynchHandler and Multiplexor,among other things,internally execute the wait strategy chosen by the developer.

Conclusion

TProactor provides a common,flexible,and configurable solution for multi-platform high- performance communications development. All of the problems and complexities mentioned in Appendix 2,are hidden from the developer.

It is clear from the charts that C++ is still the preferable approach for high performance communication solutions,but Java on Linux comes quite close. However,the overall Java performance was weakened by poor results on Windows. One reason for that may be that the Java 1.4 nio package is based on select() -style API. � It is true,Java NIO package is kind of Reactor pattern based on select() -style API (see [7,8 ]). Java NIO allows to write your own select() -style provider (equivalent of TProactor waiting strategies). Looking at Java NIO implementation for Windows (to do this enough to examine import symbols in jdk1.5.0/jre/bin/nio.dll),we can make a conclusion that Java NIO 1.4.2 and 1.5.0 for Windows is based on WSAEventSelect () API. That is better than select(),but slower than IOCompletionPort�s for significant number of connections. . Should the 1.5 version of Java's nio be based on IOCompletionPorts,then that should improve performance. If Java NIO would use IOCompletionPorts,than conversion of Proactor pattern to Reactor pattern should be made inside nio.dll. Although such conversion is more complicated than Reactor- >Proactor conversion,but it can be implemented in frames of Java NIO interfaces. (this the topic of next arcticle,but we can provide algorithm). At this time,no TProactor performance tests were done on JDK 1.5.

Note. All tests for Java are performed on "raw" buffers (java.nio.ByteBuffer) without data processing.

Taking into account the latest activities to develop robust AIO on Linux [9 ],we can conclude that Linux Kernel API (io_xxxx set of system calls) should be more scalable in comparison with POSIX standard,but still not portable. In this case,TProactor with new Engine/Wait Strategy pair,based on native LINUX AIO can be easily implemented to overcome portability issues and to cover Linux native AIO with standard ACE Proactor interface.

Appendix I

Engines and waiting strategies implemented in TProactor

Appendix II

All sync waiting strategies can be divided into two groups:

• edge-triggered (e.g. Linux RT signals)—signal readiness only when socket became ready (changes state);
• level-triggered (e.g. select(),/dev/poll)—readiness at any time.

Let us describe some common logical problems for those groups:

• edge-triggered group: after executing I/O operation,the demultiplexing loop can lose the state of socket readiness. Example: the "read" handler did not read whole chunk of data,so the socket remains still ready for read. But the demultiplexor loop will not receive next notification.
• level-triggered group: when demultiplexor loop detects readiness,it starts the write/read user defined handler. But before the start,it should remove socket descriptior from the set of monitored descriptors. Otherwise,the same event can be dispatched twice.
• ObvIoUsly,solving these problems adds extra complexities to development. All these problems were resolved internally within TProactor and the developer should not worry about those details,while in the synch approach one needs to apply extra effort to resolve them.

Resources

[1] Douglas C. Schmidt,Stephen D. Huston "C++ Network Programming." 2002,Addison-Wesley ISBN 0-201-60464-7

[2] W. Richard Stevens "UNIX Network Programming" vol. 1 and 2,1999,Prentice Hill,ISBN 0-13- 490012-X

[3] Douglas C. Schmidt,Michael Stal,Hans Rohnert,Frank Buschmann "Pattern-Oriented Software Architecture: Patterns for Concurrent and Networked Objects,Volume 2" Wiley & Sons,NY 2000

[4] INFO: Socket Overlapped I/O Versus Blocking/Non-blocking Mode. Q181611. Microsoft Knowledge Base Articles.

[5] Microsoft MSDN. I/O Completion Ports.
http://msdn.microsoft.com/library/default.asp?url=/library/en- us/fileio/fs/i_o_completion_ports.asp

[6] TProactor (ACE compatible Proactor).
www.terabit.com.au

[7] JavaDoc java.nio.channels
http://java.sun.com/j2se/1.4.2/docs/api/java/nio/channels/package-summary.html

[8] JavaDoc Java.nio.channels.spi Class SelectorProvider
http://java.sun.com/j2se/1.4.2/docs/api/java/nio/channels/spi/SelectorProvider.html

[9] Linux AIO development
http://lse.sourceforge.net/io/aio.html,and
http://archive.linuxsymposium.org/ols2003/Proceedings/All-Reprints/Reprint-Pulavarty-OLS2003.pdf

See Also:

Ian Barile "I/O Multiplexing & Scalable Socket Servers",2004 February,DDJ

Further reading on event handling
- http://www.cs.wustl.edu/~schmidt/ACE-papers.html

The Adaptive Communication Environment
http://www.cs.wustl.edu/~schmidt/ACE.html

Terabit Solutions
http://terabit.com.au/solutions.php

About the authors

Alex Libman has been programming for 15 years. During the past 5 years his main area of interest is pattern-oriented multiplatform networked programming using C++ and Java. He is big fan and contributor of ACE.

Vlad Gilbourd works as a computer consultant,but wishes to spend more time listening jazz :) As a hobby,he started and runs www.corporatenews.com.au website.