Presentation on theme: "Apache Axis2 Evaluation l Goal: u To evaluate Axis2 performance and usability in the context of its possible use in a future Globus Toolkit release. l."— Presentation transcript:
Apache Axis2 Evaluation l Goal: u To evaluate Axis2 performance and usability in the context of its possible use in a future Globus Toolkit release. l Tasks: u Implement a service testing framework based on WSRF WSDL specifications using Axis2. u Implement sample services within the framework. u Benchmark performance of sample services against similar services running under GT
Test Environment l Software used: u Sun Java JDK 1.4.2_10 u Apache Ant 1.6 u Apache Axis2 1.2 u Tomcat u GT l Hardware used: u Server: Dell Precision 530 dual Xeon 2.2ghz 512KB cache 1G RDRAM, OS: Linux (Debian Sarge 3.1) u Client: Dell Precision 530 single Xeon 1.5Ghz 512KB cache 1G RDRAM, OS: Windows XP SP2 l Network environment: u Switched 100Mb Ethernet
Test Methodology l XMLBeans was used as the data binding layer for the Axis2 WSRF sample services. l Throughput of 4 different data types was measured: u xsd:int u xsd:string u xsd:complexType: in this case a GRAM Job Description type was used. u xsd:anyType: in this case an XML document containing 80KB of GLUECE information was used. The GLUECE sample data represented a PBS cluster of approximately 100 nodes.
Test Methodology, continued l Each operation was invoked 1000 times with no delay between invocations; the average and slowest round-trip times were measured. l Memory usage/allocation was tested by noting the memory footprint at startup and subsequently after 1000 iterations of a method invocation using xsd:complexType. l Tests were run using Tomcat as the HTTP server for both Axis2 and GT l All tests were run without security enabled.
Performance Details: Throughput
Performance Details: Memory Usage l Axis2 u Memory usage at startup: KB u Memory usage after 1000 iterations: KB u Total memory growth during test: KB l GT4.0.5 u Memory usage at startup: KB u Memory usage after 1000 iterations: KB u Total memory growth during test: 5108 KB
Performance Comparison: Summary l Throughput performance was approximately 2-3 times faster for Axis2 using the primitive and complex types tested. l No appreciable performance difference found when dealing with xsd:anyType. l Memory consumption of Axis2 under the stress test approximately double that of GT4.0.5 and Axis1. l Reason for the memory consumption currently unknown; it could be XMLBeans overhead, Axis2 engine overhead, or a possible combination of both.
Axis2: Pros l General Architecture u The architecture of Axis2 is designed to be modular and flexible with regard to customization on many levels. This kind of design can help to ensure the longevity of this incarnation of the Axis system. l Performance u Performance (throughput) of Axis2 does indeed seem better than Axis1 with regard to certain types of XML structures. l REST support u REST support could be an attractive feature for certain WSRF services, namely singleton or well-known service instances that do not require a resource key and therefore can be addressed using a simple URI. u For example, REST-enabled querying of a WSRF resource data aggregator such as the MDS DefaultIndexService.
Axis2: Pros, continued l Deployment model u The J2EE-style deployment model is very attractive. It allows service developers to package and deploy service versions as self-contained binary archives. u Custom handlers can also be deployed as archive modules which allow further customization of the message processing pipeline. This feature can be key in the implementation of new WS-* specifications. u The deployment model also supports hot-deployment for service archives that do not require a container restart, a widely requested feature from GT users. u Hot-Update is also supported by Axis2, whereby an already deployed service can be updated to a newer version without requiring a container restart.
Axis2: Cons l Data Binding: u Data binding options are somewhat limited, and there are many unknowns still remaining regarding the usability of code generated from WSRF schemas, since coverage of WSRF functions in the evaluation test environment is low. u The Axis2 native Axis Data Binding is not robust enough at this time to handle WSRF schemas. u XMLBeans has the most development and testing time. While it was possible to generate valid objects from WSRF schemas using XMLBeans, it required modifications to the Axis2 code generation libraries to make it work. u JAX-ME is supposedly another data binding option, but as of the last release of Axis2 (1.3), JAX-ME support was labeled as experimental.
Axis2: Cons, continued l Code Generation: u WSDL support in Axis2 codegen module is not as flexible as it should be when it comes to standards compliance. Specifically, the biggest issue is that the codegen module does not support multiple port types defined within a single service. u This is a big problem for many of the WS-* specifications, where multiple ports are often defined. The Axis1 tooling supports multiple ports per service, so why not Axis2? This seems like a big step backwards in usability. u Note that it is possible to work around the issue by invoking wsdl2java multiple times on the same WSDL file, specifying each time the name of the port type to be generated, however this is a very clumsy and cumbersome workaround -- especially when dealing with large WSDL files that contain many port types. u Axis developers seem indifferent to fixing this, though it is a legitimate concern for users. See and for more information about this issue.
Axis2: Cons, continued l Subjective Opinions: u Axis2 is big and ambitious. In some ways, it is a bit bleeding-edge. The architecture is so flexible that it essentially tries to be everything to everybody. That is a lot of ground for the Axis developers to cover. Developer priorities and user priorities are not always in sync, so there is no guarantee that the development focus will always be in the same areas that we require. u The Axis2 code base is robust, but still highly in flux. There is quite a bit of development still going on in many areas. While this is generally a good thing, it seems like there is a bit of disparity between the various components. Some components are stable and production quality, while others are beta quality or worse. This mixed-bag situation could cause problems for a large porting effort.
Conclusion l Axis2 is very promising, but still relatively immature as a SOAP platform. While on the surface it may seem like there are many good reasons to attempt a port of Globus to Axis2, looking deeper we can see that there are a large number of factors that still need to be addressed. l Axis2 in its currently released form may be suitable for applications that have relatively simple requirements, or have total control of service interface design. l However, in the case of a large, complex application such as Globus, which relies on published WSDL interfaces and standards compliance, it is not clear that Axis2 in its current state is going to provide a robust enough solution to warrant a major porting effort at this time.
Future Directions l Track Axis2 development, in particular with regard to the code generation and data binding modules. As the code generation issues seem to be the major stumbling block, if these issues could be resolved adequately then the viability of a porting effort could be reassessed. l Update the test environment to match the current release of Axis2 and re-run the performance tests. The current test environment was developed against Axis2 1.2; it may be worth the relatively small effort required to make it compatible with the current Axis2 release, 1.3. l Test JAX-ME data binding and compare performance and usability to XMLBeans.