Hiding legacy software using Perl and SOAP as glue Alasdair Allan University of Exeter, Exeter, U.K. Abstract The worst nightmare a software developer can face is a major re-engineering of a mature system. We'd all much rather sit down and write new code than carry out software archaeology on a half understood system written by a dozen different people. This is where Perl can come in, wrapping legacy code in XS and SOAP means that you can hide the horror a mature system behind a clean interface. Perl makes an excellent glue language, and implementing new systems on top of your legacy code no longer means that you have to know the exotic internals of the legacy system. By using web services as building blocks new functionality can be written quickly and efficiently. Old code never dies, it just gets hidden.

Saturday 11th Dec. 2004London Perl Workshop 1 Encapsulating knowledge Your code base is the fossil record of your organisation The worst mistake you can make is to try and re-implement your existing code base from scratch

Saturday 11th Dec. 2004London Perl Workshop 2 Throwing code away… A very painful example from my own experience is “Chunking” This wasn’t even because we thought we could do it better, we simply thought it wasn’t relevant to the modern world We should have modified, not discarded

Saturday 11th Dec. 2004London Perl Workshop 3 Second system effect Re-implementing things can be a lot more costly than you would expect Second System Effect n. (sometimes, more euphoniously, `second-system syndrome') When one is designing the successor to a relatively small, elegant, and successful system, there is a tendency to become grandiose in one's success and design an elephantine feature-laden monstrosity.

Saturday 11th Dec. 2004London Perl Workshop 4 Software archaeology Software naturally forms layers Underneath the convenience layer you’ll normally find a more powerful layer which is usually much harder to understand

Saturday 11th Dec. 2004London Perl Workshop 5 Code reuse There are two main approaches, Wrapping code Software as services Both you and the code will benefit from, Refactoring

Saturday 11th Dec. 2004London Perl Workshop 6 Refactoring Why should you refactor? To fix architectural problems To remove inefficiency Just plain ugly is not a good reason.

Saturday 11th Dec. 2004London Perl Workshop 7 Just plain ugly Code is often ugly for a reason Every bug fix makes the code harder to understand and much uglier Refactoring just to make the code pretty could remove added goodness

Saturday 11th Dec. 2004London Perl Workshop 8 Wrapping code The traditional way to hide legacy code is to wrap it up behind a convenience layer In Perl this is usually done using XS and lately the Inline::* modules

Saturday 11th Dec. 2004London Perl Workshop 9 Refactoring by stages Once you wrap components you can remove them in a piece meal fashion This is especially true if you use the “software as services” paradigm

Saturday 11th Dec. 2004London Perl Workshop 10 The Book… Extending and Embedding Perl Tim Jenness & Simon Cozens Manning, ISBN Costs £21.17 at Amazon.co.uk “…the canonical book for this type of programming” -- Alasdair Allan

Saturday 11th Dec. 2004London Perl Workshop 11 Perl XS It looks like a couple of talks on the Advanced track will be discussion XS in depth, so I’m not going to bother Thank goodness for that…

Saturday 11th Dec. 2004London Perl Workshop 12 Inline Module Inline lets you write Perl subroutines in other programming languages like C, C++, Java, Python, Tcl and even Assembly. You don't need to compile anything. All the details are handled transparently so you can just run your Perl program like normal.

Saturday 11th Dec. 2004London Perl Workshop 13 What’s going on? 1.Your Perl module reads the code from the appropriate place usually below the DATA handle 2.An MD5 checksum is calculated for the code in question 3.This checksum and other information are compared with the XS modules previously generated by Inline.

Saturday 11th Dec. 2004London Perl Workshop 14 What’s going on? 4.If the checksum does not match, an XS module is generated based on the functions and arguments in the inlined code. 5.The module is built and and installed into a local directory 6.If the checksum matched, the relevant module is loaded

Saturday 11th Dec. 2004London Perl Workshop 15 Inline::C use Inline C; hello_world(’Your Name Here'); __END__ __C__ void hello_world(char* name) { printf("Hello %s!\n", name); }

Saturday 11th Dec. 2004London Perl Workshop 16 Inline::Python use Inline Python; print " = ", add(9, 16), "\n"; __END__ __Python__ def add(x,y): return x + y

Saturday 11th Dec. 2004London Perl Workshop 17 Inline::Python and Objects use Inline Python; my $obj = new Myclass(); __END__ __Python__ from mylibrary import myclass as Myclass

Saturday 11th Dec. 2004London Perl Workshop 18 Inline::Java use Inline Java; my $obj = new Example( ‘some data’ ); print $obj->get_data(). "\n”; __END__ __Java__ public class Example { private String data = null; public Example( String s){ data = s; } public String get_data(){ return data ; }

Saturday 11th Dec. 2004London Perl Workshop 19 Inline::Java and STUDY use Inline ( Java => 'STUDY', STUDY => ['java.util.HashMap'] ); my $hm = new java::util::HashMap(); $hm->put("key", "value"); my $val = $hm->get("key"); print $val. "\n";

Saturday 11th Dec. 2004London Perl Workshop 20 Inline::* Inline::C Inline::Java Inline::Python Inline::Tcl and others…

Saturday 11th Dec. 2004London Perl Workshop 21 Software as services A different approach to hiding legacy code is the “software as services” paradigm Add yet another layer on top of the wrapped legacy code so that the interface to the outside world becomes language neutral Perl is good at this…

Saturday 11th Dec. 2004London Perl Workshop 22 SOAP::Lite See for details The latest release is V0.65 Beta 2, although this is not yet on CPAN. Adds, amongst other things, MIME and DIME support

Saturday 11th Dec. 2004London Perl Workshop 23 Google Web Services See use SOAP::Lite; my $key = " "; my $wsdl = " my $query = "foo"; my $google = SOAP::Lite->service( $wsdl ); my $result = $google->doGoogleSearch( $key, $query, 0, 10, "false", "", "false", "", "latin1", "latin1"); my $results = $result->{'estimatedTotalResultsCount’}; print “About $results returned\n”;

Saturday 11th Dec. 2004London Perl Workshop 24 The Book… Programming Web Services with Perl Randy Ray & Pavel Kulchenko O’Reilly, ISBN Costs £19.95 at Amazon.co.uk Now slightly out of date, but still a good source of information about the SOAP::Lite modules

Saturday 11th Dec. 2004London Perl Workshop 25 The Mailing Lists… See for details, Main list is on Yahoo! Groups, see groups.yahoo.com/group/soaplite/ But there is also soaplite-announce and soaplite-devel lists hosted at SourceForge

Saturday 11th Dec. 2004London Perl Workshop 26 Amazon Web Services A good example of the software as services paradigm

Saturday 11th Dec. 2004London Perl Workshop 27 ItemSearch Service sub make_request { my $keywords = shift; my $request_type = \SOAP::Data->value( SOAP::Data->name('Keywords')->value($keywords), SOAP::Data->name('SearchIndex')->value('Books')); my $itemsearch_request = SOAP::Data->value( SOAP::Data->name('SubscriptionId’) ->value($subs_id), SOAP::Data->name('Request')->value($request_type)); my $aws_handle = SOAP::Lite->service("$aws_wsdl"); $aws_handle->ItemSearch($itemsearch_request); my $som = $aws_handle->call(); return $som; }

Saturday 11th Dec. 2004London Perl Workshop 28 Building our Complex Type my $request_type = \SOAP::Data->value( SOAP::Data->name('Keywords')->value($keywords), SOAP::Data->name('SearchIndex')->value('Books')); my $itemsearch_request = SOAP::Data->value( SOAP::Data->name('SubscriptionId’) ->value($subs_id), SOAP::Data->name('Request')->value($request_type)); Builds the following complex type $subs_id $keywords Books

Saturday 11th Dec. 2004London Perl Workshop 29 Calling the Service my $aws_handle = SOAP::Lite->service("$aws_wsdl"); $aws_handle->ItemSearch($itemsearch_request); my $som = $aws_handle->call(); return $som; Creates a SOAP object calls the Amazon service using the WSDL found at the URL Returns a SOAP::SOM object containing the results of the query

Saturday 11th Dec. 2004London Perl Workshop 30 A simple SOAP Server The simplest way is to use an existing web server, the following would end up as cgi-bin/service.cgi #!/usr/bin/perl use SOAP::Lite; use SOAP::Transport::HTTP; use ANY_OTHER_MODULE_NEEDED; SOAP::Transport:: ->dispatch_to(‘/path/to/perl/modules/’) ->handle;

Saturday 11th Dec. 2004London Perl Workshop 31 dispatch_to( ) A collection of Perl modules, dastardly{aa}: ls modules/ drwxr-xr-x2 nobody users4.0K Sep / drwxr-xr-x6 rootroot4.0K Oct 11 21:56../ -rw-rw-r--1 nobodyusers 118 Sep Echo.pm -rw-rw-r--1 nobodyusers 85 Sep Ping.pm package Ping; sub ping { $class = shift; return “ACK”; }

Saturday 11th Dec. 2004London Perl Workshop 32 Easily consumed… You don’t need to write WSDL to use the service, use SOAP::Lite; my $soap = new SOAP::Lite(); $soap->uri(‘ $soap->proxy(‘ my $result; eval { $result = $soap->ping(); }; if ( ) { print $result->faultstring(); exit; } print $result->result();

Saturday 11th Dec. 2004London Perl Workshop 33 Asynchronous web services SOAP::Lite does not support.NET asynchronous callbacks But you can implement asynchronous web services using the module Use contextual web services and persistent state to keep track of things by hand This isn’t as hard as it sounds…

Saturday 11th Dec. 2004London Perl Workshop 34 Authentication See WebService::TicketAuth for a full blown solution Alternatively you could do a light weight implement of authentication using HTTP cookies by sub-classing the relevant SOAP::Transport module. Look at for an implementation of the later

Saturday 11th Dec. 2004London Perl Workshop 35 No more language neutrality If you can serialise it you can send it over a wire. Data::Dumper can be a powerful tool my $dumper = new Data::Dumper([$object], [qw($object)]); my $serialised = $dumper->Dump();\ and at the far end, my $object = eval $serialised; if ( ) { print “Warning: Cannot de-serialise object\n”; }

Saturday 11th Dec. 2004London Perl Workshop 36 SOAP::Lite has problems Interoperability - Although have a look at the SOAP::WSDL module if you need.NET interoperability Complex types - You’ll find that the SOAP::Data::Builder should simplify things WSDL - Perl is loosely typed, this means no automatic WSDL generation

Saturday 11th Dec. 2004London Perl Workshop 37 So why did I talk about Inline? Because of the lack of “proper” WSDL support I’m currently using a mixture of Java and Perl services to hide legacy Fortran (and C) Perl serves as an excellent glue language for this task

Saturday 11th Dec. 2004London Perl Workshop 38 Grid Services? See WSRF::Lite is the follow on work from OGSI::Lite, it implements the Web Service Resource Framework which has effectively replaced OGSI, for details on WSRF visit

Saturday 11th Dec. 2004London Perl Workshop 39 Lots of code, broken compiler? If you have a large mature code base but the limits of the underlying infrastructure are starting to cause problems, then perhaps you should Re-implement the compiler, not your code No, seriously…

Saturday 11th Dec. 2004London Perl Workshop 40 Re-implement the system? The alternative is to rewrite the entire system in a new language but, Experience shows that such projects often (almost always?) fail You loose years (decades?) of customised business logic You make a lot of people very unhappy

Saturday 11th Dec. 2004London Perl Workshop 41 Use Parrot? Implementing functional compilers for 4GL languages on top of Parrot has been done before (err, once) See “Building a Parrot Compiler” by Dan Sugalski at the O’Reilly’s OnLamp.com For simpler languages, or problems, you could target the compiler for Perl 5 as Dan did initially

Saturday 11th Dec. 2004London Perl Workshop 42 Problems with Legacy Code Layering - the higher level routines will believe the output of the lower level ones. Bugs - You still have all the old bugs, although on the bright side you, you don’t have any new ones. Backdoors - The possibility of backdoors in the lower level code.

Saturday 11th Dec. 2004London Perl Workshop 43 Conclusions Perl is an excellent glue language You can use it to hold together the unlikeliest collection of different pieces of code It may be ugly but this is better than loosing decades of “smartness” Your code has evolved, don’t lose that