1. Pluto Building a large financial system in PERL

2. What is Pluto? Life insurance system Keeps track of mutual funds Handles 5 200 000 accounts Handled EUR 7 000 000 000 Process all accounts in 12 hours Written in Perl

3. What is PPM? Each person has a personal account Can select 5 funds Can switch any time Can get pension paid when 61

4. History PPM had a EUR 10 000 000 contract to buy a system from CSC CSC failed to deliver on time We were asked to deliver a 2nd system Started Novemeber-99 delivery 16th of March 2000

5. Challenges? Lots of accounts Lots of money Lots of media focus Must be right Short time

6. Advantages Business specs: Read the law and Interfaces Language? Hardware? Database? We don’t care! We need a system!

7. Why Perl? Visual Basic C/C++ C# Java 4GL

8. Advantages with Perl Quick to develop Robust No compiling Easy to get started Good parsing features Works well with Oracle Performance...

9. But won’t Perl be slow... Tests showed that about 10% of the execution time is spent in Perl and 90% in Oracle and calls to Oracle. RegExp and hashes makes for efficient algorithms Quick to program leaves time to optimize algorithms

10. Platform Linux on HP Hardware Oracle 8i CVS Production Platform: HP-UX/Linux

11. Architecture Perl DBI/DBD Oracle No N-tier complexity

12. Project Team Project lead Developers (5-7) Actuarial (2) Test (2) Legal specialist (1) Accountant (1)

13. Priorities No errors! No erroneous data in database Trace all transactions High performance Flexibility and functional depth

14. Peer Reviews Code written by one developer Code tested by author Author is responsible that code works Checked by a second developer

15. Code should be Easy to understand Correct Efficient Consistent

16. Challenges Large database (30 GB) Large numbers and exact calculations Handle crashes Trace transactions Lots of combinations No errors

17. Large database Tables with 200 000 000 records Full table scan takes 15 minutes Larger than RAM Read person  Physical read  10ms One processing takes about four reads Gives 25 pps which is too slow Index Organized Tables

18. Large numbers Amounts with 2 decimals Fund quantities with 6 decimals Value can be up to 50 000 000 000 Need 10+6 decimals, i.e. double is not enough BigInt/BigFloat too slow (1000 calc/s) Oracle too slow (1000 calc/s) Scaled integers

19. Crashes happen Power failure (None so far) OS/Hardware crashes (Some found) Disc crashes (None so far) Oracle bugs (There are some) Perl bugs (None so far) Our bugs (Some found)

20. How to handle Use Oracle transactions with great care Can not have one transaction per execution Keep track of which line we have read Always think of crashes Have not restored from backup yet...

21. Trace transaction Never update (almost) Insert new record Insert transactions

22. Murphy's law confirmed If there is one chance in a million that it will happen… it happens five times! Corrections... (we had five the first day) And corrections of corrections...

23. And then... CSC was late Could not deliver all functionality PPM Decided to use Pluto But had to pay CSC EUR 15 000 000 They might get a new chance in Denmark...

24. Advantages with Perl Portable No compiling Easy to do version handling Stable Easy to learn (Cost is not a major advantage)

25. Proven Perl is stable Perl code can be maintained (if used carefully) Perl and Oracle Interfaces well Perl systems can be developed quickly Perl can be used to build large and complex systems

26. Questions?