1. Copyright OASIS, 2005 / 2006 Standards and Open Source Tools for Implementing Loosely Coupled SOA XML, metadata techniques and micro-services tools SOA CoP MITRE, MacLean, VA October 2006 David RR Webber, Chair OASIS CAM TC drrwebber@acm.org http://drrw.net

2. Copyright OASIS, 2005 / 2006 Overview n Few people stop to understand what the formal definition of loose coupling actually is, why it is beneficial and if indeed solution providers are actually delivering on the original vision? n Worse many implementations and approaches appear to be delivering tight- coupling with XML as constricting as the original issues of EDI and CORBA based systems! n Government is looking for information agility and metadata based information integration in the context of SOA today, so how can this be achieved?

3. Copyright OASIS, 2005 / 2006 Interactive models Conventional models Changing the "Just write code" paradigm Analysis Specification Design Coding Test Maintenance Developers Adaptation, integration Installation Use Needs Requirements Procurement Business Users Installation Use Needs Requirements Procurement Business Users Analysis Specification Design Coding Test Maintenance Developers

4. Copyright OASIS, 2005 / 2006 Standards based work today n OMG – UML / XMI model-based n JSR - DROOLS n JSR – API for Rule Engines n ISO Common Logic n OCL n OASIS CAM, Registry, BCM n CEFACT WGs – UCM, CCTS, UMP

5. Copyright OASIS, 2005 / 2006 What about “Architecture”? n Many Architecture models out there n FERA - Federated Enterprise Registry Arch’ n Simple service registry model n Architecture “blue-print” templates n Business-Centric Methodology (BCM) n OMG – model driven architecture n W3C – web content / document architecture n Alignment of mechanisms around XML- scripting techniques

6. Copyright OASIS, 2005 / 2006 Operational Challenges n Need to formulize and manage partner agreements both inside and outside enterprise n Speed-up ability to integrate with partners by providing intuitive business-centric tools n Provide common point of reference for partners to ensure consistent operational practices n Facilitate re-use across enterprise by providing templates of proven solutions (micro-services) n Provide open standards-based approach that can be accessed by implementation systems n Ability to support remote ad hoc connections

7. Copyright OASIS, 2005 / 2006 External Systems Registry SOA Operational Details Network Content Transport Security Manage Business Systems Workflow Payload Handling Content Versions MoUs Business Rules Certificates Identity Role + Context Profiles Industry Semantics Partners EventsActions Errors Scripts Staged Synch Asynch Fire Wall Virus Check Receive Send Dispatch Errors Validation Business Rules

8. Copyright OASIS, 2005 / 2006 Unravelling the Gorgonian Knot? n It’s all a bit overwhelming!!! n Where to start? How to differentiate? n How can I go from business requirements to technology realization? Divide the problem into layers! Tackle each layer, and ensure interaction between layers is consistent with open interfaces with known roles and context What can open standards provide?

9. Copyright OASIS, 2005 / 2006 Business Agreement Languages (RINs) Choreography & Coordination Lang. (BPSS ) Collaboration Profile Agreement (CPA) Context Driven Information Exchanges (CAM) Semantic Registry Attaining SOA Through Component Layers

10. Copyright OASIS, 2005 / 2006 Typical Deployment Model

11. Copyright OASIS, 2005 / 2006 1. Memorandum of Understanding - MoU Business Managers Meeting of the Minds Business Goals Define Scope Rough Timeline Creation / Best Practice Wizard Knowing Business Context is vital! 2. Collaboration Protocol Agreements Operations Managers Precise Choreography Automated Messaging Parameters ebMS transport WSDL web services 3. On-line Registry Systems Administrators Manage operational use Provide automation access

12. Copyright OASIS, 2005 / 2006 Information Exchange Integration requirements n Outward facing messaging systems n Formal agreement profiles for business participants n Business process workflow definitions n Information exchange rules n Registry to hold agreements, definitions, scripts… n Internal integration routing and dispatch methods n User interfacing for entry and control

13. Copyright OASIS, 2005 / 2006 Registry - the Enterprise Artifacts Motivation Time People Specifications Schema Workflow Contract Directory Services Collaboration Partner Profiles - CPP Collaboration Partner Profiles - CPP 2 1 3 4 5 Presentation Collaboration Partner Agreements- CPA Collaboration Partner Agreements- CPA Artifact relationships Content Assembly Mechanism - CAM Content Assembly Mechanism - CAM BP Specification Data/Codes Services/Functions Network XForms MSH/SOAP Source: BCM Lubash Pyramid Verbs Messages Rules Events Process Roles Transport Routing, Packaging Transport Routing, Packaging Nouns Core Components Core Components WSDL

14. Copyright OASIS, 2005 / 2006 Example – Data Integration Micro-Services Loose Coupling of Information Exchanges

15. Copyright OASIS, 2005 / 2006 XML Micro-service Functionality n Structure Validation + Versioning Control n Content Semantics n Business Rules n Context / Role aware processing n Extensions l Database Persistence l Transformation services l Registry terms dictionary l Include mechanism support

16. Copyright OASIS, 2005 / 2006 CAM Component Architecture CAMCAM XML Parser / DOM XML Parser / DOM Built-inFunctionsBuilt-inFunctions XPathhandlerXPathhandler SQLpersistenceSQLpersistence TermsRegistryTermsRegistry Rule Engine Post-Processing / Errors Post-Processing XML-awareXML-aware EXTENSIONS

17. Copyright OASIS, 2005 / 2006 jCAM Component Architecture CAMCAM XML Parser / DOM XML Parser / DOM Built-inFunctionsBuilt-inFunctions XPathhandlerXPathhandler SQLpersistenceSQLpersistence TermsRegistryTermsRegistry Rule Engine EXTENSIONS jaxen xerces e.g. DROOLS, Others… Saxon - xslt planned using AJAX / ebXML Post-Processing / Errors Post-Processing XML Data Mapping XML-awareXML-aware cam XX

18. Copyright OASIS, 2005 / 2006 Dynamic Component Linkage n Using Maven allows for interchangeable extension component support: <component-implementation class='uk.org.jcam.processor.dataObjects.Template'/> <component-implementation class='uk.org.jcam.processor.dataObjects.DataFile'/> <component-implementation class='uk.org.jcam.processor.validator.DefaultValidator'/> <component-implementation class='uk.org.jcam.processor.trimmer.DefaultTrimmer'/> <component-implementation class='uk.org.jcam.processor.adorner.DefaultAdorner'/> <component-implementation class='uk.org.jcam.drools.DroolsDataValidator'/>

19. Copyright OASIS, 2005 / 2006 Technology Solution Summary n XML-aware services: l Structure validation l Context driven versioning and includes l XPath l DOM n Rule-based services: l 30+ built-in functions l Integration with rule engines (e.g. DROOLS) l Call-out to Java methods n Data persistence / transformation services: l SQL l xslt

20. Copyright OASIS, 2005 / 2006 Use Case Analysis What a typical CAM deployment looks like

21. Copyright OASIS, 2005 / 2006 Business Functionality n Creating reliable domain solution templates for Communities of Interest (CoI) n Providing rapid deployment and simple change support n Handling local context drivers and flexibility n Linking business process, partner collaborations and transaction exchanges into legally verifiable and deterministic solutions n Opening up business integration rules so that partners can confirm their usage n Validating and processing content with open public specification and rules base

22. Copyright OASIS, 2005 / 2006 Versioning Challenges n If the schema version changes – how to ensure it does not break our in place validations? n How to rapidly adapt to rule changes in a production environment? n How to develop user context driven deep version control and re-use of sub-components? n Enhance and automate Test release cycle by improving transparency for bug fix process and expose change deltas to speed testing process?

23. Copyright OASIS, 2005 / 2006 Grants Application Example XML content XSD schema from Grants.gov 2 Dynamic Structure rules Structure sub-form components with includes allows re-use 1 RR SF424 SBIR STTR PHS398 RR Budget RR Submitter Profile Business Validation rules Error Response formatting Data Persistence Rules 3 Message Delivery to Agency System CAM process Oracle RDBMS SQL tables Researcher Prepares Grant Application and submits XML to Grants.gov Agency develops CAM template of validations rules and business content handling Government form definitions from Grants.gov

24. Copyright OASIS, 2005 / 2006 Interaction of Rule Components XML instance content Exception Reporting SQL Tables Error logging 3 Data Mapping Component SQL commands 4 2 XML mapping rules script 1 Query Agent

25. Copyright OASIS, 2005 / 2006 Rule Driven Transaction Handling Message Delivery XML content XSD schema Structure validation 2 1 Dynamic Structure rules Structure sub-components with includes allows re-use Delivery Control Match Message Type to CAM Rule-base through enveloping Business Rules Rule Driven Error Reporting Error Logging Result logging Email notifications XML-based error outputting 3 4 Persistence Rules for SQL SQL tables Application Triggers 6 7 SQL commands Oracle API 5 8

26. Copyright OASIS, 2005 / 2006 jCAM validation template samples n Simple template checking structure n Extended template with context rule examples n Template Editor in Eclipse Simple Template Simple Template Extended Template Extended Template Sample Run Sample Run Sample Run Sample Run

27. Copyright OASIS, 2005 / 2006 What’s Next? n Context XML and context driven assembly n BIE, ACC, CCs and CAM structure components n Referencing Registry dictionaries - business nouns n W3C RIF / OMG PRR / OWL / DROOLS semantic rules n Eclipse support via CAM open source editor n XSD auto-mapping support

28. Copyright OASIS, 2005 / 2006 www.jcam.org.uk camprocessor.sourceforge.net www.drools.org www.oasis-open.org/committees/cam Resources:

29. Copyright OASIS, 2005 / 2006 A special mention for our contributors to the CAM and jCAM work: UK – Martin Roberts with BTplc USA – Sidhartha Nagolu with AC-Tech / NIH Credits: