Presentation is loading. Please wait.

Presentation is loading. Please wait.

Presented to: SOA Brown Bag #12 By: Paul Caron, SWIM T&E Lead Date: December 13, 2011 Federal Aviation Administration System Wide Information Management.

Similar presentations


Presentation on theme: "Presented to: SOA Brown Bag #12 By: Paul Caron, SWIM T&E Lead Date: December 13, 2011 Federal Aviation Administration System Wide Information Management."— Presentation transcript:

1 Presented to: SOA Brown Bag #12 By: Paul Caron, SWIM T&E Lead Date: December 13, 2011 Federal Aviation Administration System Wide Information Management (SWIM) Service-Oriented Architecture (SOA) Testing

2 2 Federal Aviation Administration SOA Testing December 13, 2011 Agenda Traditional Testing Migration to SOA Conceptual SOA Traditional vs. SOA Testing SOA Testing Solutions Anticipated Challenges

3 3 Federal Aviation Administration SOA Testing December 13, 2011 Characteristics of Traditional Testing In most cases, non-agile but well established proven process Single-system focused: not dependant on other systems within your business domain Tightly coupled Well-defined interfaces specified in the Interface Control Document (ICD) and Initial Requirements Document (IRD) Limited security constraints, if any Test program is based on the common Verification & Validation (V&V) test model Test team only needs to be technically savvy on the system under test, and not the entire business domain Test Tool Strategy is needed because systems are usually furnished with custom tools Test requirements only apply to the system/subsystem and interfaces

4 4 Federal Aviation Administration SOA Testing December 13, 2011 Stand Alone Server Architecture Applications share data and functions on an isolated single machine, no loose coupling Each application communicates with others in a point- to-point, one off interface design Difficult to upgrade or replace

5 5 Federal Aviation Administration SOA Testing December 13, 2011 Client Server Architecture Server applications share data and functions on a single machine, no loose coupling Each server based application communicates with others in a point-to-point, one-off interface design Supports multiple users

6 6 Federal Aviation Administration SOA Testing December 13, 2011 Distributed Server Architecture Server applications share data and functions across multiple server machines, possible loose coupling Each server based application communicates with others in a point-to-point, intranet, or internet design Supports multiple users and supplies composite data

7 7 Federal Aviation Administration SOA Testing December 13, 2011 Web Services Server Architecture Server applications share data and functions across multiple server machines, possible loose coupling External data and/or function provided externally Each server based application communicates with others in a point-to-point, intranet, or internet design Supports multiple users, supplies composite data

8 8 Federal Aviation Administration SOA Testing December 13, 2011 SOA Data collections and function hosted by internal and external servers loosely coupled Collections often communicate using a single interface Reuse and upgrades are less challenging Multiple remote users

9 9 Federal Aviation Administration SOA Testing December 13, Test Complexity Year Software Complexity Timeline

10 10 Federal Aviation Administration SOA Testing December 13, 2011 The Concept of SOA From Oasis (www.oasis-open.org),www.oasis-open.org SOA is defined as – An architectural style whose goal is to achieve loose coupling among interacting software agents/services. SOA enables business flexibility in an interoperable, technology- agnostic manner. It consists of a composite set of business-aligned services that support a flexible and dynamically re-configurable end- to-end business processes realization using interface-based service description. SOA is a commonly chosen implementation architecture process for today's largely distributed businesses or systems.

11 11 Federal Aviation Administration SOA Testing December 13, 2011 Some Benefits of SOA are: Reducing integration expense: uses open standards, not expensive proprietary middleware Increasing asset reuse: legacy systems can be easily wrapped as services and exposed Increasing business agility: services and systems may be rapidly developed, tested, and delivered Reduction of business risk Loosely coupled: a modified service does not affect other service or client Interoperable: platform-neutral

12 12 Federal Aviation Administration SOA Testing December 13, 2011 Components that make up SOA SOA services are intended to be reusable and can be shared within the entire business domain Service provider Service consumer Service registry Governance Lead Administrator

13 13 Federal Aviation Administration SOA Testing December 13, 2011 SOA Characteristics SOA transport protocols: –Hypertext Transfer Protocol (HTTP) –HTTP Secure (HTTPS) –Java Message Service (JMS) Messaging: –Extensible Markup Language (XML) –Web Services (WS)-Addressing –SOAP Interface: –Web Service Description Language (WSDL) Quality of Service (QoS): –Security –Reliable messaging –AuthN / AuthZ –Policies –Etc. Governance: –National Airspace System (NAS) Service Registry/Repository (NSRR)

14 14 Federal Aviation Administration SOA Testing December 13, 2011 Common SOA Testing Challenges No visibility below the User Interface to isolate errors Data driven business logic is service embedded Inaccessible external services Inability to test components that do not have an available User Interface Need for continuous validation of application functionality Inability to test composite solutions due to limited access or availability of dependent services and data needed for testing Inadequate or incomplete testing, resulting in costly problem identification and debugging once released into production

15 15 Federal Aviation Administration SOA Testing December 13, 2011 Common SOA Testing Challenges (cont.) Poor collaboration between development and Quality Assurance (QA), with minimal to no reuse of test assets between unit, functional, regression and performance testing Security testing needs to intensify as threat grows Test team structure will require alignment to business domains (processes) and not to technologies to ensure that agility and speed on SOA testing do not compromise overall service quality Organizations will have to develop and maintain Test Assets for key services to guarantee performance and security throughout the entire test coverage cycles

16 16 Federal Aviation Administration SOA Testing December 13, 2011 Traditional Testing vs. SOA Testing Similarities: Both Traditional and SOA methodologies can utilize the success proven V-Model The V-Model enforces testing discipline throughout the project life cycle Both can utilize the top down or bottom up test approach

17 17 Federal Aviation Administration SOA Testing December 13, 2011 Traditional Testing vs. SOA Testing Differences: TraditionalSOA Limited scope (system, subsystem, interfaces)Business-wide, based on client size or subscriptions Black Box testingWhite Box testing Non-agile: follows a rigid process Flexible, parallel development is a common industry practice Expensive: utilizing proprietary middleware Less expensive: incorporating open source products and reusable services Testing within the system boundary Testing involve elements inside and outside of system boundary A/B level requirement testingService-component-level testing Commonly available test tools Customizable, environmentally adaptable SOA based test tools are needed Platform specificUniversal platform Traditional systems troubleshooting is straight forward and failures are easy to identify SOA Troubleshooting is more complex: end-to-end testing involving multiple systems

18 18 Federal Aviation Administration SOA Testing December 13, 2011 Traditional Testing vs. SOA Testing TraditionalSOA Only requirements and regulations need to be validated and verified Governance is involved and industry standards apply (Quality of Availability (QoA) on performance, regulatory policies, business policies, audit policies, and infrastructure policies) Small scale security testing, usually reserved at the end of test schedule SOA security testing process requires continuous validation throughout the project cycle Test data are easily to control and configure. Test data generated and modified at different systems, therefore hard to control (data aggregation) Creating a fall-back backup configuration for the traditional test bed is simple Creating a fall-back configuration for entire SOA type distribution system is complex especially when open source products are involved (frequent version updates) Larger portion of testing occurs after the development phase of the project SOA testing occurs from the beginning alongside the project business and requirements development Regression testing occurs only when major functional requirement changes occur or when bugs are fixed (intensive but less frequent) Regression testing is needed when software updates occurs and only affects the updated service units – more frequent, but lightly scaled Differences continued:

19 19 Federal Aviation Administration SOA Testing December 13, 2011 SOA Testing Solutions Service Level Testing (SLT) emphasis Security Testing and the project lifecycle SOA and NAS experts leverage Test Tool Strategy implementation

20 20 Federal Aviation Administration SOA Testing December 13, 2011 SLT Emphasis –Test emphasis at the SLT level over Governance testing, service-component level testing, integration-level testing, process/orchestration-level testing, system-level testing, and Security testing Formal Code Peer reviews –Ensures standards and compliance are met –Identifies potential interoperability, performance, and security defects and weaknesses Continuous Functional, Performance and Security tests against the Services –Requires the use of test tools and/or test harnesses Quality Entrance and Exit Criteria for SLT –Ensures service usability

21 21 Federal Aviation Administration SOA Testing December 13, 2011 Security Testing and the Project Lifecycle Security requirements should be established Security risk assessments should be performed during the technical design phases Technical deliverables should be validated in accordance with the groups security standards Security tests should be performed at the service level and not just the delivered integrated system level Security audits should be performed and reported on periodically

22 22 Federal Aviation Administration SOA Testing December 13, 2011 SOA and NAS Experts Leverage Subject Matter Experts / Domain Experts –NAS expertise –SOA domain expertise –Software development expertise Certified Resources –Certified individuals –Certified tools Software Support –Commercial Off-The-Shelf (COTS) support contracts –Open Source community and committers Knowledge Sharing –Content repositories (SWIM Wiki, Knowledge Services Network (KSN)) –Working groups

23 23 Federal Aviation Administration SOA Testing December 13, 2011 Test Tool Strategy Implementation SOA Test Tools –iTKO LISA, soapUI XML Tools –Altova XMLSpy, XML Copy Editor Integrated Development Environments (IDEs) –FUSE IDE for Camel, Eclipse IDE, NetBeans IDE, IntelliJ IDEA Web Consoles and Browsers –Web consoles for ActiveMQ, Camel, Oracle AS, Glassfish, etc. –Fuse HQ, Lightweight Directory Access Protocol (LDAP) browsers, management and monitoring tools

24 24 Federal Aviation Administration SOA Testing December 13, 2011 Return On Investment

25 25 Federal Aviation Administration SOA Testing December 13, 2011 Business and Financial Metrics Efficiencies associated with service reuse Integration time savings Related opportunity costs Cost savings/avoidance Reduction in project and maintenance cost SOA ROI ($) = Cost Savings/Efficiencies Achieved - All SOA-Related Investments SOA ROI (%) = SOA ROI ($) All SOA-Related Investments Source: Leo Shuster, who is responsible for National City Bank's SOA

26 26 Federal Aviation Administration SOA Testing December 13, 2011 SOA Performance Metrics Overall ROI can be projected by per service ROIs Overall ROI can be projected by per service revenue Monitor service growth rate/reuse to ensure reuse maximization Design to deploy business agility Mean time to service change agility measurement Reliability mean time to failure Service vitality and revenue tracked for 12 months

27 27 Federal Aviation Administration SOA Testing December 13, 2011 Anticipated Future Challenges Services Metadata –Employing Governance in regards to managing and providing information on how services interact Level of Security –Protection against theft of sensitive data and application infrastructure vulnerabilities Test Coverage –Continuous services testing by producers and consumers to ensure expected functionality Scalability –Specialized support for persistence, failover, and load balancing XML Message Size –Additional bandwidth and resources for parsing/processing Increase in Connections –Connection management for server burden relief –Shared services hosting to avoid latency from additional hops

28 28 Federal Aviation Administration SOA Testing December 13, 2011 Conclusion Traditional Testing has its place Migrate to SOA where appropriate SOA brings benefits such as loose coupling, code/services reuse, and remote users SOA test products are numerous, mature and vary for many different use cases SOA challenges have been identified with many modeled solutions available to pursue

29 29 Federal Aviation Administration SOA Testing December 13, 2011 Questions and Comments

30 30 Federal Aviation Administration SOA Testing December 13, 2011 Back up

31 31 Federal Aviation Administration SOA Testing December 13, 2011 Keys to a Successful SOA Solution Definition of SOA Find Business Partners Aim for Critical Mass Franchise


Download ppt "Presented to: SOA Brown Bag #12 By: Paul Caron, SWIM T&E Lead Date: December 13, 2011 Federal Aviation Administration System Wide Information Management."

Similar presentations


Ads by Google