Developing a Global Testing Framework for SAP Mark S. Smith and Parul D. Parikh

2 Topics Purpose and approach Company overview Testing Framework −Vision −Content −Approach −Table of Contents Discussion

3 Presenters Mark S. Smith – US Special Projects Director −Manages the shared Automated Regression Testing (ART) team −Manages the shared SAP Basis and Infrastructure team −Has been involved in various SAP projects as test manager Parul D. Parikh – ART Team Lead −Manages the daily activities of a team that utilizes HP/Mercury’s suite of tools to automate and execute testing for projects and production support −Has been involved in various SAP projects in testing, functional design, and learning capacities

4 Purpose BP tends to staff SAP projects with new resources each time −Once employees learn about systems testing, they move back to the business −If we use an experienced contract resource, they leave after the project ends We are attempting to create a testing framework that can be used by any SAP test manager We don’t think we know all the answers or angles −We are looking to benefit from your experience and knowledge −Tell us what is good in the approach −Tell us what won’t work −Tell us what can be improved

5 Today’s Approach I will provide some context on the environment inside of BP I will describe the testing framework as proposed I would then like to enter into a discussion around the proposal Thus, I would like to limit questions during the initial part of the session to those for clarity or understanding, then really get into the bulk of the discussion towards the end

6 BP BP is a multi-national Oil and Gas Super Major It recent years, it has grown through mergers and acquisitions −Amoco −Arco −Veba/Aral −Castrol This created a company with multiple legacy systems and over 114 accounting book of records In 2004, BP re-organized to create the structure on the next slide This re-organization provided the FC&A (Financial Controls and Accounting) function with the authority to declare SAP as the mandated ERP for accounting −FC&A then created the GFT, Global Financial Template −FC&A also created the MIF, Management Information Framework to report the GFT in a consistent format

7 Lord John Browne/ Tony Hayward CEO E&PR&M Gas Power & Renew- ables Inter- national Supply & Trading Functions FC&AHRTaxIT BP Organization and SAP Projects Segment = Customer Facing Business Functions = Shared Service GFT P2P MIF GFT DART M&S GM MIF Legacy SAP GFT MIF HR MI GFT = Global Financial Template MIF = Management Information Framework

8 Testing Framework – Vision As you can see, there are many different large scale SAP programmes in progress Each programme has multiple releases to various countries set up as individual projects Each project has an obvious need to do testing By providing a common framework, we can −Incorporate the experience and best practices of FC&A −As well as insights gathered today −Incorporate, as much as possible, the experience and best practices of other areas in BP −Jumpstart the creation of any project with a robust testing approach −Execute robust testing in a common way across the projects −Provide a standard to evaluate the testing efficacy of an existing project to inform the Go/No Go decision

9 Testing Framework – Content Applies to all systems testing associated with the project, but concentrates on SAP −Technical testing −Unit/string −Stress/performance −Security −Operational Acceptance (back up/restore, DR, etc.)

10 Testing Framework – Content cont. −Business testing −Regression −Conversion −Functional −Integration −EIT (Expanded Interface Test – running crucial interfaces in the proper order with production sized input files) −GAIT (Group Accounts Information Telex, feeding financial information to the corp. consolidation system) −Mock close (closing a financial period using the actual people who do the work) −Pre-flight (a limited, prioritized test between dress rehearsal cut over and Go Live) −UAT (user acceptance test)

11 Testing Framework - Approach Create and agree the “Table of Contents” of framework (see next slide) Build out documents, templates, processes, procedures, etc. −Utilizing good pre-existing material whenever possible Store the framework on a common SharePoint site that can be accessed by anyone in the corporation Provide on-going support for the projects about the framework

12 Testing Framework – Proposed Table of Contents Purpose Generic testing theory −Why test? −How much to test? Testing models −V-model −W-model −Risk-based Test phases −Phases and definitions −What each test phase attempts to answer −Appropriate entrance and exit criteria −Testing input to Go/No Go decision criteria

13 Testing Framework – Proposed Table of Contents – cont. Test processes −Traceability matrix −Test and defect tracking −Testing tools and automation −Quality assurance of the testing −CMMi aspects of testing SAP testing nuances Challenges of testing at BP

14 Discussion Questions Has anyone else in the room done this? What’s missing from the Framework? What was your approach? What were your challenges? What worked for you? What didn't work? How did you get support from the business/projects to take on the Framework? Did you go incrementally? Or try and implement the entire Framework from the start? What was the feedback? How is it evolving?

15 Comments? Thank you for your participation Mark S. Smith Parul D. Parikh