1. Brad Bigelow, SHAPE Arend Smit, M&I/Partners Bigelow/Smit – NATO IV&V: Work in ProgressNASA 2013 IV&V Annual Workshop, 11 September 20131

2.  NATO is a political and military alliance whose primary goals are:  The collective defence of its members  The maintenance of a democratic peace in the North Atlantic area  Established in 1949  28 Member Nations  Albania, Belgium, Bulgaria, Canada, Croatia, the Czech Republic, Denmark, Estonia, France, Germany, Greece, Hungary, Iceland, Italy, Latvia, Lithuania, Luxembourg, the Netherlands, Norway, Poland, Portugal, Romania, Slovakia, Slovenia, Spain, Turkey, the United Kingdom, and the United States.  Headquarters in Brussels, Belgium Bigelow/Smit – NATO IV&V: Work in ProgressNASA 2013 IV&V Annual Workshop, 11 September 20132

3. Software Engineering Conference sponsored by the NATO Science Committee, Garmisch, Germany, 7-11 October 1968 Bigelow/Smit – NATO IV&V: Work in ProgressNASA 2013 IV&V Annual Workshop, 11 September 20133

4.  Established 1 July 2012  Primary agency for developing and operating NATO’s civil and military information services and networks  Over 3,500 military and civilian staff  Over 30 locations in Europe, USA and Afghanistan Bigelow/Smit – NATO IV&V: Work in ProgressNASA 2013 IV&V Annual Workshop, 11 September 20134

5. NATO Military Commands NATO Military Operations NATO Nations Bigelow/Smit – NATO IV&V: Work in ProgressNASA 2013 IV&V Annual Workshop, 11 September 20135

6.  Began in 2001  Driven by concern of some NATO member nations over the capacity and independence of in-house QA functions  Initially focused on QA during prime contractor execution  Review of key deliverables  Oversight of prime contractor testing & independent testing  Support to project team during milestone reviews

7.  Use of contracted IV&V delivered real benefits:  Faster turn-around on draft deliverables  More consistent/higher quality technical support  Greater attention to test-driven design and development

8.  Two opportunities for improvement:  Broadening scope from QA of prime contractor activities to QA of overall project scope—including NATO internal factors  Bringing IV&V support on board during preparation of the prime contractor solicitation  Better quality control over complex solicitation materials  Statements of Work, System Requirements Specifications  Opportunity to review winning bidder’s material prior to contractor award  Enabled mitigation of some risks during pre-award discussions

9.  In 2007, NATO launched an initiative to establish a program management and integration capability for the Bi-SC AIS  Bi-SC AIS  Bi-Strategic Command Automated Information Systems  One of NATO’s largest investment programs—over 500M Euro  Covers all Information Services and Systems used in NATO’s Military Commands  Supporting 15,000 users in 28 nations  10+ year rolling program  Over 100 active projects

10.  IV&V support staff reported to program manager, not project managers  Issues elevated, resources made more quickly  IV&V resources managed as a pool at the program level  Enabled program to prioritize and shift IV&V support to different projects  Same contractor for both program and IV&V support  Ensured higher degree of integration among project

11.  Pooled approach to IV&V resources has several advantages:  Better workload balance—fewer peaks and valleys  Compensates for differences in project timelines  Ability to concentrate less common/higher cost experts  Management at the program level enables better adaptation to changing project needs  Some projects have continuing QA challenges, but others experience “QA crises” that can be resolved through prompt intervention

12.  120,000 M 2 complex  Construction started in October 2010  Planned to be completed early 2016  A secure, collaborative, energy-efficient building  NATO C&I Agency is implementing the network/IT infrastructure

13.  For four security domains, ANWI provides  Processing/storage  Wired and wireless LAN switching  Communications (telephone, teleconferencing, site security communications, IPTV, GSM)  End-user equipment  Core information services  Cross domain support for external networks interconnection and information services interoperability

14.  Integration with systems/services external ANWI scope  Migration of services to new building  Support for migration of business applications  Integration Test Services (ITS)  Baselined reference suite of functions  Host Nations test their capabilities prior to deployment in new building

15.  Support in pre-contract award phase  Focus on major risk areas during execution  Security accreditation  Migration  Service based specification, implementation, and acceptance  Introduction of new capability  Unified Communications and Collaboration  Service Management  Cross Domain services  Monitor prime contractor’s Integration Test Services

16. Questions? Brad Bigelow Principal Technical Advisor, J6 Supreme Headquarters Allied Powers Europe (SHAPE) (former Bi-SC AIS Program Manager, NATO C&I Agency) brad.bigelow@shape.nato.int Bigelow/Smit – NATO IV&V: Work in ProgressNASA 2013 IV&V Annual Workshop, 11 September 201316 Arend Smit, PhD IT and QA consultant M&I/Partners arend.smit@mxi.nl