1. COMOPTEVFOR U.S. Navy OT&E
Transformational T&E Initiatives
Steven K. Whitehead
Technical Director

2. OT Initiatives Early Test and Resource Planning Integrated Test
Why do these initiatives?
EA and Spiral Development has created a tremendous compression on traditional T&E schedules
T&E has become more complex with Systems of Systems testing becoming more prevalent.
CNO mandate to reduce T&E cost by 20%.
Eliminate large execution year test bills.

3. Early Test and Resource Planning
Initiative 1 - Early Test and Resource Planning
What is it?
T&E framework and resource summary prior to MS-B.
Framework developed from Capability Development Document (CDD), System Threat Assessment Report (STAR), and Mission Essential Task Lists (METLs)
Use common terms and measures
Compatible with fleet training community and other service/OSD test organizations – allows for portability of data.
All test objectives traceable to capabilities documents

4. Early Test and Resource Planning
Initiative 1 - Early Test and Resource Planning
How will we do it?
“ Test by Objective”, scenario based design
METLs to define mission areas and scenarios
MOEs will be primarily METL based
Operational Vignettes to define variability of the mission areas and to create test events
Vignettes will identify: test assets, required ranges, instrumentation needs, limitations, M&S requirements

5. Example Operational Vignette
Scenario Description
Scenario #: NMETL Ref: NTA 1.2.8
Title: Armed Reconnaissance – Desert Environment
Scenario Description: Two AH-XXs will conduct armed Recon on a suspected Red country supply line. The Blue aircraft will launch from a forward deployed friendly airfield. Intelligence shows the Red convoy traveling a desert road between two critical supply junctions. The convoy is known to posses MANPADs and there are SAM sites placed strategically along the ingress and egress routes. The Blue aircraft will navigate the recon area, avoiding threats, locate the threat, use the laser to designate the target, and neutralize the threat (simulated) using the CATM Hellfire missile.
Run Plan/Geometry:Target Location
Target Location
xx.xx Lat/xx.xx Long
China Lake

6. Example Operational Vignette (cont)
Test Articles
1. AH-XX Quantity – 2
Target:
Dual track truck. A wooden mockup that meets DIA standards
will be used
Personnel carrier. A Ford panel truck will be used to simulate
the PC
Threats:
MANPAD (Actual/Surrogate) – Surrogate will be the XXXX
SA-X (Real/Surrogate) – Surrogate will be the XXX
Countermeasures:
Infrared masking for truck
Test Site:
China Lake R-2508
Weapons Required:
1. AGM-163 Hellfire Quantity – 4
Expendables required:
xxx-38 Flares Quantity - 20
YYY-39 Chaff Quantity – 50
Support Articles:
Marine spotters
Instrumentation Requirements:
Laser Spot Tracker
TSPI
AGM-163 TM download/recording station
Limitations:
SA-XX is currently not available either as an actual unit or surrogate. CTEIP funds are needed to develop this surrogate.
Modeling and Simulation:
The DUMASS model will be used to predict the AH-XX laser performance against the target. This will aid in determining the most stressing run-in angles for the AH-XX.
Projected execution time:
Planning – 1 hour
range time – 3 hours
post mission debrief and data recording – 2 hours

7. Early Test and Resource Planning
Major thrust of Framework planning is to execute a combinatorial analysis on the vignettes to identify synergies.
Synergized Operational Vignettes and Resources
Combinatorial
Analysis
Planned Operational Vignettes and Resources
Output of Framework – Complete definition of Operational Test
requirements and fully developed resource summary

8. Integrated Test Initiative 2 - Integrated Test (IT) What is it? – Why?
Performing concurrent CT, DT and OT uniformly over a continuum.
Why?
An effort to keep T&E relevant in Evolutionary Acquisition environment
More efficient use of test resources and schedules
Earlier injection of deficiency corrections into current and subsequent builds
CNO mandate to reduce T&E costs by 20%

9. Integrated Test Traditional Test Program Integrated Test Program MS A
MS B
MS C
FRP
Contractor Test
Gov’t DT OA
OPEVAL
TECHEVAL
Integrated Test Program
In the traditional test program, contractor tests and developmental tests were performed independently from OT. Techevals and Independent Verification and Validations were performed as “dress rehearsals” for OPEVAL. There was an inherent fear of divulging information that could be used against a program. The idea was to conduct the developmental test efforts and when the program was deem mature enough to field, it was “thrown over the fence” to the OT community.
Operational requirements documents were often times out of date when test planning phase was conducted. This created discontinuities between the systems end configuration (and maybe even from what the user could live with) and the test as planned against the requirements document. This is primarily due to the difficulties of updating the ORD, which sometimes took months.
In the new era of Integrated Test, the capabilities documents will define the end item configuration and useful military capability configuration for test planning. The CDD will become the foundation for the development of the T&E Framework for both DT and OT and the CPD should purely be a refinement of the CDD based end item configuration. There should be a convergence of test requirements, not an excursion as a result of the CPD development.
The integrated test plan development process will scope and plan all testing for the increment. By constructing a single, integrated test plan, all test members will have total insight to each other’s requirements and synergies can be obtained. The integrated test process will eliminate a vast amount of test paperwork including multiple OTRRs, concept of test briefs, and multiple test plans.
Instead of the traditional OA reports, which take a considerable effort to produce, a new OT product, called an Interim Assessment Report, will be used to assess and document the programs progress. The frequency of reporting will be pre agreed to by the MDA, OTA and PM. This report will have a flexible format and should generally be less lengthy than current OA reports while still conveying the appropriate information.
MS A
MS A
MS B
MS B
MS C
FRP
INTEGRATED TEST
INTEGRATED TEST
PLAN DEVELOPMENT
OPEVAL
Schedule
Reduction
INTERIM ASSESSMENT
REPORTS

10. Integrated Test Approach
How will we do it?
Utilize the Framework process discussed earlier and impose process on all test activities.
Test and Evaluation Working Groups (TEWGs) will merge objectives and identify synergies.
Common test plan is proposed for capturing the details of the IT.
Contractor test requirements can be parsed from common test matrix and placed in statement of work (SOW)
Test plan matrix reviewed periodically to ensure test objectives are being met.

11. Benefits to Early Test Planning/IT
Mutual understanding of test requirements
Better understanding of contractor test requirements – allows for better SOW preparation
Identifies limitations and mitigation strategies early
Early identification of system deficiencies.
What costs $1 to fix today costs $10,000 to fix tomorrow
Identification of test costs prior to Acquisition Program Baseline Agreements formally issued
Better use of navy test assets and resources
Preliminary calculations predict a 15% reduction in T&E
Schedule by implementing these initiatives!

12. Challenges Implementation of contractors in the IT process
Resourced to be responsive to program changes?
Cultural issues
How high are the stovepipes?
How willing are the IT members (and those externally who influence them ) willing to change their philosophies?
How does the program handle discovery of major anomalies?
Configuration control of design.
When is the system “good enough” to freeze?
Can we test w/o “freezing” the configuration?
What is “Production Representative”? How much regression testing is needed?

13. Early Test Planning/IT Pilot Programs
VXX Presidential Helicopter Pilot Program
Currently planned as totally integrated test team
Fast Track program with two defined increments
Framework planning sessions begin Mar 04
Testing begins mid FY-05
Approval in principle from DOT&E
Other programs targeted as pilot candidates:
DD(X)
Littoral Combat Ship (LCS)
Advanced Hawkeye
T-AKE
F-18 HOL software
Multi-role Maritime Aircraft (MMA)

14. QUESTIONS ?