1. Technical Performance Measures Module Space Systems Engineering, version 1.0
SOURCE INFORMATION:
The material contained in this lecture was developed by Lisa Guerra of NASA’s Exploration Systems Mission Directorate while on assignment in the Department of Aerospace Engineering at the University of Texas at Austin. As part of a course entitled, Space Systems Engineering, the lecture was piloted at UT-Austin in Spring 2008.
The content that follows was also reviewed and edited by Dr. Paul Graf, Adjunct Professor at the University of Colorado at Boulder.

2. Module Purpose: Technical Performance Measures
To define Technical Performance Measure (TPM).
To show how TPM trends are used to predict delivered system performance.
To describe how TPMs are used to monitor project progress and, when compared with standard resource contingency values, highlight when corrective action should be considered.
To provide example TPMs from current NASA development projects.

3. Technical Performance Measures
TPMs are measures of the system technical performance that have been chosen because they are indicators of system success. They are based on the driving requirements or technical parameters of high risk or significance - e.g., mass, power or data rate.
TPMs are analogous to the programmatic measures of expected total cost or estimated time-to-completion. There is a required performance, a current best estimate, and a trend line.
Actual versus planned progress of TPMs are tracked so the systems engineer or project manager can assess progress and the risk associated with each TPM.
The final, delivered system value can be estimated by extending the TPM trend line and using the recommended contingency values for each project phase.
The project life trend-to-date, current value, and forecast of all TPMs are reviewed periodically (typically monthly) and at all major milestone reviews.

4. Tracking Technical Performance Measures
Tracking TPMs and comparing them against typical resource growth provides an early warning system designed to detect deficiencies or excesses.
Contingency allocations narrow as the design matures.
TPMs that violate their contingency allocations or have trends that do not meet the final performance should trigger action by the systems engineer.
Current Best Estimate
Trend
Mass Contingency Plan
Mass Allocation 5% 2%
15%
20%
35%
Mass
Contingency violated, decisions are needed!
Is the trend dependable and no action is needed?
Act now to avoid more drastic action in the future?
See INCOSE Systems Engineering Handbook 3.1; 2007 figure 7-5.
Technical Performance Measures (TPM) express the objective performance requirements, are evaluated at decision gate reviews, and may be used to assess the risk position of the project. TPM provide visibility into important project technical parameter status to enable effective management enhancing the likelihood of achieving the technical objectives of the project. Limit the number of TPM being monitored to critical issues. Collecting too many measures without knowing how they can be used wastes time and resources, and, even worse, the really useful values may become lost in the ocean of data. The Systems Engineering Plan (SEP) should define the approach to TPM.
Without TPM, a project manager could fall into the trap of relying on cost and schedule status alone, with perhaps the verbal assurances of technical staff to assess project progress. This can lead to a product developed on schedule and within cost that does not meet all key requirements. Values are established to provide limits that give early indications if a TPM is out of tolerance, as illustrated in Figure 7-5.
Today
Time
Concept
PDR
CDR
Test
Launch

5. Chandra Mass TPM System Requirements Review to Launch
Source: NASA Systems Engineering Handbook; 2007; Figure 6.7•5 Use of the planned profile method for the weight TPM with rebaseline in Chandra Project

6. Design Contingencies
Design contingencies are largest during concept exploration and uniformly shrink as the project matures. For example, mass contingencies are typically 35% at SRR, 20% at PDR, 15% at CDR and 2% at the launch readiness review.
Why? Contingencies are used to account for development risks, interface uncertainties, and less than perfect design fidelity. As the design becomes more established and the team has greater confidence in their estimates for resource use or system performance, less contingency is needed.
The trends of past, successful projects have been used to create guidelines for new projects.
Why not carry even more contingency? Say 50% mass contingency at PDR to cover an even greater range of possible risks against system mass. With greater contingencies there is less allocation for the design - greater contingencies make the design problem harder. So there is a balance between contingency for risk management and allocation for design flexibility.
This slide describes why margins decrease with project phase (design maturity).
As a system goes from concept to engineering development unit (EDU) to test unit to actual flight hardware, the fidelity of its performance gets better understood and the margins decrease.
They are called design contingencies to be consistent with the NASA Space Science Announcement of Opportunities definitions of margin and contingency.
Note that this point can also be made in the margins lecture.

7. Contingency Guidelines for Common TPMs For Different Project Phases
NASA Green Book, op cit.

8. JWST Key Technical Performance Measures
Observatory Mass Margin
Observatory Power Margin
Observing Efficiency
OTE Wave-front Error
Wave-front Error Stability
Strehl Ratio
Sensitivity
Image Motion
Stray Light Levels
Cryogenic Thermal Margins
Commissioning Duration
Data Volume / Link Margin
Momentum Acceleration
Note that the TPMs above use ‘margin’ instead of ‘contingency.’
The TPMs are monitored at various frequencies from once a month to once every 4 months.
Minimum contingency / reaction limits for many of these TPMs are taken from GSFC-STD-1000 (aka “The Gold Rules”) as a primary source. If guidance is not provided by this source then one of these are used:
- Interpolation / extrapolation of GSFC-STD-1000
- Other Standard Aerospace Sources or common industry practice
- JWST specific risk / uncertainty assessment
Contingency percents are: (Requirement – Estimate) / (Estimate)*100%
Cryogenic Thermal Rejection Margin = (T4Required – T4Predicted) / T4Predicted where T is temperature measured at the cryogenic radiators
Both the established quantitative value and the “current” status for each TPM are documented in the monthly report from the observatory prime contractor (NGST).
James Webb Space Telescope (JWST)

9. JWST TPM - Mass

10. JWST TPM – Mass Reserve

11. JWST TPM – Power 6 year Power System Capability = 1826 Watts
Spacecraft + OTE Allocation ( ) = 932 Watts
ISIM + Cryocooler Allocation ( ) = 740 Watts
Power Margin (Estimate vs. Allocated) = 25 %
Notes: 5/05: ISIM allocation changed to 740 W
12/06: Power Margin being carried as Load Margin not Solar Array Margin (Golden Rules Compliance)
4/07: Solar Array Capability decrease due to 1 wing baseline
8/07: Cryocooler separated from ISIM, Solar Array Capability increased

12. Module Summary: Technical Performance Measures
TPMs are measures of the system technical performance that have been chosen because they are indicators of system success.
The trends of past, successful projects have been used to create contingency guidelines for new projects.
Tracking TPMs and comparing them against typical resource growth provides an early warning system designed to detect deficiencies or excesses.
TPMs that violate their contingency allocations or have trends that do not meet the final performance should trigger action by the systems engineer.
The final, delivered system value can be estimate by extending the TPM trend line and using the recommended contingency values for each project phase.
There is a balance between contingency for risk management and allocation for design flexibility. This balance is apparent since contingency allocations shrink as designs mature.

13. Backup Slides for Technical Performance Measures Module

14. Technical Performance Measures
TPM Basics
Parameter for meeting key requirements and constraints.
Sound engineering parameter that is always tracked regardless of mission, such as mass margin or milestone achievements.
TPMs are usually tracked over the development life cycle of a project.
TPM trends over time usually compare a planned profile with the actual profile…planning is very important in order to meet specified targets.
TPMs are usually reported monthly or quarterly in management/engineering status meetings.
TPM Sources
Responsible NASA Center guidance (e.g., GSFC STD-1000 “The Golden Rules”)
Industry Practices
Mission-specific risk assessments
Note: Check out CxPO Program Management Requirements Document for info on TPMs, no examples but lots of process.

15. JWST TPM - Mass

16. JWST TPM – Strehl Ratio Science Requirement:
L1-14 The Observatory, over the field of view (FOV) of the Near-Infrared Camera (NIRCam) shall be diffraction limited at 2 micrometers defined as having a Strehl Ratio greater than or equal to 0.8.
Definition: The modern definition of the Strehl ratio is the ratio of the observed peak intensity at the detection plane of a telescope or other imaging system from a point source compared to the theoretical maximum peak intensity of a perfect imaging system working at the diffraction limit. This is closely related to the sharpness criteria for optics defined by Karl Streh. Unless stated otherwise, the Strehl Ratio is usually defined at the best focus of the imaging system under study.

17. JWST TPM – Wavefront Error