1. DISCUSSION DRAFT ONLY Data Management METRICS for NNDC and CLASS David Hermreck

2. DISCUSSION DRAFT ONLY Metrics - Context Revisit appropriate operational performance metrics in an environment with an operational CLASS. CLASS and NNDC metrics are currently overlapping. Metrics should focus on core functions. Need both Development and Operations Metrics Will use a “bank” analogy to better understand CLASS and NNDC Operations roles

3. DISCUSSION DRAFT ONLY CLASS Development Metrics CLASS development requires Capability Maturity Model Integration (CMMI) level 3 – this model provides many potential metrics Potential metrics could include: Number of Change Requests implemented on time and budget Major software releases delivered to operations on time and budget Other CMMI metrics

4. DISCUSSION DRAFT ONLY Operations Metrics - The Bank Analogy Safe and Secure Preservation just storage CLASS is…is NOT…

5. DISCUSSION DRAFT ONLY The Bank Analogy Safe and Secure Preservation Stewardship CLASS is…is NOT…

6. DISCUSSION DRAFT ONLY The Bank Analogy Wholesale Access Retail Access CLASS is… is NOT…

7. DISCUSSION DRAFT ONLY The Bank Analogy Retail Access: Checks Debit Cards Branch Banking ATM access Public usability CLASS is…NNDC is… “Interbank” Transactions “Owner” Deposits “Owner” Withdrawal Commercial Access “Read Only” Expert Users Primary

8. DISCUSSION DRAFT ONLY CLASS Service CLASS Services – Ingest Archive Storage “One NOAA” Access Coordinated OAIS with NNDC NODCNCDCNGDC Climate Perspective Ocean Perspective Geophysical Perspective Raw IngestRaw Access XML? Per submission agreements IT Services Layer Domain Expert Layer Preservation Layer Stewardship Layer

9. DISCUSSION DRAFT ONLY Service Metrics CLASS Metrics – Data Stored (PB, # sets) Data Accessed (# inquiry, volume) Data Ingested Latency Preservation Activity NODCNCDCNGDC Climate Perspective Ocean Perspective Geophysical Perspective Raw IngestRaw Access XML? computer to computer Per submission agreements IT Services Layer Domain Expert Layer Preservation Layer Stewardship Layer Customers Served Tailored portals Stewardship actions Metadata enhancement Note that NNDC stewarded (“owned”) data can be delivered to end-users without passing through the NNDC owner’s site. Domain Portals & Expert User Support

10. DISCUSSION DRAFT ONLY NNDC Metrics: Questions re. CLASS Does CLASS support: subsetting capabilities? (Does this require an inappropriate understanding of content?) data mining? Regeneration (e.g., producing an intermediate data form in response to a query?) Can CLASS change (obsolete) the external “look and feel” of data access (e.g., no command line access)? Can CLASS obsolete “old” access methods (e.g., dial-up modems, 8” diskettes, 2” tape, etc.?) How do these impact CLASS metrics?

11. DISCUSSION DRAFT ONLY NNDC Metrics: Questions re. CLASS Is CLASS independently responsible for reversible transformations (perhaps as part of media migration)? Is this an “operations” question? Can CLASS independently do irreversible transformations, if required for data preservation? Can CLASS move obsolete editions to lower care levels? How does CLASS measure closely coupled data transfers (e.g., for reprocessing)? How do these impact NNDC metrics?

12. DISCUSSION DRAFT ONLY Metrics CLASS metrics would move toward: infrastructure/wholesale storage and preservation NNDC metrics would move toward: Value-added/retail Stewardship focused CLASS metrics and NNDC metrics should (eventually) be distinguishable. However, CLASS should report nonintermediated access statistics for NNDC owned/stewarded datasets. Metrics need further development.

13. DISCUSSION DRAFT ONLY CLASS Metrics ?? Volume in and out by time, by NNDC Volume stored Collection Inventory Inventory changes Preservation activity Data flows & latency Storage and bandwidth reserves

14. DISCUSSION DRAFT ONLY NNDC Metrics ?? Number & quality of Value- added interfaces (usage?) Datasets reprocessed or enhanced # datasets at highest level of maturity Volume served/total of data at highest level of maturity Customer liaison contacts Metadata enhancements

15. DISCUSSION DRAFT ONLY Conclusion Eventually, NNDC and CLASS metrics should be distinct. More work is needed to identify the “right” metrics to measure effectiveness. Coordinate with other data centers in NASA and USGS on metrics Good metrics are HARD! – multiple measures are required