ESDSWG meeting – 10/21-23/2008 Metrics Planning and Reporting (MPAR) WG Breakout Summary H. K. (Rama) Ramapriyan NASA GSFC Clyde Brown NASA LaRC Co-Chairs, MPARWG 8 th Earth Science Data Systems Working Group Joint Working Group Meeting Wilmington, DE October 20-22, 2009

ESDSWG meeting – 10/21-23/2008 MPAR Working Group  Attendance – 23  Topics Discussed  Discussions with Martha Maiden  Metrics baseline review  FY 09 Reporting  Core and Community metrics collection

ESDSWG meeting – 10/21-23/2008 MPARWG Breakout Sessions ItemWho Day 1 - Session 1: FY09 Activities and Progress IntroductionRama HQ / Program view of MetricsMaiden Martha Maiden's Four Questions and MPARWG responsesHunolt E-Books vs Metrics - What is Reported Via EachRama Action / Open Items from Oct 2008 workshopRama Summary of FY09 Metrics ReportingHunolt Day 2 - Session 2: Metrics and MEaSUREs MEaSUREs Telecon ResultsRama Metrics Baseline ReviewHunolt / All Day 2 - Session 3: Metrics Collection - Core and Community Overview of Metrics Collection MCT & EMSRama Introduction for MEaSURES and Changes to MCTRandy Barth Transition to "MCT2" and EMS for Metrics ReportingKevin Murphy Day 2 - Session 4: TBD, Action Items and Work Plan Continued discussion of metrics baselineHunolt/All

ESDSWG meeting – 10/21-23/2008 Discussions with Martha Maiden  Martha emphasized importance of metrics for her reporting to her management and OMB  Metrics from both Community systems and Core systems are important  User satisfaction metrics – American Consumer Satisfaction Index is an important metric watched by OMB  Aggregates and highlights from core and community metrics are useful  Impact metrics are very useful, but a better format is needed  Encouraged use of metrics for the benefit of the Projects themselves

ESDSWG meeting – 10/21-23/2008 Actions Planned in Response  Provide separate aggregates for community systems that include trends, so community systems are not lost in the totals.  Provide periodic reports in a form useful to Martha  Program level summaries and trends  Highlights from individual community systems. These could be based on the regular metrics, e.g. noting significant growth in users, distribution, addition of new projects or services, etc.  Projects could use supplemental comments to highlight / call attention to items.  The form in which this information is provided to Martha could be quad-chart or a short pdf ‘newsletter’.

ESDSWG meeting – 10/21-23/2008 Metrics Baseline Review Products and Services Metrics - Common Metrics:  Metric 1 - Distinct Users – No change  Metric 2 – Distinct Users by Class – No change  Metric 3 – Products Distributed – Added option of breaking down by product types  Metric 4 – Product Types Available  Metric 5 – Volume Distributed – after considerable discussion, left unchanged  Metric 6 – Volume Available – after considerable discussion, left unchanged  Metric 7 – no longer used  Metric 11 – Services Provided  Metric 12 – Services Available – need explanatory examples for projects to use this better

ESDSWG meeting – 10/21-23/2008 Metrics Baseline Review  Project Defined (Project Specific) Metrics (up to 4, defined by each project, metrics 100, 101, 102, 103)  Programmatic Metrics:  Metric 8 – Science Focus Areas Supported  Metric 9 – Applications Areas Supported  Metric 10 – Education categories Supported  Impact Metrics – decided to recommend use of quad chart format, but will develop strawman examples and validate with Martha  New Citation Metric  Deborah Smith (RSS, Inc) presented an approach they have been using to collect citations of their products  Recommend use of citation metric to be used optionally

ESDSWG meeting – 10/21-23/2008 FY 09 Reporting  22 REASoN; 15 ACCESS; 8 MEaSUREs [Many of the REASoN and ACCESS Projects completed during FY 09  Impact metrics – 3 from 2 REASoN and 1 MEaSUREs Projects  We expect 10 or more MEaSUREs projects to start reporting in FY 10  Generally, projects report when they start distributing data to users  Last REASoN and ACCESS Projects are expected to finish in FY 10  Upcoming ACCESS reporting - TBD

ESDSWG meeting – 10/21-23/2008 Core and Community Metrics Collection  Core system metrics  EOSDIS Data Center metrics collection using EMS was demonstrated  Internal use of metrics in ESDIS for planning purposes  Network flows vs. requirements are analyzed using periodic “pinging” and measuring actual data flows  Community metrics  Collected using MCT  FY 09 improvements to MCT discussed (e.g., 508 compliance)  Transition to EMS-based collection of community metrics discussed – needs further examination  Recommended prototyping automated collection with two separate (diverse) projects using log files