1. Douglas Mach Science and Technology Institute, Universities Space Research Association, Huntsville, AL, USA GOES-R GLM Post Launch Product Tests (PLPTs)

2. GLM L2+ Product Science Validation Baseline Schedule 2 LEGEND Launch & Orbit Raising Operations - Post-launch Testing (PLT) - 180 Days Outgas System Performance Operational Test (SPOT) 12 Days30 Days 60 Days 6 Mo Post-Launch Assessment Review (PLAR) and Handover Readiness Review (HRR): Handover S/C Ops to OSPO 30 Days PLPT Ex- tended Val =8.4 Yrs Ops Beta Stage Testing Provisional Stage TestingValidated Stage Testing Peer-Stakeholder L1b & KPP Product Validation Review Handover L1b & KPP Products to OSPO Peer-Stakeholder L2+ Product Validation Reviews Handover L2+ Products to OSPO Products Not Yet Available to Public Beta Validated Product Provisionally Validated Product Fully Validated Product PLT Cal/INR 32 Days * 28 Days PLT End + 4 Months L+103 Days L+73 Days 9 Mo PLT End + 12 Months May 28, 2015 * Data blackout due to COOP test and Station Change to 105W.

3. GLM PLPT Types The MSFC GLM CWG IV&V team has 4 major categories of PLPTs –Validate GLM Product with Reference Data Set Comparisons (is the GLM actually meeting specifications for detecting lightning (DE), rejecting false alarms (FAR), and locating the lightning in time and space?) –Algorithm Testing (is the GLM algorithm correctly filtering, geolocating, and clustering the raw Level 0 (L0) data, both backgrounds and lightning, so that GLM is meeting specifications?) –Instrument Navigation and Registration (INR) Testing (is the GLM navigation correct so that GLM is meeting specifications?) –Long Term Baseline Testing (what is the current sensitivity of GLM, how does it compare to the Lightning Imaging Sensor (LIS) and the Optical Transient Detector (OTD), and how does it change over the years?) 3

4. GLM L2+ PLPT Sets GLM_L2_001 series: Validate GLM DE and FER using Well Characterized Medium to Long Range Networks with Medium to High Detection Efficiencies and Low False Alarm Rates –The various medium to long range networks primarily detect cloud to ground (CG) flashes with medium to high accuracy over the contiguous US (and beyond) GLM_L2_002 series. Validate GLM DE and FER using Well Characterized Short Range Networks with High Detection Efficiencies and Low False Alarm Rates –The various shot range networks (primarily VHF) detect total lightning with high accuracy over their limited ranges 4

5. GLM L2+ PLPT Sets (continued) GLM_L2_003 series. Validate GLM DE and FER using Well Characterized Very Long Range Systems with Low (to no) Flash Detection Efficiencies but Medium Storm Detection Efficiencies –There are locations within the FOV of GLM that are not covered by any lightning detecting system with decent flash level detection (e.g., open ocean areas). For these areas, validation will need to use bootstrapping methods (described later) and the use of systems that do not detect flashes (or do so at a very low DE) GLM_L2_004 series. Validate GLM DE and FER using Well Characterized Very Short Range Optical Systems with Very High Flash Detection Efficiencies and Low False Alarm Rates –The Airborne GLM Simulator (AGS), which consists of an optical detector array and the Lightning Instrument Package (LIP) flown on high altitude aircraft (e.g., ER-2, Global Hawk, etc.) detects total lightning with very high detection efficiency and very low false alarm rate in a very small area around the aircraft platform location and time within the GLM FOV 5

6. GLM L2+ PLPT Sets (continued) GLM_L2_005 series. Validate GLM DE and FER using Well Characterized Orbital Based Optical Systems with High Flash Detection Efficiencies and Low False Alarm Rates –The various orbital based, optical systems detect total lightning with high detection efficiency and low false alarm rate in a small area around the satellite footprint within the GLM FOV GLM_L2_006 series. Validate GLM DE and FER using Well Characterized Ground Based Electric Field Networks with High Flash Detection Efficiencies and Low False Alarm Rates –The various ground based electric field systems detect total lightning with very high detection efficiency and very low false alarm rate in a very small area around the array location and time within the GLM FOV 6

7. GLM L2+ PLPT Sets (continued) GLM_L2_007 series. Validate and Tune GLM L0-L1b Filter Parameters –Withdrawn GLM_L2_008 series. Validate GLM Continuing Current Detection Mode –Withdrawn GLM_L2_009 series. Validate GLM L1b-L2 Cluster/Filter Algorithm Using Spec Code and Comparing Outputs –This test will take the L1b output of GLM, process the events with the spec cluster/filter code produced by the GLM-AWG group and compare the output of the spec code with the output of the operational code to verify that the operational code is reproducing the spec code clustering results 7

8. GLM L2+ PLPT Sets (continued) GLM_L2_010 series. Validate GLM L0-L1b Filter Algorithms Using Spec Code and Comparing Outputs –The GLM L0-L1b operational code has a number of filters that can be toggled off/on and the output compared to external code outputs. To test the filters, two identical streams of L0 data are to be run through the L0-L1b code. One version has the individual filter off and the other has the individual filter on. The stream with the filter off is then filtered by an external version of the filter. The results are compared GLM_INR_011 series. Validate GLM INR With Comparisons to Well Located Ground Points GLM_RAD_012 series. Validate GLM Background Deep Convective Cloud Radiances and Radiance Trending With Comparisons to Other Measures of Background Radiances GLM_RAD_013 series. Validate GLM Event Radiances and Radiance Trending With Comparisons to Other Measures of Event Radiances 8

9. GLM PLPT Data Sources 9 ACTUAL DATAMinimum Needs 1,6 Maximum Needs 2 GLM L1b-L2 PLPT Reference 5 Data Name Data Avail Data FromData QuantityDurationData QuantityData Duration ATDnetMOUUK Met Office10/100/1000 flashes in GLM FOV1/5/10 dayNo Set MaximumFull Extended Validation Period001a, 011l BrazilDATMOUINPE10/100/1000 flashes in GLM FOV1/5/10 dayNo Set MaximumFull Extended Validation Period001b, 011l ENTLNIn houseEarth Networks10/100/1000 flashes in GLM FOV1/5/10 dayNo Set MaximumFull Extended Validation Period001e, 011l GLD360In houseVaisala10/100/1000 flashes in GLM FOV1/5/10 dayNo Set MaximumFull Extended Validation Period001g, 011l NLDNIn houseVaisala10/100/1000 flashes in GLM FOV1/5/10 dayNo Set MaximumFull Extended Validation Period001n, 011l RinDATMOUINPE10/100/1000 flashes in GLM FOV1/5/10 dayNo Set MaximumFull Extended Validation Period001r, 011l STARnetMOUUniv. Sao Paulo10/100/1000 flashes in GLM FOV1/5/10 dayNo Set MaximumFull Extended Validation Period001s, 011l NALMAIn houseNASA Server10/100/1000 flashes in GLM FOV1/5/10 dayNo Set MaximumFull Extended Validation Period002a, 011l TLMAMOUEnvironment Canada10/100/1000 flashes in GLM FOV1/5/10 dayNo Set MaximumFull Extended Validation Period002c, 011l DCLMAIn houseNASA Server10/100/1000 flashes in GLM FOV1/5/10 dayNo Set MaximumFull Extended Validation Period002d, 011l FCLMAMOUColorado State Univ.10/100/1000 flashes in GLM FOV1/5/10 dayNo Set MaximumFull Extended Validation Period002f, 011l NGLMAMOUGeorgia Tech10/100/1000 flashes in GLM FOV1/5/10 dayNo Set MaximumFull Extended Validation Period002g, 011l HLMAMOUTAMU10/100/1000 flashes in GLM FOV1/5/10 dayNo Set MaximumFull Extended Validation Period002h, 011l KSCLMAMOUNASA Server10/100/1000 flashes in GLM FOV1/5/10 dayNo Set MaximumFull Extended Validation Period002k, 011l NMLMAMOUNew Mexico Tech10/100/1000 flashes in GLM FOV1/5/10 dayNo Set MaximumFull Extended Validation Period002n, 011l OKLMAMOUOU-CIMMS10/100/1000 flashes in GLM FOV1/5/10 dayNo Set MaximumFull Extended Validation Period002o, 011l WTLMAMOUTexas Tech10/100/1000 flashes in GLM FOV1/5/10 dayNo Set MaximumFull Extended Validation Period002t, 011l WILMAIn houseNASA Server10/100/1000 flashes in GLM FOV1/5/10 dayNo Set MaximumFull Extended Validation Period002w, 011l

10. GLM PLPT Data Sources (continued) 10 ACTUAL DATAMinimum Needs 1,6 Maximum Needs 2 GLM L1b-L2 PLPT Reference 5 Data Name Data Avail Data FromData QuantityDurationData QuantityData Duration ABIIn houseNASA Server2/20/200 thunderstorms in GLM FOV1/5/10 dayNo Set MaximumFull Extended Validation Period003a, 011a NEXRAD RadarMOUNOAA-NWS2/20/200 thunderstorms in GLM FOV1/5/10 dayNo Set MaximumFull Extended Validation Period003n MSG SEVERIMOUEUMETSAT2/20/200 thunderstorms in GLM FOV1/5/10 dayNo Set MaximumFull Extended Validation Period003s WWLLNIn houseUniv. of Washington2/20/200 thunderstorms in GLM FOV1/5/10 dayNo Set MaximumFull Extended Validation Period003w AGSIn houseUAH, MSFC10/100/1000 flashes in GLM FOV1/5/10 dayNo Set MaximumFull Extended Validation Period004a, 011l, 013a ISS-LISIn houseNASA Server10/100/1000 flashes in GLM FOV1/5/10 dayNo Set MaximumFull Extended Validation Period005i, 011l, 013i TARANIS 4 MOUCNES10/100/1000 flashes in GLM FOV1/5/10 dayNo Set MaximumFull Extended Validation Period005t, 011l HAMMAIn houseUAH10/100/1000 flashes in GLM FOV1/5/10 dayNo Set MaximumFull Extended Validation Period006h, 011l KSCFMAMOUNASA Server10/100/1000 flashes in GLM FOV1/5/10 dayNo Set MaximumFull Extended Validation Period006k, 011l GLM L2 Events, Groups, Flashes In houseNASA Server1000/10000/100000 events in GLM FOV1/5/10 dayNo Set MaximumFull Extended Validation Period009c GLM L0 EventsIn houseNASA Server1000/10000/100000 events in GLM FOV1/5/10 dayNo Set MaximumFull Extended Validation Period010 series GLM L1b EventsIn houseNASA Server1000/10000/100000 events in GLM FOV1/5/10 dayNo Set MaximumFull Extended Validation Period010 series GLM Geolocated Backgrounds TBD 1000/10000/100000 pixels in GLM FOV1/5/10 dayNo Set MaximumFull Extended Validation Period011a, 011b, 011g Laser BeaconTBD No Set MaximumFull Extended Validation Period011b Landmarks 7 TBD No Set MaximumFull Extended Validation Period011g ISS-LIS BackgroundsIn houseNASA Server1000/10000/100000 pixels in GLM FOV1/5/10 dayNo Set MaximumFull Extended Validation Period012i TRMM-LIS Backgrounds 3 In houseNASA Server1000/10000/100000 pixels in GLM FOV1/5/10 dayNo Set MaximumFull Extended Validation Period012t TRMM-LIS 3 In houseNASA Server100/1000/10000 flashes in GLM FOV1/5/10 dayNo Set MaximumFull Extended Validation Period013t

11. GLM PLPT Data Sources (notes) 11 1. For Beta/Provisional/Fully Validated levels. 2. There is no such thing as too much data. Based on lightning availability within the GLM FOV, various PLPTs will be activated in an attempt to validate as much of the GLM FOV as possible within the shortest period of time possible. 3. Instrument no longer in orbit, will use historical data for validation. 4. Instrument not likely to be available for initial PLPT period. Will be used in extended validation period if available. If not available even during extended validation, then the data (and related PLPTs) will not be activated. 5. Data sources can be used for more than one PLPT. 6. Given the "target of opportunity" nature of lightning data sources, the beta/provisional/fully validated data amounts are more of a guideline than hard limits. 7. Landmarks to be derived from various TBD sources.

12. GLM L2+ PLPT General Concepts Not all PLPTs may be activated –Weather at the time of the PLPT period will determine which ones will be utilized –Targets of Opportunity If multiple, possibly conflicting PLPTs can be activated, priority will be given to PLPTs in this general order: –Ground truth systems that are optical based (apples to apples, e.g., Airborne GLM Simulator (AGS), ISS-LIS) –Regions with no prior PLPT lightning (both temporal and spatial) –Ground truth systems with the highest DE, lowest FAR for total lightning –Note, these priorities are “flexible” Field program based PLPTs (e.g., using AGS) would take priority over all others due to limited time/location of field program data 12

13. GLM L2+ PLPT Summary GLM L2+ PLPTs consist of 4 types –Validate GLM Product –Algorithm Testing –Instrument Navigation and Registration (INR) Testing –Long Term Baseline Testing Goal of GLM L2+ PLPTs is to help validate the GLM data for operational use –Reach Beta Validation (minimally validated, and may still contain significant errors) within 1 month of start of PLPT period –Reach Provisional Validation (analyses are sufficient for qualitative, and limited quantitative, determination of product fitness-for-purpose) within 4 months of the end of the PLPT period –Reach Full Validation (analyses are sufficient for full qualitative and quantitative determination of product fitness-for-purpose) within 1 year of end of the PLPT period 13

14. Bootstrapping A Method to Achieve Validation of GLM Product Over Full FOV 14

15. Bootstrapping No “Ground Truth” system is exactly like GLM Most systems will have LOWER DE than GLM Some systems approach the DE of GLM (such as LMA’s) LMA’s have limited coverage of GLM FOV How can we validate GLM data in areas without well characterized ground truth systems with high DE and low FAR? 15

16. Bootstrapping Step One: Validate GLM DE/FAR with well characterized ground truth systems with high DE and low FAR in areas where they have coverage –LMA –AGS –ISS-LIS Step Two: Demonstrate “In Family” DE/FAR of well characterized ground truth systems with medium DE and high DE in areas where both systems have coverage –NLDN & LMA –ENTLDN & AGS Step Three: Extend “In Family” DE/FAR values into areas where there is only well characterized but medium DE ground truth systems –NLDN –ENTLDN Step Four: Demonstrate “In Family” DE/FAR values in areas where there are both medium DE and low DE systems –NLDN & WWLLN –GLD360 & ENTLDN Step Five: Further extend “in Family” DE/FAR into areas where the ONLY systems that are available are those with low DE –WWLLN –ABI 16

17. Bootstrapping 17

18. Bootstrapping 18

19. Questions? 19

20. Acronyms ABI = Advanced Baseline Imager ATDNet = Arrival Time Difference NETwork AGS = Airborne GLM Simulator ATBD = Algorithm Theoretical Basis Document AWG = Algorithm Working Group BrazilDAT = Sistema Brasileiro de Detecção de Descargas Atmosféricas CWG = Calibration Working Group DCC = Deep Convective Cloud DCLMA = District of Columbia Lightning Mapper Array DE = Detection Efficiency ENTLN = Earth Networks Total Lightning Network FAR = False Alarm Rate FCLMA = Fort Collins Lightning Mapper Array FM = Field Mill GHRC = Global Hydrology Resource Center GLD360 = Vaisala Global Lightning Dataset GLM = Geostationary Lightning Mapper GOES-R = Geostationary Operational Environmental Satellite R-Series GPA = Ground Processing Algorithm HAMMA = Huntsville Area Marx Meter Array HLMA = Houston Lightning Mapper Array INR = Image Navigation and Registration IR = Infrared ISS = International Space Station IV&V = Independent Verification and Validation KSC = Kennedy Space Center KSCLMA = Kennedy Space Center Lightning Mapper Array 20 LIS = Lightning Imaging Sensor LM = Lockheed Martin LZSS = Level Zero Storage System MOU = Memorandum Of Understanding MSFC = Marshal Space Flight Center MSG = Meteosat Second Generation NALMA = North Alabama Lightning Mapper Array NEXRAD = NEXt-generation RADar NGLMA = North Georgia Lightning Mapper Array NLDN = National Lightning Detection Network NMLMA = New Mexico Lightning Mapper Array NOAA = National Oceanic and Atmospheric Administration NSSTC = National Space Science and Technology Center OKLMA = OKlahoma Lightning Mapper Array OTD = Optical Transient Detector PLPT = Post Launch Product Test RINDAT = Rede Integrada Nacional de Detecção de Descargas Atmosféricas SEVERI = Spinning Enhanced Visible and Infrared Imager STARNet = Sferics Timing And Ranging NETwork TARANIS = Tool for the Analysis of RAdiations from lightNIngs and Sprites TBD = To Be Determined TLMA = Toronto Lightning Mapper Array TRMM = Tropical Rainfall Measuring Mission VaLiD = Validate Lightning Data WILMA = Wallops Island Lightning Mapper Array WTLMA = West Texas Lightning Mapper Array WWLLN = World Wide Lightning Locating Network