1 U.S. Department of the Interior U.S. Geological Survey National Center for EROS Remote Sensing Technologies Group The Proposed USGS Plan for Digital Data Quality Assurance Jon Christopherson SAIC, Contractor to the USGS EROS ASPRS Camera Calibration Session March 9 th, 2005

2 U.S. Department of the Interior U.S. Geological Survey Historical Role USGS providing Camera Calibration service since 1973 Not a “Certificate” or certification Simply a report of camera characterization Contained pass/fail criteria Camera meets basic performance levels Deliverable in those days was often just the film Film required camera parameters to derive products Barest minimum of product quality assurance! And only if the product was film

3 U.S. Department of the Interior U.S. Geological Survey Changing Roles Film is often not the delivered product today Instead products derived from that film Orthos, mosaics, GIS layers, etc. USGS Camera Calibration Report still required Necessary, but not sufficient to assure quality Many more processes involved with producing products today. Can be multiple firms in the chain from photon to final product What has USGS Camera Cal report provided? What more is needed?

4 U.S. Department of the Interior U.S. Geological Survey The USGS Proposed Plan USGS proposed plan is: Focused on Sensors Focused on Processes Focused on the End Products End Goal is to Assure End Product Quality

5 U.S. Department of the Interior U.S. Geological Survey Focus on Sensors Focus on Sensors - “Sensor Systems” Without sensors there are no processes or products Increasing capabilities and complexities will bring challenges The USGS is considering “Type Certification” of digital systems Certification would include not only sensors and associated systems, but also the entire “chain” USGS would like to work with manufacturers & developers to certify product creation chain(s) e.g. Leica ADS-40 with ADS40-specific calibration technique(s), ADS40 processing software, etc.

6 U.S. Department of the Interior U.S. Geological Survey Sensors (cont.) Type certification would include understanding: Sensor & system designs Operating parameters, limitations Operating constraints – what is necessary to deliver intended quality Designers/Manufacturers know this best USGS works with them to learn system capabilities USGS to understand Mfr’s verification process Observes/Participates in testing, validating methods, verifying accuracy

7 U.S. Department of the Interior U.S. Geological Survey Sensors (cont.) Goal is to understand: The capabilities of a system The requirements for successful operation Requirements for calibrations, methods, frequency, etc. What level of data quality can this system reliably provide?

8 U.S. Department of the Interior U.S. Geological Survey Focus on Processes All products are the result of processes The quality of aerial mapping imagery is at least as dependent on the processes as on sensors Process control at least as important to customers as sensors Processes make/break the quality chain

9 U.S. Department of the Interior U.S. Geological Survey Processes (cont.) Processes include: Flight planning, data collection & recording Configuration maintenance of systems/software Maintenance of sensor/systems Calibration procedures, history Data handling, particularly steps where data is processed Processing software versions, etc. Any other external inputs All of this information must be documented Much of this is ideally in the metadata associated with the final product

10 U.S. Department of the Interior U.S. Geological Survey Processes (cont.) Manufacturers are key to helping develop standard processes Flyers, industry, ASPRS/ISPRS are key also Quality demands that processes are controlled – and documented Something like an ISO-9000 certification? Or an ASPRS-certified Professional Practices?

11 U.S. Department of the Interior U.S. Geological Survey Focus on the End Products The ultimate goal is to assure the quality of the final product Community has developed a reliance on USGS Camera Calibration report Implies that a good camera was used More is needed Community needs to know that good cameras, systems and processes were used throughout!

12 U.S. Department of the Interior U.S. Geological Survey What This All Means Hardware/Software Certification burden is transferred from many individual flyers to (relatively) few manufacturers Allows more in-depth understanding of each system, working directly with technologists Helps keep burden of hardware/software performance on those who developed it Burden on flyers/producers now shifted to processes Insists that defined processes be carried out and documented Aids in troubleshooting, maintaining high standards Flyers are/should be doing this anyway

13 U.S. Department of the Interior U.S. Geological Survey What This All Means (cont.) USGS to work with ASPRS to promote education of new standards for use by consumers Many more factors affect data quality, data usefulness Education required to best obtain & use new capabilities Old “Camera Cal Report required ” boilerplate doesn’t cover it all! USGS Certification and documented processes means more - a Greater Assurance of Data Quality We all benefit from that!

14 U.S. Department of the Interior U.S. Geological Survey We Are Not Alone EuroSDR is exploring similar concepts and processes for Europe Professional organizations (ASPRS/ISPRS) are in general agreement so far Could it be possible to come up with universal standards & practices?

15 U.S. Department of the Interior U.S. Geological Survey Work To Be Done: USGS to continue building cooperation: With Manufacturers With partners (IADIWG, EuroSDR, ASPRS, ISPRS) With Data Providers (flyers & processors) With consumers/users USGS to continue exploring/developing new calibration & quality assurance tools & methods Make tools available to public as appropriate USGS to continue developing infrastructure Continue assessing new systems, capabilities USGS Needs Your Input and Ideas!

16 U.S. Department of the Interior U.S. Geological Survey End

17 U.S. Department of the Interior U.S. Geological Survey “Classes” of Digital Data Industry needs “Classes” of Digital Products Applies to those generating, processing, selling, buying and working with these data Can apply to aerial, satellite, even lead to standards for non- imaging products, e.g. LIDAR. Will develop different quality “classes” of digital mapping data Based on resolution & accuracy Initial Classes for geometric and spatial quality Followed soon (?) by radiometric quality USGS offers to lead standards development, with ASPRS/ISPRS

18 U.S. Department of the Interior U.S. Geological Survey Examples of Data Classes The following are notional examples to illustrate what data classes could be: Class 0: No geometric/geodetic accuracy implied, “Pretty pictures” Class A: Ground Sample Distance (GSD) > 1.0 meter, geometric accuracies <0.5pixel size Class B: Ground Sample Distance (GSD) < 1.0 meter, geometric accuracies <0.4m Class C: GSD < 0.5m, geometric/geodetic <0.2m Class D: GSD < than 0.15m, geometric/geodetic <0.1m Class E: (Reserved for future capabilities)

19 U.S. Department of the Interior U.S. Geological Survey Work Needed to be Done USGS to establish contacts points for manufacturers, begin interaction Funding – the longer we wait, the more it costs USGS partially funded in FY05 Establish IADIWG funding mechanisms required for FY05 USGS and IADIWG need to define funding mechanisms for FY06 and FY07 Define infrastructure needed In-situ ranges Hardware and Software requirements Laboratories – radiometry especially Establish guidelines, policy, standards, and boiler plate specifications IADIWG involved in standards development & approval IADIWG to begin specifying data “classes” in their work, requesting USGS standards, educating users & industry