1. Usability Issues Facing 21st Century Data Archives Joey Mukherjee and David Winningham joey@swri.org

2. Current Archiving Goal MissionTeam Raw Data Processed Data Write Papers Data Iteration Quality Data Archive Future Scientists Quality Data

3. Current Archiving Reality MissionTeam Raw Data Processed Data Write Papers Data Iteration Data Subsets Permanent Archive Future Scientists Unchecked Data Home Institution Archive Public Data

4. New Goal MissionTeam Raw Data Processed Data Write Papers Data Iteration Processed Data Archive Future Scientists Processed Data

5. Standardizing HOWTO Make it easy Make it useful Make it extensible

6. Make it Easy Reading / writing files must be super easy (i.e. cheap!) –Either with tools or libraries Tools can be command line or GUI

7. Make it Useful How do I look at it? –Plots/Analysis What else can I do with it? –Read into IDL, Matlab, Excel, etc. Must have immediate benefits

8. Make it Extensible Must be possible for others to add value added services Must be able to hold varieties of data Must agree to give up control on content

9. Case Studies: HTML Easy to create! Once done, look at in browser Embrace / Extend

10. Case Studies: SPASE Creation is slow and difficult Once created, no real benefits yet VxOs have embraced, no one extended yet

11. Case Studies: IDFS Until recently, difficult to create, complex Once in, easy to look at, use, archive, etc. Somewhat extensible

12. Things right with IDFS Efficient Self documenting Calibrations stored in text file Science units derived instead of stored Little to no reprocessing ever needed

13. Other IDFS Benefits Can store most types of space physics data from raw telemetry to highly processed science units Reversible from science units to raw telemetry Usable by data processor, scientist, and data archiver

14. Things wrong with IDFS Overly complex format and API Not enough support in other tools - poor buy-in Analysis routines merged with the file format - tried to do too much!

15. Implementation Plan Develop a simple file format that can contain any and all types of time series space physics data Develop tools that allow someone to create and inspect files in this format Merge in the best parts of IDFS, CDF, netCDF, HDF, FITS, etc... without breaking paradigm of simplicity

16. Simple File Format Format might already exist: –HDF5 –XML –JSON –Other data models?

17. Making it useful Get buy-in from visualization tools (SDDAS, DataShop, VisBard, IDL DLM, etc.) Get buy-in from archives sites (PDS, PSA, NSSDC, etc.) Seed money is essential

18. Advantages Providers Users Management

19. Advantages: Providers Instrument teams now have something to work toward Can develop expertise

20. Advantages: Users Quick ways to create plots or access data Expertise again!

21. Advantages: Management Homogenous archives are infinitely easier to manage and maintain Value added services are a natural extension of quality archives

22. Conclusion Why now? Because SPASE is gaining traction, this is the next logical step. This will save money for everyone in the long run. Everyone benefits with value added services.