Presentation on theme: "EXtensible Data Cataloging System XDCS. 1 Problem The radar software group relies on capturing very high speed data of low- level digital signals for."— Presentation transcript:
eXtensible Data Cataloging System XDCS
1 Problem The radar software group relies on capturing very high speed data of low- level digital signals for development. Examples: –GRA 5500 ADC capture: 4 GB per hour –GWX 70 ADC capture: 100 GB per hour –GWX 7000 ADC capture: > GWX 70 Some of these captures take considerable effort to acquire.
2 What is XDCS? Efficient, scalable storage for very large files Consistent, well-defined retrieval of the contents of very large files Relate file contents between different sources and formats Document and annotate file contents
3 Why not network folders? File storage must scale in the range of 2 TB to 10 TB per year. It is difficult to maintain organization in a free-form file/folder structure. Finding, downloading, and re-parsing individual files larger than a gigabyte wastes time when all that’s needed is a few megabytes. No content-relevant searching.
4 Desired Features Abstract physical file storage to ease scaling and organization on the user Retrieve selectable segments of files Stand-alone GUI for common activities C and Python libraries allow streaming partial files directly over the network Web interface for browsing/retrieval (and administration/monitoring) File format translation, combining, and filtering Attach supporting files to capture sets Offline parse/tag workflow
5 System Diagram
6 Typical Workflow
8 Project Setup Examples
9 Typical Workflow
10 GUI Demonstration
11 Export Modules Ability to plug in different export formats
12 Future Plans Web browse/search/retrieve interface GUI polish/iteration –Parametric search and visualization Project setup automation/verification Portable library development Import/export plug-in API Server performance, scalability analysis Data acquisition front-end Software/project update tools