1. SWGData and Software Access - 1 UCB, Nov 15/16, 2006 THEMIS SCIENCE WORKING TEAM MEETING Data and Software Access Ken Bromund GST Inc., at NASA/GSFC

2. SWGData and Software Access - 2 UCB, Nov 15/16, 2006 Overview THEMIS website will make all levels of data -- Level0 (L0), L1, L2 and key parameter data -- available to the public. Seamlessly interface with other websites which provide data sub-sets or specialized data. –CDAWeb –Experimenters’ FTP sites. –Possibly CDAWeb mirror sites (ASI data). THEMIS websites will provide software necessary for loading all THEMIS data types into IDL, display, and basic analysis on most common platforms. –Goal is to create a platform-independent software distribution. –Work on most platforms supported by IDL –Goal is to create a complete, IDL-only code distribution. –IDL code will be tested on Windows, Mac OS X, Linux, Solaris. Overview

3. SWGData and Software Access - 3 UCB, Nov 15/16, 2006 Local Data Access Local data access depends on having data in proper tree structure. THEMIS software will access data in local cache by timespan and data type: –Interface to local cache will be based on information available from directory tree structure and times in file names. –User does not need to know directory tree and file name structure. –Cdf_file_name IDL procedure does this for daily files. –L0 file interface is TBD –Option to have THEMIS load_th[a-g]_L[0-3] procedures check remote FTP sites for data missing or out-of-date in local cache. Remote users will be able use the THEMIS software to access data in their local cache when off line.

4. SWGData and Software Access - 4 UCB, Nov 15/16, 2006 Data Distribution Data will be available from FTP sites at Berkeley and CDAWeb, and possibly other sites. Web browser or anonymous FTP (mget) -- must re-create directory structure manually. THEMIS cache procedure: IDL interface to FTP sites which creates a local cache of all files for a given data type and timespan. –Will copy files into proper data structure using IDL sockets to access FTP or HTTP server. –Interface to FTP sites will be built on CDAWeb Web Services. –Any FTP site can be indexed by CDAWeb: CDAWeb can then be used to provide locations of files at remote FTP sites.

5. SWGData and Software Access - 5 UCB, Nov 15/16, 2006 Web services provide a unified interface to the THEMIS SOC archive, the CDAWeb archive and other data repositories. Data Distribution Architecture THEMIS SOC FTP L0, L1, L2 CDAWeb FTP Key Param, L2, …? CDAWeb Web Services Other Remote FTP L2, …? THEMIS cache Procedure Local Cache Metadata Data Request URL Response CDAWeb creates and uses an XML description of the layout and format of a remote FTP or HTTP site Load Timespan for Data Type Into IDL THEMIS load_ procedure Look for remote data? Data Files Pre-Load Timespan for Data Type Into Cache

6. SWGData and Software Access - 6 UCB, Nov 15/16, 2006 Software Distribution Three layers of access:.zip files of stable, tested releases of complete software. Public HTTP access to Subversion (SVN) code repository for read-only access to latest software. –Source tree can be browsed in any web browser. –Complete software distribution can be downloaded and kept up-to- date using SVN client software (freely available). –Alternatively, can download daily.zip files based on latest version of repository. Restricted SVN+SSH access to SVN for developers/contributers. –SVN maintains version control and facilitates collaboration with other THEMIS developers. –Requires installation of SVN client. –Develop/modify code on your own platform, and upload it to the THEMIS repository. –Public/Private SSH key pair must be generated to obtain access. –You keep the private key. THEMIS SOC registers a copy of your public key to give you write-access to the SVN server. –THEMIS SOC can generate a key for you, and send you the private key.

7. SWGData and Software Access - 7 UCB, Nov 15/16, 2006 Subversion Clients Access to the THEMIS SOC code repository has been tested with the following SVN clients SVN command line on Linux, Mac OS X (should also work on PC, if interested) –Fastest running interface. –May be the simplest solution if your interaction with SVN will be limited to checking out the latest software. –Can be scripted, e.g. to get latest code every night. Cross-platform GUI: SmartSVN –Java-based, looks the same on all platforms: PC, Linux, Mac OS X. –Basic checkout, update, commit, etc. features are available for free. –Some useful features (e.g. checking for available updates, repository browsing) are available only when a license is purchased (~$60). Windows-only GUI: TortoiseSVN –Integrated with Windows Explorer. –All features of SVN are implemented and availble for free.

8. SWGData and Software Access - 8 UCB, Nov 15/16, 2006 Subversion Training THEMIS SOC will work with you to get up-to-speed with SVN Choice of using SVN yourself or letting the SOC facilitate code assimilation. Crib sheets available for installation and use of GUI and command line interfaces. Some SVN concepts such as global version numbers, branching, copying and tagging are new, even to users of other version control systems (CVS, SCCS, Code Manager) Installation / training at remote sites can be facilitated with VNC. –Enables remote users to view the same desktop. –VNC viewers available on PC/Linux/MacOSX.