14 October 2003ADASS 2003 – Strasbourg1 Resource Registries for the Virtual Observatory R.Plante (NCSA), G. Greene (STScI), R. Hanisch (STScI), T. McGlynn (NASA/GSFC), W. O’Mullane (JHU), R. Williams (Caltech), R. Williamson (NCSA) T HE US N ATIONAL V IRTUAL O BSERVATORY

14 October 2003ADASS 2003 – Strasbourg2 The role of Resource Registries Used to discover and locate resources—data and services—that can be used in a VO application Resource: anything that is describable and identifiable. –Besides data and services: organizations, projects, software, … –Presently concerned with simple set of resource types Registry: a list of resource descriptions –Expressed as structured metadata to enable automated processing and searching

14 October 2003ADASS 2003 – Strasbourg3 Selected Requirements Allow user to select resources that are likely to pertain to a scientific question Select resources based on characteristics… –Type of resource: catalogs, image archives, EPO, services –Coverage in space, time, and frequency –Where data comes from, who curates it Dynamic: resources will come and go Distributed: Should not depend on a single point of failure or single view of the VO. Preserve the data providers’ control over their data –Curators control what gets registered, content, updates –Allow integration with existing resource management Allow extension to new types of resources

14 October 2003ADASS 2003 – Strasbourg4 IVOA Registry Working Group (RWG) IVOA = International Virtual Observatory Alliance Common, global approach to registries Work packages: –Science requirements and use cases –Resource metadata –Registry interfaces –Prototyping Distributed model for registries

14 October 2003ADASS 2003 – Strasbourg5 Local Publishing Registry Local Searchable Registry Full Searchable Registry Local Publishing Registry Full Searchable Registry Data Centers VO Projects Specialized Portals & Services Registry Model

14 October 2003ADASS 2003 – Strasbourg6 Local Publishing Registry Local Searchable Registry Full Searchable Registry Local Publishing Registry Full Searchable Registry Data Centers VO Projects Specialized Portals & Services Registry Model harvest (pull)

14 October 2003ADASS 2003 – Strasbourg7 Local Publishing Registry Local Searchable Registry Full Searchable Registry Local Publishing Registry Full Searchable Registry Data Centers VO Projects Specialized Portals & Services Registry Model harvest (pull) replicate

14 October 2003ADASS 2003 – Strasbourg8 Local Publishing Registry Local Searchable Registry Full Searchable Registry Local Publishing Registry Full Searchable Registry Data Centers VO Projects Specialized Portals & Services Registry Model harvest (pull) replicate selective harvesting

14 October 2003ADASS 2003 – Strasbourg9 Local Publishing Registry Local Searchable Registry Full Searchable Registry Local Publishing Registry Full Searchable Registry Data Centers VO Projects Specialized Portals & Services Client Applications search queries Registry Model

14 October 2003ADASS 2003 – Strasbourg10 Local Publishing Registry Local Searchable Registry Full Searchable Registry Local Publishing Registry Full Searchable Registry Data Centers VO Projects Specialized Portals & Services Client Applications search queries Registry Model

14 October 2003ADASS 2003 – Strasbourg11 Local Publishing Registry Local Searchable Registry Full Searchable Registry Local Publishing Registry Full Searchable Registry Data Centers VO Projects Specialized Portals & Services Client Applications search queries Registry Model

14 October 2003ADASS 2003 – Strasbourg12 NVO Prototype Registry To support a Data Inventory Service (DIS) What is known about a position in the sky? –Use a registry to locate and query standard services: Cone Search Services: querying catalogs Simple Image Access Services: querying image archives and cutout services (see P3.18, McGlynn et al.; Components –Publishing Registries –Searchable Registry –Resource Metadata –Harvesting Protocol –Populated with service descriptions

14 October 2003ADASS 2003 – Strasbourg13 Resource Metadata Under development within the IVOA RWG The standard comes in two parts: –Prose document that defines concepts independent of an encoding scheme see P3.5, Hanisch et al. “Resource Metadata for the VO” –XML Schemas Draws on Dublin Core metadata –An interdisciplinary standard for core resource metadata Schema to stablize this month

14 October 2003ADASS 2003 – Strasbourg14 Resource Metadata: XML Schema Classes of Resources Organisation, DataCollection, Service, Registry –Specific classes inherit from generic Organized into separate schemas: –Core resource metadata: VOResource –Various extensions schemas containing specific types Capable of describing… –Data centers, research organizations, missions, observatories –Data collections, archives –VO standard services: Cone Search, Simple Image Access –Existing Browser/CGI-based services

14 October 2003ADASS 2003 – Strasbourg15 Publishing Registries: getting information into registries Two publishing registries established at Caltech and NCSA. Motivation: –Register Simple Image Access Services –Develop techniques for easy registration Resource descriptions stored as XML documents using VOResource schema P3.22 Williamson & Plante

14 October 2003ADASS 2003 – Strasbourg16 Harvesting Interface Adopted Open Archives Initiative (OAI) Protocol for Metadata Harvesting –HTTP/CGI-based protocol for exposing metadata to harvesters (e.g. searchable registries) Advantages: –Existing, field-tested design we didn’t have to re-invent –Fairly easy to implement –Existing tools for emitting and harvesting metadata –Exposes our metadata to larger digital library community Caltech & NCSA Registries use existing tool that implements OAI interface. To use, we had to: Store XML descriptions in files in a directory structure Provide an XSL style sheet to convert to Dublin Core XML See P3.22; Williamson & Plante;

14 October 2003ADASS 2003 – Strasbourg17 Curator uses another site’s registry –Good for a few resources whose descriptions are fairly static VORegistry-in-a-box: –Deployable package that allows a data provider to run own registry “out of the box” See P3.22; Williamson & Plante; –Good for larger number of resources that might be updated often Curator builds own OAI interface –Good for very large number of resources –Automate XML generation using site’s existing information management tools Models for Registering Resources

14 October 2003ADASS 2003 – Strasbourg18 Searchable Registry Searchable Registry was set up at JHU/STScI see P3.8 Green et al., “Searchable Registry for the NVO” OAI harvester collects resource descriptions –from Publishing Registries at Caltech & NCSA –Loads data into relational database SOAP Web Service interface –Searching Currently provides specialized querying useful for DIS –Re-harvest request To get updated records from publishing registries

14 October 2003ADASS 2003 – Strasbourg19 Local Publishing Registry Full Searchable Registry Local Publishing Registry Caltech JHU/STScI harvest (pull) Data Inventory Service search for services Registry Model NCSA DIS

14 October 2003ADASS 2003 – Strasbourg20 Cone Search Service Cone Search Service Simple Image Access Simple Image Access Local Publishing Registry Full Searchable Registry Local Publishing Registry Caltech JHU/STScI harvest (pull) Data Inventory Service search for services Registry Model NCSA DIS Cone Search Service Simple Image Access Data Providers

14 October 2003ADASS 2003 – Strasbourg21 Summary We built a working prototype registry system to support an end-user VO service –Distributed Publishing and Searchable components –Encoded descriptions using emerging VO XML standard schemas –OAI Harvesting Standard deployed easily –Used to discover Cone Search and SIA services What’s next: Interoperable registries IVOA-wide –Stablize XML metadata standard –Standardize registry interfaces