Presentation is loading. Please wait.

Presentation is loading. Please wait.

17 Nov 2003Australia VO - ATNF1 Metadata and Registries: Describing and Finding VO Resources R. Hanisch 1, R.Plante 2, G. Greene 1, A.E. Linde 3, T. McGlynn.

Similar presentations


Presentation on theme: "17 Nov 2003Australia VO - ATNF1 Metadata and Registries: Describing and Finding VO Resources R. Hanisch 1, R.Plante 2, G. Greene 1, A.E. Linde 3, T. McGlynn."— Presentation transcript:

1 17 Nov 2003Australia VO - ATNF1 Metadata and Registries: Describing and Finding VO Resources R. Hanisch 1, R.Plante 2, G. Greene 1, A.E. Linde 3, T. McGlynn 4, W. OMullane 5, A.M.S. Richards 6, R. Williams 7, R. Williamson 2, E. C. Auden 8,K. T. Noddle 3 1) Space Telescope Science Institute 2) National Center for Supercomputing Applications 3) University of Leicester 4) NASA Goddard Space Flight Center 5) The Johns Hopkins University 6) Jodrell Bank Observatory 7) California Institute of Technology 8) Mullard Space Science Laboratory T HE US N ATIONAL V IRTUAL O BSERVATORY

2 17 Nov 2003Australia VO - ATNF2 Resource Metadata A resource is any VO entity that can be described and given a name and unique identifier –Data collection (archive) –Catalog or collection of catalogs –Organization –Software packages –Bandpass filter functions –Services Services are VO resources that can be invoked by a user or software agent to perform some action on their behalf Metadata describes VO resources. This metadata generally includes information the user or a computer program needs to determine if a resource is of interest and how a service is invoked.

3 17 Nov 2003Australia VO - ATNF3 Resource Metadata Resource metadata is described by –A prose document that defines concepts independent of an encoding scheme –XML Schemas that encode metadata and metadata relationships Draws on Dublin Core metadata –An interdisciplinary standard for core resource metadata http://dublincore.org http://dublincore.org Can be categorized –Identity –Curation –General content – Collection/service content – Data quality – Service invocation

4 17 Nov 2003Australia VO - ATNF4 Resource Metadata

5 17 Nov 2003Australia VO - ATNF5 Resource Metadata Example Identity metadata TitleSloan Digital Sky Survey ShortNameSDSS Identifierivo://stsci.edu/mast/sdss Curation metadata PublisherSpace Telescope Science Institute/MAST PublisherIDivo://stsci.edu/mast CreatorSloan Digital Sky Survey Consortium Creator.Logohttp://archive.stsci.edu/images/sdss_logo.gif ContributorSloan Digital Sky Survey Consortium Date2001-06-15 VersionSDSS EDR ReferenceURLhttp://archive.stsci.edu/sdss/index.html Contact.NameArchive Branch, Space Telescope Science Institute Contact.Address3700 San Martin Drive, Baltimore, MD 21218 USA Contact.Emailarchive@stsci.edu Contact.Telephone +1-410-338-4547 General content metadata Subjectgalaxies, quasars, stars, CCD photometry, spectroscopy, redshift, sky surveys DescriptionThe Sloan Digital Sky Survey is using a dedicated 2.5-m telescope and a large format CCD camera to obtain images of over 10,000 square degrees of high Galactic latitude sky in five broad bands (u', g', r', i' and z', centered at 3540, 4770, 6230, 7630, and 9130 Å, respectively)… Source2002AJ….123..485S TypeSurvey, Catalog, EPOResource ContentLevelResearch Relationshipmirror-of RelationshipIDivo://sdss.org/sdss/edr Required keywords shown in red Collection and service content metadata FacilityApache Point Observatory, Sloan 2.5-m Telescope InstrumentFive-band clocked CCD camera Coverage.Spatialpolygon (FK5, 145.17, 1.25, 235.9, 1.25, 235.9, -1.25, 145.17, 1.25) or polygon (FK5, 250.71, 66.29, 267.0, 66.29, 267.0,52.15, 250.71, 66.29) or polygon (FK5, 350.43, 1.17, 360.0, 1.17,360.0, -1.25, 350.43, -1.25) or polygon (FK5, 0.0, 1.17, 56.37, 1.17, 56.37, -1.25, 0.0, -1.25) Coverage.RegionOfRegard0.0001 Coverage.SpectralOptical Coverage.Spectral.Bandpassu, g, r, i, z Coverage.Spectral.MinimumWavelength400.e-9 Coverage.Spectral.MaximumWavelength850.e-9 Coverage.Temporal.StartTime1999-12-25 Coverage.Temporal.StopTime2001-07-15 Coverage.Depth3.e-6 Coverage.ObjectDensity6.e4 Coverage.ObjectCount2.e7 Coverage.SkyFraction0.01 Resolution.Spatial0.00028 Resolution.Spectral5000 Resolution.Temporal120 UCDNot Provided Formattext/xml RightsPublic Data quality metadata DataQualityA Uncertainty.Photometric3.e-7 Uncertainty.Spatial0.00003 Uncertainty.Spectral1.e-11 Uncertainty.Temporal0.1

6 17 Nov 2003Australia VO - ATNF6 Resource Metadata Example Service metadata Service.InterfaceURLhttp://archive.stsci.edu/cgi-bin/sdss/catalog.html Service.BaseURLhttp://archive.stsci.edu/cgi-bin/sdss/catalog Service.HTTPResultstext/xml Service.StandardIDivo://ivoa.net/Services/ConeSearch Service.StandardURLivo://www.ivoa.net/Documents/REC/ConeSearch.html Service.MaxSearchRadius0.2 Service.MaxReturnRecords5000

7 17 Nov 2003Australia VO - ATNF7 Resource Metadata: XML Schema Classes of Resources Organization, DataCollection, Service, Registry –Specific classes inherit from generic Organized into separate schemas: –Core resource metadata: VOResource –Various extensions schemas containing specific types Capable of describing… –Data centers, research organizations, missions, observatories –Data collections, archives –VO standard services: Cone Search, Simple Image Access –Existing Browser/CGI-based services

8 17 Nov 2003Australia VO - ATNF8 The Role of Resource Registries Used to discover and locate resourcesdata and servicesthat can be used in a VO application Registry: a list of resource descriptions –Expressed as structured metadata to enable automated processing and searching Registries are themselves VO Resources

9 17 Nov 2003Australia VO - ATNF9 Registry Requirements Allow user to select resources that are likely to pertain to a scientific question Select resources based on characteristics… –Type of resource: catalogs, image archives, EPO, services –Coverage in space, time, and frequency –Where data comes from, who curates it Dynamic: resources will come and go Distributed: Should not depend on a single point of failure or single view of the VO. Preserve the data providers control over their data –Curators control what gets registered, content, updates –Allow integration with existing resource management Allow extension to new types of resources

10 17 Nov 2003Australia VO - ATNF10 IVOA Registry Working Group (RWG) Common approach to registries Work packages –Science requirements and use cases –Resource metadata –Registry interfaces –Prototyping Distributed model for registries

11 17 Nov 2003Australia VO - ATNF11 Local Publishing Registry Local Searchable Registry Full Searchable Registry Local Publishing Registry Full Searchable Registry Data Centers VO Projects Specialized Portals & Services Registry Model

12 17 Nov 2003Australia VO - ATNF12 Local Publishing Registry Local Searchable Registry Full Searchable Registry Local Publishing Registry Full Searchable Registry Data Centers VO Projects Specialized Portals & Services Registry Model harvest (pull)

13 17 Nov 2003Australia VO - ATNF13 Local Publishing Registry Local Searchable Registry Full Searchable Registry Local Publishing Registry Full Searchable Registry Data Centers VO Projects Specialized Portals & Services Registry Model harvest (pull) replicate

14 17 Nov 2003Australia VO - ATNF14 Local Publishing Registry Local Searchable Registry Full Searchable Registry Local Publishing Registry Full Searchable Registry Data Centers VO Projects Specialized Portals & Services Registry Model harvest (pull) replicate selective harvesting

15 17 Nov 2003Australia VO - ATNF15 Local Publishing Registry Local Searchable Registry Full Searchable Registry Local Publishing Registry Full Searchable Registry Data Centers VO Projects Specialized Portals & Services Client Applications search queries Registry Model

16 17 Nov 2003Australia VO - ATNF16 Local Publishing Registry Local Searchable Registry Full Searchable Registry Local Publishing Registry Full Searchable Registry Data Centers VO Projects Specialized Portals & Services Client Applications search queries Registry Model

17 17 Nov 2003Australia VO - ATNF17 Local Publishing Registry Local Searchable Registry Full Searchable Registry Local Publishing Registry Full Searchable Registry Data Centers VO Projects Specialized Portals & Services Client Applications search queries Registry Model

18 17 Nov 2003Australia VO - ATNF18 NVO Prototype Registry To support a Data Inventory Service (DIS) What is known about a position in the sky? –Use a registry to locate and query standard services: Cone Search Services: querying catalogs Simple Image Access Services: querying image archives and cutout services Components –Publishing Registries –Searchable Registry –Resource Metadata –Harvesting Protocol –Populated with service descriptions

19 17 Nov 2003Australia VO - ATNF19 Publishing Registries: getting information into registries Two publishing registries established at Caltech and NCSA. Motivation: –Register Simple Image Access Services –Develop techniques for easy registration Resource descriptions stored as XML documents using VOResource schema

20 17 Nov 2003Australia VO - ATNF20 Harvesting Interface Adopted Open Archives Initiative (OAI) Protocol for Metadata Harvesting –HTTP/CGI-based protocol for exposing metadata to harvesters (e.g. searchable registries) Advantages: –Existing, field-tested design we didnt have to re-invent –Fairly easy to implement –Existing tools for emitting and harvesting metadata –Exposes our metadata to larger digital library community

21 17 Nov 2003Australia VO - ATNF21 Curator uses another sites registry –Good for a few resources whose descriptions are fairly static e.g. @NCSA: http://nvo.ncsa.uiuc.edu/nvoregistration.html VORegistry-in-a-box: –Deployable package that allows a data provider to run own registry out of the box http://nvo.ncsa.uiuc.edu/VO/software –Good for larger number of resources that might be updated often Curator builds own OAI interface –Good for very large number of resources –Automate XML generation using sites existing information management tools Models for Registering Resources

22 17 Nov 2003Australia VO - ATNF22 Searchable Registry Searchable Registry was set up at JHU/STScI http://skyserver.pha.jhu.edu/devel/registry OAI harvester collects resource descriptions –from Publishing Registries at Caltech & NCSA –Loads data into relational database SOAP Web Service interface http://skyserver.pha.jhu.edu/devel/registry/registry.asmx –Searching Currently provides specialized querying useful for DIS –Re-harvest request To get updated records from publishing registries

23 17 Nov 2003Australia VO - ATNF23 Local Publishing Registry Full Searchable Registry Local Publishing Registry Caltech JHU/STScI harvest (pull) Data Inventory Service search for services Registry Model NCSA DIS

24 17 Nov 2003Australia VO - ATNF24 Cone Search Service Cone Search Service Simple Image Access Simple Image Access Local Publishing Registry Full Searchable Registry Local Publishing Registry Caltech JHU/STScI harvest (pull) Data Inventory Service search for services Registry Model NCSA DIS Cone Search Service Simple Image Access Data Providers

25 17 Nov 2003Australia VO - ATNF25 Summary We built a working prototype registry system to support an end-user VO service –Distributed Publishing and Searchable components –Encoded descriptions using emerging VO XML standard schemas –OAI Harvesting Standard deployed easily –Used to discover Cone Search and SIA services Whats next: Interoperable registries IVOA-wide –Implement newly agreed-upon Resource Metadata standard and VOResource XML schema –Demonstrate harvesting and replication –Populate registries with broad base of VO resources –Standardize registry query interfaces


Download ppt "17 Nov 2003Australia VO - ATNF1 Metadata and Registries: Describing and Finding VO Resources R. Hanisch 1, R.Plante 2, G. Greene 1, A.E. Linde 3, T. McGlynn."

Similar presentations


Ads by Google