1. AgStandards Gauri Salokhe FAO, UN 21 May 2002 1 AgStandards Gauri Salokhe FAO, UN 21 May 2002 “the Consultation….. recognized the crucial importance of standards for addressing some of the key problems affecting access to agricultural information and data exchange on the Internet. This includes the implementation of international agreements on content management, classification schemes and agreed vocabularies ” COAIM 2000.

2. AgStandards Gauri Salokhe FAO, UN 21 May 2002 2 AgStandards Gauri Salokhe FAO, UN 21 May 2002 AgStandards Initiatives November 2000: workshop in Brussels (FAO and OneWorld Europe) to initiate discussion about a Metadata schema. July 2001: concept paper about an agricultural ontology service

3. AgStandards Gauri Salokhe FAO, UN 21 May 2002 3 AgStandards Gauri Salokhe FAO, UN 21 May 2002 Do we need common Metadata? Do we need Metadata at all?

4. AgStandards Gauri Salokhe FAO, UN 21 May 2002 4 AgStandards Gauri Salokhe FAO, UN 21 May 2002 Benefits of Metadata? simply put, “data about data” it enables effective, efficient, and accurate use of data sets and data collections It is the departure point for every kind of discovery system –paper based  catalogues –digital  machine readable repositories

5. AgStandards Gauri Salokhe FAO, UN 21 May 2002 5 AgStandards Gauri Salokhe FAO, UN 21 May 2002 Benefits of Metadata? although informal knowledge is still with the data collectors, metadata can be used by the “re-user” to understand the intellectual content It is a documentation of collected data, understandable many years after its collection

6. AgStandards Gauri Salokhe FAO, UN 21 May 2002 6 AgStandards Gauri Salokhe FAO, UN 21 May 2002 Metadata Roles - Creators moving from an unorganized pool or resources to organized pool organize knowledge about a domain –presenting an higher level of abstraction allows maintenance at both data set and collection levels

7. AgStandards Gauri Salokhe FAO, UN 21 May 2002 7 AgStandards Gauri Salokhe FAO, UN 21 May 2002 Metadata Roles – System Designers allows for managing the storage and access of the resources Facilitates communication between different domains when they all define their application specific information in an agreed way: –Syntax  XML, RDF –Structure  AgMES DTD, RDF Schema –Semantics  AgMES gives flexibility to provide more relevant query results through targeted searching –only title field, only subjects from AGROVOC

8. AgStandards Gauri Salokhe FAO, UN 21 May 2002 8 AgStandards Gauri Salokhe FAO, UN 21 May 2002 Metadata Roles– System Designers alleviates issues related to full-text searches by providing data on data –ex. Publication date vs. any date appearing in the document permits navigable display of resources as topics trees –subject gateways, alphabetical browsing by authors eases the process of harvesting different databases through understanding of common format of resource description –cross the multi-lingual barrier

9. AgStandards Gauri Salokhe FAO, UN 21 May 2002 9 AgStandards Gauri Salokhe FAO, UN 21 May 2002 Metadata Roles – Searchers find more useful and relevant resources independent of what they describe –ex. multimedia – can you make a full-text? –however, garbage in = garbage out! get adequate information about the content of the resource – classifications, keywords, date of publication, etc. get information on the status of the rights concerning the resource –copyright, terms of use, price, etc.

10. AgStandards Gauri Salokhe FAO, UN 21 May 2002 10 AgStandards Gauri Salokhe FAO, UN 21 May 2002 Metadata Roles– Searchers gives capability to browse resources –by subject, by country, by year Informed decisions can be made since it can allow for selection of relevant resource –25 min. tape vs. 143 min. tape

11. AgStandards Gauri Salokhe FAO, UN 21 May 2002 11 AgStandards Gauri Salokhe FAO, UN 21 May 2002 Why do we need common metadata? We are able to say that in a given application dc:title means “A name given to the resource” –And for example, not…A title given to a person, such as “Sir” or “Ms.” Standardization ensures that we are talking about the same thing!

12. AgStandards Gauri Salokhe FAO, UN 21 May 2002 12 AgStandards Gauri Salokhe FAO, UN 21 May 2002 Example: Common Environment Parallel search queries to search across systems arc:title my:titlebest:title Query to search title element for “Shrimp Production in Thailand” wrapper DB-A DB-C DB-B

13. AgStandards Gauri Salokhe FAO, UN 21 May 2002 13 AgStandards Gauri Salokhe FAO, UN 21 May 2002 Example: Ideal Environment Single search across systems Query to search dc:title tag for “Shrimp Production in Thailand” dc:title DB-A DB-C DB-B

14. AgStandards Gauri Salokhe FAO, UN 21 May 2002 14 AgStandards Gauri Salokhe FAO, UN 21 May 2002 Example: Achievable Compromise Single search across systems Query to search dc:title tag for “Shrimp Production in Thailand” arc:title my:titlebest:title DB-A DB-C DB-B AgMES compliant XML mapping

15. AgStandards Gauri Salokhe FAO, UN 21 May 2002 15 AgStandards Gauri Salokhe FAO, UN 21 May 2002 Summary - Metadata Common metadata allows us to : –Give lexical words a meaning –Facilitate easy exchange between systems –Facilitate resource discovery and request access for it –Recombine content to be used for different purposes inventory across databases –Reduction of cost by using standardized tools AgMES document DC.Dot It can also facilitate automatic processing and manipulation of information –Ex. send an email using all fields

16. AgStandards Gauri Salokhe FAO, UN 21 May 2002 16 AgStandards Gauri Salokhe FAO, UN 21 May 2002 Metadata Standards Its not lack of standards, its that we have too many of them!!! –Dublin Core, DOI,, EPICS, RDF and more Most are developed either by: –standards body (ISO) –a community of practice (DCMI) –a company (local use) need to move up

17. AgStandards Gauri Salokhe FAO, UN 21 May 2002 17 AgStandards Gauri Salokhe FAO, UN 21 May 2002 Metadata Standards Dublin Core –15 simple elements –Qualified DC ex. date modified Why DC? –Its goal is resource discovery which meets our requirements. –We also need to avoid reinventing the wheel

18. AgStandards Gauri Salokhe FAO, UN 21 May 2002 18 AgStandards Gauri Salokhe FAO, UN 21 May 2002 Development of AgMES AgMES – Agricultural Metadata Element Set –http://www.fao.org/agris/agmes/http://www.fao.org/agris/agmes/ Specifies elements necessary for precise and complete description of resources within the Agricultural Domain Can’t we just use DC Qualified? –No, its not domain specific

19. AgStandards Gauri Salokhe FAO, UN 21 May 2002 19 AgStandards Gauri Salokhe FAO, UN 21 May 2002 When do we need something new? Use existing single schema where possible else Use an existing domain specific schema else Borrow a term from another namespace else Create local namespace and recommend an extension to existing element (schemes or refinements) else Create an element

20. AgStandards Gauri Salokhe FAO, UN 21 May 2002 20 AgStandards Gauri Salokhe FAO, UN 21 May 2002 A Sample from AgMES Element Qualifier Element Refinement (s) Encoding Schemes/ Controlled List (DC) Title (DC) Alternative (DC) Subject (AGLS) Availability (AGS) Creator (AGS) Corporate Author (AGS) Personal Author (AGS) Conference (AGS) Title Supplement (AGS) Number (AGS) Place (AGS) AGROVOC (AGS) CABIT (AGS) ASFA (AGS) NAL (AGS) ASC (AGS) CABIC (AGS) Classification (AGS) Thesaurus

21. AgStandards Gauri Salokhe FAO, UN 21 May 2002 21 AgStandards Gauri Salokhe FAO, UN 21 May 2002 Benefits of AgMES? Gives meaning to our data across the organization – Semantics – Interoperability Domain specific controlled vocabularies –AGROVOC –ASFA As more and more information becomes available, discovery gets harder –a common tool to sort through them

22. AgStandards Gauri Salokhe FAO, UN 21 May 2002 22 AgStandards Gauri Salokhe FAO, UN 21 May 2002 What are Schemas? Schemas –Describe concepts and existing relationships between those concepts (Ontologies); –Can be encoded in machine processable way. Namespace schema –Allows for declaration of “newly defined” terms and their definitions Application Profile schema –Allows for declaration of “use” of terms by specific applications – Ex. DC, AgMES – Ex. AGRIS AP, CGIAR AP

23. AgStandards Gauri Salokhe FAO, UN 21 May 2002 23 Overview SCHEMAS DC NS DCQ NS AgMES NS AGLS NS AGRIS APS

24. AgStandards Gauri Salokhe FAO, UN 21 May 2002 24 Why an AGRIS application Profile? AGRIS is an information exchange network –More than 50 active centres –70.000 references a year –Very heterogeneous centres with different local needs and systems The existing AGRIS standards (from 1975) are not sufficient –Too much focused on one central database –Not considering digital objects –Not compliant to other Standards (DC, OAI-MHP) –Too much linked to a specific database structure and specific processing software (AGRIN) –Demand for XML based input procedures

25. AgStandards Gauri Salokhe FAO, UN 21 May 2002 25 AgStandards Gauri Salokhe FAO, UN 21 May 2002 AGRIS Application Profile Will present a common format on which information between centres can be exchanged Publishing the metadata allows us to specify what AGRIS metadata means Based primarily on accepted standards : DC, AGLS Specifications will be made available in following formats –NarrativeNarrative –DTDDTD –XML SchemaXML Schema –RDF Schema

26. AgStandards Gauri Salokhe FAO, UN 21 May 2002 26 AgStandards Gauri Salokhe FAO, UN 21 May 2002 AGRIS Application Profile DC.Title with xml:lang attribute to indicate the language in which the title is written AGS Subject Classification with ASC as schemes; XML seq is used to make sure that we retain the order DC Date Issued English Title [120,220] French Title [122,222] Spanish Title [124,224] Other Title [126,226] Primary subject Category [7] Secondary Subject Category [8] Imprint: Date of Publication [263, 463] Analytical, Monograph, Serials

27. AgStandards Gauri Salokhe FAO, UN 21 May 2002 27 AgStandards Gauri Salokhe FAO, UN 21 May 2002 AGRIS Application Profile English Title [120,220] French Title [122,222] Spanish Title [124,224] Other Title [126,226] Primary subject Category [7] Secondary Subject Category [8] Imprint: Date of Publication [263, 463] Analytical, Monograph, Serials Water Agua Water 2000

28. AgStandards Gauri Salokhe FAO, UN 21 May 2002 28 AgStandards Gauri Salokhe FAO, UN 21 May 2002 Benefits of AGRIS AP Problem 1 : focused on one central database – AP provides opportunities for distributed approach – new AGRIS AP standards take into account digital objects – based on DC standards which allows us to be OAI-MHP compliant as well – standardized form provides for improved accessibility and visibility in a more open environment. – file structure takes into account unnecessary duplication of information in records – platform independent storage and exchange of information – XML DTD and Schema are provide Problem 2 : No consideration for digital objects Problem 3 : Not compliant to other Standards Problem 4 : Linked to a specific database structure and specific processing software (AGRIN) Problem 5 : Demand for XML based input procedures

29. AgStandards Gauri Salokhe FAO, UN 21 May 2002 29 Past, Present, Future CG Infofinder DTD based partly on AGMES specifications FAO needs : Application Profiles for metadata on different levels based on a common standard as evidenced in the Finder workshop The AgMES, containing our domain specific elements, will be made available for the COAIM with an example AP –AGRIS AP

30. AgStandards Gauri Salokhe FAO, UN 21 May 2002 30 Ontology aided Querying and browsing WAICENT Virtual Library Qualified Metadata (App.Profile 2) DocRep- DB Library Catalogue Others Distributed Metadata-Repositories (WebApps) Simple (unqualified) metadata (App.Profile 1) EIMSCDS Others Distributed Metadata-Repositories Simple metadata (App.Profile 1) Distributed Multimedia Online Repositories Wordindex produced by automatic system/HTML metadata FAO web Domains Ranked and ordered Resultset Infofinder 1. Validation against Ontology (Metadata schema, Controlled vocabularies, Categorization) 2. Ranking for subject relevance 3. Ranking for metadata security 4. Production of Output

31. AgStandards Gauri Salokhe FAO, UN 21 May 2002 31 AgStandards Gauri Salokhe FAO, UN 21 May 2002 More Information http://www.fao.org/agris/agmes/ Agricultural Metadata Standards Homepage http://www.dublincore.org/ Dublin Core Metadata Initiative http://www.naa.gov.au/recordkeeping/gov_online/agls/cim/cim_man ual.html Australian Government Locator Service Metadata Set