Exchange of trade data using SDMX COMTRADE joint UNSD-OECD project 9 January 2007

Provide a coherent source of merchandise trade statistics data Eliminate duplication in provision and processing of data Develop common data processing standards and tools Share development costs Nearly identical integrated systems for data collection and processing at both agencies OECD processes member country data for OECD UNSD processes data for all other countries Content of complete databases in each Organisation is synchronised via the Data Exchange Suite (DES) DES = web service & synchronisation program (ComtradeTools) DECISIONS !!! Use SDMX as data exchange standard for UNSD  OECD data transfers !!! Use SDMX as data dissemination format Comtrade - overview

Comtrade - screens

Comtrade - content UN Comtrade is a main disseminator of annual commodity trade statistics - data since 1962 over 150 reporters 1.3billion records Over 150 reporters per year (>95% of the World Trade) On average, more than 2 billion records downloaded every month Newly published data synchronized with other international organizations such as World Bank, ECLAC, OECD, ITC, UNCTAD, etc via Web Services Imports, Exports, Re-exports, Re- imports Commodity by Trading Partner Annual data Value in US dollars Net weight and secondary quantities

Comtrade – key elements Classifications and Commodity Codes HS (Harmonized System) and SITC (Standard International Trade Classification) Reporter country (ISO digit numeric) Partner country (ISO digit numeric) Trade Flow (Export, Import, Re-export, Re-import) Time (Year) Unit of measurements (Weight, Currency) Observations (Net weight, Net value) Comtrade uses SDMX cross-sectional format v1.0. Each commodity classification has a specific Key Family. Therefore, there are 6 key families: HS2002, HS1996, HS1992, SITC Rev.3, SITC Rev.2, and SITC Rev.1 Explanatory notes are integrated with Comtrade SDMX. This is a beta version. The SDMX structure can be modified. Comtrade Web Services are open Site License users. For further information please go to

Comtrade – key families Data series defined in SDMX Key Families element structure:Dimension

Comtrade – data exchange Server: Comtrade web servicesComtrade web services –Internet-based, HTTP protocol, REST architecture Client: –Can access Comtrade web service via web browser –and/or use ComtradeTools – a command-line program for data transfer/conversion applications SDMX cross-sectional format, schemas specific to each classification - see UN site for more infoSDMX cross-sectional formatinfo Web service can compress data on the fly (70-80%) Access security based on IP address

Comtrade – data transfer

Comtrade – SDMX downloads Data can be downloaded in SDMX format directly, e.g.: –Classification: H2 = Harmonized System 2002 –Reporter: 36 = Australia; Partner 251 = France –Years: 2003 and 2004; Commodity: Total trade =36&y=2004,2003&cc=TOTAL&p=251&comp=false –Classification : S2 = SITC2 –Reporter: 36 = Australia; Partner 0 = World –Year : 2004; Commodity: Total trade

Comtrade - tools Requires Win2000 or higher Standalone command-line program for Comtrade data download; requires additional software to synchronize SQL database Downloads data in SDMX-ML format, can transform into CSV (or import into SQL Server) Available from ssetup.zip

Comtrade – tools usage ComtradeTools /px:s2 /r:36 /p:0 /cc:TOTAL /y:2004 ComtradeTools /px:H2 /r:251 /cc:10* /y:2004 ComtradeTools /px:H2 /r:251 /cc:10* /y:2004 /action:DownloadAndConvertToCSV /outputDirectory:c:\temp\ ComtradeTools /help ComtradeTools /year:2003 /action:DownloadDataSetList ComtradeTools /r:360 /y:2003 /px:H1 /source:Comtrade /format:SDMX /action:DownloadAndImportToSQLDB

Comtrade – conclusions, future A production application, based on SDMX data exchange format and web services SDMX cross-sectional format – standardised V1.0 Rapid - exchange can be compressed by web service Can convert to CSV format, and upload/synchronise with SQL database - no specialised SDMX knowledge necessary for end-users SDMX is becoming “just a part of the plumbing” UN Comtrade SDMX v.2.0 is in progress Data collection via SDMX: would need to be simple, via a generic format to avoid multiple schemas User-friendly interface for ComtradeTools Aligning with other trade SDMX trade projects such as CDS (Common Data Set) from WTO