1. DATA QUALITY AND META DATA
PRESENTED AT THE TRAINING COURSE ON ‘INTERNATIONAL TRADE STATISTICS’
BRUNEI DARUSSALAM
20-22 SEPTEMBER 2016

2. 1. IMTS 2010 RECOMMENDATIONS 2. DATA QUALITY 3. QUALITY REPORT OUTLINE
4. METADATA
5. DISSEMINATION

3. IMTS 2010 RECOMMENDATIONS

4. 1. IMTS 2010 RECOMMENDATIONS i. Systematic approach to data quality
ii. Dimensions of data quality
iii. Contents of quality reports
iv. Quality indicators
v. Cross-country data comparability 4 4

5. DATA QUALITY

6. WHY Data Quality is Important
Credibility
To ensure reliable and accurate data for input into BOP and GDP
And also for policy making
Will distort statistical analysis if included.
e.g : Wrong country code…

7. Validation Errors Data is certainly incorrect
2. DATA QUALITY
Validation Errors
Data is certainly incorrect
May cause processing Problems
Cannot be included in statistical analysis
Must be removed/corrected
Examples
text characters in value field
invalid commodity or country code

8. Credibility Errors Data is valid but possibly incorrect
2. DATA QUALITY – cont.
Credibility Errors
Data is valid but possibly incorrect
Can be processed
Examples:
inflated value (currency) or quantity (*000)
valid but wrong commodity code
Will distort statistical analysis if included.
e.g : Wrong country code…

9. Interim data of External Trade Statistics
2. DATA QUALITY – cont.
Validation Steps
Invalid
Validity Check
Electronic Entry
Valid
Auto-correction
Credibility Check
Server JP
Credible
Server SMK
Interim data of External Trade Statistics

10. Quality Controls 2. DATA QUALITY – cont. (a) Coverage check (b)
Front-end check
(c)
Back-end check 10 10

11. (a) Coverage Check Electronic Hardcopy 2. DATA QUALITY – cont.
Statements from Free Zone Authority, Power Producer Company, Petroleum Company, MAS, Air Asia, etc)
Hardcopy 11

12. (b) Front-end Check High value transactions Cross-check Range-check
2. DATA QUALITY – cont.
(b) Front-end Check
High value transactions
Cross-check
Range-check
Masterfile-check
Computation-check 12

13. EXAMPLES OF VALIDATION CHECK

14. High Value Transactions
BXXXXW
COMPANY ABC
JALAN ABCXYZ
High Value Transactions
COMPANY XYZ
ADDRESS OVERSEAS
AZ9999
SYARIKAT XXX
TAN AH BAH

15. Country of Origin COMPANY ABC TAN AH BAH JALAN ABCXYZ XXXX
BXXXXW
COMPANY ABC
JALAN ABCXYZ XXXX
XXXX YYYYYYYYYYYYY
NEGERI AA
Country of Origin
COMPANY XXXMMMM
JALAN ABCXYZ XXXX
XXXX YYYYYYYYYYYYY
AZ9999
COMPANY XXXMMMM
TAN AH BAH

16. Unit of Quantity COMPANY ABC TAN AH BAH BXXXXW JALAN ABCXYZ XXXX
XXXX YYYYYYYYYYYYY
NEGERI AA
Unit of Quantity
COMPANY XXXMMMM
JALAN ABCXYZ XXXX
XXXX YYYYYYYYYYYYY
NEGERI CINA
AZ9999
COMPANY XXXMMMM
TAN AH BAH

17. Country of destination
BXXXXW
COMPANY ABC
JALAN ABCXYZ XXXX
Country of destination
COMPA
JALAN ABCXYZ XXXX
XXXX YYYYYYYYYYYYY
NEGERI AA
AZ9999
COMPANY XX
TAN AH BAH

18. Channel COMPANY ABC TAN AH BAH BXXXXW JALAN ABCXYZ XXXX
XXXX YYYYYYYYYYYYY
NEGERI AA
COMPANY XXXMMMM
JALAN ABCXYZ XXXX
AZ9999
COMPANY XXXM
TAN AH BAH

19. Code COMPANY ABC TAN AH BAH BXXXXW JALAN ABCXYZ XXXX XXXY COMPANY XXX
AZ9999
COMPANY XXXMMMM
TAN AH BAH

20. Quality Check (METSS) COMPANY ABC TAN AH BAH AUP CHECK BXXXXW
JALAN ABCXYZ XXXXXXXXXXXX
XXXX YYYYYYYYYYYYYXXX
COMPANY XXXMMMMMMM
AUP CHECK
JALAN ABCXYZ XXXXXXXXXXX
XXXX YYYYYYYYYYYYYXXXXX
AZ9999
COMPANY XXXMMMMMMMMM
TAN AH BAH

21. Quality Check (METSS) cont.
BXXXXW
COMPANY ABC
JALAN ABCXYZ XXXX
XXXX YYYYYYYYYYYYY
COMPANY X
JALAN ABCXYZ XXXXMMMMM
XXXX YYYYYYYYYYYY
Error in Tariff Code
Change HS  AHTN
AZ9999
COMPANY XXXMMMMMMMMM
TAN AH BAH
ASEAN country

22. (c) Back-end Check 2. DATA QUALITY – cont.
Imports and Exports by Country
Imports and Exports by Customs Station/ Channel
Imports Exports Ratio
Cross-check with Other Countries
Checking of Selected Products (Aircraft, Rubber, Petroleum, Copper, etc) 22

23. QUALITY REPORT - MALAYSIA

24. Benchmark to UK’s Quality Report
Malaysia External Trade Statistics Quality Report was sent to ASEAN Secretariat in August 2013 – in line with the recommendation of IMTS 2010.
Benchmark to UK’s Quality Report

25. Quality Dimensions 3. QUALITY REPORT 2. Accuracy
1. Timeliness
2. Accuracy
3. Accessibility & Clarity
4. Comparability
5. Coherence

26. Quality Indicators 3. QUALITY REPORT – cont. Timeliness
- Advanced Release Calendar (ARC)
YEAR
MONTHLY METS
ANNUAL METS
2009
5-6 weeks
10 months
2010
2011-current
9 months

27. Quality Indicators – cont.
3. QUALITY REPORT – cont.
Quality Indicators – cont.
2) Accuracy 2.1 Accuracy Characteristics 2.2 Controls using primary information/ sources 2.3 Controls using secondary information/source 2.4 Confidentiality 2.5 Actions to improve accuracy

28. Quality Indicators – cont.
3. QUALITY REPORT – cont.
Quality Indicators – cont.
2) Accuracy – cont.
2.1 Accuracy Characteristics
i) Statistical Value
Exports – free on board (f.o.b.)
Imports – cost, freight and insurance(c.i.f.)
ii) Level of thresholds
From January 2013, single transactions with a value ‘below RM1,000’ for Peninsular Malaysia are excluded except for major commodities and selected goods.

29. Quality Indicators – cont.
3. QUALITY REPORT – cont.
Quality Indicators – cont.
2) Accuracy – cont.
2.1 Accuracy Characteristics – cont.
iii) Exclusions
Transit and goods for repair
iv) Revisions
- The monthly data are provisional when 1st released and subject to revision in the following month.
- For annual, it will be released 9 months after the reference year.

30. Quality Indicators – cont.
3. QUALITY REPORT – cont.
Quality Indicators – cont.
2) Accuracy – cont.
2.2 Controls using primary information/sources
i) General information
ii) Information on controls and errors

31. Quality Indicators – cont.
3. QUALITY REPORT – cont.
Quality Indicators – cont.
2) Accuracy – cont.
2.2 Controls using primary information/sources
i) General information
a) Data providers
b) Number of transactions
YEAR
IMPORTS
% CHANGE
EXPORTS
2009
1,340,407 -
842,529
2010
1,447,348
8.0%
977,095
16.0%
2011
1,479,002
2.2%
994,848
1.8%
2012
1,555,057
5.0%
1,118,218
12.0%

32. Quality Indicators – cont.
3. QUALITY REPORT – cont.
Quality Indicators – cont.
2) Accuracy – cont.
2.2 Controls using primary information/sources
i) General information – cont.
c) Electronic declarations
YEAR
IMPORTS
EXPORTS
2009
85.0%
82.0%
2010
95.0%
93.0%
2011
96.0%
2012
97.3%
96.8%

33. Quality Indicators – cont.
3. QUALITY REPORT – cont.
Quality Indicators – cont.
2) Accuracy – cont.
2.2 Controls using primary information/sources
i) General information – cont.
d) Information on controls and errors
- Validity checks
- Credibility checks
- Methodology for credibility checks

34. Quality Indicators – cont.
3. QUALITY REPORT – cont.
Quality Indicators – cont.
2) Accuracy – cont.
2.2 Controls using primary information/sources
i) General information – cont.
d) Information on controls and errors
- Validity checks
- Credibility checks
- Methodology for credibility checks

35. Quality Indicators – cont.
3. QUALITY REPORT – cont.
Quality Indicators – cont.
2) Accuracy – cont.
2.3 Controls using secondary information/source
i) Statement on the imports/exports of oil and gas
ii) Statement on the imports of tobacco
iii) Report of total values and total number of transactions declared in Free Commercial Zone

36. Quality Indicators – cont.
3. QUALITY REPORT – cont.
Quality Indicators – cont.
2) Accuracy – cont.
2.4 Confidentiality
- Statistics Act 1965 (Revised – 1989)
Actions to improve accuracy

37. Quality Indicators – cont.
3. QUALITY REPORT – cont.
Quality Indicators – cont.
3) Accessibility and clarity
3.1 Accessibility
- Publications
- METS Online
Clarity
- Documentation available to users in all publications

38. Quality Indicators – cont.
3. QUALITY REPORT – cont.
Quality Indicators – cont.
4) Comparability and Issues
4.1 Comparability over space
- Methodological discrepancies in reporting
- Triangular trade
- Exchange rate
- Reporting time lags
- Classification of goods
- Confidentiality
Comparability over time

39. Quality Indicators – cont.
3. QUALITY REPORT – cont.
Quality Indicators – cont.
5) Coherence
- Inclusion of ‘ Specific Movements’ data into METS
- Mirror statistics analysis

40. 4. METADATA
The term metadata defines all information used to describe other data.
It is recommended that countries at least cover the following categories of metadata for their international merchandise trade statistics:
(i) description of all underlying concepts and definitions, including the trade system used,
and deviations from international standards, if any;
(ii) legal framework, institutional arrangements and description of data sources;
(iii) description of data collection and data processing procedures;
(iv) description of estimation methods;
(v) data dissemination policy including release and revision schedules;
(vi) description of the all data fields/ variables (reference period, trade flow, commodity
classification used, valuation, currency, quantity (net weight), weight unit used,
supplementary quantity, supplementary quantity unit used, partner country (origin, last
known destination, consignment) (this applies to all forms of data dissemination);
(vii) explanations and footnotes concerning the data as required, i.e. informing about
revisions, break in series and application of confidentiality;
(viii) quality reporting

41. 5. DISSEMINATION
The dissemination of data and metadata, therefore, should be treated as an integral part of the national programme of trade statistics and carried out with great care and attention to the needs of users, while at the same time ensuring adequate confidentiality of data providers.
It is recommended that passive confidentiality be applied as much as possible unless the use of active confidentiality is already the established, desired and accepted practice.
Reference period. It is recommended that countries make their data available on a calendar period basis, according to the Gregorian calendar and consistent with the present IMTS 2010 recommendations.
Countries are encouraged to issue their first releases of data as follows:
(a) Monthly totals of exports and imports within 45 days after the end of the
month, at least by major trading partners and basic commodity breakdown;
(b) Quarterly data within 60 days after the end of the quarter;
(c) Annual data within 90 days after the end of the year
Revision policy. Countries are encouraged to develop a revision policy that is well designed, carefully managed, transparent and well coordinated with other areas of statistics, and hence allows users to cope with revisions in a systematic manner.
It is recommended that countries choose the dissemination format that best suits their user needs.
It is further recommended that international merchandise trade statistics be accessed
through the electronic dissemination databases maintained by the responsible agency.