1. Metadata Infrastructure and Standardisation in New Zealand
Evelyn Wareham Manager, Information Management Statistics New Zealand
METIS Working Session, May 2013

2. Overview Statistics New Zealand’s transformation
Metadata Infrastructure programme
What we’ve achieved
What’s next
Our approach
Challenges and lessons learnt
7 May 2013
E Wareham METIS 2013

3. Statistics 2020
Statistics NZ is currently undertaking an organisation-wide programme of change, Statistics 2020 Te Kāpehu Whetū, to create the statistical system of the future.
This programme has been developed to address challenges and opportunities in order to create a more dynamic, responsive, and sustainable organisation.
The Strategic Plan sets out what Statistics NZ is seeking to achieve over the next 10 years and how we plan to get there. The plan contains four key strategic priorities.
Five key benefits will also be delivered, these are:
1. Continued supply of important and trusted statistics
2. An agile and responsive NSO able to respond to changing needs
3. Costs to government, businesses, and households minimised
4. Increased use of government data
5. Government has confidence that its investment in official statistics is value-for-money
7 May 2013
E Wareham METIS 2013

4. 10 Year Transformation 160+ current projects
Major infrastructure development
Platforms replacing fragmented systems
Standardisation
Outwards orientation
Culture change
Traditionally business units across Statistics NZ stood alone and implemented systems for their needs. While the business units were operating well, they did not work together to create optimal efficiency for the organisation.
As a result many of Statistics NZ's systems and software are now out-of-date, not supported by vendors, and only a few IT staff understand them. Resolving this system legacy problem is a major priority for the Board, as this issue could seriously affect Statistics NZ's reputation, plus its delivery and leadership roles in the official statistics system.
The future plan is founded on effective and standardised technology, shared capability and common use platforms.
With the legacy migration plan, the Platform roadmaps provide a staged transition from the old siloed architecture to the future state architecture allowing continued delivery of statistical outputs and expected time and cost savings.
Standardisation is about developing statistical infrastructures and approaches to clusters of “like” statistics, and functions (e.g. collection and analysis) within the statistical production process… Standardisation is not a one size fits all approach. Rather it focuses on natural production clusters or integrated functions which have similar characteristics, thereby facilitating the introduction of a common, standardised approach to their management.
23/11/2018
E Wareham ALGIM 2012

5. Phases of Transformation
23/11/2018
E Wareham ALGIM 2012

6. Metadata Infrastructure Vision
Information flows efficiently through the statistical business process, enabling more agile statistical production.
Allowing us to be more customer-focused and respond more quickly to new information needs
Making sure info resource required to support statistical production are available to be easily used and reused and where they’re not available, easily developed/created

7. We see an environment where …
Less effort is needed to transport information across systems and adopt powerful new tools
Less effort is needed to discover and understand existing data sources and information
Our tools encourage standardisation and data (and metadata) re-use
Good metadata management tools and systems
Is a key foundational of this environment

8. Previous Situation
Currently, information does not flow particularly efficiently between systems or across the statistical process
We store key pieces of information that are by their nature structured and precisely defined – like questions, topic defintions, concepts, data item definitions- that are produced as part of the statistical process in Lotus Notes documents in DocONE libraries, in Excel spreadsheets, on our website on various pages and in various reports.
And heaven help us if we want to re-use older data or use information from another part of the business - we often have to go on time-consuming treasure hunt searching through a variety of DocONE libraries and Notes databases or rely on the memory of key staff members.
This means that when we want to use these pieces of information, for example, to load a processing system or a dissemination tool, etc. we have to manually massage that information into a new form (e.g. by cutting and pasting from Word or Lotus documents into an Excel spreadsheet) to meet the particular format requirements of a particular software packages or legacy database system or particular process.

9. Pain Points
In terms of where this occurs, bottleneck before dissemination
We often have to do this searching or manual re-shaping of information between phases of the statistical business process – particularly right before we disseminate data – and before we collect as well. So we merrily go along with production process and then we get to one of these points we have to do a bunch of work. effort is spent on these parts of the process

10. The Future

11. Look Reuse Create Information approach
High – level approach to solving these problems -
Our goal, then is to develop tools – software, standards, guidance, etc. – that allow us to better utilize our information to help us through the process, i.e. utilise
key pieces of information that we naturally produce during the statistical process in an active way to help automate or drive some processes
The organization should operate on the principle that– rather than making creation of new information resources in various shapes (documents, excel templates, load files for each system or process) the default, our systems should make re-using information the default
So we want a model that make it easier to
In the first place, LOOK for information or data
RE-USE existing information or data
And then only if something is not available, CREATE new a resources
This will both save us money and support standardisation

12. Systems
And looking in a bit more depth at the individual systems or tools that may be part of that metadata box or layer, these are the systems that would comprise such a layer.
We’ve already built 1 & 2

13. Progress to date
April 2012: Colectica deployed, training starts
June 2012: CARS, Micro-economic & Household platform metadata surfaced in Colectica
June 2012: Standards and guidelines developed
Oct 2012: Classification System development begins
Oct : Data custodians update information in Colectica
October 2012: RPD process happens via Colectica
May 2013: OSS metadata improvement pilot conducted
May 2013: Colectica Portal available on external website

14. Improvements to come
Apr 2014: Classification Management system developed
Dec2013: Services to allow other systems to access metadata developed
Question library?
Concept library?
Metadata standardisation

15. Coming Soon!
23/11/2018
E Wareham ALGIM 2012

16. Our Approach
Series of stages: No big bang
Partnerships with innovative vendors
International collaboration
Standardisation
80:20, not perfection
Culture change, not just systems

17. Challenges and Lessons Learnt
Selling the benefits – these may be to others, not to those doing the work
Tailoring the message – do they need to understand DDI, SDMX and GSIM?
Continuity and getting the timing right – must connect to legacy systems, but look to the future
Can’t do it alone – partnerships are critical to success
7 May 2013
E Wareham METIS 2013

18. evelyn.wareham@stats.govt.nz www.stats.govt.nz
Questions?