Data Archiving at the U.S. Central Bank Linda F. Powell Board of Governors of the Federal Reserve System Research and Statistics Division IASSIST Conference May 2005

Challenges Collection/Acquisition Quality Storage Documentation Cataloguing Dissemination

Data Collection / Acquisition Surveys Uniform storage and retrieval Uniform documentation Expensive collection process Purchase data from vendorsor acquire data from other sources Various formats and technology Unique documentation Licensing agreements

Data Collection through Storage – Micro Data Surveys Edit data at FRS Data Transmission Data Transmission FRS Fincl Data Repos. FRS Struct Data Repos. Reporter (e.g. bank) Process Data at FRS Display Data Reports

Data Collection through Storage – Micro Data Purchased from Vendors Comes in a variety of formats and modes of access Some data are processed and stored in SAS datasets Other data are only available through proprietary software Licensing agreements vary, affecting who has access to data and how the data can be stored

Data Collection through Storage – Macro Data Acquired from other public sources A limited amount is purchased from vendors Micro data are aggregated to create macro data Most production macro data are stored in FAME databases (time series data)

Data Collection through Storage – Data Quality Not all data are created equal Macro data are edited for reasonableness and consistency Survey micro data are edited for reasonableness, consistency, and compliance with business rules Vendor data are generally reviewed for accuracy by the end users. Resolution of problems varies by product

Data Documentation – Micro Data Purchased data have individual documentation Survey data have a uniform set of documentation compiled in the Micro Data Reference Manual (MDRM)

MDRM Each accounting or economic concept gets a number The same number is used across surveys for the same concept Each survey gets a mnemonic Surveys can have sub mnemonics to show distinctions between the series within a survey

MDRM Example Total Assets of an institution is collected on many surveys Total assets always has the number 2170 For finance companies the variable name is DFCR2170 Commercial banks have: RCON2170 for domestic total assets RCFD2170 for foreign total assets

MDRM Information

Data Documentation – Macro Data The FAME database can be self documenting Location, contacts, periodicity, units, … Data stored in FAME use a hierarchical nomenclature E.g. Disposable Outlays of Personal Income data are labeled Y.P.D.O

What Data are Available? Survey Micro Data are recorded in MDRM Macro Data are listed in FAME Other data are recorded in Data & News Catalogue All purchased data Some databases created in-house (e.g. merger adjusted financial data and supervisory data) Surveys performed by external organizations There is some overlap with MDRM and FAME

Data Dissemination Data are stored on various platforms in different applications Survey micro data are stored on mainframe and replicated on SQL server and Unix/Linux Users write programs against databases Some in-house end-user applications Macro data stored in FAME users write programs against FAME database Vendor purchased data are accessed through the vendor software

Data Dissemination – Data and News Catalogue Division: Monetary Affairs Database: TINY Vendor: Federal Reserve Board Data Contact: John Doe License Contact: Jane Doe License Information: Board Unix user Form of Access: Unix Network: Description: Merger adjusted Call Report data. Data are from the quarterly Reports of Condition and Income (Call Reports) for insured domestic commercial banks and nondeposit trust companies. The data consolidate information from foreign and domestic offices and have been adjusted to take account of mergers. Status: Active Contract Keyword(s): Accounting, Bank Mergers,Bank Profits, Banking Structure, Banks and Banking, Call Report, Deposits, Financial Institutions, Financial Statements, Interest Rates TINYFederal Reserve Board Link to database or documentation Link to vendor website

Data Dissemination Important factors to make data dissemination easier Uniform storage facilities High quality meta data Using transmission protocols and standards XBRL (financial micro data) SDMX (economic macro data)

In Summary As the volume and diversity of data increase the need for organized, flexible, and well documented data archives becomes more critical.