Navigating Your Way Through the EFT, Nesstar and Beyond 20/20 (WDS) Jen Scharf April 29, 2016

DLI’s data products The DLI’s data are kept in the following products: The EFT (Electronic File Transfer) service Nesstar Beyond 20/20, WDS (Web Data Server) Statistics Canada • Statistique Canada 14/04/2018

So what’s what? THE EFT The Electronic File Transfer (EFT) service by Statistics Canada is a corporate solution for exchanging files Requires that each user have their own unique user ID and password. Houses the data files (PUMFs), and aggregate data, for the majority of StatCan surveys that are available to the DLI. The DLI EFT site is the primary method used to disseminate the DLI Collection. Only the DLI contact has access to the EFT. Statistics Canada • Statistique Canada 2016-04-29

So what’s what? NESSTAR Nesstar is a web-based exploration, extraction and analysis tool for social science data. Provides access to the Data Liberation Initiative’s (DLI) collection of public use microdata files (PUMFs). Allows authorized users to access both PUMFs and metadata for master files. Uses the Data Documentation Initiative (DDI) standard which has controlled vocabulary, consistent terminology, is highly accessible, reusable, and is detailed metadata. The data for the Masterfile are available in RDS (research data centre). Statistics Canada • Statistique Canada 14/04/2018

So what’s what? BEYOND 20/20 (WDS) Web-based multi-dimensional table viewer. Dissemination of multiple data products in a variety of formats (PDF, Excel, IVT, Word). User friendly: No knowledge of SPSS, SAS or other complicated Statistical programs required. Contains aggregate data only! Statistics Canada • Statistique Canada 14/04/2018

But what do those terms means? Master file - "Pure" data set created by the author division. All variables and cases are available for analysis in the master file. Metadata - Defines and describes the structure and meaning of information resources, and the context and systems in which they exist. It is used to support efficient and effective management of these information resources over time. PUMF - A master file that has undergone modification to minimize the possibility of disclosing a respondent's identity. Aggregate data – Information derived directly from microdata files or from statistical aggregate files. As opposed to microdata files, aggregate statistics do not record information at the level of individual units of observation. In other words, they are the result of grouping data at an aggregate or macro level (eg., persons in a specific region). Statistics Canada • Statistique Canada 14/04/2018

Where do I find the data I need? Step 1: Determine the type of data you are interested in: Master files  RDC PUMFs  Nesstar/EFT Generic tables/aggregate data  WDS/EFT Step 2: Check the website! Many surveys on the Restricted page are defined as being PUMF (through Nesstar) or Tables (through WDS) Check the Daily, summary tables and CANSIM. Masterfiles are only for advance users. Statistics Canada • Statistique Canada 14/04/2018

The DLI “Restricted Access Web Site” Links directly to the WDS Links directly to Nesstar Provide the link or how to get to this page. Statistics Canada • Statistique Canada 14/04/2018

Let’s break it down! The EFT 166 surveys comprised of data files, PUMFs, tables, etc. Essentially the “warehouse” for the DLI files. All files received from subject matter are placed in the EFT, PUMFs are staged and sent to Nesstar. Excel tables/IVT tables are sent directly (as is) to the WDS. Statistics Canada • Statistique Canada 14/04/2018

Where are the files? “Most” files received from subject matter are placed in the “other” folder Michael is working on a proposal to re-organise the file structures. Statistics Canada • Statistique Canada 14/04/2018

Let’s break it down! Nesstar Contains PUMFs Master files that have been decompressed for confidentiality and coded using the DDI standard. Essentially, takes the original (staged) PUMF provided by subject matter and makes it more user friendly. Contains a listing of master files (metadata) that are available through the RDC’s. Statistics Canada • Statistique Canada 14/04/2018

Where are the files? PUMFs are coded using the DDI standard to produce easy-to-understand results Statistics Canada • Statistique Canada 14/04/2018

Where are the files? Nesstar also contains a listing of all Master Files available through the RDCs Statistics Canada • Statistique Canada 14/04/2018

Let’s break it down! Beyond 20/20 (WDS) Aggregate data only. No PUMFs or metadata. Geography – maps  the WDS contains thousands! Excel tables, IVT tables, Word documents, PDF files. Statistics Canada • Statistique Canada 14/04/2018

Where are the files? Files are organized by category Need a map? The WDS has thousands! Statistics Canada • Statistique Canada 14/04/2018

Where does DLI’s data go? Files received from Subject Matter areas PUMFs Staged through the EFT DDI coded and placed in Nesstar (data and metadata) Excel tables/IVT tables/PDFs/Word documents Uploaded “as-is” to the EFT Renamed and added to the WDS Statistics Canada • Statistique Canada 14/04/2018

Quick reference guide!  Survey Name EFT (all files) Nesstar (PUMFs) WDS (aggregate files) Aboriginal Peoples Survey  Canadian Business Patters Canadian Community Health Survey Census of Population General Social Survey Geography maps Homicide Survey Inter-corporate Ownership Labour Force Survey National Household Survey Survey of Household Spending Statistics Canada • Statistique Canada 14/04/2018

Need more info? The DLI’s online Survival Guide is a great tool! Check out Section 6: “Accessing and Citing DLI Data” for a more information! http://www.statcan.gc.ca/eng/dli/guide/toc/3000279 Statistics Canada • Statistique Canada 14/04/2018

In conclusion Although there is no “rule” as to which surveys can be found where, the easiest thing to remember is to start by determining WHAT KIND of data you are looking for FIRST. You will only find PUMFs and metadata in Nesstar, and aggregate data in the WDS. The EFT houses most files that come from subject matter, typically in “data only” form (SAS files). Statistics Canada • Statistique Canada 14/04/2018

Questions? Statistics Canada • Statistique Canada 2016-04-29