INFO 4470/ILRLE 4470 Social and Economic Data Populations and Frames John M. Abowd and Lars Vilhuber February 7, 2011

Outline Basic definitions Demographic sampling frames Coverage and duplication Sampling frame maintenance 2/7/2011(c) 2011 John M. Abowd and Lars Vilhuber, All Rights Reserved 2

Basic Definitions Target population Out-of-scope population Frame population Geography Population demographics Business entity demographics Government entities 2/7/2011(c) 2011 John M. Abowd and Lars Vilhuber, All Rights Reserved 3

Target Population The “target population” is any entity satisfying the set of conditions that specify the universe. “Universe” and “Target Population” are synonyms. Example 1: “Human population” All people, male and female, child and adult, living in a given geographic area at a particular date. Example 2: “Flu query population” Historical logs of online web search queries submitted between fixed dates containing only the specified key words and originating only from the specified URLs. 2/7/2011(c) 2011 John M. Abowd and Lars Vilhuber, All Rights Reserved 4

Out-of-scope Population An entity that is outside either the geographic region under study or fails to meet another specific restriction imposed on the target population. Example 1: When the in-scope population is “persons age 16 or over living in households,” persons age 15 or younger and all persons living in group quarters are out of scope. Example 2: When the in-scope population is “flu queries,” all queries that originate from URLs not on the list are out-of-scope. 2/7/2011(c) 2011 John M. Abowd and Lars Vilhuber, All Rights Reserved 5

Frame Population Set of target population, or universe, entities that can be selected into a sample or census. Also called a sampling frame. The frame population or sampling frame is the physical manifestation of the universe—if an entity is not on the frame (or one of the frames for multi-frame sampling), then it cannot be in the census or survey. Simple cases: (single frame sampling) a list of all addresses to be sampled; list of all people to be sampled; list of all businesses to be sampled. Complex sample designs use multiple frame populations to get better coverage of the target population or universe. Complex frame example: ACS, CPS or SIPP Complex query frame example: Google Flu Trents 2/7/2011(c) 2011 John M. Abowd and Lars Vilhuber, All Rights Reserved 6

Geography The geography of a sampling frame assigns to every latitude and longitude a fundamental geographic area Geographic entities can be assembled uniquely by aggregating geographic areas The basic geographic entity for the U.S. Census is the “block” 2/7/2011(c) 2011 John M. Abowd and Lars Vilhuber, All Rights Reserved 7

U.S. Census Geography 2/7/20118

Query-based Frames Begin with a search log Filter queries by origin URL Filter queries by search terms Rank search term relevance using external indicators 2/7/2011(c) 2011 John M. Abowd and Lars Vilhuber, All Rights Reserved 9

Entities In sampling frame development every geographic location (latitude and longitude) that contains a structure (natural or man-made) capable of originating economic activity is classified as a domicile, business, or both Entities are placed in the frame by declaring the target population to be humans beings (all domiciles including group quarters), queries (originating from URL list, containing search terms) 2/7/2011(c) 2011 John M. Abowd and Lars Vilhuber, All Rights Reserved 10

Population Demographics Human populations are usually categorized by current living quarters when designing demographic sampling frames Distinguish between household living quarters and group living quarters 2/7/2011(c) 2011 John M. Abowd and Lars Vilhuber, All Rights Reserved 11

Household Living Quarter Definitions Housing unit: A house, an apartment, a mobile home or trailer, a group of rooms, or a single room occupied as separate living quarters, or if vacant, intended for occupancy as separate living quarters. Separate living quarters are those in which the occupants live separately from any other individuals in the building and which have direct access from outside the building or through a common hall. For vacant units, the criteria of separateness and direct access are applied to the intended occupants whenever possible. Household: A household includes all the people who occupy a housing unit as their usual place of residence. 2/7/2011(c) 2011 John M. Abowd and Lars Vilhuber, All Rights Reserved 12

Group Living Quarters Group quarters: all people not living in households. There are two types of group quarters: institutional (for example, correctional facilities, nursing homes, and mental hospitals) and non-institutional (for example, college dormitories, military barracks, group homes, missions, and shelters). 2/7/2011(c) 2011 John M. Abowd and Lars Vilhuber, All Rights Reserved 13

Group Quarters Population Includes all people not living in households. Includes those people residing in group quarters as of the date on which a particular survey was conducted. Two general categories the institutionalized population which includes people under formally authorized supervised care or custody in institutions at the time of enumeration (such as correctional institutions, nursing homes, and juvenile institutions) the non-institutionalized population which includes all people who live in group quarters other than institutions (such as college dormitories, military quarters, and group homes). The non-institutionalized population includes all people who live in group quarters other than institutions. 2/7/2011(c) 2011 John M. Abowd and Lars Vilhuber, All Rights Reserved 14

Demographic Sampling Frames Comprehensive mapping of location of the in- scope population with standardized geography Comprehensive list of domiciles in the geographic area covered Characteristics of inhabitants of the domiciles (stratifying variables) 2/7/2011(c) 2011 John M. Abowd and Lars Vilhuber, All Rights Reserved 15

Coverage Domicile address lists are collected from multiple sources for households and group quarters The U.S. Census Bureau collects domicile addresses into the MAF (Master Address File) The U.S. Census Bureau collects physical locations into a set of files known as the TIGER (Topologically Integrated Geographic Encoding and Referencing system) 2/7/2011(c) 2011 John M. Abowd and Lars Vilhuber, All Rights Reserved 16

Refreshing and Unduplication Refreshing a demographic sampling frame consists of collecting information on new housing units (households or group quarters) and purging information on housing units that have been taken out of service Unduplication is the process of expending resources to reduce the probability that an entity is present more than once in a sampling frame 2/7/2011(c) 2011 John M. Abowd and Lars Vilhuber, All Rights Reserved 17

Duplication and Unduplication When a demographic sampling frame is refreshed, addresses are added from multiple sources (tax records, new construction permits, etc.) The same address may appear more than once Some locations may appear to have no domiciles located on them but actually contain households or group quarters 2/7/2011(c) 2011 John M. Abowd and Lars Vilhuber, All Rights Reserved 18

What is Sampling Frame Maintenance? The sampling frame or frame population defines the actual entities from the target population with a positive probability of inclusion in the survey or census Demographic and economic activity change the frame dynamically The biggest investment that statistical agencies and private firms make is the creation and maintenance of frame populations 2/7/2011(c) 2011 John M. Abowd and Lars Vilhuber, All Rights Reserved 19

Housing Frame Census MAF Housing address list updates How do you combine the information? 2/7/2011(c) 2011 John M. Abowd and Lars Vilhuber, All Rights Reserved 20

Census Master Address File (Census 2000) Started with 1990 Census Address Control File (ACF) Add US Postal Service Delivery Sequence File (DSF) Unduplication (one record per address) No deletions yet 100% block canvas (independent of LUCA) Physical visit of address; verification of use Address confirmed or deleted Missing addresses added if residential This procedure slightly modified for the 2010 Census 2/7/2011(c) 2011 John M. Abowd and Lars Vilhuber, All Rights Reserved 21

Census MAF (2) Local Update of Census Addresses (LUCA; independent of 100% canvas) Voluntary program by county and local governments Adds and deletes by address Special statutory exception to Title 13 for address improvement States are also authorized in 2010 Census to cooperate 2/7/2011(c) 2011 John M. Abowd and Lars Vilhuber, All Rights Reserved 22

Census MAF (3) New construction To include new construction since 100% canvas and LUCA Updated DSF Lists of new housing construction from local governmental units Arbitration of units found by LUCA not in 100% canvas (field verification) Unduplication of new construction list Field staff visits to determine Census status (summer 2009) 2/7/2011(c) 2011 John M. Abowd and Lars Vilhuber, All Rights Reserved 23