Historical Counties: Visualizing Change with Tracking Analyst MGIS Capstone Kevin Horne Advisor – Mark Lindberg University of Minnesota May 13, 2008

research overview research overview research overview research overview project phases project phases project phases project phases tracking analyst tracking analyst tracking analyst tracking analyst animation series animation series animation series animation series data preparation data preparation data preparation data preparation recommendations recommendations recommendations questions questions questions agenda Historical Counties: Visualizing Change with Tracking AnalystHorne 2008

research overview models key dates key dates snapshot NHGIS shapefiles temporal feature classes space-time composite space-time composite lowest common geography atomic units reconfigure based on atomic history NHGIS development Historical Counties: Visualizing Change with Tracking AnalystHorne 2008 China Historical GIS (Berman 2003)

research overview visualization techniques techniques “frozen wards”“frozen wards” cartogramscartograms census/election data census/election data temporal symbologytemporal symbologytemporal symbologytemporal symbology time seriestime series animationanimation applications applications VoxelViewerVoxelViewer MapTimeMapTime Tracking AnalystTracking Analyst Historical Counties: Visualizing Change with Tracking AnalystHorne 2008 VoxelViewer (Harrower 2002) 2004 U.S. Presidential Election Results by population (Gastner, Shalizi, & Newman, M. E. J., 2004)

project phases Historical Counties: Visualizing Change with Tracking AnalystHorne 2008 script animation China Historical GIS Data Model (Berman 2002) model

project phases Historical Counties: Visualizing Change with Tracking AnalystHorne 2008 animation

tracking analyst Historical Counties: Visualizing Change with Tracking AnalystHorne 2008 time window properties date field; tracking field symbology properties playback manager

interactive visualization interactive visualization default properties for animations default properties for animations available in ArcMap ‘data view’ available in ArcMap ‘data view’ Historical Counties: Visualizing Change with Tracking AnalystHorne 2008

current date always changing always changing can be directly manipulated can be directly manipulated Historical Counties: Visualizing Change with Tracking AnalystHorne 2008

start date & end date derived from data derived from data can be adjusted can be adjusted Historical Counties: Visualizing Change with Tracking AnalystHorne 2008

playback rate establish rate of change for current date establish rate of change for current date Jan 1, 1790 Jan 1, 1800 Jan 1, 1810… Jan 1, 2000 Historical Counties: Visualizing Change with Tracking AnalystHorne 2008

playback manager 10 years yields 3600 days (corrected in version 9.2) 10 years yields 3600 days (corrected in version 9.2) use days  current dates Jan 1, Jan 2, or Jan 3 extra empty frame at end of animation extra empty frame at end of animation reduce end date by one time interval (10 years) speed increases/reduces frame count speed increases/reduces frame count should see all frames – just faster or slower Historical Counties: Visualizing Change with Tracking AnalystHorne 2008

date field / tracking field initialize at load time initialize at load time date field date field requiredrequired January 1, YYYYJanuary 1, YYYY tracking field tracking field optionaloptional required for labelingrequired for labeling ability to modify ability to modify new in ArcGIS 9.2new in ArcGIS 9.2 Historical Counties: Visualizing Change with Tracking AnalystHorne 2008

time window temporal query temporal query relative to Playback Manager’s current daterelative to Playback Manager’s current date not required not required grab most current event (new in 9.2)grab most current event (new in 9.2) typical time window typical time window 4 days (ArcGIS 9.0)4 days (ArcGIS 9.0) day intervals day intervals 1 day (ArcGIS 9.2)1 day (ArcGIS 9.2) 10-year intervals 10-year intervals Historical Counties: Visualizing Change with Tracking AnalystHorne 2008

symbology settings time window time window ageage select vs. classify select vs. classify advanced options advanced options additional labelingadditional labeling basic symbology basic symbology data exclusion data exclusion Historical Counties: Visualizing Change with Tracking AnalystHorne 2008

animation series Historical Counties: Visualizing Change with Tracking AnalystHorne 2008 “B” series – Expanded Study Area “C” series – Change “A” series – Basic 7-State Study Area “D” series – Frozen Boundaries

animation series ”A” Series – Basic 7-State Study Area clipped study area clipped study area map types map types boundariesboundaries county populationcounty population issues issues clipped feature valuesclipped feature values what’s over the edge?what’s over the edge? Historical Counties: Visualizing Change with Tracking AnalystHorne 2008

animation series ”B” Series – Expanded Study Area view entire features view entire features map types map types boundariesboundaries county populationcounty population county population densitycounty population density issues issues single decadesingle decade classificationclassification Historical Counties: Visualizing Change with Tracking AnalystHorne 2008

animation series ”C” Series – Change temporal comparisons temporal comparisons map types map types boundariesboundaries county populationcounty population county population densitycounty population density issues issues symbologysymbology features across decadesfeatures across decades Historical Counties: Visualizing Change with Tracking AnalystHorne 2008

animation series ”D” Series – Frozen Boundaries “frozen wards” “frozen wards” map types map types populationpopulation population densitypopulation density issues issues interpolationinterpolation identity  sliversidentity  slivers tolerance 1 meter tolerance 1 meter Historical Counties: Visualizing Change with Tracking AnalystHorne 2008

data preparation Historical Counties: Visualizing Change with Tracking AnalystHorne 2008 D2 County Population Density (frozen boundaries) C5 Change in County Population Density (graduated symbols)

data preparation C5 Change in County Population Density (graduated symbols) Historical Counties: Visualizing Change with Tracking AnalystHorne 2008 (6) select state & territory features (12) create county features based on population (9) separate state & territory features (10) select county features (8) create date label features (4) assign temporal fields (5) aggregate features (7) identify max extent (2) download population tables from NHGIS (1) download shapefiles from NHGIS (3) create study area (11) select statistical data (14) calculate population values (13) add tracking and change fields (15) add population density fields (16) calculate population density values layer properties layer properties

Historical Counties: Visualizing Change with Tracking AnalystHorne 2008 Add Field C_YEAR Add Field C_DATE Add Field YR_GISJOIN Calculate Field shapefile year Calculate Field January 1, [C_YEAR] Calculate Field [C_YEAR] & [GISJOIN] (4) assign temporal fields states by year with dates counties by year from NHGIS states by year from NHGIS counties by year with dates (5) aggregate features Merge Python script states all years merged together counties all years merged together C5 Change in County Population Density (graduated symbols)

Historical Counties: Visualizing Change with Tracking AnalystHorne 2008 (6) select state & territory features ArcMap ‘Select by Loc.’ [crossed by outline of] ArcMap ‘Select by Loc.’ [are contained by] Merge ArcToolbox states all years merged together study area states crossed by/ contained by study area max extent (7) identify max extent Dissolve (9) separate state & territory features ArcMap ‘Select by Attr.’ [GISJOIN] ends with ‘5’ selected territories selected states C5 Change in County Population Density (graduated symbols)

Historical Counties: Visualizing Change with Tracking AnalystHorne 2008 max extent (10) select county features selected territories selected states ArcMap ‘Select by Loc.’ [are contained by] counties all years merged together Add Join to territories Add Join to states selected counties [no population] (in territories) selected counties [no population] (in states) merged population tables population tables by year from NHGIS (11) select statistical data Merge via model in Model Builder Add Join to selected counties selected counties updated (in territories) selected counties updated (in states) C5 Change in County Population Density (graduated symbols)

problem when excluding features, valid features are also excluded. work around separate features with population from those without. Historical Counties: Visualizing Change with Tracking AnalystHorne 2008 selected counties updated (in territories) selected counties updated (in states) selected counties with population selected counties without population Select counties with population Select counties with no population (12) create county features based on population Merge state & territory counties with population Select state & territory counties with no population C5 Change in County Population Density (graduated symbols)

Historical Counties: Visualizing Change with Tracking AnalystHorne 2008 selected counties with population (13) add tracking and change fields Add Field PREV_YR_ TRACKID Add Field PREV_POP Add Field CHG_POP Add Field YR_TRACKID Add Field TRACKID Calculate Field (C_YEAR – 10) + TRACKID Calculate Field [C_YEAR] + [TRACKID] Calculate Field values depend upon feature relationships selected counties with population Calculate Field PREV_POP = counties PREV. COUNTY_POP Calculate Field CHG_POP = [COUNTY_POP] – [PREV_POP] Add Join to WITH POP PREV Copy Features create WITH POP PREV (14) calculate population values Remove Join selected counties without population change selected counties with population PREV selected counties with population change C5 Change in County Population Density (graduated symbols) Brookings County

Historical Counties: Visualizing Change with Tracking AnalystHorne 2008 selected counties with population selected counties with pop density change [NEG] selected counties with population PREV selected counties with pop density change [POS] (15) add population density fields Add Field CHG_ SQ_MILES Add Field PREV_ SQ_MILES Add Field CHG_PERSON_ SQ_MILE Add Field PERSONS_ SQ_MILE Add Field SQ_MILES Calculate Field [COUNTY_POP] / [SQ_MILES] Add Field PREV_ PERSONS_ SQ_MILE ArcMap Calculate Geometry Calculate Field CHG_SQ_MILES = SQ_MILES – PREV_SQ_M… Calculate Field PREV_SQ_ MILES = PREV. SQ_MILES Calculate Field PREV_PERSONS _SQ_MILE = PREV.PERS… Add Join to WITH POP PREV (16) calculate population density values Remove Join Calculate Field CHG_PERSONS _SQ_MILE = PERSONS… Select/Export CHG_PERSONS_ SQ_MILE >= 0 Select/Export CHG_PERSONS _SQ_MILE < 0 C5 Change in County Population Density (graduated symbols)

layer properties C5 Change in County Population Density (graduated symbols) Historical Counties: Visualizing Change with Tracking AnalystHorne 2008 data prep date label features (w/date ranges) max extentstudy area selected counties with pop density change (+/-) C_DATE TRACKID 4 days quantities – graduated symbols CHG_PERSONS_SQ_MILE POS layer – Jenks (4) NEG layer – Jenks(3) C_DATE GISJOIN 4 days features – single symbol (for boundaries) state & territory features

data preparation D2 County Population Density (frozen boundaries) Historical Counties: Visualizing Change with Tracking AnalystHorne 2008 (2) create atomic county features (3) interpolate population values (4) dissolve frozen county features (5) calculate population density (1) select frozen county features layer properties layer properties

Historical Counties: Visualizing Change with Tracking AnalystHorne 2008 (1) select frozen county features max extent 2000 county from NHGIS Clip selected frozen counties (2) create atomic county features Identity selected counties with population atomic counties atomic counties with population (3) interpolate population values Add Field ID_SQ_MILES Add Field P_ID_ORIG Add Field ID_POP ArcMap Calculate Geometry Calculate Field [ID_SQ_MILES]/ [SQ_MILES] Calculate Field [P_ID_ORIG] * [COUNTY_POP] D2 County Population Density (frozen boundaries)

Historical Counties: Visualizing Change with Tracking AnalystHorne 2008 atomic counties with population (4) dissolve frozen county features Dissolve SUM fields: ID_POP ID_SQ_MILES frozen counties with population (5) calculate population density Add Field F_SQ_MILES Add Field F_DENSITY ArcMap Calculate Geometry Calculate Field [SUM_ID_POP]/ [F_SQ_MILES] frozen counties with population density D2 County Population Density (frozen boundaries)

layer properties D2 County Population Density (frozen boundaries) Historical Counties: Visualizing Change with Tracking AnalystHorne 2008 data prep state & territory features county features without population date label features max extentstudy area C_DATE GISJOIN 1 day features – single symbol (for boundaries) frozen county features (f0) features – single symbol (for all other boundaries) frozen county features (f4) with pop. density C_DATE FRZJOIN 1 day quantities – graduated colors F_DENSITY Jenks (7)

recommendations Methodology Historical Counties: Visualizing Change with Tracking AnalystHorne 2008 data preparation data preparation download  aggregate  selectdownload  aggregate  select minimize time reformatting base data minimize time reformatting base data maximize time using/evaluating formatted base data maximize time using/evaluating formatted base data process development process development batch processing available in ArcGIS 9.2batch processing available in ArcGIS 9.2 create model and ask…create model and ask… “does it do the job for a single feature class?” “does it do the job for a single feature class?” “does it automatically assign proper names to output?” “does it automatically assign proper names to output?” Yes  DONEYes  DONE No  export to Python; refactor; test in PythonWinNo  export to Python; refactor; test in PythonWin small is goodsmall is good

recommendations ArcGIS – Tracking Analyst Historical Counties: Visualizing Change with Tracking AnalystHorne 2008 feature labeling feature labeling provide date labeling capabilitiesprovide date labeling capabilities created point feature class with dates created point feature class with dates created point feature class with dates created point feature class with dates provide full complement of label properties for temporal layersprovide full complement of label properties for temporal layers data classification data classification correct data exclusion problemscorrect data exclusion problems playback manager playback manager handle speed without adjusting current datehandle speed without adjusting current date correct data selection/data classification discrepancycorrect data selection/data classification discrepancy handle extra empty frame at the end of animationshandle extra empty frame at the end of animations allow users to modify time window for multiple layers at onceallow users to modify time window for multiple layers at once

recommendations National Historical GIS Historical Counties: Visualizing Change with Tracking AnalystHorne 2008 shapefiles shapefiles add tracking date (e.g. 01/01/2000)add tracking date (e.g. 01/01/2000) Tracking Analyst date field Tracking Analyst date field add year to GISJOIN (e.g. ‘G ’)add year to GISJOIN (e.g. ‘G ’) link aggregated temporal features to aggregated attribute data link aggregated temporal features to aggregated attribute data add tracking identifiers (e.g. ‘T ’, ‘T ’)add tracking identifiers (e.g. ‘T ’, ‘T ’) TRACKING_ID, PREV_TRACKING_ID TRACKING_ID, PREV_TRACKING_ID link features across time; link features to temporal neighbors link features across time; link features to temporal neighbors metadata metadata document minimum tolerance required to handle sliver polygonsdocument minimum tolerance required to handle sliver polygons 1 meter – for Identity tool 1 meter – for Identity tool web application web application simplify census data download for base data – Quick Tablessimplify census data download for base data – Quick Tables required 22 individual requests for county aggregate data required 22 individual requests for county aggregate data

questions thank you! thank you! Historical Counties: Visualizing Change with Tracking AnalystHorne 2008

references cited Berman, L. (2002). Simplified spatio-temporal data model for CHGIS. Unpublished manuscript. Berman, L. (2003). Data model for historical GIS: The CHGIS time series. Unpublished manuscript Gastner, M. T., Shalizi, C. R. & Newman, M. E. J. (2004). Maps and cartograms of the 2004 US presidential election results. Retrieved March 13, 2007, 2007, from Harrower, M. (2002). Visualizing change: Using cartographic animation to explore remotely-sensed data. Cartographic Perspectives, 39, Hunt, L. G., & Chambers, C. D. (1976). The heroin epidemics: A study of heroin use in the united states, New York: Spectrum. Kuby, M., Harner, J., & Gober, P. (2004). Tracking the AIDS epidemic in the united states: Diffusion through space and time. Human geography in action (pp ). New York: John Wiley & Sons. Historical Counties: Visualizing Change with Tracking AnalystHorne 2008

thank you Historical Counties: Visualizing Change with Tracking AnalystHorne 2008

Historical Counties: Visualizing Change with Tracking AnalystHorne 2008

Historical Counties: Visualizing Change with Tracking AnalystHorne TRACKING_ID TRACKING_ID

select vs. classify time window classes playback rate feature 20 years 0 to 10 Years 10 to 20 Years 10 years January 1, 1900 selected on… classified as… January 1, to 10 years January 1, to 20 years January 1, 1920 NONE Historical Counties: Visualizing Change with Tracking AnalystHorne 2008

temporal symbology Historical Counties: Visualizing Change with Tracking AnalystHorne 2008 Peak Heroin Use in the United States: 1968 to 1973 (Hunt & Chambers, 1976) AIDS rates v distance from source node (Kuby, Harner, & Gober, 2004) isochrony multivariate