Archived ITS Data A New Resource for Operations, Planning and Research Robert L. Bertini Portland State University Robert L. Bertini Portland State University

Archived ITS Data 2 IntroductionIntroduction “Data are too valuable to use only once.”

Archived ITS Data 3 ADUS Is Born ITS technologies collect dataITS technologies collect data –Real time control »Incident management »Traffic signal systems »Traveler information –Also useful if saved and accessible –Data already being collected—incentive for storing them for future use. Difficulty not in collecting data but in gaining access to that dataDifficulty not in collecting data but in gaining access to that data US DOT Archived Data User Service (ADUS)US DOT Archived Data User Service (ADUS) –Managing ITS data beyond ITS –Careful management of data for various stakeholders ITS technologies collect dataITS technologies collect data –Real time control »Incident management »Traffic signal systems »Traveler information –Also useful if saved and accessible –Data already being collected—incentive for storing them for future use. Difficulty not in collecting data but in gaining access to that dataDifficulty not in collecting data but in gaining access to that data US DOT Archived Data User Service (ADUS)US DOT Archived Data User Service (ADUS) –Managing ITS data beyond ITS –Careful management of data for various stakeholders

Archived ITS Data 4 Who Can Use ITS Data? Fourteen Stakeholders IdentifiedFourteen Stakeholders Identified –Transportation planning –Transportation system monitoring –Air quality analysis –MPO/state freight and intermodal planning –Land use/growth management planning –Transportation administrators and policy analysis –Traffic management –Transit management –Construction and maintenance –Safety planning and administration –CVO –Emergency management –Transportation research –Private sector Fourteen Stakeholders IdentifiedFourteen Stakeholders Identified –Transportation planning –Transportation system monitoring –Air quality analysis –MPO/state freight and intermodal planning –Land use/growth management planning –Transportation administrators and policy analysis –Traffic management –Transit management –Construction and maintenance –Safety planning and administration –CVO –Emergency management –Transportation research –Private sector

Archived ITS Data 5 Data Poor to Data Rich Traffic surveillance Fare/toll systems Incident management Traffic video EnvironmentalCVO Traffic control Highway/rail Emergency response Traffic surveillance Fare/toll systems Incident management Traffic video EnvironmentalCVO Traffic control Highway/rail Emergency response ITSDataArchivesITSDataArchives Performance Monitoring –National reporting –Performance-based planning –Evaluations –Public Reactions Long Range Planning –TRANSIMS –IDAS –Four step models –Transit routes Operations Planning –Incident management –ER deployment –Signal timing –Transit service Travel Time Forecasting –Customized route planning –ATIS Advisories Other Stakeholder Functions –Safety –Land use –Air quality –Maintenance management Performance Monitoring –National reporting –Performance-based planning –Evaluations –Public Reactions Long Range Planning –TRANSIMS –IDAS –Four step models –Transit routes Operations Planning –Incident management –ER deployment –Signal timing –Transit service Travel Time Forecasting –Customized route planning –ATIS Advisories Other Stakeholder Functions –Safety –Land use –Air quality –Maintenance management

Archived ITS Data 6 ADUSADUS Development and evaluation of operations strategiesDevelopment and evaluation of operations strategies –Detailed data from ADUS Performance monitoringPerformance monitoring –Continuous and direct measurements of actual conditions Advanced operation productsAdvanced operation products –Sophistication leads to more data requirements –Short term traffic prediction –Customized route planning Next generation of planning and operations modelsNext generation of planning and operations models –Require more detailed information Development and evaluation of operations strategiesDevelopment and evaluation of operations strategies –Detailed data from ADUS Performance monitoringPerformance monitoring –Continuous and direct measurements of actual conditions Advanced operation productsAdvanced operation products –Sophistication leads to more data requirements –Short term traffic prediction –Customized route planning Next generation of planning and operations modelsNext generation of planning and operations models –Require more detailed information

Archived ITS Data 7 ADUSADUS ITS produce continuous dataITS produce continuous data Continuous data allows measurement of reliabilityContinuous data allows measurement of reliability Reliability is key to management of transportation systemReliability is key to management of transportation system Use of ITS data requires creativityUse of ITS data requires creativity Requires data to be stored and made accessibleRequires data to be stored and made accessible ITS produce continuous dataITS produce continuous data Continuous data allows measurement of reliabilityContinuous data allows measurement of reliability Reliability is key to management of transportation systemReliability is key to management of transportation system Use of ITS data requires creativityUse of ITS data requires creativity Requires data to be stored and made accessibleRequires data to be stored and made accessible

Archived ITS Data 8 ADUSADUS Management of the transportation system cannot be done without knowledge of its performance

Archived ITS Data 9 ADUSADUS Early involvement of stakeholdersEarly involvement of stakeholders Design ADUS as original function of ITS deploymentDesign ADUS as original function of ITS deployment Build ADUS into ITS from the startBuild ADUS into ITS from the start National ITS ArchitectureNational ITS Architecture Few operational examplesFew operational examples Consider the following set of questions….Consider the following set of questions…. Early involvement of stakeholdersEarly involvement of stakeholders Design ADUS as original function of ITS deploymentDesign ADUS as original function of ITS deployment Build ADUS into ITS from the startBuild ADUS into ITS from the start National ITS ArchitectureNational ITS Architecture Few operational examplesFew operational examples Consider the following set of questions….Consider the following set of questions….

Archived ITS Data 10 Archive Creation Question: What data are to be stored?Question: What data are to be stored? –Raw data –Summary statistics –Examples »Volume and lane occupancy, or »Estimated speed Question: What data are to be stored?Question: What data are to be stored? –Raw data –Summary statistics –Examples »Volume and lane occupancy, or »Estimated speed Credit: M. Hallenbeck, Washington DOT

Archived ITS Data 11 Archive Creation How much data gets stored?How much data gets stored? –All raw data –Only summary statistics –Something in between (e.g., aggregated data) –Samples of the data (raw or summary statistics) –All variables, or only some (tag IDs) How much data gets stored?How much data gets stored? –All raw data –Only summary statistics –Something in between (e.g., aggregated data) –Samples of the data (raw or summary statistics) –All variables, or only some (tag IDs)

Archived ITS Data 12 Archive Creation At what level of aggregationAt what level of aggregation –Lowest level collected »Individual vehicle passages (controller) »20 second intervals »5 minute intervals »15 minute intervals »Higher »More than one level At what level of aggregationAt what level of aggregation –Lowest level collected »Individual vehicle passages (controller) »20 second intervals »5 minute intervals »15 minute intervals »Higher »More than one level

Archived ITS Data 13 Archive Creation Issues that impact decision:Issues that impact decision: –What use is planned for the data? –How large is storage requirement? –Cost/speed of processing raw data to more useful form –How much additional data is needed to convert the “raw” data into useful information? –Privacy concerns? Issues that impact decision:Issues that impact decision: –What use is planned for the data? –How large is storage requirement? –Cost/speed of processing raw data to more useful form –How much additional data is needed to convert the “raw” data into useful information? –Privacy concerns?

Archived ITS Data 14 Archive Creation Example: Tag ObservationsExample: Tag Observations Raw data: tag ID, location, time and dateRaw data: tag ID, location, time and date Store all of the above?Store all of the above? Store O/D pairs?Store O/D pairs? Travel times?Travel times? Privacy of tag ID?Privacy of tag ID? Speeds? (distance between readers)Speeds? (distance between readers) Example: Tag ObservationsExample: Tag Observations Raw data: tag ID, location, time and dateRaw data: tag ID, location, time and date Store all of the above?Store all of the above? Store O/D pairs?Store O/D pairs? Travel times?Travel times? Privacy of tag ID?Privacy of tag ID? Speeds? (distance between readers)Speeds? (distance between readers)

Archived ITS Data 15 Archive Creation Example: Fleet AVL InformationExample: Fleet AVL Information –Raw data: Vehicle ID, location, time, and date –ID may not describe route and run »Need schedule information, operations info. »Relationships change every day »Routes can change every schedule change, need historical information Example: Fleet AVL InformationExample: Fleet AVL Information –Raw data: Vehicle ID, location, time, and date –ID may not describe route and run »Need schedule information, operations info. »Relationships change every day »Routes can change every schedule change, need historical information

Archived ITS Data 16 Archive Creation How and why is aggregation performed?How and why is aggregation performed? –Quality control –Assumptions made –Details lost –Costs and benefits uncertain How and why is aggregation performed?How and why is aggregation performed? –Quality control –Assumptions made –Details lost –Costs and benefits uncertain

Archived ITS Data 17 Quality Control Not all collected data is validNot all collected data is valid Can the archive identify bad or questionable data?Can the archive identify bad or questionable data? How are these judgments indicated?How are these judgments indicated? How/are users informed of these conditions?How/are users informed of these conditions? How are “bad” data identified?How are “bad” data identified? –Sensor output –Checks against historical data –Checks against expected ranges –Other comparisons Not all collected data is validNot all collected data is valid Can the archive identify bad or questionable data?Can the archive identify bad or questionable data? How are these judgments indicated?How are these judgments indicated? How/are users informed of these conditions?How/are users informed of these conditions? How are “bad” data identified?How are “bad” data identified? –Sensor output –Checks against historical data –Checks against expected ranges –Other comparisons

Archived ITS Data 18 Quality Control What do you do with “questionable” data?What do you do with “questionable” data? –Construction –Weather –Major incidents What resources are needed to investigate “questionable” data?What resources are needed to investigate “questionable” data? Does this affect willingness to share data?Does this affect willingness to share data? How do you handle missing/bad data?How do you handle missing/bad data? Does this change if you areDoes this change if you are –Storing raw data –Only storing summary data –Storing both What do you do with “questionable” data?What do you do with “questionable” data? –Construction –Weather –Major incidents What resources are needed to investigate “questionable” data?What resources are needed to investigate “questionable” data? Does this affect willingness to share data?Does this affect willingness to share data? How do you handle missing/bad data?How do you handle missing/bad data? Does this change if you areDoes this change if you are –Storing raw data –Only storing summary data –Storing both

Archived ITS Data 19 User Access Who gets access to the data?Who gets access to the data? Classes of users and permission processClasses of users and permission process How do users get access to the data?How do users get access to the data? How do you communicateHow do you communicate –What data (variables) are available –What geographic locations are available –What quality issues exist –How the data can (should) and can not (should not) be used Who gets access to the data?Who gets access to the data? Classes of users and permission processClasses of users and permission process How do users get access to the data?How do users get access to the data? How do you communicateHow do you communicate –What data (variables) are available –What geographic locations are available –What quality issues exist –How the data can (should) and can not (should not) be used

Archived ITS Data 20 User Access Meta DataMeta Data –Data about data (self describing) Truth-in-DataTruth-in-Data –The principal that says you will be honest with users about »What data are real »What data are interpolated »What data are missing and have/have not been replaced, and how those data were replaced Meta DataMeta Data –Data about data (self describing) Truth-in-DataTruth-in-Data –The principal that says you will be honest with users about »What data are real »What data are interpolated »What data are missing and have/have not been replaced, and how those data were replaced

Archived ITS Data 21 User Access Do you trust users to use data correctly?Do you trust users to use data correctly? –At what level of summarization? –Site specific data isn’t always representative of reality How easy do you make their retrieval of data?How easy do you make their retrieval of data? –Cost implications of that task –Political benefits/costs of providing access Do you trust users to use data correctly?Do you trust users to use data correctly? –At what level of summarization? –Site specific data isn’t always representative of reality How easy do you make their retrieval of data?How easy do you make their retrieval of data? –Cost implications of that task –Political benefits/costs of providing access

Archived ITS Data 22 User Access Mechanism used to provide accessMechanism used to provide access –CD-ROM (Arizona) –Web access –File transfer on request –Real time data transfer Cost to user for access?Cost to user for access? Mechanism used to provide accessMechanism used to provide access –CD-ROM (Arizona) –Web access –File transfer on request –Real time data transfer Cost to user for access?Cost to user for access?

Archived ITS Data 23 CommunicationsCommunications How do you communicate with potential users?How do you communicate with potential users? –Staff time –On-line help –None –Other How do you communicate with potential users?How do you communicate with potential users? –Staff time –On-line help –None –Other

Archived ITS Data 24 PrivacyPrivacy Privacy concerns grow with increased user access and sensitivity of data being collectedPrivacy concerns grow with increased user access and sensitivity of data being collected –Personal IDs »Vehicle tags »Driver identification (union issues) Privacy concerns grow with increased user access and sensitivity of data being collectedPrivacy concerns grow with increased user access and sensitivity of data being collected –Personal IDs »Vehicle tags »Driver identification (union issues)

Archived ITS Data 25 Who Pays? ITS systems are paid for by those who operate the systemITS systems are paid for by those who operate the system Often the greatest use for the archive is a different groupOften the greatest use for the archive is a different group –Control of resources –Ownership –Willingness to cooperate ITS systems are paid for by those who operate the systemITS systems are paid for by those who operate the system Often the greatest use for the archive is a different groupOften the greatest use for the archive is a different group –Control of resources –Ownership –Willingness to cooperate

Archived ITS Data 26 Vision for a Portland ADUS

Archived ITS Data 27 Traditional Performance Measures Traditional measuresTraditional measures –Do not describe the complexity of what is happening on the roadway –Are not easily understood by most decision makers and/or the public –Examples: »V/C Ratios: based on limited data, poor mechanism for showing changing conditions during the day »LOS: based on limited data, not meaningful over space, misunderstood »Travel time and delay: based on limited sample, or imperfect calculations Traditional measuresTraditional measures –Do not describe the complexity of what is happening on the roadway –Are not easily understood by most decision makers and/or the public –Examples: »V/C Ratios: based on limited data, poor mechanism for showing changing conditions during the day »LOS: based on limited data, not meaningful over space, misunderstood »Travel time and delay: based on limited sample, or imperfect calculations

Archived ITS Data 28 EB Highway 26

Archived ITS Data 29 Loop Detector Health

Archived ITS Data 30 Average Daily Traffic

Archived ITS Data 31 Average Speed

Archived ITS Data 32 Average Speed + Reliability

Archived ITS Data 33 Percent Lane Miles Congested

Archived ITS Data 34 Demand Vs. Capacity

Archived ITS Data 35 Daily Congestion

Archived ITS Data 36 Frequency of Congestion

Archived ITS Data 37 VHTVHT

Archived ITS Data 38 Travel Time

Archived ITS Data 39 Fusing AVL With Loop Data

Archived ITS Data 40 Fusing AVL With Travel Time

Archived ITS Data 41 Travel Time Reliability

Archived ITS Data 42 Occupancy Contours

Archived ITS Data 43 Contour Plot

Archived ITS Data 44 Contour Plot

Archived ITS Data 45 Performance Measures When truck volume and weight data become available for freeways, these same matrices (and some assumptions) can be used to compute:When truck volume and weight data become available for freeways, these same matrices (and some assumptions) can be used to compute: –Truck hours of delay –Truck miles of delay –Ton-miles of delay –Value of freight delay When truck volume and weight data become available for freeways, these same matrices (and some assumptions) can be used to compute:When truck volume and weight data become available for freeways, these same matrices (and some assumptions) can be used to compute: –Truck hours of delay –Truck miles of delay –Ton-miles of delay –Value of freight delay

Archived ITS Data 46 Performance Measures Each time we use our new tools to answer a question, we develop new ways to display that informationEach time we use our new tools to answer a question, we develop new ways to display that information The goal is to make that informationThe goal is to make that information –Easier to understand –More accurate of “real life” Each time we use our new tools to answer a question, we develop new ways to display that informationEach time we use our new tools to answer a question, we develop new ways to display that information The goal is to make that informationThe goal is to make that information –Easier to understand –More accurate of “real life”

Archived ITS Data 47 Example: FAST FAST system architecture incorporates capability to receive, collect, and archive ITS-generated operational data including: ·incident data ·traffic volumes ·vehicle speeds ·vehicle classification ·travel lane occupancy Data will be stored at periodic intervals, and will be remotely accessible by partner agencies via communication links. Data flows are defined in the FAST regional system architecture, which is consistent with the ITS National Architecture. The ADUS implementation will focus on a centralized concept where relevant data is captured, archived, and provided in a summary format to stakeholders and other FAST ITS subsystems. FAST system architecture incorporates capability to receive, collect, and archive ITS-generated operational data including: ·incident data ·traffic volumes ·vehicle speeds ·vehicle classification ·travel lane occupancy Data will be stored at periodic intervals, and will be remotely accessible by partner agencies via communication links. Data flows are defined in the FAST regional system architecture, which is consistent with the ITS National Architecture. The ADUS implementation will focus on a centralized concept where relevant data is captured, archived, and provided in a summary format to stakeholders and other FAST ITS subsystems. Nevada DOT Archived Data User Service (ADUS)

Archived ITS Data 48 ConclusionsConclusions Archived ITS DataArchived ITS Data Performance Evaluation and Measurement ClearinghousePerformance Evaluation and Measurement Clearinghouse Experiment With Different MeasuresExperiment With Different Measures Freeways as a Starting PointFreeways as a Starting Point ArterialsArterials TransitTransit Integrate Into TMC Decision SupportIntegrate Into TMC Decision Support PeMS successfully implemented at Caltrans Districts 7 & 12PeMS successfully implemented at Caltrans Districts 7 & 12 Archived ITS DataArchived ITS Data Performance Evaluation and Measurement ClearinghousePerformance Evaluation and Measurement Clearinghouse Experiment With Different MeasuresExperiment With Different Measures Freeways as a Starting PointFreeways as a Starting Point ArterialsArterials TransitTransit Integrate Into TMC Decision SupportIntegrate Into TMC Decision Support PeMS successfully implemented at Caltrans Districts 7 & 12PeMS successfully implemented at Caltrans Districts 7 & 12

Archived ITS Data 49 ConclusionConclusion Thank You!