Archived Data Management System Study Advisory Committee Meeting May 14, 2003

Preliminary Data Analysis 2002 AADT Estimation TRIMARC ARTIMIS

Procedure 1.Data extraction and screening Applied quality check criteria used in mobility monitoring study by TTI Problem encountered: The rounding of occupancy in the segment file created problems. Potential solution: Eliminating the vehicle length criteria

Procedure (cont’d) 2.Summarize 15min volume to hourly volume  If there are at least 2 15min records, calculate hourly volume by adding or extrapolating;  Otherwise, mark hourly volume as missing/null.

Procedure (cont’d) 3.Summarize hourly volume to daily volume  If there are 24 hourly volume records for a day, add them.  If there are hourly volume records for a day, impute the hourly volumes for those “missing” hours based on hourly distribution at the same site throughout the year, and then add them.  Otherwise, mark the day as missing/null.

Procedure (cont’d) 4.For each site, screen out daily volumes on those days of week based on ±1.5σ from the day-of- week mean. 5.Calculate day-of-week volume distribution using yearly average for each day of week.

Procedure (cont’d) 6.Calculate AADT using the AASHTO formulation (I) where:  VOL = daily traffic for day k, of day-of-week i, and month j  i = day of the week  j = month of the year  k = 1 when the day is the first occurrence of that day of the week in a month, 4 when it is the fourth day of the week.  n = the number of days of that day of the week during that month (usually between 1 and 5, depending on the number of missing data).

Quality Control Criteria Quality Control Test and DescriptionSample Code with Threshold ValuesAction Controller error codes  Special numeric codes that indicate that controller or system software has detected an error or a function has been disabled. If VOLUME={code} or OCC={code} or SPEED={code} where {code} typically equals “-1” or “255”  Set values with error codes to missing/null, assign missing value flag/code. No vehicles present  Speed values of zero when no vehicles present  Indicates that no vehicles passed the detection zone during the detection time period. If SPEED=0 and VOLUME=0 (and OCC=0)  Set SPEED to missing/null, assign missing value code  No vehicles passed the detection zone during the time period. Consistency of elapsed time between records  Polling period length may drift or controllers may accumulate data if polling cycle is missed.  Data collection server may not have stable or fixed communication time with field controllers. Elapsed time between consecutive records exceeds a predefined limit or is not consistent  Action varies. If polling period length is inconsistent, volume-based QC rules should use a volume flow rate, not absolute counts. Duplicate records  Caused by errors in data archiving logic or software process. Detector and date/time stamp are identical.  Remove/delete duplicate records. QC1-QC3: Logical consistency tests  Typically used for date, time and location.  Caused by various types of failures. If DATE={valid date value} (QC1) If TIME={valid time value} (QC2) If DET_ID={valid detector location value} (QC3)  Write to off-line database and/or remove records with invalid date, time or location values. QC4: Maximum volume  Traffic flow theory suggests a maximum traffic capacity. If VOLUME > 17 (20 sec.) If VOLUME > 25 (30 sec.) If VOLUME > 250 (5 min.) If VPHPL > 3000 (any time period length)  Assign QC flag to VOLUME, write failed record to off- line database, set VOLUME to missing/null. QC5: Consecutive identical volume values  Research and statistical probability indicates that consecutive runs of identical data values are suspect.  Typically caused by hardware failures. No more than 8 consecutive identical volume values  Assign QC flag to VOLUME, OCCUPANCY and SPEED; write failed record to off-line database; set VOLUME, OCCUPANCY and SPEED to missing/null

Quality Control Criteria (cont’d) Quality Control Test and DescriptionSample Code with Threshold ValuesAction QC6: Maximum occupancy  Empirical evidence suggests that all data values at high occupancy levels are suspect.  Caused by detectors that may be “stuck on.” If OCC > 95% (20 to 30 sec.) If OCC > 80% (1 to 5 min.)  Assign QC flag to VOLUME, OCCUPANCY and SPEED; write failed record to off-line database; set VOLUME, OCCUPANCY and SPEED to missing/null QC7: Minimum speed  Empirical evidence suggests that actual speed values at low speed levels are inaccurate. If SPEED < 5 mph  Assign QC flag to SPEED, write failed record to off- line database, set SPEED value to missing/null QC8: Maximum speed  Empirical evidence suggests that actual speed values at high speed levels are suspect. If SPEED > 100 mph (20 to 30 sec.) If SPEED > 80 mph (1 to 5 min.)  Assign QC flag to SPEED, write failed record to off- line database, set SPEED value to missing/null Maximum reduction in speed  Empirical evidence suggests that speed reductions greater than some maximum value are suspect. If SPEED n+1 < (0.45  SPEED n )  Assign QC flag to SPEED, write failed record to off- line database, set SPEED value to missing/null QC9: Multi-variate consistency  Zero speed values when volume (and occupancy) are non-zero  Speed trap not functioning properly If SPEED = 0 and VOLUME > 0 (and OCC > 0)  Assign QC flag to SPEED, write failed record to off- line database, set SPEED value to missing/null QC10: Multi-variate consistency  Zero volume values when speed is non-zero.  Unknown cause. If VOLUME = 0 and SPEED > 0  Assign QC flag to VOLUME, write failed record to off- line database, set VOLUME to missing/null

Quality Control Criteria (cont’d) Quality Control Test and DescriptionSample Code with Threshold ValuesAction QC11: Multi-variate consistency  Zero speed and volume values when occupancy is non-zero.  Unknown cause. If SPEED = 0 and VOLUME = 0 and OCC > 0  Assign QC flag to VOLUME, OCCUPANCY and SPEED; write failed record to off-line database; set VOLUME, OCCUPANCY and SPEED to missing/null QC12: Truncated occupancy values of zero  Caused when software truncates or rounds to integer value  Calculate maximum possible volume (MAXVOL) for an occupancy value of “1”: If OCC = 0 and VOLUME > MAXVOL where MAXVOL=(2.932*ELAPTIME*SPEED)/600  Assign QC flag to VOLUME, OCCUPANCY and SPEED; write failed record to off-line database; set VOLUME, OCCUPANCY and SPEED to missing/null QC13: Minimum average effective vehicle length (AEVL)  This test computes an avg. vehicle length using volume, occupancy and speed. The test then assumes a minimum avg. length. If AEVL < 9 ft. where AEVL=(SPEED  OCC  52.8  ELAPTIME) /(VOLUME*3600)  Assign QC flag to VOLUME, OCCUPANCY and SPEED; write failed record to off-line database; set VOLUME, OCCUPANCY and SPEED to missing/null QC14: Maximum average effective vehicle length (AEVL)  This test computes an avg. vehicle length using volume, occupancy and speed. The test then assumes a minimum avg. length. If AEVL > 60 ft. where: AEVL=(SPEED  OCC  52.8  ELAPTIME) /(VOLUME*3600)  Assign QC flag to VOLUME, OCCUPANCY and SPEED; write failed record to off-line database; set VOLUME, OCCUPANCY and SPEED to missing/null

Data Screening Summary 2002 Volume Data AnalysisTRIMARCARTIMIS Total number of segments within KY5382 Total number of EXPECTED 15min data records1,857,1202,873,280 Total # of 15min records1,727, Completeness percentage93.0%19.0% Quality CheckController error codes00 No vehicle present19,26569,221 Consistency of elapsed time between records 0 0 Duplicate record5,79275 Invalid date, time1,2420 Maximum volume (750) Consecutive identical volume ,472 Maximum occupancy (80%)38520 Multi-variate consistency (Vol = 0,Speed > 0)1,5510 Multi-variate consistency (Speed = 0,Vol=0,OCC<>0)00 Truncated occupancy values of zero113 Minimum average effective length (9ft)156,164N/A Maximum average effective length (60ft)33,711N/A Total flagged ,522 Total # of valid records AFTER QUALITY CHECK1,503,070361,323 % passing quality check (compared to raw data)87.0%66.3%

Data Screening Summary (cont’d) 2002 Volume Data AnalysisTRIMARCARTIMIS 15min records used for hourly volume calculation1,492,768361,323 Total number of hourly records factored from 15min data391,44282,917 Average # of hours per segment7,3861,011 % of 15min records used to calculate hourly volume86.4%66.3% 15min records used for daily volume calculation1,392,101352,682 % of 15min records used to calculate daily volume80.6%64.7% Hourly records used for daily volume calculation359,79380,715 Total number of daily records grouped from hourly data15,4533,361 Average # of days per segment29241

TRIMARC Data Quality

ARTIMIS Data Quality

Alternative Methods  Calculate Monthly ADT (MADT) and multiply it with the monthly factor to get AADT (II)  Estimate MADT using two-week’s of “good” data (with minimum 15min records marked as missing/null) and multiply it with the monthly factor to get AADT (III)

2003 Monthly Factor (Division of Planning, KYTC) JANFEBMARAPRMAYJUNJULAUGSEPOCTNOVDEC 1 - Rural InterstateAll Week Weekend Weekday Rural GeneralAll Week Weekend Weekday Urban GeneralAll Week Weekend Weekday Rural RecreationAll Week Weekend Weekday Urban InterstateAll Week Weekend Weekday

TRIMARC Summary (Method I)

TRIMARC Summary (Method II)

TRIMARC Summary (Method II) (cont’d)

ARTIMIS Summary (Method I)

ARTIMIS Summary (Method II)

ARTIMIS Summary (Method II) (cont’d)

AADT Estimates at Sample Sites TRIMARCARTIMIS SegmentSKYI71001SSEGK DeviceWBR039LFC076(N) MPI I-71/ State AADT (one direction) % of State #86500% of State # AADT Estimates I Detector data only % % II MADT + MF31850 Sep %75835 Jan % III Two week + MF (all week) Sep %76952 Jan % Two week + MF (weekday) Sep %80453 Jan % Two week + MF (weekend) Sep %64208 Jan %

Observations  Data availability varied by sites in  The rounding of occupancy data may have caused large amount of data being screened out by vehicle length criteria, which were hence dropped when processing ARTIMIS data.

Observations (cont’d)  Generally, the AADT estimates obtained from method II are closer to the State figure when more valid days are present for a month.  Significant differences exist between AADT estimates obtained using monthly factors for all week, weekday, and weekend (method III).