Using an Odometer and a GPS Panel to Evaluate Travel Behaviour Changes Peter Stopher and Natalie Swann Institute of Transport and Logistics Studies The University of Sydney and Camden FitzGerald Parson Brinkerhoff, Sydney April 2007

Using an Odometer and a GPS Panel to Evaluate... 2 Introduction  Policies often aimed at changing travel behaviour  Rarely are the changes monitored to see if the policy was successful  Often the changes expected are within the error levels of self- report surveys

April 2007Using an Odometer and a GPS Panel to Evaluate... 3 Introduction  A policy of major interest in Australia is Voluntary Travel Behaviour Change (VTBC or TravelSmart)  Based on improving household information about alternatives to current travel  Expectation is that it will result in reduced VKT by private car and possibly increased walk, bicycle, transit use, and car occupancy

April 2007Using an Odometer and a GPS Panel to Evaluate... 4 Introduction  TravelSmart is a relatively cheap strategy compared to capital projects  Would be very cost-effective if it reduced car VKT by even 3-4 percent  Standard diary surveys have been shown to be in error by 20 percent or more on average  Self-reporting of distance travelled is potentially even more in error

April 2007Using an Odometer and a GPS Panel to Evaluate... 5 An Innovative Approach  Use of panels to monitor change  Measurement of the panel begins prior to policy introduction  Panel continues until some period of time after introduction  Method one – quarterly odometer survey  Method two – annual or semi- annual GPS survey

April 2007Using an Odometer and a GPS Panel to Evaluate... 6 An Innovative Approach  Panels – recruit households who are asked to repeat the survey at prescribed intervals  Attrition – recruit additional households in each wave  Panel reduces error in measuring change  Panel members report changes in demographics on each wave

April 2007Using an Odometer and a GPS Panel to Evaluate... 7 An Innovative Approach  Odometer survey – request households to provide current odometer readings from each household vehicle and the date on which they take the reading  If a car is bought or sold, then beginning/ending odometer reading is requested and date of acquisition/sale  Readings are requested every quarter

April 2007Using an Odometer and a GPS Panel to Evaluate... 8 An Innovative Approach  GPS survey involves carrying a GPS device for up to 28 days  Everyone over age 14 in household is asked to take a device  Survey is repeated either annually or semi-annually

April 2007Using an Odometer and a GPS Panel to Evaluate... 9 An Innovative Approach  Three elements to the pilot evaluation  Quarterly odometer panel of 200 households in South Australia and 200 in Victoria  Six-monthly 28-day GPS survey of 50 households in South Australia  Focus groups on participants in GPS panels  Six waves of the odometer panel completed  Two waves of GPS panel completed  Focus Groups completed

Odometer Surveys

April 2007Using an Odometer and a GPS Panel to Evaluate Odometer Pilots Six waves completed

April 2007Using an Odometer and a GPS Panel to Evaluate Odometer Panels Response rates were as follows: Returned/Estimated Eligible (%) W1W2W3W4W5W6 SA New Recruit HHs NA Continuing HHs NA VIC New Recruit HHs NA12.7NA Continuing HHs NA Overall, similar patterns of recruitment and response in Melbourne and Adelaide

April 2007Using an Odometer and a GPS Panel to Evaluate Odometer Panels - Response Comparison * These are all TravelSmart households Negligible difference between 3 month and 4 month frequency Returned/Estimated Eligible (%) W1W2W3W4W5W6 SA Main New Recruit HHs *NA Continuing HHs NA NA Add On SA New Recruit HHs NA Continuing HHs NA Add On VIC New Recruit HHs NA12.7NA Continuing HHs NA

April 2007Using an Odometer and a GPS Panel to Evaluate Odometer Panels – Demographic Bias? The demographic profile of the sample from wave to wave is remarkably stable. There may be some coverage error, which can be corrected by applying weighting factors, but there does not appear to be any systematic non-response error.

April 2007Using an Odometer and a GPS Panel to Evaluate Odometer Panels – Elapsed Time – SA Attribute Waves 2-1 Waves 3-2 Waves 4-3 Waves 5-4 Waves 6-5 Waves 6-1 Elapsed Days Standard Deviation in Days No. of Valid Observations No. Missing/Refused 66 (20%) 92 (23%) 104 (24%) 114 (26%) 116 (27%) 113 (35%) Total No. of Vehicles

April 2007Using an Odometer and a GPS Panel to Evaluate Odometer Panels – Elapsed Time – Victoria AttributeWaves 2-1 Waves 3-2 Waves 4-3 Waves 5-4 Waves 6-5 Waves 6-1 Elapsed Days Standard Deviation in Days No. of Valid Observations No. Missing/Refused 97 (26%) 143 (35%) 129 (32%) 138 (33%) 140 (33%) 126 (35%) Total No. of Vehicles

April 2007Using an Odometer and a GPS Panel to Evaluate Odometer Panels – Average VKT – SA Attribute Waves 2- 1 Waves 3-2 Waves 4-3 Waves 5-4 Waves 6-5 Waves 6-1 Average Daily VKT Standard Deviation No. of Valid Observations No. Missing/Refused 66 (20%) 92 (23%) 104 (24%) 114 (26%) 116 (27%) 113 (35%) Total No. of Vehicles

April 2007Using an Odometer and a GPS Panel to Evaluate Odometer Panels – Average VKT –Victoria Attribute Waves 2- 1 Waves 3-2 Waves 4-3 Waves 5-4 Waves 6-5 Waves 6-1 Average Daily VKT Standard Deviation No. of Valid Observations No. Missing/Refused 97 (26%) 143 (35%) 129 (32%) 138 (33%) 140 (33%) 126 (35%) Total No. of Vehicles Strong evidence of success in collecting Odometer readings – not Trip meter readings

April 2007Using an Odometer and a GPS Panel to Evaluate Odometer Survey - Conclusions  Method works in both a small and large urban area  Response rates are higher than normal for a postal survey  Good evidence for successful reporting of Odometer readings and not trip-meter readings  The sample does not become more biased over time, although rotation of the panel may still be preferable.

GPS Surveys

April 2007Using an Odometer and a GPS Panel to Evaluate GPS Panel  Set up as 50 households in SA  Two waves conducted with 28-day task, six months apart  Third wave set up with 15-day task, six months later

April 2007Using an Odometer and a GPS Panel to Evaluate GPS Panels Panel WaveSA Panel (50) Wave 1Wave 2“Wave 3” Recruited (New to the wave)57170 Completed50140 Continuing (Recruited)3544 Continuing (Completed) Total Complete Households Households Failing to comply4 (8%)4 (9%)1 (3%)

April 2007Using an Odometer and a GPS Panel to Evaluate GPS Panels – Days of Data by Person Number of DaysWave 1 Pilot Panel (50) Wave 2 Pilot Panel (46) All Days1 (1%)39 (44%) Up to 6 days per week13 (12%)21 (24%) 5 to less than 6 days10 (10%)7 (8%) 3 to less than 5 days27 (25%)8 (9%) 1 to less than 3 days32 (30%)9 (10%) More than 0 and less than 124 (22%)5 (6%) Total10789 (100%)

April 2007Using an Odometer and a GPS Panel to Evaluate GPS Survey Analysis  Analysis has been done at several levels  Disaggregate – person by day  Aggregated to days of the week  Aggregated to persons  Aggregated further to households  In wave 2, some households overlapped the Easter Holidays  Person days that appeared to have extensive travel on one or two days in the period were dropped from disaggregate analysis

April 2007Using an Odometer and a GPS Panel to Evaluate GPS Panel – Trip Data Per Person Day Statistic (sd in brackets) Wave 1 PanelWave 2 Panel* All Days Week- days Week- ends All Days Week- days Week- ends Number of Valid Obs Mean Number of Trips3.85 (3.44) 3.64 (3.30) 4.35 (3.71) 3.80 (3.74) 3.68 (3.71) 4.07 (3.80) Mean PKT25.6 (33.0) 24.1 (31.79) 29.2 (35.5) 25.8 (36.36) 25.0 (34.37) 27.6 (40.81) Mean Travel Time55.3 (51.6) 53.0 (50.36) 60.8 (54.10) 50.6 (52.24) 49.8 (51.89) 52.7 (53.08) * 4 households were found to have undertaken excessive (probably holiday) travel on two or more days around the Easter period in 2006 and were excluded from these statistics

April 2007Using an Odometer and a GPS Panel to Evaluate GPS Panel – Trip Data Per Person by Day of Week Statistic (sd in brackets) Wave 1 PanelWave 2 Panel All Days Week- days Week- ends All Days Week- days Week- ends Number of Valid Obs Mean Number of Trips3.96 (2.37) 3.77 (2.22) 4.44 (2.65) 3.99 (2.61) 3.90 (2.59) 4.21 (2.62) Mean PKT25.0 (22.73) 23.3 (22.49) 29.2 (22.86) 29.3 (37.96) 28.7 (40.15) 30.8 (31.86) Mean Travel Time54.8 (38.14) 52.2 (37.26) 61.5 (39.59) 51.8 (41.85) 51.0 (42.30) 54.0 (40.73) Note that even the large values are included in Wave 2.

April 2007Using an Odometer and a GPS Panel to Evaluate GPS Panel – Trip Data Per Person by Day of Week Note that even the large values are included in Wave 2. Statistic (sd in brackets) Wave 1 PanelWave 2 PanelWave 1-Wave 2 Covariance All Days Number of Valid Obs Mean Number of Trips 3.91 (1.77) 3.96 (1.58) 1.66 Mean PKT 24.7 (15.80) 25.0 (14.76) 69.6 Mean Travel Time 48.8 (25.55) 54.7 (24.45) 246.4

April 2007Using an Odometer and a GPS Panel to Evaluate GPS Panels – Trip Data Per Household Statistic (sd in brackets) Wave 1 Pilot Panel Wave 2 Pilot Panel Wave 1-Wave 2 Covariance All Days Number of Valid Obs Mean Number of Trips per person 4.06 (1.50) 3.94 (1.64) 2.11 Mean PKT24.8 (14.03) 24.3 (12.52) 92.1 Mean Travel Time54.9 (22.94) 47.8 (20.70) 273.3

April 2007Using an Odometer and a GPS Panel to Evaluate GPS Duration  Panel Survey suggests that:  Recruitment rates are not affected much between 7 and 28 days  Also, early drop out is not much affected by the length of GPS recording  Statistics tend to stabilise after about days  We have determined that there is no halo effect in the first few days  Ideal appears to be a 15-day GPS

April 2007Using an Odometer and a GPS Panel to Evaluate GPS Frequency  Attrition for three waves of the six-monthly GPS was not markedly different than for two waves of the annual GPS survey  Annual participants have received interim mailings  Non-compliance rate is about double in the annual survey (8% versus under 4%)

Focus Groups

April 2007Using an Odometer and a GPS Panel to Evaluate Focus Groups - Themes Six themes were derived from the group discussions: 1.Respondents’ understanding of the survey task; 2.The form and functions of the devices; 3.Patterns of respondent behaviour in undertaking the task; 4.Reactions to the survey documents and survey administration; 5.Respondent attitudes and perceptions of issues relevant to the study; and 6.Curiosities about the study displayed by respondents.

April 2007Using an Odometer and a GPS Panel to Evaluate Focus Groups - Outcomes  Newsletter - to encourage and educate respondents  Sourcing new devices – to improve performance  Refining survey documentation – to further clarify the GPS task

April 2007Using an Odometer and a GPS Panel to Evaluate Conclusions  Both panel surveys are feasible methods  Odometer survey can show small differences in average VKT with suitable sample sizes  GPS survey can provide a much greater wealth of data on effects of the policy  Other work shows much smaller sample sizes are required for GPS versus Odometer