U.S. Department of Transportation Federal Highway Administration MODULE 4 Volume III – Guidelines for Applying Microsimulation Modeling Software

2 U.S. Department of Transportation Federal Highway Administration VOLUME III ●Recommend a project management process applicable to all traffic studies regardless of tool used. ●Provide overview of Project Management Process using a Microsimulation Example ●Discuss the importance of model calibration ●Share examples and experiences

3 U.S. Department of Transportation Federal Highway Administration GOALS OF A GOOD MODELING PROCESS ●Based on a clearly defined purpose and objective ●Develop consensus –Elected officials, transportation officials and citizens ●Identify trade-offs - better decision making ●Measure performance of alternatives –Do they meet general goals of the project. ●Improve design and evaluation time and costs

4 U.S. Department of Transportation Federal Highway Administration PROJECT MANAGEMENT PROCESS ●Scope Project (Purpose, scope and approach) ●Data Collection ●Base Model Development ●Error Checking ●Calibration (Compare model MOE’s to field data) ●Alternative Analysis ●Final Report and Technical Documentation

5 U.S. Department of Transportation Federal Highway Administration Handout

6 U.S. Department of Transportation Federal Highway Administration Courtesy of MNDOT

7 U.S. Department of Transportation Federal Highway Administration

8 U.S. Department of Transportation Federal Highway Administration TYPICAL QUESTIONS TO CONSIDER ●How should the project scope and physical limits be established? ●What are the project objectives? ●Which traffic analysis tool is best? ●How do I pick a traffic analysis tool? ●What are the available resources? ●Is sufficient expertise available to develop the model?

9 U.S. Department of Transportation Federal Highway Administration SCOPE PROJECT ●Identify Project Purpose and Need –High Level Planning –Detailed Design –Operational Improvements

10 U.S. Department of Transportation Federal Highway Administration SCOPE PROJECT ●Identify Limits of Project –Physical Construction –Operational Area of Influence –Model Limits? ●Estimate Data Collection –Available Data Sources –Additional Data Needs –Analysis Year(s) and Time Period(s) ●Estimate Level of Effort –Commensurate to Purpose and Need, Investment, and Safety or Failure Risk Figure Courtesy of MNDOT

11 U.S. Department of Transportation Federal Highway Administration PROJECT SCOPE THE REVIEWING AGENCY’S POSITION ●Have needs and expectations been met? ●Have appropriate project tasks and deliverables been identified?  Is there a focus on documentation, including the model development process and calibration process?  Have the needs of the target audience been identified?

12 U.S. Department of Transportation Federal Highway Administration QUESTIONS ON PROJECT SCOPE AND MODEL SELECTION ●Discuss what was considered in the selection of the microsimulation tool. ●Does the scope of work reflect these considerations? ●Will the scope of work generate a report that is soundly supported by model results that we can base engineering decisions on? ●Any additional questions?

13 U.S. Department of Transportation Federal Highway Administration

14 U.S. Department of Transportation Federal Highway Administration DATA COLLECTION ●The Quality of the Data Will Influence Analysis ●Use Data Which is Measurable in the Field

15 U.S. Department of Transportation Federal Highway Administration DATA COLLECTION ●Base Mapping –Institutionally Acceptable –Eases any Referenced to other Activities ●Field Review –Identify Hot Spots –Confirm Operations / Timing Sheets –Confirm Geometry / “Ad-hock” Lanes

16 U.S. Department of Transportation Federal Highway Administration DATA COLLECTION ●Traffic Volumes –Peak Periods ●Could be > 1 hr ●Saturated Conditions – consider periods before / after peak –Turning Movements –15 minute intervals, (or available interval) –Data should be no more than 2 years olds –Balance Counts – Influences the Outcome

17 U.S. Department of Transportation Federal Highway Administration GEOMETRIC DATA Source: Introduction to Corsim Training Course Workshop 3 ●Lane widths ●Speeds ●Length of accel/decel lanes ●Length of turn bays ●Exclusive turn movements

18 U.S. Department of Transportation Federal Highway Administration CONTROL DATA ●Actuation/Pre-timed ●Phase Operations ●Green times ●Vehicle extensions Source: Introduction to Corsim Training Course Workshop 5

19 U.S. Department of Transportation Federal Highway Administration DATA FOR CALIBRATION ●Depending on the Scope of the Project, Two or More of the Following Are Needed : Mainline Volumes (Every X Feet) Mainline Speeds (Every X Feet) Travel Times (Link or Between Occupancy/Density Pairs) Bottleneck Capacity (Measured, Not Theoretical) Entrance Ramp Queues Intersection Queues and Queue Discharge Rates Source: Chapter 9 Model Calibration John Hourdakis, Center for Transportation Studies University of Minnesota

20 U.S. Department of Transportation Federal Highway Administration FIELD OBSERVATIONS ●Verify Geometry –Has road been widened? –Have lanes been added? –Have movements been restricted? ●Verify Signal Phasing/Timing –Do signals operate as documented? ●Verify Speed Limits –Posted speeds –Prevailing speeds

21 U.S. Department of Transportation Federal Highway Administration DATA COLLECTION ●Sources of Data / Studies –Instrumented Systems (TMC / TOC/Temp “RTMS”) –Planning / Permanent Count Stations –Automatic Vehicle Location / Identification Systems –Crash Databases –Manual Counts –Speed Studies –Queue Observations –O-D Studies –Truck/Freight Data

22 U.S. Department of Transportation Federal Highway Administration QUESTIONS ON DATA COLLECTION ●Is the necessary data available? ●Does additional data need to be collected? ●What resources are available for data collection in a timely manner? ●Have any data assumptions been made and have these assumptions been documented?

23 U.S. Department of Transportation Federal Highway Administration

24 U.S. Department of Transportation Federal Highway Administration BASE MODEL DEVELOPMENT ●Use Structured / Consistent Process ●Information Management – Establish QA/QC –Potential Volume –Means to Trace Input, Output and Errors –Spreadsheets –Schematics and Images –Text Files –Automated Programs –Node Numbering Convention –Assumptions

25 U.S. Department of Transportation Federal Highway Administration BASE MODEL DEVELOPMENT ●Each Tool Is Different ●Follow Recommendations of Developer ●Capitalize on the Capabilities of the Tool ●Multiple Parameters Include –Driver Behavior –Vehicle Characteristics ●Consider Limitations of the Tool

26 U.S. Department of Transportation Federal Highway Administration BASE MODEL DEVELOPMENT ●Develop Link-Node Diagram and Lane Schematic Figure Courtesy of MNDOT

27 U.S. Department of Transportation Federal Highway Administration BASE MODEL DEVELOPMENT ●Placement of Nodes Will Influence Output (Measure of Effectiveness) 1500’ Lane 2 Lane 1 Direction of travel 1500’ Please Be Aware

28 U.S. Department of Transportation Federal Highway Administration BASE MODEL DEVELOPMENT ●Code Network – Inputs Vary by Model –Link – Node Diagram –Traffic Volume –Origin – Destination –Turn Volume / Percentage –Signal Timing / Operational Characteristics Variations During Analysis Period?

29 U.S. Department of Transportation Federal Highway Administration BASE MODEL DEVELOPMENT ●Code Network – Inputs Vary by Model –Link – Node Diagram –Traffic Volume –Origin – Destination –Turn Volume / Percentage –Signal Timing / Operational Characteristics May Warrant Multiple Time Intervals

30 U.S. Department of Transportation Federal Highway Administration BASE MODEL DEVELOPMENT ●Develop Input for for Multiple Time Intervals ●15 Min Intervals

31 U.S. Department of Transportation Federal Highway Administration BASE MODEL DEVELOPMENT ●Run Model with Multiple Time Interval Data ●What About Multiple Runs?

32 U.S. Department of Transportation Federal Highway Administration BASE MODEL DEVELOPMENT Why Multiple Runs? CORSIM is a stochastic model, which means that random numbers are assigned to driver and vehicle characteristics and to decision making processes. The MOEs that are obtained from a simulation are the result of a specific set of random number seeds. For example, one set of random number seeds may result in three very conservative drivers driving side by side on a three-lane roadway blocking more aggressive drivers behind them. The resulting MOE would reflect a lower average speed then has been observed in the real world. Relying on the MOE generated from a single run of CORSIM may be misleading. To gain a better understanding of network performance the network should be simulated several times using different sets of random number seeds. The resulting distribution of MOEs should then be an accurate representation of the network performance –TSIS Users Guide

33 U.S. Department of Transportation Federal Highway Administration BASE MODEL DEVELOPMENT Why Multiple Runs? For serious simulation applications, it is recommended that multiple runs be performed with different random number seeds… The MOEs can then be averaged with some clever spreadsheet manipulations. –SimTraffic Users Guide

34 U.S. Department of Transportation Federal Highway Administration QUESTIONS ON BASE MODEL DEVELOPMENT ●How does the coding of the network affect the output? ●Do we need to make any assumptions or do we need additional data? ●Does the existing/base year model reflect the geometric and operational characteristics of those seen in the field? ●Are the limits of the simulated network broad enough to capture the effect of the alternatives in question?

35 U.S. Department of Transportation Federal Highway Administration

36 U.S. Department of Transportation Federal Highway Administration ERROR CHECKING ●Review Input Data ●Check basic network connectivity for consistency ●Review Vehicle Characteristics and Performance Data ●Review Turning Movements ●Review Traffic Signal Operations

37 U.S. Department of Transportation Federal Highway Administration ERROR CHECKING ●Run Model at Low Volume / Short Time Period ●Review Animation to Confirm Geometry ●Review Animation to Confirm Operations –Single vehicle –Low volume –Full volume ●Review Model Warnings and Errors

38 U.S. Department of Transportation Federal Highway Administration ERROR CHECKING Upon Completion of Base Model Development, Including Assessment with Multiple Runs YOU HAVE A WORKING MODEL

39 U.S. Department of Transportation Federal Highway Administration

40 U.S. Department of Transportation Federal Highway Administration QUESTIONS ON ERROR CHECKING ●What process should be used to identify and correct errors during base model development? ●What type of internal QA/QC process was used to assure all errors were identified?

41 U.S. Department of Transportation Federal Highway Administration BREAK

42 U.S. Department of Transportation Federal Highway Administration

43 U.S. Department of Transportation Federal Highway Administration WHY CALIBRATE? ●Computers Cannot Magically Replicate Reality! –Simulation Models Are Designed to be General –Driver Behavior and Road Characteristics Depend on ●Location i.e. Minnesota vs California ●Vehicle Characteristics (Horsepower, Size, etc.) ●Weather Conditions (Dry, Wet, Ice, etc.) ●Microscopic Simulators Can Adapt and Replicate Almost Any Condition if the Model Parameters Are Properly Adjusted –What is Realistic and What is Not? Source: Chapter 9 Model Calibration John Hourdakis, Center for Transportation Studies University of Minnesota

44 U.S. Department of Transportation Federal Highway Administration OBJECTIVE OF CALIBRATION To improve the ability of the model to accurately reproduce local traffic conditions.

45 U.S. Department of Transportation Federal Highway Administration COMPARE MODEL TO FIELD DATA ●Calibration of Existing Models –Sometimes Referred to as Existing Condition ●Base Model is Calibrated When: –Volume, Density and other Operational Observations are Satisfactorily Replicated –Statistical Tests Support Such a Determination

46 U.S. Department of Transportation Federal Highway Administration ADJUST PARAMETERS ●Calibration Involves: –Modification of Default Values (Parameters) –Consideration of the Sensitivity of Parameters –Reliance on a Sound Base Model –Representative Field Data

47 U.S. Department of Transportation Federal Highway Administration ADJUST PARAMETERS ●Suggestions –Limit the Modification of Multiple Parameters per Iteration and document changes between runs –Modify Known Global and Link Level Parameters –Consider Impact of Global Parameters on Individual Links –Consider Impact of Link Parameters on Upstream and Downstream Locations –Use Caution When Modifying Unknown Parameters –Carefully Note all Parameter Modifications

48 U.S. Department of Transportation Federal Highway Administration ADJUST PARAMETERS ●Sample Global Parameters –Vehicle Entry Headway –Fleet Composition –Driver Behavior ●Sample Link Parameters –Free Flow Speed –Warning Sign Locations –Mean Start Up Delay

49 U.S. Department of Transportation Federal Highway Administration COMPARE MODEL TO FIELD DATA ●What is Satisfactory? –A statistical test is suggested –Volumes Match –Density/Occupancy Match –Speeds Match –Field Observations Occur in Model –“False” bottlenecks are cleared ●Post Processor ●Spreadsheets

50 U.S. Department of Transportation Federal Highway Administration CALIBRATION ISSUES ●Very Important For Model Accuracy and Robustness ●Accuracy Depends on Measurement Granularity –Averages Over Several Days is a Bad Choice –Might Need Additional Information to be Collected in Turbulent Sections (Bottlenecks, Weaving Areas, etc.) ●Simulation Objective Affects Calibration –When Adaptive Control Strategies Are Simulated, Stricter Calibration is Needed –Modeling of an Isolated Interchange in Rural Minnesota Will be Restrictive Source: Chapter 9 Model Calibration John Hourdakis, Center for Transportation Studies University of Minnesota

51 U.S. Department of Transportation Federal Highway Administration CALIBRATION ISSUES ●CAN BE TIME CONSUMMING PROCESS –Currently Simulators Do Not Provide a Methodology or Tools to Assist in Calibration –Often Users End Up in Endless Trial-and-Error Cycles –Sporadic Attempts Made in Literature to Streamline the Process But: ●Focused on a Particular Simulator ●Too Complex or too Naive to be Effectively Used in Practice ●No Widely Accepted Methods/Standards Currently Available Source: Chapter 9 Model Calibration John Hourdakis, Center for Transportation Studies University of Minnesota

52 U.S. Department of Transportation Federal Highway Administration BEFORE CALIBRATION! ●Check Geometry For Correctness –Disjoined Sections –Stuck Vehicles (Sizes of Accel/Decel Lanes) –Verify Location of Detectors ●Check Input For Accuracy –Entrance Volume Comparison (Perfect Match) –Exit Volume Comparison (Match Sum Over All Hours) –Volume Totals on Mainline Stations Should Match Source: Chapter 9 Model Calibration John Hourdakis, Center for Transportation Studies University of Minnesota

53 U.S. Department of Transportation Federal Highway Administration COMPARE MODEL TO FIELD DATA ●Once Existing Condition is Calibrated –Reflect Calibration Data in Alternatives Analysis –May Proceed with Alternatives Analysis

54 U.S. Department of Transportation Federal Highway Administration I-494 SOUTHBOUND CALIBRATION EXAMPLE ●CORSIM Model Developed to Evaluate: –I-494 Expansion From TH 5 to TH 55 –I-394 Interchange Modification ●Model limits –I-494 From Fish Lake Interchange (I-94) to Valley View Road –I-394 From Central Avenue to TH 169/General Mills Interchange –17 Service Interchanges –3 Systems Interchanges Source: Chapter 9 Corsim Calibration Jaimison Sloboden, P.E., Short Elliot Hendrickson, Inc.

55 U.S. Department of Transportation Federal Highway Administration SB I-494 TRY 1 Default Car Following Factors Estimated Free Flow Speeds Source: Chapter 9 Corsim Calibration Jaimison Sloboden, P.E., Short Elliot Hendrickson, Inc. Handout

56 U.S. Department of Transportation Federal Highway Administration SB I-494 TRY 12 Car Following Factors Modified Free Flow Speeds Modified Source: Chapter 9 Corsim Calibration Jaimison Sloboden, P.E., Short Elliot Hendrickson, Inc.

57 U.S. Department of Transportation Federal Highway Administration I-494 SOUTHBOUND MOE’S Try 1 MOE’s Try 12 MOE’s Source: Chapter 9 Corsim Calibration Jaimison Sloboden, P.E., Short Elliot Hendrickson, Inc.

58 U.S. Department of Transportation Federal Highway Administration

59 U.S. Department of Transportation Federal Highway Administration QUESTIONS ON CALIBRATION ●Were multiple runs used to calibrate the model? –Why or why not? ●Was the calibration process documented, including what parameters were modified and why? ●Why were default parameters modified and how was the user specified value selected?

60 U.S. Department of Transportation Federal Highway Administration

61 U.S. Department of Transportation Federal Highway Administration ALTERNATIVES ANALYSIS ●Why do we conduct an Alternatives Analysis? –Environmentally required –Looking for best operational strategy –What is the purpose? ●What do we consider? –Assess No-Build and Build Concepts –Design Year, Mid-Year(s), Opening Year and Existing Year –What else?

62 U.S. Department of Transportation Federal Highway Administration ALTERNATIVE ANALYSIS ●Amount of Output Can Be Overwhelming ●Convey Characteristics of Alternative –Highlight Problem Areas –Highlight Opportunity ●Utilize Tabular Summaries ●Utilize Animation ●Utilize Schematics

63 U.S. Department of Transportation Federal Highway Administration ALTERNATIVE ANALYSIS ●Reflect Upon the No-Build with Build Alternatives ●A “Relative Comparison” Normalizes Output ●Consider Impacts of Geometric and Operational Strategies ●Baseline Demand Forecast ●Capacity constraint, uncertainty. ●Selection of MOE’s ●Multiple Replications ●Correcting Biases with Results ●Comparison to HCM

64 U.S. Department of Transportation Federal Highway Administration Discussion of Examples

65 U.S. Department of Transportation Federal Highway Administration ●Step 1: Baseline Demand Forecast –5-year forecast was estimated – assumed straight line growth –2% per year uncompounded –10% increase for the corridor –No anticipated capacity constraints ●Step 2: Generation of Alternatives –2 alternatives test – No Build and Build –Build alternative – ramp metering on the 2 eastbound freeway on-ramps –No-Build alternative – no ramp metering ALTERNATIVE ANALYSIS

66 U.S. Department of Transportation Federal Highway Administration ●Step 3: Selection of MOEs –MOEs selected for evaluation ●VMT ●VHT ●Delay (vehicle-hours) ●Selected indicator – blocked link ●Report mean results rather than 95% worst case result ALTERNATIVE ANALYSIS

67 U.S. Department of Transportation Federal Highway Administration ●Step 4: Model Application –Run model 10 times for each alternative –Impact on ramp meters on route choice estimated outside the model –Manually adjusted turn percentages –Induced demand not considered to be a significant factor –Signal/meter optimization performed outside model –Separate optimizations performed for each alternative ALTERNATIVE ANALYSIS

68 U.S. Department of Transportation Federal Highway Administration ●Step 5: Tabulation of Results –10 repetitions of each alternative –Output to a spreadsheet and averaged –Post-model corrections not necessary ●No increasing queues of underserved demand found ALTERNATIVE ANALYSIS

69 U.S. Department of Transportation Federal Highway Administration ●Step 6: Evaluation of Alternatives –No Build Scenario ●Corridor delay increased 46% over existing conditions ●VMT increased 12% ●Total travel time increased 20% ●Delay increases on freeway mainline links –Build Scenario ●Delay – 3% less than No-build alternative ●Slight increase in VMT ●Freeway mainline improved – at expense of on-ramp traffic ●Improvements operationally acceptable ALTERNATIVE ANALYSIS

70 U.S. Department of Transportation Federal Highway Administration POINTS / QUESTIONS ON ALTERNATIVE ANAYLSIS ●LOS results should not be derived from microsimulation output –Address this Early in Scoping Process ●Did the alternatives analysis focus on numerical output or by viewing the animation? ●How was the preferred alternative selected? –What are the operational benefits?

71 U.S. Department of Transportation Federal Highway Administration

72 U.S. Department of Transportation Federal Highway Administration FINAL REPORT ●Measures of Effectiveness –Each Tool May Be Slightly Different –Difference in HCM Methodology ●Utilize Animation ●Convey Analysis Results ●Explain Measure of Effectiveness ●Capitalize on the Relative Comparison

73 U.S. Department of Transportation Federal Highway Administration FINAL REPORT ●Address the Original Question / Problem ●Tie Back to the Purpose and Need ●Clarify the Geometric Modifications ●Clarify the Operational Strategy ●Identify the Operational Area of Influence

74 U.S. Department of Transportation Federal Highway Administration QUESTIONS ON THE FINAL REPORT ●Does the final report address the needs of the target audience? ●Do the reported numerical values have meaning to the target audience? ●Does the simulated analysis support the engineering decision?

75 U.S. Department of Transportation Federal Highway Administration Summary ●Traffic Analysis Tools Program ●Traffic Analysis Toolbox –Volume I – Primer –Volume II –Volume III ●Project Scoping in context of microsimulation ●Applying Guidelines to Microsimulation