NCHRP Study (30) Performance Measures for Context Sensitive Solutions – A Guidebook for State DOTs As presented by Sally Oldham at the 2005 Midwest CSS Conference

Performance Measurement CSS and CIA

Why Measure? Make CSS state of the practice not state of the art To Strengthen Agency Leadership Support for CSS Principles. To Maintain Focus on Strategic CSS Goals. To Strengthen Trust with Stakeholders and Customers.

Project Level – Key Characteristics CSS principles rooted at project level Work for one or many projects Foster team commitment Rely on collaborative self-assessment Apply at milestones in project delivery Strengthen stakeholder trust

Guiding Concepts for Project Measures Integrate measures into project development process as an aid Develop tailored consensus-based concepts to measure against –Problems and needs for all projects –Project vision or goals

Project Measures -- Key Process Focus Areas Use of multi-disciplinary teams Public engagement Project problems and needs Project vision or goals Alternatives analysis Construction and maintenance

Project Measures – Key Outcome Focus Areas Achievement of project vision or goals Stakeholder satisfaction Quality assurance review

Organizational Level - Characteristics Used to assess performance of entire organization Fewer in number than project-level measures Rely on central reporting of data Vital resource for senior management Monitored on regular schedule

Organization Measures Key Process Focus Areas Training Manuals Policies Motivation

Organization Measures – Key Outcome Focus Areas Timeframe and budget Stakeholder satisfaction

Matrix of CSS Measures

Project Level (Process)

Use of Multi-Disciplinary Teams. Suggestions for Measuring –Were the right people on the team? –Did the team function effectively? –Focus on Context Sensitive Solutions’ principles?

Public Engagement Suggestions for Measuring –Presence of a public involvement plan? –Were external champions for the project created? –Was public input sought and used at key decisions points? –Adequacy of DOT expertise and resources? –Quality of public involvement strategy?

Project Problems, Opportunities and Needs. Suggestions for Measuring –Support for statement of Problems, Opportunities and Needs? –Linkage of Problems, Opportunities and Needs to evaluation of alternatives?

Achievement of Project Vision or Goals. Suggestions for Measuring –Match between original Problems, Opportunities and Needs statement and final project? –Consistency with Local Plans? –Tracking and adherence to project commitments? –Were project Vision or Goals met? –Does the project support community values? –Are environmental resources preserved or enhanced? –Did the project leverage other resources?

Alternatives Analysis Suggestions for Measuring –Adequacy of range of alternatives developed? –Existence of criteria for evaluation of alternatives? –Design considerations: Design speed Level of service Safety –Need for redesign –Multi-modal considerations

Construction & Maintenance Were construction staff involved with the project team at all key milestones? Was a list of commitments to stakeholders maintained throughout the planning and design phases and incorporated into construction documents prior to beginning construction? Was the project monitored to ensure that commitments were acted on? Were there many requests for change orders during construction?

Construction & Maintenance Were maintenance staff involved with the project team at all key milestones? Were maintenance needs/requirements taken into consideration when alternatives were evaluated? Is a maintenance plan in place to ensure that the project investment will be maintained? As a reflection of community buy-in and support, has the local government or has a local organization agreed to maintain some portion of the project improvements?

Project Level (Outcomes)

Achievement of Project Vision and Goals Match between original Problems, Opportunities and Needs statement and final project? Tracking and adherence to project commitments? Were project Vision or Goals met? Does the project support community values? Are environmental resources preserved or enhanced? Did the project leverage other resources?

Stakeholder Satisfaction Suggestions for Measuring –Tailored surveys of key stakeholders –Achievement of consensus during project? –Impacts of construction

Quality Assurance Review Suggestions for Measuring –Maryland SHA Charrettes –Maryland SHA Peer Reviews Impacts of construction –Connecticut DOT Project Reviews

Organizational Level (Process)

Training Suggestions for Measuring –Quantity of training? –Focus of training? –Quality of training?

Manuals Suggestions for Measuring –Changes in manuals? –Effectiveness of manual changes?

Policies Suggestions for Measuring –Changes in policies? –Effectiveness of policy changes?

Staff Motivation Strategies Suggestions for Measuring –NYDOT CSS awards program –Utah’s management through performance plan requirements

Organizational Level (Outcomes)

Timeframe and Budget Suggestions for Measuring –What proportion of projects is completed on, or ahead of schedule? –Were few or no project redesigns required? –Were low-build options selected? –Were there added costs attributed to changes in scope mid-way through the design process? –Were there cost overruns during construction attributable to changes in design during the construction phase?

Stakeholder Satisfaction Suggestions for Measuring –Aggregate results of project-level surveys, etc. –Conduct opinion polls; focus groups

Creating Measures: Two Key Ingredients Leadership –Executive management must show considerable support –Need a day-to-day champion –Create a work group (internal & external members Strategic Planning –CSS should be consistent with any strategic planning efforts

Implementing Measures: A Tailored, Collaborative, Self- Assessment Approach. May likely rely on self-administered surveys –team members and –stakeholders. “Charrette” style sessions may be a practical strategy

Timing Organization-wide –Quarterly or once a year Project-level –Process: At project milestones and at project completion –Outcomes: after project completion

Tips For Getting Started Create a Champion for CSS Measurement Start Small Incorporate Feedback from External Sources Focus on Planning and Preliminary Design Measures for Small Projects are as Important as Those for Large Projects.

Other Ways to Measure Utilize the language of the “Thinking Beyond the Pavement” workshop. Now a part of SAFETEA-LU Process related Design related

SAFETEA-LU TITLE VI--Transportation Planning And Project Delivery Amends Title 23 § 109

Characteristics of the Process Contributing to Excellence Open, honest, early, continuous communication Multidisciplinary team Involve full range of stakeholders in project scoping phase.

Characteristics of the Process Contributing to Excellence Tailor processes to meet the circumstances/project. Multiple Alternatives; reach consensus. Be committed to the process. Valued resources are understood before engineering design is started. Use a full range of tools.

Qualities of Excellence in Transportation Design The project: –Satisfies the purpose and needs as agreed to by a full range of stakeholders. –Is a safe facility both for the user and the community. –Is in harmony with the community, natural resources and values of the area.

Qualities of Excellence in Transportation Design The project: –Exceeds the expectations of both designers and stakeholders –Involves efficient and effective use of the resources (time, budget, community) of all involved parties. –Is designed and built with minimal disruption to the community. –Is seen as having added lasting value to the community.

FHWA CSS Criteria Written CSS commitment and/or policy. Technical staff is trained in CSS approach, both field and central offices, and across disciplines with a substantial portion of staff trained in CSS for project development. At a minimum, all new projects are being developed in accordance with CSS concepts, consistent with scope, size, and type of project.

FHWA CSS Criteria Early, continuing, and interactive public involvement throughout the project development process. Interdisciplinary teams are involved in the process from the beginning until the end.

CIA Measures CUTR, U. of South Florida October 2005

Basic Structure Application of Core Measures Checklist of Questions Suggested Public Involvement Methods

Sample Checklist Question

You can find the full report at: center/Completed_Proj/Summary_ PTO/FDOT_BC353_28_rpt.pdf