Jake Smithwick Arizona State University

Jake Smithwick Arizona State University
Best-Value Overview Jake Smithwick Arizona State University

Similarities? Can you award to someone other than the lowest priced?
Do you receive proposals? Do you receive past performance information or references? Do you interview? Are you allowed to negotiate with a vendor before award?

What is Best-Value? Win-Win Client: Outsource to experts
Higher performance Less management and resources Vendor Control of project/service Ability to increase profit by maximizing their efficiency

Traditional Model RESULTS Client assumes they have expertise
Client makes decisions on what they think they need to be successful Client prepares minimum standards Client prepares documents that describe everything they need/require Client competes vendors, but primary award factor is cost Inconsistent performance Inefficiencies Lack of performance metrics Lack of accountability RESULTS

Best-Value Model Experts know more about the project/service than the client Client does their best job at defining what they think they need Experts act in the clients best interest Expert can optimize the service by reducing inefficiencies Think about the project, use past experience, and layout a specific plan Focus on the key personnel, lessons learned, and risk mitigation What the Owner Described What the Consultant Specified What the Contractors Installed What the Owner Really Needed

Factors For Success Fair (state/follow rules) Open
Impartial and Transparent (minimize evaluator bias / provide debriefing) Efficient (minimize efforts) Award based on value 6

Best Value System

Best Value System High Level Overview Details Proposal ($) Schedule
Past Performance Team Qualifications Risk Assessment Value Assessment Interviews Pre-Planning & Clarification Award Weekly Reporting Post Award Metrics Final Documentation Update PPI

Phase 2 is Most Critical High Level Overview Details
Vendors Are Experts Verify Everything 9 9

What is the Clarification Period? (Proactive vs Reactive)
Minimize All Surprises!!!

What Could Cause a Surprise
Delivering something that doesn’t work Delivering something that isn’t what the client is expecting Delivering something that isn’t what the client needed Requiring the client to do something (that they did not know they had to do) Requiring things from the client that they cannot provide Expecting that something will happen as planned Assuming that things are clear and understandable Assuming that things will be done/occur as planned Changes that impact cost Changes that impact time Poor satisfaction 11

Guide Us / Help Us See & Understand
12

How Can We Minimize Surprises
Carefully preplan the project in detail Coordinate the project/service with all critical parties Prepare a detailed project plan (work plan, staffing, implementation, etc) Revisit the sites to do any additional investigating Prepare a detailed project schedule identifying critical milestones Cost Verification Detailed cost breakdown Identify why the cost proposal may be significantly different from competitors Review big-ticket items Value added options Identify all assumptions Prepare a list of all proposal assumptions 13

How Can We Minimize Surprises
Align expectations Identify any potential deal breakers Clearly identify what is included and excluded in the proposal Client roles and responsibilities Any contract terms and conditions Identify how the vendor will track and document their performance Performance metrics & Weekly risk reports Identify and Mitigate All Risks Client concerns/risks Other proposers risks Previous project risks Uncontrollable risks 14

Phase 2 Financial Summary Project Plan Project Risks/Concerns
Assumptions Performance Metrics Contract 15 15

Best Value System

Setting Up The RFP RFP Approach Make it simple!
Don’t do things just because “it has been done that way in the past” Structure Identify your goals, expectations, desired outcomes, requirements Tell them everything you know about your current conditions Tell them how you will evaluate them Tell them your terms and conditions Use Forms/Attachments (standardize, easier to evaluate, less likely to forget)

Vendor Proposals Past Performance Information Written Proposal
Cost/Schedule

Best Value Process Filter 1 Filter 2 Filter 3 Filter 4 Filter 5
Proposal Evaluations Filter 2 Interview Key Personnel Filter 3 Prioritization (Identify Best Value) Filter 4 Cost Reasonableness Check Filter 5 Pre-Award & Clarification Project Execution Risk Reporting & Close Out Rating 1 1 2 2 Contract Award 3 3 4 4 Evaluation Criteria Price / Cost / Fee Project Capability Risk Assessment Value Added Past Performance Information (PPI) Pre Award Activities Training Kickoff Meeting Plan & Clarify Summary Meeting Project Execution - Weekly Risk Report - Director Report - Performance Meas. - Close Out Ratings Short List prior to Interviews (if necessary) Total Evaluation Scores are determined Decision Matrix to confirm Selection of the potential Best Value Proponent Copyright Arizona State Univ. 2013

Phase 1 Evaluation Weights

Past Performance Information
PPI will be collected on the following Entities: The Firm Project Manager (Individual) Critical Subcontractors Vendor ENTITY Prepare Reference List Step 1 Prepare and Send Survey Questionnaires to Past Clients Step 2 Step 3 Collect/Receive Completed Surveys Enter data into Reference List Step 4 Package all material (Reference List and Surveys) and Submit Step 5 21

Written Proposal Goal is to minimize work / keep process efficient
Minimize marketing material or general information Only focus on the specific project Only look at Risks and Value Added Ideas

Critical Formatting Requirements
In order to minimize any bias, the evaluated proposal documents MUST NOT contain any names that can be used to identify who Proposer is (such as company names, personnel names, project names, or product names). 23

Blind Submittals = capability to meet requirements
Phase 1 Blind Submittals Project Capability = capability to meet requirements Simple, concise, support w/ performance metrics Risk Plan = risk don’t control Value Added = added scope

Phase 1 Project Capability Differentiate capability to meet the requirements of this project Identify "vision” or “plan” for the alignment of expertise over the duration of the project Alignment of Resources Cash flow Selection of critical subcontractors Address cash flow on the project, and how it is integrated into the schedule Use either verifiable performance metrics or best value practices with performance measurements

Bring Value to the Project
VENDOR 1 RISK: Noise from our demolition may result in student/staff complaints (since we will be doing demo in an in-operational library during finals week). SOLUTION: We will work with the user to minimize the impact of noise from demolition. VENDOR 2 SOLUTION: To minimize this risk, we have planned to demolition during off hours and weekends. We will also install rubber sheets on the floors to diminish noise and vibrations. Both solutions can be performed within your budget. We have used a similar approach on 10 projects in the last year, with 0 noise complaints. 26 26

Project Capability Submittal Example, cont’d
Phase 1 Project Capability Submittal Example, cont’d Capability Claim: Roofing contractor. Documented Performance: Our roofing contractor has a PPI score of 9.5 out of 10. Capability Claim: Our roofing subcontractor has proven long-term performance to eliminate leaks caused by the roofing system, at a very high level of customer satisfaction. Documented Performance: During the past 15 years, the roofing contractor has completed 65 roofing projects (similar size and scope to this project). 99% of the projects were completed on-time, and 100% of the roofs do not currently leak. The average age of these roofs is 12 years. The average overall customer satisfaction rating is 9.5 out of 10.

Phase 1 Risk Plan Prioritize major risk items on this project that the submitter does NOT control Explain how proposer will mitigate, manage, and/or minimize the risk from occurring Method of minimizing contract transactions The risk should be described in non-technical terms and should contain enough information to understand why the risk is a valid risk. Proposer must also explain how it will avoid or minimize the risks from occurring. Solutions must be nontechnical, logical, easily understood, or contain verifiable performance information.

Example of Solutions Risk: Getting water to the site
Type: Risk Assessment Plan 1 Coordination with [water company] is critical. We will coordinate and plan with [water company] as soon as the award is made to make sure that we get water to the site to irrigate the fields. Plan 2 We will coordinate and schedule the water with [water company]. However, based on past experience there is a high risk they will not meet the schedule (the water company does not meet schedule over 90% of the time). We will have temporary waterlines setup and ready to connect to the nearby fire hydrant to irrigate until [water company] is ready. We will also have water trucks on-site if there is problems with connecting the lines. In this example, the project was a park renovation that included installation of several recreations fields. The greatest risk was to get water to the fields in order to grow the grass. One contractors solution was to “coordinate with the water company as soon as possible” CLICK The next contractors solution was to also coordinate, but if the coordination failed, they would have temporary waterlines available, and would also have water trucks on-site if there were any issues with the temporary lines. Once again, we are looking for differential between vendors. We are trying to identify vendors that can prove to the owner that they have experience.

Explaining a Plan VENDOR 1 VENDOR 2
RISK: Since this is a revenue based contract, the greatest risk is that the student population on campus does not grow as expected (which will impact our financial projections). SOLUTION: We will implement a plan based on our past expertise to stimulate student growth and enhance our marketing plan to the students (which we have done at several accounts). VENDOR 2 SOLUTION: We expect that we can still increase revenues even if the student population does not increase as predicted. At two similar accounts, we increased growth by more than 10% in two years, despite flattening enrollment, through concept and menu updates, increasing convenience (speed of service), convenience store additions, and expanding points of sale.

Phase 1 Value Added Plan Opportunity to identify any value added options or ideas that may benefit the Owner Respondent should identify: What the Owner may have excluded or omitted from its scope How these options or ideas have been successful through verifiable performance information and/or best value practices MUST have a cost impact (and possibly schedule impact) Use either verifiable performance metrics or best value practices with performance measurements Value add ideas are NOT included in the base cost proposal

Why Value Add Plan? Provide ways to keep project at or below budget
Phase 1 Why Value Add Plan? Provide ways to keep project at or below budget Modifications to requirements to meet budget Specific cost ($) savings Supported by metrics (high performance) Increase customer satisfaction / performance

Scope is Above Budget Owner’s Scope (-$ value add) Owner’s Budget ($$)
Phase 1 Scope is Above Budget Owner’s Scope (-$ value add) Owner’s Budget ($$)

Intent Doesn’t Match Scope
Phase 1 Intent Doesn’t Match Scope Owner’s Budget ($$) Owner’s Intent (+$ value add) Owner’s Scope

Phase 1 Meeting the intent Reroofing this building will not stop all water leaks. The majority of the leaks are caused by cracks in the parapet walls, broken/missing glass, and poor caulking. For an additional $20K and 3 weeks in schedule we can replace and repair all of these items. This approach in 8 similar projects has worked 100% of the time in stopping all leaks. 35

Things to Avoid General Statements:
Our employees will wear safety equipment (hard hats, vests, etc) Safety: Our goal is to have no accidents or deaths We will put a fence around the site to prevent outside access Marketing data: Our company is known worldwide as a leader in… We will use our long history to… We will use state-of-the-art process to… Technical data: The system we propose has 200% elongation and 600psi tensile strength The product will pass the ASTM-568a test. Transferring risk back to client: We will work with the owner to resolve issues… We will have team meetings / partnering meetings with the owner…

Submittal Requirements
Phase 1 Submittal Requirements Submittal Document Maximum Page Length Project Capability Plan Two (2) pages Risk Plan Value Add Plan Must NOT contain ANY identifying information Respondents are NOT allowed to re-create, re-format, or modify the template (cannot alter font size, font type, font color; add colors, pictures, tables, diagrams, etc)

Proposal Summary Proposal Format: Limited to 6 pages
No names in the proposals Key Advantages: Gives advantage back to high performers Minimizes bias / Fair environment Attracts high performers Minimizes marketing Focus on the actual project (risks, value, expertise) Minimizes evaluation efforts (time and expertise)

How The Submittal Process Works
Average Score Proposal Form (1 page) Non-Evaluated Documents Submittal Contracting Officer Purchasing Proposal Form (1 page) Evaluated Documents Evaluation Members

Evaluation Committee Will be used to evaluate specific portions of the Proposal Evaluators will not be provided with the names of any Proposers, product names, cost, or any additional information Evaluators will independently (not as a group or consensus) review and score the items comparatively to one another Objective of the scoring is to not make a decision (looking for “dominant” differential) Evaluations will be scored on a 1/5/10 scale “10” = Dominantly higher value than the average (clearly shows differential) “5” = About average (insufficient information to make a clear decision) “1” = Dominantly below the average (clearly shows differential) 40

Key Personnel Interviews
The Client will conduct interviews with the Project Manager and Site Superintendent from each of the Shortlisted Offerors. No substitutes, proxies, phone interviews, or electronic interviews will be allowed. Individuals who fail to attend the interview on the date/time specified will be given a “1” score. Interviews are expected to last approximately 15 minutes per individual. No other individuals (from the Offeror’s organization) will be allowed to sit in or participate during the interview session. Interviewees may not bring notes or handouts. The Client will interview individuals separately. Interviewees will be prohibited from making any reference to their proposed financial contributions. 42 42 42

Interview Comments Goal Is To Minimize Risk
“I have no idea why I am here today”…“My boss called me last night and told me to show up for this interview” - $10 Million Project “I did not participate at all in preparing our proposal” “I am not currently employed by this company, but if we win this project, they will then hire me” - $25 Million Service Project “I have never managed a project of this size/scope” - $30 Million Project “There is no risk on this project” - Large IT Project “The greatest risk that I always face, is how to accomplish all of the things that our sales team promised we could do” – Cleanroom Design Project 43

Final Prioritization 45

Cost Reasonableness Best-Value is within budget Dominance Check Also
Phase 1 Cost Reasonableness Best Value Prioritization Best-Value is within budget Yes Yes No Proceed to highest ranked proposal within budget Yes Yes No Best-Value is the lowest price Dominance Check Also Best-Value is within 10% of next highest ranked firm Yes Yes No Best-Value can be justified based on other factors Yes Yes No Proceed to Pre-Award Go with Alternate Proposal or Cancel

Phase 2

Phase 2 is Most Critical High Level Overview Details
Vendors Are Experts Verify Everything

Impact of Pre-Award (General Services Administration)
No CRITERIA PRE AWARD (None) PRE AWARD (<10 Days) PRE AWARD (>10 Days) 1 Number of projects analyzed 11 4 6 2 Average PA duration (days) 7 22 3 Total awarded cost $14,244,385 $1,997,933 $7,996,954 Total awarded schedule 1,822 699 674 5 Average Overall Change Order Rate 44% 11% 12% Average Overall Project Delay Rate 92% 93% 25% The Pre-Award Period has been shown to: Minimize cost increases by 72% Minimize project delays by 72% 51 51

Example - Software Project

Overview 11,970 students; 832 staff 12 member districts
First project in March 2011

Change Orders 74% fewer change orders 40% less of the contingency used
Characteristic Facility 1 (Traditional) Facility 2 (Best Value) Total number of change orders 422 110 Total cost of change orders $1,523,902 $1,448,243 Percent of contingency budget used 8.0% 4.8% Overall change order rate N/A 5.0% 74% fewer change orders 40% less of the contingency used Contractor much more proactive Contractor becomes a leader in the model

Best Value System AWARD

Best Value System

Weekly Risk Reporting System
Spreadsheet that documents all risks on the service Risk = Anything that may impact cost or schedule. Risks can be caused by the Offeror or the Client Report must be submitted on Friday of every week (until service is complete) The WRRS does not substitute or eliminate weekly progress reports or any other traditional reporting systems or meetings (that the Offeror may perform or may be required to perform).

Risk Management by Contractor
Director Director Procurement 1 Director Procurement 2 Director PM 1 Director PM 2 Director PM 3 Director PM 4 Director Contractor 1 Director Contractor 3 Director Contractor 6 Director Contractor 8 Director Contractor 2 Director Contractor 4 Director Contractor 7 Director Contractor 4 Director Contractor 5 Director Contractor 2 Director Contractor 8 Director Contractor 6 Director Contractor 8 Director Contractor 2 Director Contractor 9 Director Contractor 1

University of Minnesota
4th Largest University in U.S. (69,000 students) 5 major campuses 3.6 Million Square Feet (classroom and research space) Founded in 1851 $3 Billion in Revenues (tuition, research, sales, etc) Partnered with Capital Planning in 2005 to increase accountability and document performance

Contractor Generated Reports
161 projects Reports submitted once per week via System would then pull the data from each spreadsheet into a master report (“Directors Report”) Data can be used to generate a wide variety of information Individual Projects External Contractors External Designers Client Project Managers Client Procurement Officers Other Internal Staff Selection Process (LB/BV) Delivery Method (DBB, DB) Entire organization

Overall Program 63 63

Report – Top 10 Riskiest Projects
64 Report – Top 10 Riskiest Projects

Analysis of Contractors
65 65

Report – End Users / Customers
66 66

Report – Analysis of Risks
67 67

Roadmap to Sustainability
Keys to Sustainability Agency with wide impact Vendor buy-in Initial high performance results Application of philosophy Early Adopters (UMN) Expansion into non-Construction 2013 2005 Industry Support 2007 Other Users Begin Testing Contractors become source of BV

Early Adopters Year 1 Performance Lessons Learned
Performance Criteria Results Awarded Cost ($M) $3.1M Completed Projects 8 Overall Vendor Cost Increase 0.4% Overall Vendor Schedule Increase 0.9% Contractors are very high performing WRR provides project documentation that UMN never had NEXT STEP: establish PIPS as procurement model

Best Value System identify expertise document expectations
measure against expectations Proposal ($) Past Performance Team Qualifications Risk Assessment Value Assessment Interviews Pre-Planning & Clarification Award Weekly Reporting Post Award Metrics Final Documentation Update PPI

Questions Can a project be any worse if you:
Educate vendors on thinking in your best interest Try to hire the best vendor Require the vendor to carefully preplan a project Require the vendor to document their performance on a weekly basis 71

Case Studies

Tri-U Furniture Consortium
Best Value Approach to Commodity Contracts

Background Consortium to procure furniture:
Arizona State University (ASU) Northern Arizona University (NAU) University of Arizona (UA) Goals: standard ordering, better pricing, high performance $7M / annually (all project types)

Buyer Satisfaction Increases
Survey ASU NAU UA Overall Baseline 43 surveys 72% 64% 74% 70% Best Value 132 surveys 93% 92% 33% increase in buyer satisfaction Less performance deviation at each university Alignment of expectations Pre-planning with other trades

Best Value Structure dealer clients On-time On-budget
Project On-time On-budget Client satisfaction Expectations met

Performance Report Documentation of all projects
General Overview Dealer A Dealer B Dealer C Overall Total Number of Projects 137 624 783 1,544 Total Awarded Cost $ 2,977,029 $ 6,591,214 $ 21,085,727 $ 30,653,970 Average Awarded Cost $ ,384 $ ,153 $ ,172 $ ,436 Cost Analysis (Projects >= $100K) Overall Change Order Rate 0.1% 0.0% Client Designer Dealer Unforeseen Schedule Analysis (Projects >= $100K) Overall Schedule Delays 3.7% 6.9% 4.6% 5.5% 3.8% 1.5% 0.8% Satisfaction Ratings Overall Client Satisfaction 9.3 9.2 9.1 Total Number of Surveys 56 29 50 135 Documentation of all projects Measurement and accountability of key players Improves coordination between trades

State of Hawaii (1998-2002) 193 projects
In 2002, an internal audit was performed: Total number of awarded PIPS roofs: 96 100% would rather use the PIPS process over low-bid process (55 DOE users) PIPS average performance rating 9.6 (10 max) Projects were 6% under budget Projects finished approximately 35% faster (than LB) 98% were completed on time PIPS contractors were approximately two times more productive In last 4 years, there has been no roof leaks PIPS was 14% cheaper than low-bid 78

City of Peoria (2004-2009) Number of Best-Value Procurements: 55
Estimated Budget: $389 Million (DB/CMAR/JOC) (Wastewater, Office Buildings, Fire Station, Parks, Roadway) (AE Services, Radio, Maintenance, Software) Average Number of Proposals per Project: 6 Number of Completed Projects: 10 ($193 Million) Overall C/O Rate: 0.5% (compared to 14%) Overall Delay Rate: 6% (compared to 35%) Final Results: 93% Satisfaction (compared to 20%) Final Results: 9.1 Rating (10 max) 5 Projects where money returned 79

ASU Management Requirement
ASU Dining 16 year Four campuses $1 Billion Criteria Year 1 (From Incumbent) Year 2 (From Year 1) Year 3 (From Year 2) Year 4 (From Year 3) Sales 14% Increase 11% Increase 24% Increase 13.5% Increase Commission 23% Increase 6% Increase 20% Increase 22% Increase ASU Management Requirement Reduced 79% -- Student Satisfaction 37% Increase 1% Decrease 9% Increase 3% Increase

University of Minnesota
Award Analysis: Number of Best-Value Procurements: 161 Awarded Cost: $50.6M (11% below average cost) Average Number of Proposals: 4 Projects Where Best-Value was also Lowest Cost: 53% 85% of projects were awarded to vendor with highest / second highest RAVA Plan (7.3 vs 5.9) Performance Information: Contractor Impacts: 0% Change Orders / 4% Delay Vendor post project rating: 9.6 Average Contractor Increase in Profit: 5% UMN PM management requirements: -65% Traditional History of UMN: 32% projects on schedule 32% projects on budget $17 million in claims 81 81

Satisfaction / Performance
University of Alberta Project Value Cost Savings Schedule Impacts Satisfaction / Performance Custodial Services (campus-wide) $18M $2M 10% 5.5% performance Improvement 10 (out of 10) 2. DB Construction (Research Facility) $30M $8-12M 25% 14-18 months 9.7 (out of 10) 3. Design Services (Building Redevelopment) $4M $500k 12% 0% Cost & Schedule CO’s $190k in Value Added Options Had high levels of success with these 3 projects Brian Stewart, Director of Procurement will be here, talking more on Thursday about his experiences 82 82

Key Takeaways Change in thinking is the most critical (and most difficult) Get executive support Minimize decision making Stick to the process Get educated

Comments / Questions W W W . P B S R G . C O M 84 84

Jake Smithwick Arizona State University

Similar presentations

Presentation on theme: "Jake Smithwick Arizona State University"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Jake Smithwick Arizona State University

Similar presentations

Presentation on theme: "Jake Smithwick Arizona State University"— Presentation transcript:

Similar presentations

About project

Feedback