Presentation is loading. Please wait.

Presentation is loading. Please wait.

Best-Value Overview Jake Smithwick Arizona State University.

Similar presentations


Presentation on theme: "Best-Value Overview Jake Smithwick Arizona State University."— Presentation transcript:

1 Best-Value Overview Jake Smithwick Arizona State University

2 Similarities? Can you award to someone other than the lowest priced? Do you receive proposals? Do you receive past performance information or references? Do you interview? Are you allowed to negotiate with a vendor before award?

3 What is Best-Value? Win-Win Client: – Outsource to experts – Higher performance – Less management and resources Vendor – Control of project/service – Ability to increase profit by maximizing their efficiency 3

4 Traditional Model Client assumes they have expertise Client makes decisions on what they think they need to be successful Client prepares minimum standards Client prepares documents that describe everything they need/require Client competes vendors, but primary award factor is cost 4 Inconsistent performance Inefficiencies Lack of performance metrics Lack of accountability

5 Best-Value Model Experts know more about the project/service than the client Client does their best job at defining what they think they need Experts act in the clients best interest Expert can optimize the service by reducing inefficiencies Think about the project, use past experience, and layout a specific plan Focus on the key personnel, lessons learned, and risk mitigation 5 What the Owner Described What the Owner Really Needed What the Consultant Specified What the Contractors Installed

6 Factors For Success Fair (state/follow rules) Open Impartial and Transparent (minimize evaluator bias / provide debriefing) Efficient (minimize efforts) Award based on value 6

7 7 Best Value System

8 8 Proposal ($) Schedule Past Performance Team Qualifications Risk Assessment Value Assessment Interviews Pre-Planning & Clarification Award Weekly Reporting Post Award Metrics Final Documentation Update PPI High Level Overview Details

9 9 Phase 2 is Most Critical High Level Overview Verify EverythingVendors Are Experts Details

10 What is the Clarification Period? (Proactive vs Reactive) Minimize All Surprises!!!

11 What Could Cause a Surprise Delivering something that doesn’t work Delivering something that isn’t what the client is expecting Delivering something that isn’t what the client needed Requiring the client to do something (that they did not know they had to do) Requiring things from the client that they cannot provide Expecting that something will happen as planned Assuming that things are clear and understandable Assuming that things will be done/occur as planned Changes that impact cost Changes that impact time Poor satisfaction 11

12 12 Guide Us / Help Us See & Understand

13 How Can We Minimize Surprises Carefully preplan the project in detail – Coordinate the project/service with all critical parties – Prepare a detailed project plan (work plan, staffing, implementation, etc) – Revisit the sites to do any additional investigating – Prepare a detailed project schedule identifying critical milestones Cost Verification – Detailed cost breakdown – Identify why the cost proposal may be significantly different from competitors – Review big-ticket items – Value added options Identify all assumptions – Prepare a list of all proposal assumptions 13

14 How Can We Minimize Surprises Align expectations – Identify any potential deal breakers – Clearly identify what is included and excluded in the proposal – Client roles and responsibilities – Any contract terms and conditions Identify how the vendor will track and document their performance – Performance metrics & Weekly risk reports Identify and Mitigate All Risks – Client concerns/risks – Other proposers risks – Previous project risks – Uncontrollable risks 14

15 15 Phase 2 Financial Summary Project Plan Project Risks/Concerns Assumptions Performance Metrics Contract

16 16 Best Value System

17 Setting Up The RFP Approach – Make it simple! – Don’t do things just because “it has been done that way in the past” Structure – Identify your goals, expectations, desired outcomes, requirements – Tell them everything you know about your current conditions – Tell them how you will evaluate them – Tell them your terms and conditions – Use Forms/Attachments (standardize, easier to evaluate, less likely to forget) RFP

18 Vendor Proposals Past Performance Information Written Proposal Cost/Schedule

19 Filter 1 Proposal Evaluations Filter 2 Interview Key Personnel Filter 4 Cost Reasonableness Check Filter 5 Pre-Award & Clarification Project Execution Risk Reporting & Close Out Rating Filter 3 Prioritization (Identify Best Value) Contract Award Evaluation Criteria - Price / Cost / Fee - Project Capability - Risk Assessment - Value Added - Past Performance Information (PPI) Short List prior to Interviews (if necessary) Pre Award Activities - Training - Kickoff Meeting - Plan & Clarify - Summary Meeting Total Evaluation Scores are determined Decision Matrix to confirm Selection of the potential Best Value Proponent Project Execution - Weekly Risk Report - Director Report - Performance Meas. - Close Out Ratings Best Value Process Copyright Arizona State Univ

20 Evaluation Weights 20 Phase 1

21 Past Performance Information PPI will be collected on the following Entities: – The Firm – Project Manager (Individual) – Critical Subcontractors 21 Vendor ENTITY Prepare and Send Survey Questionnaires to Past Clients Step 2 Step 3 Collect/Receive Completed SurveysPrepare Reference List Step 1 Enter data into Reference List Step 4 Package all material (Reference List and Surveys) and Submit Step 5

22 Written Proposal Goal is to minimize work / keep process efficient Minimize marketing material or general information Only focus on the specific project Only look at Risks and Value Added Ideas

23 Critical Formatting Requirements In order to minimize any bias, the evaluated proposal documents MUST NOT contain any names that can be used to identify who Proposer is (such as company names, personnel names, project names, or product names). 23

24 Blind Submittals Risk Plan Value Added Project Capability Simpleconcise performance metrics Simple, concise, support w/ performance metrics = risk don’t control = capability to meet requirements = added scope Phase 1

25 Project Capability Differentiate capability to meet the requirements of this project Identify "vision” or “plan” for the alignment of expertise over the duration of the project – Alignment of Resources – Cash flow – Selection of critical subcontractors Address cash flow on the project, and how it is integrated into the schedule Use either verifiable performance metrics or best value practices with performance measurements Phase 1

26 26 Bring Value to the Project VENDOR 1 – RISK: Noise from our demolition may result in student/staff complaints (since we will be doing demo in an in-operational library during finals week). – SOLUTION: We will work with the user to minimize the impact of noise from demolition. VENDOR 2 – RISK: Noise from our demolition may result in student/staff complaints (since we will be doing demo in an in-operational library during finals week). – SOLUTION: To minimize this risk, we have planned to demolition during off hours and weekends. We will also install rubber sheets on the floors to diminish noise and vibrations. Both solutions can be performed within your budget. We have used a similar approach on 10 projects in the last year, with 0 noise complaints.

27 Project Capability Submittal Example, cont’d Capability Claim:Roofing contractor. Documented Performance: Our roofing contractor has a PPI score of 9.5 out of 10. Capability Claim:Our roofing subcontractor has proven long-term performance to eliminate leaks caused by the roofing system, at a very high level of customer satisfaction. Documented Performance: During the past 15 years, the roofing contractor has completed 65 roofing projects (similar size and scope to this project). 99% of the projects were completed on-time, and 100% of the roofs do not currently leak. The average age of these roofs is 12 years. The average overall customer satisfaction rating is 9.5 out of 10. Phase 1

28 Risk Plan Prioritize major risk items on this project that the submitter does NOT control Explain how proposer will mitigate, manage, and/or minimize the risk from occurring Method of minimizing contract transactions The risk should be described in non-technical terms and should contain enough information to understand why the risk is a valid risk. Proposer must also explain how it will avoid or minimize the risks from occurring. Solutions must be nontechnical, logical, easily understood, or contain verifiable performance information. Phase 1

29 Plan 1 – Coordination with [water company] is critical. We will coordinate and plan with [water company] as soon as the award is made to make sure that we get water to the site to irrigate the fields. Plan 2 – We will coordinate and schedule the water with [water company]. However, based on past experience there is a high risk they will not meet the schedule (the water company does not meet schedule over 90% of the time). – We will have temporary waterlines setup and ready to connect to the nearby fire hydrant to irrigate until [water company] is ready. – We will also have water trucks on-site if there is problems with connecting the lines. Example of Solutions Risk: Getting water to the site Type: Risk Assessment

30 30 Explaining a Plan VENDOR 1 – RISK: Since this is a revenue based contract, the greatest risk is that the student population on campus does not grow as expected (which will impact our financial projections). – SOLUTION: We will implement a plan based on our past expertise to stimulate student growth and enhance our marketing plan to the students (which we have done at several accounts). VENDOR 2 – RISK: Since this is a revenue based contract, the greatest risk is that the student population on campus does not grow as expected (which will impact our financial projections). – SOLUTION: We expect that we can still increase revenues even if the student population does not increase as predicted. At two similar accounts, we increased growth by more than 10% in two years, despite flattening enrollment, through concept and menu updates, increasing convenience (speed of service), convenience store additions, and expanding points of sale.

31 Value Added Plan Opportunity to identify any value added options or ideas that may benefit the Owner Respondent should identify: – What the Owner may have excluded or omitted from its scope – How these options or ideas have been successful through verifiable performance information and/or best value practices MUST have a cost impact (and possibly schedule impact) Use either verifiable performance metrics or best value practices with performance measurements Value add ideas are NOT included in the base cost proposal Phase 1

32 Why Value Add Plan? at below budget 1.Provide ways to keep project at or below budget – Modifications to requirements to meet budget – Specific cost ($) savings – Supported by metrics (high performance) 2.Increase customer satisfaction / performance Phase 1

33 Scope is Above Budget Owner’s Budget ($$) Owner’s Scope (-$ value add) Phase 1

34 Intent Doesn’t Match Scope Owner’s Budget ($$) Owner’s Scope Owner’s Intent (+$ value add) Phase 1

35 Meeting the intent Reroofing this building will not stop all water leaks. The majority of the leaks are caused by cracks in the parapet walls, broken/missing glass, and poor caulking. For an additional $20K and 3 weeks in schedule we can replace and repair all of these items. This approach in 8 similar projects has worked 100% of the time in stopping all leaks.35 Phase 1

36 Things to Avoid General Statements: – Our employees will wear safety equipment (hard hats, vests, etc) – Safety: Our goal is to have no accidents or deaths – We will put a fence around the site to prevent outside access Marketing data: – Our company is known worldwide as a leader in… – We will use our long history to… – We will use state-of-the-art process to… Technical data: – The system we propose has 200% elongation and 600psi tensile strength – The product will pass the ASTM-568a test. Transferring risk back to client: – We will work with the owner to resolve issues… – We will have team meetings / partnering meetings with the owner…

37 Submittal Requirements Submittal DocumentMaximum Page Length Project Capability PlanTwo (2) pages Risk PlanTwo (2) pages Value Add PlanTwo (2) pages37 Phase 1 NOT Must NOT contain ANY identifying information NOT Respondents are NOT allowed to re-create, re-format, or modify the template (cannot alter font size, font type, font color; add colors, pictures, tables, diagrams, etc)

38 Proposal Summary Proposal Format: – Limited to 6 pages – No names in the proposals Key Advantages: – Gives advantage back to high performers – Minimizes bias / Fair environment – Attracts high performers – Minimizes marketing – Focus on the actual project (risks, value, expertise) – Minimizes evaluation efforts (time and expertise)

39 How The Submittal Process Works Submittal Evaluation Members Proposal Form (1 page) Non-Evaluated Documents Proposal Form (1 page) Evaluated Documents Average Score Contracting Officer Purchasing Officer

40 Evaluation Committee Will be used to evaluate specific portions of the Proposal Evaluators will not be provided with the names of any Proposers, product names, cost, or any additional information Evaluators will independently (not as a group or consensus) review and score the items comparatively to one another Objective of the scoring is to not make a decision (looking for “dominant” differential) Evaluations will be scored on a 1/5/10 scale – “10” = Dominantly higher value than the average (clearly shows differential) – “5” = About average (insufficient information to make a clear decision) – “1” = Dominantly below the average (clearly shows differential) 40

41 Filter 1 Proposal Evaluations Filter 2 Interview Key Personnel Filter 4 Cost Reasonableness Check Filter 5 Pre-Award & Clarification Project Execution Risk Reporting & Close Out Rating Filter 3 Prioritization (Identify Best Value) Contract Award Evaluation Criteria - Price / Cost / Fee - Project Capability - Risk Assessment - Value Added - Past Performance Information (PPI) Short List prior to Interviews (if necessary) Pre Award Activities - Training - Kickoff Meeting - Plan & Clarify - Summary Meeting Total Evaluation Scores are determined Decision Matrix to confirm Selection of the potential Best Value Proponent Project Execution - Weekly Risk Report - Director Report - Performance Meas. - Close Out Ratings Best Value Process Copyright Arizona State Univ

42 42 Key Personnel Interviews The Client will conduct interviews with the Project Manager and Site Superintendent from each of the Shortlisted Offerors. No substitutes, proxies, phone interviews, or electronic interviews will be allowed. Individuals who fail to attend the interview on the date/time specified will be given a “1” score. Interviews are expected to last approximately 15 minutes per individual. No other individuals (from the Offeror’s organization) will be allowed to sit in or participate during the interview session. Interviewees may not bring notes or handouts. The Client will interview individuals separately. Interviewees will be prohibited from making any reference to their proposed financial contributions.

43 Interview Comments Goal Is To Minimize Risk “I have no idea why I am here today”…“My boss called me last night and told me to show up for this interview” - $10 Million Project “I did not participate at all in preparing our proposal” “I am not currently employed by this company, but if we win this project, they will then hire me” - $25 Million Service Project “I have never managed a project of this size/scope” - $30 Million Project “There is no risk on this project” - Large IT Project “The greatest risk that I always face, is how to accomplish all of the things that our sales team promised we could do” – Cleanroom Design Project 43

44 Filter 1 Proposal Evaluations Filter 2 Interview Key Personnel Filter 4 Cost Reasonableness Check Filter 5 Pre-Award & Clarification Project Execution Risk Reporting & Close Out Rating Filter 3 Prioritization (Identify Best Value) Contract Award Evaluation Criteria - Price / Cost / Fee - Project Capability - Risk Assessment - Value Added - Past Performance Information (PPI) Short List prior to Interviews (if necessary) Pre Award Activities - Training - Kickoff Meeting - Plan & Clarify - Summary Meeting Total Evaluation Scores are determined Decision Matrix to confirm Selection of the potential Best Value Proponent Project Execution - Weekly Risk Report - Director Report - Performance Meas. - Close Out Ratings Best Value Process Copyright Arizona State Univ

45 Final Prioritization 45

46 Filter 1 Proposal Evaluations Filter 2 Interview Key Personnel Filter 4 Cost Reasonableness Check Filter 5 Pre-Award & Clarification Project Execution Risk Reporting & Close Out Rating Filter 3 Prioritization (Identify Best Value) Contract Award Evaluation Criteria - Price / Cost / Fee - Project Capability - Risk Assessment - Value Added - Past Performance Information (PPI) Short List prior to Interviews (if necessary) Pre Award Activities - Training - Kickoff Meeting - Plan & Clarify - Summary Meeting Total Evaluation Scores are determined Decision Matrix to confirm Selection of the potential Best Value Proponent Project Execution - Weekly Risk Report - Director Report - Performance Meas. - Close Out Ratings Best Value Process Copyright Arizona State Univ

47 47 Cost Reasonableness Best-Value is the lowest price Best-Value is within 10% of next highest ranked firm Best-Value can be justified based on other factors Best-Value is within budget Yes No Yes Best Value Prioritization Best Value Prioritization Yes No Go with Alternate Proposal or Cancel Proceed to Pre-Award Yes No Yes No Proceed to highest ranked proposal within budget Dominance Check Also Phase 1

48 Filter 1 Proposal Evaluations Filter 2 Interview Key Personnel Filter 4 Cost Reasonableness Check Filter 5 Pre-Award & Clarification Project Execution Risk Reporting & Close Out Rating Filter 3 Prioritization (Identify Best Value) Contract Award Evaluation Criteria - Price / Cost / Fee - Project Capability - Risk Assessment - Value Added - Past Performance Information (PPI) Short List prior to Interviews (if necessary) Pre Award Activities - Training - Kickoff Meeting - Plan & Clarify - Summary Meeting Total Evaluation Scores are determined Decision Matrix to confirm Selection of the potential Best Value Proponent Project Execution - Weekly Risk Report - Director Report - Performance Meas. - Close Out Ratings Best Value Process Copyright Arizona State Univ

49 49 Phase 2

50 50 Phase 2 is Most Critical High Level Overview Verify EverythingVendors Are Experts Details

51 Impact of Pre-Award (General Services Administration) 51 NoCRITERIA PRE AWARD (None) PRE AWARD (<10 Days) PRE AWARD (>10 Days) 1Number of projects analyzed1146 2Average PA duration (days)0722 3Total awarded cost$14,244,385$1,997,933$7,996,954 4Total awarded schedule1, Average Overall Change Order Rate44%11%12% 6Average Overall Project Delay Rate92%93%25% The Pre-Award Period has been shown to: ̶Minimize cost increases by 72% ̶Minimize project delays by 72%

52 Example - Software Project

53 Overview 11,970 students; 832 staff 12 member districts First project in March 2011

54 Change Orders 54 Characteristic Facility 1 (Traditional) Facility 2 (Best Value) Total number of change orders Total cost of change orders$1,523,902$1,448,243 Percent of contingency budget used8.0%4.8% Overall change order rateN/A5.0% 74% fewer change orders 40% less of the contingency used Contractor much more proactive Contractor becomes a leader in the model

55

56 56 Best Value System

57 Filter 1 Proposal Evaluations Filter 2 Interview Key Personnel Filter 4 Cost Reasonableness Check Filter 5 Pre-Award & Clarification Project Execution Risk Reporting & Close Out Rating Filter 3 Prioritization (Identify Best Value) Contract Award Evaluation Criteria - Price / Cost / Fee - Project Capability - Risk Assessment - Value Added - Past Performance Information (PPI) Short List prior to Interviews (if necessary) Pre Award Activities - Training - Kickoff Meeting - Plan & Clarify - Summary Meeting Total Evaluation Scores are determined Decision Matrix to confirm Selection of the potential Best Value Proponent Project Execution - Weekly Risk Report - Director Report - Performance Meas. - Close Out Ratings Best Value Process Copyright Arizona State Univ

58 58 Best Value System

59 Weekly Risk Reporting System Spreadsheet that documents all risks on the service Risk = Anything that may impact cost or schedule. Risks can be caused by the Offeror or the Client Report must be submitted on Friday of every week (until service is complete) The WRRS does not substitute or eliminate weekly progress reports or any other traditional reporting systems or meetings (that the Offeror may perform or may be required to perform). 59

60 Risk Management by Contractor Director Procurement 1 Director Contractor 1 Director Contractor 2 Director Contractor 5 Director Contractor 8 Director Contractor 3 Director Contractor 4 Director Contractor 2 Director Contractor 2 Director PM 2 Director PM 1 Director Procurement 2 Director Contractor 6 Director Contractor 7 Director Contractor 8 Director Contractor 9 Director Contractor 8 Director Contractor 4 Director Contractor 6 Director Contractor 1 Director PM 4 Director PM 3

61 University of Minnesota 4 th Largest University in U.S. (69,000 students) 5 major campuses 3.6 Million Square Feet (classroom and research space) Founded in 1851 $3 Billion in Revenues (tuition, research, sales, etc) Partnered with Capital Planning in 2005 to increase accountability and document performance

62 Contractor Generated Reports 161 projects Reports submitted once per week via System would then pull the data from each spreadsheet into a master report (“Directors Report”) Data can be used to generate a wide variety of information – Individual Projects – External Contractors – External Designers – Client Project Managers – Client Procurement Officers – Other Internal Staff – Selection Process (LB/BV) – Delivery Method (DBB, DB) – Entire organization

63 63 Overall Program

64 64 Report – Top 10 Riskiest Projects

65 65 Analysis of Contractors

66 66 Report – End Users / Customers

67 67 Report – Analysis of Risks

68 Roadmap to Sustainability Industry Support Other Users Begin Testing Early Adopters (UMN) Keys to Sustainability Agency with wide impact Vendor buy-in Initial high performance results Application of philosophy Expansion into non- Construction 2013 Contractors become source of BV

69 Early Adopters Year 1 Performance Performance CriteriaResults Awarded Cost ($M)$3.1M Completed Projects8 Overall Vendor Cost Increase0.4% Overall Vendor Schedule Increase 0.9% Lessons Learned Contractors are very high performing WRR provides project documentation that UMN never had NEXT STEP: establish PIPS as procurement model

70 70 Best Value System Proposal ($) Past Performance Team Qualifications Risk Assessment Value Assessment Interviews Pre-Planning & Clarification Award Weekly Reporting Post Award Metrics Final Documentation Update PPI identify expertise document expectations measure against expectations

71 Questions Can a project be any worse if you: – Educate vendors on thinking in your best interest – Try to hire the best vendor – Require the vendor to carefully preplan a project – Require the vendor to document their performance on a weekly basis 71

72 Case Studies

73 Tri-U Furniture Consortium Best Value Approach to Commodity Contracts

74 Background Consortium to procure furniture: – Arizona State University (ASU) – Northern Arizona University (NAU) – University of Arizona (UA) Goals: standard ordering, better pricing, high performance $7M / annually (all project types)

75 Buyer Satisfaction Increases 33% increase in buyer satisfaction Less performance deviation at each university Alignment of expectations Pre-planning with other trades75 SurveyASUNAUUAOverall Baseline 43 surveys 72%64%74%70% Best Value 132 surveys 93%92% 93%

76 Best Value Structure clients dealer Project On-time On-budget Client satisfaction Expectations met

77 Performance Report Documentation Documentation of all projects Measurement accountability Measurement and accountability of key players coordination Improves coordination between trades77 General Overview Dealer ADealer BDealer COverall Total Number of Projects ,544 Total Awarded Cost $ 2,977,029 $ 6,591,214 $ 21,085,727 $ 30,653,970 Average Awarded Cost $ 22,384 $ 11,153 $ 27,172 $ 20,436 Cost Analysis (Projects >= $100K) Overall Change Order Rate0.1%0.0% Client0.0% Designer0.0% Dealer0.0% Unforeseen0.0% Schedule Analysis (Projects >= $100K) Overall Schedule Delays0.0%3.7%6.9%4.6% Client0.0%3.7%5.5%3.8% Designer0.0% Dealer0.0% 1.5%0.8% Unforeseen0.0% Satisfaction Ratings Overall Client Satisfaction Total Number of Surveys

78 State of Hawaii ( ) 193 projects In 2002, an internal audit was performed: 1.Total number of awarded PIPS roofs: % would rather use the PIPS process over low-bid process (55 DOE users) 3.PIPS average performance rating 9.6 (10 max) 4.Projects were 6% under budget 5.Projects finished approximately 35% faster (than LB) 6.98% were completed on time 7.PIPS contractors were approximately two times more productive 8.In last 4 years, there has been no roof leaks 9.PIPS was 14% cheaper than low-bid 78

79 79 City of Peoria ( ) Number of Best-Value Procurements: 55 Estimated Budget: $389 Million – (DB/CMAR/JOC) – (Wastewater, Office Buildings, Fire Station, Parks, Roadway) – (AE Services, Radio, Maintenance, Software) Average Number of Proposals per Project: 6 Number of Completed Projects: 10 ($193 Million) – Overall C/O Rate: 0.5% (compared to 14%) – Overall Delay Rate: 6% (compared to 35%) – Final Results: 93% Satisfaction (compared to 20%) – Final Results: 9.1 Rating (10 max) – 5 Projects where money returned

80 ASU Dining 16 year Four campuses $1 Billion

81 81 Award Analysis: – Number of Best-Value Procurements: 161 – Awarded Cost: $50.6M (11% below average cost) – Average Number of Proposals: 4 – Projects Where Best-Value was also Lowest Cost: 53% – 85% of projects were awarded to vendor with highest / second highest RAVA Plan (7.3 vs 5.9) Performance Information: – Contractor Impacts: 0% Change Orders / 4% Delay – Vendor post project rating: 9.6 – Average Contractor Increase in Profit: 5% – UMN PM management requirements: -65% University of Minnesota Traditional History of UMN: ―32% projects on schedule ―32% projects on budget ―$17 million in claims

82 University of Alberta 82 ProjectValueCost Savings Schedule Impacts Satisfaction / Performance 1.Custodial Services (campus-wide) $18M$2M 10% 5.5% performance Improvement 10 (out of 10) 2. DB Construction (Research Facility) $30M$8-12M 25% months9.7 (out of 10) 3. Design Services (Building Redevelopment) $4M$500k 12% 0% Cost & Schedule CO’s $190k in Value Added Options

83 Key Takeaways Change in thinking is the most critical (and most difficult) Get executive support Minimize decision making Stick to the process Get educated83

84 Comments / Questions 84 W W W. P B S R G. C O M


Download ppt "Best-Value Overview Jake Smithwick Arizona State University."

Similar presentations


Ads by Google