Best-Value Overview Jake Smithwick Arizona State University.
-
Upload
jessie-scoble -
Category
Documents
-
view
228 -
download
3
Transcript of Best-Value Overview Jake Smithwick Arizona State University.
Best-Value Overview
Jake SmithwickArizona State University
Similarities?• Can you award to someone other than the lowest priced?
• Do you receive proposals?
• Do you receive past performance information or references?
• Do you interview?
• Are you allowed to negotiate with a vendor before award?
What is Best-Value?
• Win-Win
• Client:– Outsource to experts– Higher performance– Less management and resources
• Vendor– Control of project/service– Ability to increase profit by
maximizing their efficiency
3
Traditional Model• Client assumes they have expertise• Client makes decisions on what they think they need to be successful• Client prepares minimum standards• Client prepares documents that describe everything they need/require• Client competes vendors, but primary award factor is cost
4
Inconsistent performance
Inefficiencies
Lack of performance metrics
Lack of accountability
RESULTS
Best-Value Model• Experts know more about the project/service than the client• Client does their best job at defining what they think they need• Experts act in the clients best interest• Expert can optimize the service by reducing inefficiencies• Think about the project, use past experience, and layout a specific plan• Focus on the key personnel, lessons learned, and risk mitigation
5
What the Owner Described
What the Owner Really Needed
What the Consultant Specified
What the Contractors Installed
Factors For Success• Fair (state/follow rules)
• Open
• Impartial and Transparent (minimize evaluator bias / provide debriefing)
• Efficient (minimize efforts)
• Award based on value
6
7
Best Value System
8
Best Value System
• Proposal ($)• Schedule• Past Performance• Team Qualifications• Risk Assessment• Value Assessment• Interviews
• Pre-Planning & Clarification
• Award • Weekly Reporting• Post Award Metrics• Final Documentation• Update PPI
High Level Overview Details
9
Phase 2 is Most Critical
High Level Overview
Verify EverythingVendors Are ExpertsDetails
What is the Clarification Period? (Proactive vs Reactive)
Minimize All Surprises!!!
What Could Cause a Surprise• Delivering something that doesn’t work• Delivering something that isn’t what the client is expecting• Delivering something that isn’t what the client needed• Requiring the client to do something (that they did not know they had to do)• Requiring things from the client that they cannot provide• Expecting that something will happen as planned• Assuming that things are clear and understandable• Assuming that things will be done/occur as planned• Changes that impact cost• Changes that impact time• Poor satisfaction
11
12
Guide Us / Help Us See & Understand
How Can We Minimize Surprises• Carefully preplan the project in detail
– Coordinate the project/service with all critical parties– Prepare a detailed project plan (work plan, staffing, implementation, etc) – Revisit the sites to do any additional investigating– Prepare a detailed project schedule identifying critical milestones
• Cost Verification– Detailed cost breakdown– Identify why the cost proposal may be significantly different from competitors– Review big-ticket items– Value added options
• Identify all assumptions– Prepare a list of all proposal assumptions
13
How Can We Minimize Surprises• Align expectations
– Identify any potential deal breakers– Clearly identify what is included and excluded in the proposal– Client roles and responsibilities– Any contract terms and conditions
• Identify how the vendor will track and document their performance– Performance metrics & Weekly risk reports
• Identify and Mitigate All Risks– Client concerns/risks – Other proposers risks– Previous project risks– Uncontrollable risks
14
15
Phase 2
• Financial Summary• Project Plan• Project Risks/Concerns• Assumptions• Performance Metrics• Contract
16
Best Value System
Setting Up The RFP
• Approach– Make it simple!– Don’t do things just because “it has been done that way in the past”
• Structure– Identify your goals, expectations, desired outcomes, requirements– Tell them everything you know about your current conditions– Tell them how you will evaluate them– Tell them your terms and conditions– Use Forms/Attachments (standardize, easier to evaluate, less likely to forget)
RFP
Vendor Proposals• Past Performance Information
• Written Proposal
• Cost/Schedule
Filter 1Proposal
Evaluations
Filter 2Interview
Key Personnel
Filter 4Cost
ReasonablenessCheck
Filter 5Pre-Award &Clarification
Project ExecutionRisk Reporting & Close Out Rating
Filter 3Prioritization
(Identify Best Value)
Cont
ract
Aw
ard
Evaluation Criteria - Price / Cost /
Fee- Project
Capability- Risk Assessment- Value Added- Past
Performance Information (PPI)
Short List prior to
Interviews (if necessary)
Pre Award Activities- Training- Kickoff Meeting- Plan & Clarify- Summary
Meeting
Total Evaluation Scores are
determined
Decision Matrix to confirm Selection of
the potential Best Value Proponent
Project Execution- Weekly Risk Report- Director Report- Performance Meas.- Close Out Ratings
Best Value Process
Copyright Arizona State Univ. 2013
1
2
3
4
1
2
3
4
Evaluation Weights20
Phase 1
Past Performance Information
• PPI will be collected on the following Entities:– The Firm– Project Manager (Individual)– Critical Subcontractors
21
VendorENTITY
Prepare and Send Survey Questionnaires to Past Clients Step 2
Step 3Collect/Receive Completed Surveys
Prepare Reference ListStep 1
Enter data into Reference ListStep 4
Package all material (Reference List and Surveys) and SubmitStep 5
Written Proposal• Goal is to minimize work / keep process efficient
• Minimize marketing material or general information
• Only focus on the specific project
• Only look at Risks and Value Added Ideas
Critical Formatting Requirements• In order to minimize any bias, the evaluated
proposal documents MUST NOT contain any names that can be used to identify who Proposer is (such as company names, personnel names, project names, or product names).
23
Blind Submittals
Risk Plan
Risk Plan
Value Added
Value Added
Project Capabilit
y
Project Capabilit
y
Simple, concise, support w/ performance metrics
= risk don’t control
= capability to meet requirements
= added scope
Phase 1
Project Capability• Differentiate capability to meet the requirements of this project
• Identify "vision” or “plan” for the alignment of expertise over the duration of the project– Alignment of Resources– Cash flow– Selection of critical subcontractors
• Address cash flow on the project, and how it is integrated into the schedule
• Use either verifiable performance metrics or best value practices with performance measurements
Phase 1
26
Bring Value to the Project• VENDOR 1
– RISK: Noise from our demolition may result in student/staff complaints (since we will be doing demo in an in-operational library during finals week).
– SOLUTION: We will work with the user to minimize the impact of noise from demolition.
• VENDOR 2– RISK: Noise from our demolition may result in student/staff complaints (since
we will be doing demo in an in-operational library during finals week). – SOLUTION: To minimize this risk, we have planned to demolition during off
hours and weekends. We will also install rubber sheets on the floors to diminish noise and vibrations. Both solutions can be performed within your budget. We have used a similar approach on 10 projects in the last year, with 0 noise complaints.
Project Capability Submittal Example, cont’d
Capability Claim: Roofing contractor.
Documented Performance:
Our roofing contractor has a PPI score of 9.5 out of 10.
Capability Claim: Our roofing subcontractor has proven long-term performance to eliminate leaks caused by the roofing system, at a very high level of customer satisfaction.
Documented Performance:
During the past 15 years, the roofing contractor has completed 65 roofing projects (similar size and scope to this project). 99% of the projects were completed on-time, and 100% of the roofs do not currently leak. The average age of these roofs is 12 years. The average overall customer satisfaction rating is 9.5 out of 10.
Phase 1
Risk Plan• Prioritize major risk items on this project that the submitter does NOT
control
• Explain how proposer will mitigate, manage, and/or minimize the risk from occurring
• Method of minimizing contract transactions
• The risk should be described in non-technical terms and should contain enough information to understand why the risk is a valid risk. Proposer must also explain how it will avoid or minimize the risks from occurring.
• Solutions must be nontechnical, logical, easily understood, or contain verifiable performance information.
Phase 1
• Plan 1 – Coordination with [water company] is critical. We will coordinate and
plan with [water company] as soon as the award is made to make sure that we get water to the site to irrigate the fields.
• Plan 2– We will coordinate and schedule the water with [water company].
However, based on past experience there is a high risk they will not meet the schedule (the water company does not meet schedule over 90% of the time).
– We will have temporary waterlines setup and ready to connect to the nearby fire hydrant to irrigate until [water company] is ready.
– We will also have water trucks on-site if there is problems with connecting the lines.
Example of Solutions Risk: Getting water to the siteType: Risk Assessment
30
Explaining a Plan• VENDOR 1
– RISK: Since this is a revenue based contract, the greatest risk is that the student population on campus does not grow as expected (which will impact our financial projections).
– SOLUTION: We will implement a plan based on our past expertise to stimulate student growth and enhance our marketing plan to the students (which we have done at several accounts).
• VENDOR 2– RISK: Since this is a revenue based contract, the greatest risk is that
the student population on campus does not grow as expected (which will impact our financial projections).
– SOLUTION: We expect that we can still increase revenues even if the student population does not increase as predicted. At two similar accounts, we increased growth by more than 10% in two years, despite flattening enrollment, through concept and menu updates, increasing convenience (speed of service), convenience store additions, and expanding points of sale.
Value Added Plan• Opportunity to identify any value added options or ideas that may benefit
the Owner
• Respondent should identify:– What the Owner may have excluded or omitted from its scope– How these options or ideas have been successful through verifiable
performance information and/or best value practices
• MUST have a cost impact (and possibly schedule impact)
• Use either verifiable performance metrics or best value practices with performance measurements
• Value add ideas are NOT included in the base cost proposal
Phase 1
Why Value Add Plan?1. Provide ways to keep project at or below budget
– Modifications to requirements to meet budget– Specific cost ($) savings– Supported by metrics (high performance)
2. Increase customer satisfaction / performance
Phase 1
Scope is Above Budget
Owner’s Budget ($$)
Owner’s Scope
(-$ value add)
Phase 1
Intent Doesn’t Match Scope
Owner’s Budget ($$)
Owner’s Scope
Owner’s Intent
(+$ value add)
Phase 1
Meeting the intent
• Reroofing this building will not stop all water leaks. The majority of the leaks are caused by cracks in the parapet walls, broken/missing glass, and poor caulking. For an additional $20K and 3 weeks in schedule we can replace and repair all of these items. This approach in 8 similar projects has worked 100% of the time in stopping all leaks.
35
Phase 1
Things to Avoid• General Statements:
– Our employees will wear safety equipment (hard hats, vests, etc)– Safety: Our goal is to have no accidents or deaths– We will put a fence around the site to prevent outside access
• Marketing data:– Our company is known worldwide as a leader in… – We will use our long history to…– We will use state-of-the-art process to…
• Technical data:– The system we propose has 200% elongation and 600psi tensile strength – The product will pass the ASTM-568a test.
• Transferring risk back to client:– We will work with the owner to resolve issues…– We will have team meetings / partnering meetings with the owner…
Submittal RequirementsSubmittal Document Maximum Page Length
Project Capability Plan Two (2) pages
Risk Plan Two (2) pages
Value Add Plan Two (2) pages
37
Phase 1
• Must NOT contain ANY identifying information
• Respondents are NOT allowed to re-create, re-format, or modify the template (cannot alter font size, font type, font color; add colors, pictures, tables, diagrams, etc)
Proposal Summary• Proposal Format:
– Limited to 6 pages– No names in the proposals
• Key Advantages:– Gives advantage back to high performers– Minimizes bias / Fair environment– Attracts high performers– Minimizes marketing– Focus on the actual project (risks, value, expertise)– Minimizes evaluation efforts (time and expertise)
How The Submittal Process Works
Submittal
Evaluation Members
Proposal Form(1 page)
Non-Evaluated Documents
Proposal Form(1 page)
Evaluated Documents
Average Score
ContractingOfficer
PurchasingOfficer
Evaluation Committee• Will be used to evaluate specific portions of the Proposal
• Evaluators will not be provided with the names of any Proposers, product names, cost, or any additional information
• Evaluators will independently (not as a group or consensus) review and score the items comparatively to one another
• Objective of the scoring is to not make a decision (looking for “dominant” differential)
• Evaluations will be scored on a 1/5/10 scale– “10” = Dominantly higher value than the average (clearly shows differential) – “5” = About average (insufficient information to make a clear decision)– “1” = Dominantly below the average (clearly shows differential)
40
Filter 1Proposal
Evaluations
Filter 2Interview
Key Personnel
Filter 4Cost
ReasonablenessCheck
Filter 5Pre-Award &Clarification
Project ExecutionRisk Reporting & Close Out Rating
Filter 3Prioritization
(Identify Best Value)
Cont
ract
Aw
ard
Evaluation Criteria - Price / Cost /
Fee- Project
Capability- Risk Assessment- Value Added- Past
Performance Information (PPI)
Short List prior to
Interviews (if necessary)
Pre Award Activities- Training- Kickoff Meeting- Plan & Clarify- Summary
Meeting
Total Evaluation Scores are
determined
Decision Matrix to confirm Selection of
the potential Best Value Proponent
Project Execution- Weekly Risk Report- Director Report- Performance Meas.- Close Out Ratings
Best Value Process
Copyright Arizona State Univ. 2013
1
2
3
4
1
2
3
4
42
Key Personnel Interviews• The Client will conduct interviews with the Project Manager and Site
Superintendent from each of the Shortlisted Offerors.
• No substitutes, proxies, phone interviews, or electronic interviews will be allowed. Individuals who fail to attend the interview on the date/time specified will be given a “1” score.
• Interviews are expected to last approximately 15 minutes per individual.
• No other individuals (from the Offeror’s organization) will be allowed to sit in or participate during the interview session. Interviewees may not bring notes or handouts. The Client will interview individuals separately. Interviewees will be prohibited from making any reference to their proposed financial contributions.
Interview CommentsGoal Is To Minimize Risk
“I have no idea why I am here today”…“My boss called me last night and told me to show up for this interview” - $10 Million Project
“I did not participate at all in preparing our proposal”
“I am not currently employed by this company, but if we win this project, they will then hire me” - $25 Million Service Project
“I have never managed a project of this size/scope” - $30 Million Project
“There is no risk on this project” - Large IT Project
“The greatest risk that I always face, is how to accomplish all of the things that our sales team promised we could do” – Cleanroom Design Project
43
Filter 1Proposal
Evaluations
Filter 2Interview
Key Personnel
Filter 4Cost
ReasonablenessCheck
Filter 5Pre-Award &Clarification
Project ExecutionRisk Reporting & Close Out Rating
Filter 3Prioritization
(Identify Best Value)
Cont
ract
Aw
ard
Evaluation Criteria - Price / Cost /
Fee- Project
Capability- Risk Assessment- Value Added- Past
Performance Information (PPI)
Short List prior to
Interviews (if necessary)
Pre Award Activities- Training- Kickoff Meeting- Plan & Clarify- Summary
Meeting
Total Evaluation Scores are
determined
Decision Matrix to confirm Selection of
the potential Best Value Proponent
Project Execution- Weekly Risk Report- Director Report- Performance Meas.- Close Out Ratings
Best Value Process
Copyright Arizona State Univ. 2013
1
2
3
4
1
2
3
4
Final Prioritization
45
NO CRITERIA POINTS FIRM A FIRM B FIRM C BEST FIRM A FIRM B FIRM C
1 Total Financial Contribution 300 1,000,000$ 1,020,000$ 1,050,000$ $ 1,000,000 300 294 286
2 Interview of Onsite General Manager 300 4.2 9.3 6.4 9.3 135 300 206
3 Risk Assessment Plan 150 5.2 8.6 5.1 8.6 91 150 89
4 Value Assessment Plan 100 5.0 9.2 5.0 9.2 54 100 54
5 Team Qualifications 50 5.0 5.0 5.0 5.0 50 50 50
6 PPI – Firm (1-10 Ratings) 25 9.2 9.1 9.3 9.3 25 24 25
7 PPI – Firm (# of Surveys) 25 5 5 5 5.0 25 25 25
8 PPI – General Manager (1-10 Ratings) 25 9.4 9.1 9.5 9.5 25 24 25
9 PPI – General Manager (# of Surveys) 25 3 4 5 5.0 15 20 25
TOTAL POINTS (1,000): 720 988 785
RAW DATA FINAL POINTS
Filter 1Proposal
Evaluations
Filter 2Interview
Key Personnel
Filter 4Cost
ReasonablenessCheck
Filter 5Pre-Award &Clarification
Project ExecutionRisk Reporting & Close Out Rating
Filter 3Prioritization
(Identify Best Value)
Cont
ract
Aw
ard
Evaluation Criteria - Price / Cost /
Fee- Project
Capability- Risk Assessment- Value Added- Past
Performance Information (PPI)
Short List prior to
Interviews (if necessary)
Pre Award Activities- Training- Kickoff Meeting- Plan & Clarify- Summary
Meeting
Total Evaluation Scores are
determined
Decision Matrix to confirm Selection of
the potential Best Value Proponent
Project Execution- Weekly Risk Report- Director Report- Performance Meas.- Close Out Ratings
Best Value Process
Copyright Arizona State Univ. 2013
1
2
3
4
1
2
3
4
47
Cost Reasonableness
Best-Value is the lowest priceBest-Value is within 10% of next highest ranked firm Best-Value can be justified based on other factors
Best-Value is within budget
YesNo
YesYes
Best ValuePrioritizationBest Value
Prioritization
YesNo
Go with AlternateProposal or Cancel
Proceed toPre-Award
YesYes
YesYes
YesYes YesNo
YesNo Proceed to highest ranked proposal within budget
Dom
inan
ce C
heck A
lso
Phase 1
Filter 1Proposal
Evaluations
Filter 2Interview
Key Personnel
Filter 4Cost
ReasonablenessCheck
Filter 5Pre-Award &Clarification
Project ExecutionRisk Reporting & Close Out Rating
Filter 3Prioritization
(Identify Best Value)
Cont
ract
Aw
ard
Evaluation Criteria - Price / Cost /
Fee- Project
Capability- Risk Assessment- Value Added- Past
Performance Information (PPI)
Short List prior to
Interviews (if necessary)
Pre Award Activities- Training- Kickoff Meeting- Plan & Clarify- Summary
Meeting
Total Evaluation Scores are
determined
Decision Matrix to confirm Selection of
the potential Best Value Proponent
Project Execution- Weekly Risk Report- Director Report- Performance Meas.- Close Out Ratings
Best Value Process
Copyright Arizona State Univ. 2013
1
2
3
4
1
2
3
4
49
Phase 2
50
Phase 2 is Most Critical
High Level Overview
Verify EverythingVendors Are ExpertsDetails
Impact of Pre-Award (General Services Administration)
51
No CRITERIA PRE AWARD (None)
PRE AWARD (<10 Days)
PRE AWARD (>10 Days)
1 Number of projects analyzed 11 4 6
2 Average PA duration (days) 0 7 22
3 Total awarded cost $14,244,385 $1,997,933 $7,996,954
4 Total awarded schedule 1,822 699 674
5 Average Overall Change Order Rate 44% 11% 12%
6 Average Overall Project Delay Rate 92% 93% 25%
• The Pre-Award Period has been shown to: Z̶ Minimize cost increases by 72%Z̶ Minimize project delays by 72%
Example - Software ProjectNO CRITERIA WEIGHT VENDOR B VENDOR C VENDOR E VENDOR G
1 Cost Rating 200 4.2 2.6 5.0 9.0
2 Interview Rating 250 4.0 3.7 8.0 7.0
3 Demo Rating 100 4.2 1.0 10.0 10.0
4 RAVA Plan 250 7.2 3.4 3.4 2.8
5 Work Plan 100 7.0 5.0 6.0 7.2
6 Past Performance Rating 100 5.0 4.2 5.0 5.0
Overall Score 1,000 708 454 763 816
Vendor CodeInstallation &
Integration5-Year Support TOTAL
FIRM A $750,000 $1,250,000 $2,000,000FIRM B $745,028 $1,183,470 $1,928,498FIRM C $369,720 $1,192,135 $1,561,855FIRM D $618,050 $1,098,429 $1,716,479FIRM E $712,600 $1,612,000 $2,324,600FIRM F $732,000 $1,477,500 $2,209,500FIRM G $688,671 $379,872 $1,068,543
Average (Minus Outliers) : $719,575 $1,229,745 $1,895,841
Overview• 11,970 students; 832 staff • 12 member districts
• First project in March 2011
Change Orders54
CharacteristicFacility 1
(Traditional)Facility 2
(Best Value)
Total number of change orders 422 110
Total cost of change orders $1,523,902 $1,448,243
Percent of contingency budget used 8.0% 4.8%
Overall change order rate N/A 5.0%
74% fewer change orders
40% less of the contingency used
Contractor much more proactive
Contractor becomes a leader in the model
56
Best Value System
AWARD
Filter 1Proposal
Evaluations
Filter 2Interview
Key Personnel
Filter 4Cost
ReasonablenessCheck
Filter 5Pre-Award &Clarification
Project ExecutionRisk Reporting & Close Out Rating
Filter 3Prioritization
(Identify Best Value)
Cont
ract
Aw
ard
Evaluation Criteria - Price / Cost /
Fee- Project
Capability- Risk Assessment- Value Added- Past
Performance Information (PPI)
Short List prior to
Interviews (if necessary)
Pre Award Activities- Training- Kickoff Meeting- Plan & Clarify- Summary
Meeting
Total Evaluation Scores are
determined
Decision Matrix to confirm Selection of
the potential Best Value Proponent
Project Execution- Weekly Risk Report- Director Report- Performance Meas.- Close Out Ratings
Best Value Process
Copyright Arizona State Univ. 2013
1
2
3
4
1
2
3
4
58
Best Value System
Weekly Risk Reporting System• Spreadsheet that documents all risks on the service
• Risk = Anything that may impact cost or schedule. Risks can be caused by the Offeror or the Client
• Report must be submitted on Friday of every week (until service is complete)
• The WRRS does not substitute or eliminate weekly progress reports or any other traditional reporting systems or meetings (that the Offeror may perform or may be required to perform).
59
Risk Management by Contractor
DirectorDirector
DirectorProcurement 1
DirectorContractor 1
DirectorContractor 2
DirectorContractor 5
DirectorContractor 8
DirectorContractor 3
DirectorContractor 4
DirectorContractor 2
DirectorContractor 2
DirectorPM 2DirectorPM 1
DirectorProcurement 2
DirectorContractor 6
DirectorContractor 7
DirectorContractor 8
DirectorContractor 9
DirectorContractor 8
DirectorContractor 4
DirectorContractor 6
DirectorContractor 1
DirectorPM 4DirectorPM 3
University of Minnesota• 4th Largest University in U.S. (69,000 students)• 5 major campuses• 3.6 Million Square Feet (classroom and research space)• Founded in 1851• $3 Billion in Revenues (tuition, research, sales, etc)
• Partnered with Capital Planning in 2005 to increase accountability and document performance
Contractor Generated Reports• 161 projects
• Reports submitted once per week via email
• System would then pull the data from each spreadsheet into a master report (“Directors Report”)
• Data can be used to generate a wide variety of information – Individual Projects– External Contractors– External Designers– Client Project Managers– Client Procurement Officers– Other Internal Staff – Selection Process (LB/BV)– Delivery Method (DBB, DB)– Entire organization
63
Overall Program
64
Report – Top 10 Riskiest Projects
No ProjectRisk
Analysis Factor
Awarded CostAwarded Duration
Overall Change Order Rate
Overall Delay Rate
Percent of Late Reports
PMProcurement
Agent
1 Mayo Remodel Suite 18.4 127,338$ 58 9% 321% 47% Joe Smith Tim Nelson
2 Coffman Union 11.4 1,593,561$ 49 3% 0% 0% Jeff Nickel Kristi Jasper
3 Animal Science Stair Towers 8.8 46,300$ 42 0% 10% 10% Chuck Nelson Tim Nelson
4 WBOB Remodel 7.9 525,000$ 82 1% 96% 37% Amy Perkins Tim Nelson
5 Snyder Bldg Exterior 7.0 178,295$ 254 -4% 81% 22% Joe Smith Tim Nelson
6 Barn Clean Renovations 6.1 366,595$ 217 2% 166% 60% Joe Smith Tim Nelson
7 Steam Plant Break 5.7 48,000$ 106 3% 72% 0% Chuck Nelson Tim Nelson
8 Vet Sciences Third 5.7 187,700$ 51 3% 86% 28% Amy Perkins Tim Nelson
9 PWB Units B-295 5.7 102,140$ 142 0% 42% 29% Joe Smith Tim Nelson
10 PWB Drug Design 5.6 136,900$ 25 5% 37% 33% Jim Roberts Kristi Jasper
65
No ContractorTotal
Number of Projects
Total Awarded Cost:
Owner Change Order Rate
Owner Delay Rate
Vendor Change Order Rate
Vendor Delay Rate
Percent of Late
Reports
Vendor Performance
1 Contractor 118 3 $ 721,965 0.3% 18.1% 0.2% 66.8% 53% 120%2 Contractor 119 3 $ 220,002 0.7% 10.4% 0.0% 0.0% 69% 69%3 Contractor 120 1 $ 269,850 9.4% 303.0% 0.0% 18.2% 47% 65%4 Contractor 104 3 $ 459,225 1.6% 2.7% 0.0% 18.8% 37% 56%5 Contractor 121 1 $ 241,575 0.0% 21.9% 2.7% 50.0% 0% 53%6 Contractor 105 8 $ 1,611,015 0.3% 32.9% 0.0% 16.3% 32% 49%7 Contractor 106 9 $ 1,280,362 2.2% 31.1% 0.7% 3.2% 35% 39%8 Contractor 122 3 $ 367,650 0.0% 79.1% 0.0% 1.4% 37% 38%9 Contractor 107 1 $ 178,440 0.0% 0.0% 0.6% 11.4% 25% 37%
10 Contractor 123 2 $ 3,227,182 14.9% 0.0% -0.6% 5.4% 30% 35%11 Contractor 108 2 $ 327,295 0.0% 135.4% 0.0% 0.0% 32% 32%12 Contractor 124 1 $ 69,218 3.5% 0.0% 0.0% 0.0% 31% 31%13 Contractor 125 3 $ 1,150,738 1.9% 7.3% 0.0% 4.2% 26% 30%14 Contractor 109 5 $ 534,095 2.0% 23.2% 0.0% 0.0% 29% 29%15 Contractor 126 1 $ 323,000 3.3% 3.4% 0.0% 6.8% 22% 29%16 Contractor 110 1 $ 308,882 1.2% 24.8% 0.0% 0.0% 27% 27%
Analysis of Contractors
66
TEAM 1 (President /
University / Admin)
TEAM 2Academic Health
Center
TEAM 3Provost College
1 Total Number of Projects 19 14 52 Percent of Projects Procured Using PIPS 79% 86% 80%3 Total Awarded Cost: $5,359,995 $2,821,005 $2,353,7614 Average Number of Risks per Project 3 8 12
5 Overall Owner Impacts (Time & Cost) 7.7% 41.3% 41.1%6 Owner Change Order Rate 0.6% 3.4% 20.0%7 Owner Delay Rate 7.2% 37.8% 21.1%8 Percent of Projects without Owner Cost Changes 63% 36% 80%9 Percent of Projects without Owner Delays 68% 50% 80%
10 Overall Contractor Impacts (Time & Cost) 8.1% 19.6% 14.8%11 Contractor Change Order Rate 0.1% 0.1% -0.8%12 Contractor Delay Rate 8.0% 19.6% 15.6%13 Percent of Projects without Contractor Cost Changes 95% 93% 100%14 Percent of Projects without Contractor Delays 79% 79% 60%
15 Total Number of Completed Projects 4 2 116 Total Number of Client Surveys Returned 3 2 117 Percent of Projects Evaluated by Client 75% 100% 100%18 Average PM Post Project Rating of Contractor 6.75 10 1019 Average Client Post Project Rating of Contractor 7.7 8.5 8.020 Average Client Post Project Rating of CPPM 10.7 8.5 7.0
Contractor Impacts
Owner Impacts
Satisfaction Ratings
General Overview
Report – End Users / Customers
67
Report – Analysis of Risks
Risk CategoryNumber of
RisksImpact to
CostImpact to Schedule
Percent Impact to
Cost
Percent Impact to Schedule
1) Client Impacts 114 $660,369 1,200 59% 46%Client Scope Change / Decision 111 660,369$ 976 59% 37%
Client Requested Delay 3 -$ 224 0% 9%
2) CPPM Impacts 135 $329,425 885 30% 34%Design Issue 48 189,876$ 230 17% 9%
CPPM Issue (Codes / Permits) 36 46,140$ 170 4% 7%
CPPM Issue (Energy Mgmt) 2 47,533$ 30 4% 1%
CPPM Issue (Hazardous / Health & Safety) 8 35,407$ 118 3% 5%
CPPM Issue (NTS) 8 10,018$ 64 1% 2%
CPPM Issue (Contract / Payment) 11 -$ 132 0% 5%
CPPM Issue (Other) 22 451$ 141 0% 5%
3) Contractor Impacts 43 $21,005 411 2% 16%Contractor Issue 11 -$ 101 0% 4%
Contractor Oversight of Design 9 21,005$ 38 2% 1%
Contractor Issue with Supplier / Sub 23 -$ 272 0% 10%
4) Unforeseen Impacts 19 $102,544 111 9% 4%311 1,113,343$ 2,607
Roadmap to Sustainability
2005
2007
2008 - 2012
Industry Support
Other Users Begin
Testing
Early Adopters
(UMN)
Keys to Sustainability• Agency with wide impact• Vendor buy-in• Initial high performance
results• Application of philosophy
Expansion into non-
Construction2013
Contractors become
source of BV2012-2013
Early Adopters
Year 1 PerformancePerformance Criteria Results
Awarded Cost ($M) $3.1M
Completed Projects 8
Overall Vendor Cost Increase 0.4%
Overall Vendor Schedule Increase 0.9%
Lessons Learned• Contractors are very
high performing
• WRR provides project documentation that UMN never had
• NEXT STEP: establish PIPS as procurement model
70
Best Value System
• Proposal ($)• Past Performance• Team Qualifications• Risk Assessment• Value Assessment• Interviews
• Pre-Planning & Clarification
• Award • Weekly Reporting• Post Award Metrics• Final Documentation• Update PPI
identify expertise
document expectations
measure against expectations
Questions• Can a project be any worse if you:
– Educate vendors on thinking in your best interest– Try to hire the best vendor– Require the vendor to carefully preplan a project– Require the vendor to document their performance on a weekly basis
71
Case Studies
Tri-U Furniture Consortium
Best Value Approach to Commodity Contracts
Background• Consortium to procure furniture:
– Arizona State University (ASU)– Northern Arizona University (NAU)– University of Arizona (UA)
• Goals: standard ordering, better pricing, high performance
• $7M / annually (all project types)
75
Buyer Satisfaction Increases
• 33% increase in buyer satisfaction• Less performance deviation at each university• Alignment of expectations• Pre-planning with other trades
Survey ASU NAU UA OverallBaseline43 surveys 72% 64% 74% 70%
Best Value132 surveys 93% 92% 92% 93%
Best Value Structureclients
dealer
ProjectProjectProjectProjectProject
• On-time• On-budget• Client
satisfaction• Expectations
met
Performance Report Documentation
of all projects
Measurement and accountability of key players
Improves coordination between trades
77
General Overview Dealer A Dealer B Dealer C Overall
Total Number of Projects 137 624 783 1,544
Total Awarded Cost $ 2,977,029 $ 6,591,214 $ 21,085,727 $ 30,653,970
Average Awarded Cost $ 22,384 $ 11,153 $ 27,172 $ 20,436
Cost Analysis (Projects >= $100K)
Overall Change Order Rate 0.1% 0.0% 0.0% 0.0%
Client 0.0% 0.0% 0.0% 0.0%
Designer 0.0% 0.0% 0.0% 0.0%
Dealer 0.0% 0.0% 0.0% 0.0%
Unforeseen 0.0% 0.0% 0.0% 0.0%
Schedule Analysis (Projects >= $100K)
Overall Schedule Delays 0.0% 3.7% 6.9% 4.6%
Client 0.0% 3.7% 5.5% 3.8%
Designer 0.0% 0.0% 0.0% 0.0%
Dealer 0.0% 0.0% 1.5% 0.8%
Unforeseen 0.0% 0.0% 0.0% 0.0%
Satisfaction Ratings
Overall Client Satisfaction 9.3 9.2 9.1 9.2
Total Number of Surveys 56 29 50 135
State of Hawaii (1998-2002)
• 193 projects • In 2002, an internal audit was performed:
1. Total number of awarded PIPS roofs: 962. 100% would rather use the PIPS process over low-bid process (55 DOE users)3. PIPS average performance rating 9.6 (10 max)4. Projects were 6% under budget5. Projects finished approximately 35% faster (than LB)6. 98% were completed on time7. PIPS contractors were approximately two times more productive8. In last 4 years, there has been no roof leaks9. PIPS was 14% cheaper than low-bid
78
79
City of Peoria (2004-2009)
• Number of Best-Value Procurements: 55
• Estimated Budget: $389 Million– (DB/CMAR/JOC)– (Wastewater, Office Buildings, Fire Station, Parks, Roadway)– (AE Services, Radio, Maintenance, Software)
• Average Number of Proposals per Project: 6
• Number of Completed Projects: 10 ($193 Million)– Overall C/O Rate: 0.5% (compared to 14%)– Overall Delay Rate: 6% (compared to 35%)– Final Results: 93% Satisfaction (compared to 20%)– Final Results: 9.1 Rating (10 max)
– 5 Projects where money returned
ASU Dining
CriteriaYear 1(From
Incumbent)Year 2
(From Year 1)Year 3
(From Year 2)Year 4
(From Year 3)
Sales 14% Increase 11% Increase 24% Increase 13.5% Increase
Commission 23% Increase 6% Increase 20% Increase 22% Increase
ASU Management Requirement Reduced 79% -- -- --
Student Satisfaction 37% Increase 1% Decrease 9% Increase 3% Increase
• 16 year• Four campuses• $1 Billion
81
• Award Analysis:– Number of Best-Value Procurements: 161– Awarded Cost: $50.6M (11% below average cost)– Average Number of Proposals: 4– Projects Where Best-Value was also Lowest Cost: 53%– 85% of projects were awarded to vendor with highest / second highest
RAVA Plan (7.3 vs 5.9)
• Performance Information:– Contractor Impacts: 0% Change Orders / 4% Delay– Vendor post project rating: 9.6– Average Contractor Increase in Profit: 5%– UMN PM management requirements: -65%
University of Minnesota
Traditional History of UMN: ― 32% projects on schedule― 32% projects on budget― $17 million in claims
University of Alberta
82
Project Value Cost Savings
Schedule Impacts
Satisfaction / Performance
1. Custodial Services (campus-wide)
$18M $2M10%
5.5% performanceImprovement
10 (out of 10)
2. DB Construction (Research Facility)
$30M $8-12M25%
14-18 months 9.7 (out of 10)
3. Design Services (Building Redevelopment)
$4M $500k12%
0% Cost & Schedule CO’s
$190k in Value Added Options
83
Key Takeaways• Change in thinking is the most critical (and most difficult)
• Get executive support
• Minimize decision making
• Stick to the process
• Get educated
Comments / Questions
84
W W W . P B S R G . C O M