Presentation is loading. Please wait.

Presentation is loading. Please wait.

Copyright  2013 CIFE TAC 2013 Statistical Analysis of KPIs: the Missing Link in the VDC Decision-making Process Calvin Kam (PI)- Civil and Environmental.

Similar presentations


Presentation on theme: "Copyright  2013 CIFE TAC 2013 Statistical Analysis of KPIs: the Missing Link in the VDC Decision-making Process Calvin Kam (PI)- Civil and Environmental."— Presentation transcript:

1 Copyright  2013 CIFE TAC 2013 Statistical Analysis of KPIs: the Missing Link in the VDC Decision-making Process Calvin Kam (PI)- Civil and Environmental Engineering Martin Fischer (PI)- Civil and Environmental Engineering Sadri Khalessi (PI)- Statistics Devini Senaratna (RA)- Statistics

2 Copyright  2013 CIFE TAC 2013 Statistical Analysis of KPIs: the Missing Link in the VDC Decision-making Process 2 A set of statistical models for innovative: benchmarking, prediction of performance and adoption, decision-prioritization and in Virtual Design and Construction projects

3 Copyright  2013 CIFE TAC 2013 Statistical Analysis of KPIs: the Missing Link in the VDC Decision-making Process Problem Statement Why?

4 Copyright  2013 CIFE TAC 2013 Statistical Analysis of KPIs: the Missing Link in the VDC Decision-making Process “What Gets Measured, Gets Done” –Peter Drucker What do AEC firms need to “Get Done” using VDC? 4 Minimal waste in the process Shorter total duration Tight synchronization between design & fabrication Last responsible moment benefit from VDC More alternatives evaluated earlier

5 Copyright  2013 CIFE TAC 2013 Statistical Analysis of KPIs: the Missing Link in the VDC Decision-making Process Ref: McGraw and Hill (2012): The Business Value of BIM AEC may not see the value in KPIs for VDC 5 Ref: Kam, McKinney, Xiao & Senaratna (2013): The Formulation and Validation of the VDC Scorecard

6 Copyright  2013 CIFE TAC 2013 Statistical Analysis of KPIs: the Missing Link in the VDC Decision-making Process Apparel industries: 80%VDC in AEC: 40% AEC firms monitor KPIs for VDC less than other industries Ref: Esin, et al. (2009) Are KPIs and Benchmarking actively used among organizations of the Swiss apparel industry to assure revenue? 6 Ref: Kam, McKinney, Xiao & Senaratna (2013): The Formulation and Validation of the VDC Scorecard

7 Copyright  2013 CIFE TAC 2013 Statistical Analysis of KPIs: the Missing Link in the VDC Decision-making Process What information do AEC firms want from the KPIs?

8 Copyright  2013 CIFE TAC 2013 Statistical Analysis of KPIs: the Missing Link in the VDC Decision-making Process Frequency of Integrated Meetings (Monthly) Frequency of Integrated Meetings (Monthly) Extent of Stakeholder Involvement (% of Stakeholder involved in VDC) Extent of Stakeholder Involvement (% of Stakeholder involved in VDC) AEC project teams want to know where they stand relative to other AEC project teams Where do we stand relative to other projects? One data-driven benchmark that combines KPIs? 8

9 Copyright  2013 CIFE TAC 2013 Statistical Analysis of KPIs: the Missing Link in the VDC Decision-making Process Frequency of Integrated Meetings Frequency of Integrated Meetings Extent of Stakeholder Involvement Extent of Stakeholder Involvement AEC project teams want to quantify and understand interdependencies among KPIs for VDC How much more involved would the stakeholders be? How much more frequently would the stakeholders attend the integrated meetings 9

10 Copyright  2013 CIFE TAC 2013 Statistical Analysis of KPIs: the Missing Link in the VDC Decision-making Process Our on-going research using the VDC Scorecard Tool Ref: Kam, McKinney, Xiao & Senaratna (2013) A survey based benchmarking tool cases and counting 50+indicators 13 countries 11 types of facilities More cases for statistical modeling Adding information to existing cases Data-driven benchmarks Data-driven prioritization of KPIs More cases for statistical modeling Adding information to existing cases Data-driven benchmarks Data-driven prioritization of KPIs What more do we need?

11 Copyright  2013 CIFE TAC 2013 Statistical Analysis of KPIs: the Missing Link in the VDC Decision-making Process 10 Divisions VDC Scorecard PlanningAdoptionPerformanceTechnology 4 Areas Objective Organization QualityQuantity Integration Preparation Maturity Standard Coverage Process Documentatio n Stakeholders Reduced Design Errors Stakeholder Survey Communicatio n Budget Depth Guideline Level of Detail Project MetricTraining Reduced Labor Post Occupancy S. Interoper- ability Tool Breadth Benchmark Model Use Life Cycle Broader Context Etc. 50+ Measures Etc. 1 Score 11 VDC Scorecard measures many types of KPIs

12 Copyright  2013 CIFE TAC 2013 Statistical Analysis of KPIs: the Missing Link in the VDC Decision-making Process 12 Our research shows many interesting interdependencies between KPIs Which KPI impacts VDC performance most? Stakeholders involved in VDC decision-making* Percentage of funds allocated for VDC** Percentage of funds allocated for VDC** Efficient technological use of VDC High Correlation No Correlation 6 Area Combinations 45 Division Combinations 1542 Measure Combinations P-values: * ; **0.2

13 Copyright  2013 CIFE TAC 2013 Statistical Analysis of KPIs: the Missing Link in the VDC Decision-making Process Decisions based on anecdotal learning’s only won't be as strong as making decisions that are also supported by statistically established insights Breadth Involvement Model Use Life Cycle Depth Reduced Design Errors Stakeholder satisfaction Reduced Labor Safety Stakeholders Project Meetings Benchmark Measure Combinations

14 Copyright  2013 CIFE TAC 2013 Statistical Analysis of KPIs: the Missing Link in the VDC Decision-making Process Raw Data (KPIs) “What gets measured, gets done “ How can AEC firms benefit from this advantage ? Information = Statistical analysis of KPIs + VDC expert opinion 14

15 Copyright  2013 CIFE TAC 2013 Statistical Analysis of KPIs: the Missing Link in the VDC Decision-making Process What we propose What?

16 Copyright  2013 CIFE TAC 2013 Statistical Analysis of KPIs: the Missing Link in the VDC Decision-making Process Ref: neural.cs.nthu.edu.tw We will benchmark with a variant of Principal Component Analysis (PCA) Few Features 16 Many KPIs Few KPIs for benchmarking and ranking Many Features Picks up the most important signals

17 Copyright  2013 CIFE TAC 2013 Statistical Analysis of KPIs: the Missing Link in the VDC Decision-making Process 17 Clustering to find features about “projects like mine” Features of projects that are similar to mine Features of projects that are different to mine

18 Copyright  2013 CIFE TAC 2013 Statistical Analysis of KPIs: the Missing Link in the VDC Decision-making Process Clustering groups of projects with similar KPIs together Used in Recommendation Engines Methods: K-means, Hierarchical, Two-Step Clustering Projects with similar features will group together Feature Combination Score 1 Feature Combination Score 2 Numerical statistical measures built using a collection of KPIs 18

19 Copyright  2013 CIFE TAC 2013 Statistical Analysis of KPIs: the Missing Link in the VDC Decision-making Process Observation for Cluster Groups 1 and 3: Performance Adoption Common Drawbacks : 1.Interoperability problems 2.Inconsistent Level of Detail for model uses 3. Only native formats used for model exchange 19 GOOD Technological Coverage but POOR Technological Integration

20 Copyright  2013 CIFE TAC 2013 Statistical Analysis of KPIs: the Missing Link in the VDC Decision-making Process Statistical predictive models to identify interdependencies  Factor Analysis  Structural Equation Modeling  Classification Trees  Canonical Correlations  Correspondence Analysis  Factor Analysis  Structural Equation Modeling  Classification Trees  Canonical Correlations  Correspondence Analysis For Prediction and Prioritization For Prediction and Prioritization 20

21 Copyright  2013 CIFE TAC 2013 Statistical Analysis of KPIs: the Missing Link in the VDC Decision-making Process Structural Equation Modeling will model interdependencies Ref: Lee, Kleinman, (2003): Statistical choices and apparent work outcomes in auditing Example of a Structural Equation model for Job Dis- Satisfaction 21

22 Copyright  2013 CIFE TAC 2013 Statistical Analysis of KPIs: the Missing Link in the VDC Decision-making Process How are we going to do it? How? 22

23 Copyright  2013 CIFE TAC 2013 Statistical Analysis of KPIs: the Missing Link in the VDC Decision-making Process Preliminary Work (Aug 2013) Survey Design (Sep-Oct 2013) Data Preparation (Nov 2013) Model Building (Dec-Feb 2013) Validation and Refining (Feb 2013) Timetable 23  Addition of KPIs to Scorecard  Testing of statistical methods on sample data  Surveying108 cases (Online – Web survey)  Data cleaning and cross checking  Organizing the data for analysis  Descriptive Analysis and assumption checks  Statistical model based on SEM, FA, CHAID  Test data  Using expert opinions

24 Copyright  2013 CIFE TAC 2013 Statistical Analysis of KPIs: the Missing Link in the VDC Decision-making Process Dissemination of Findings 24 Two Industry Workshops (Kam, Khalessi, Fischer) November CIFE workshop February CIFE workshop Contributions towards: Statistical KPI wiki for AEC CIFE members (AEC firms) National BIM Standards and McGraw Hill Smart Market Report Publications New findings to be posted on vdcscorecard.stanford.edu Journal paper publication

25 Copyright  2013 CIFE TAC 2013 Statistical Analysis of KPIs: the Missing Link in the VDC Decision-making Process The data-flow process of the proposed study AEC Professionals VDC Scorecard Model Analyses KPIs and their Correlations Prioritized, Customized feedback based on KPIs Statistical Model 25

26 Copyright  2013 CIFE TAC 2013 Statistical Analysis of KPIs: the Missing Link in the VDC Decision-making Process Goals Long Term Goal: Automation of this process for faster decision-making; and to make decision-making more consistent across an organization Short Term Goal: Statistical analysis of KPIs for VDC decision-making 26

27 Copyright  2013 CIFE TAC 2013 Statistical Analysis of KPIs: the Missing Link in the VDC Decision-making Process References Abeysekera, S. (2006) Multivariate methods for index construction Alarcón, L., Mourgues, C., O’Ryan, C., and Fischer, M. (2010) Designing a Benchmarking Platform to select VDC/BIM Implementation Strategies Bloom, N. and Van Reenen, J. (2006) Measuring and explaining management practices across firms and countries Bloom, N. and Van Reenen, J. (2007) Why do management practices differ across firms and countries? Esin, C., Von Bergen, M., and Wüthrich, R. (2009) Are KPIs and Benchmarking actively used among organisations of the Swiss apparel industry to assure revenue? Kam, C., McKinney, B., Xiao, Y., Senaratna, D. (2013) Kam, C., Xiao, Y., McKinney, B., Senaratna, D. (2013) McGraw and Hill (2012) The Business Value of BIM. Lee, Kleinman, (2003): Statistical choices and apparent work outcomes in auditing Li, W., Fischer, M., Schwegler, B., Bloom, N., Van Reenen, J. (2012) Zevenbergen, J., Gerry, J., and Buckbee, G., (2006) Automation KPIs Critical for Improvement of Enterprise KPIs. 27

28 Copyright  2013 CIFE TAC 2013 Statistical Analysis of KPIs: the Missing Link in the VDC Decision-making Process Don't you want to find out about which VDC practices will give you the best results? 28

29 Copyright  2013 CIFE TAC 2013 Statistical Analysis of KPIs: the Missing Link in the VDC Decision-making Process Thank You! 29


Download ppt "Copyright  2013 CIFE TAC 2013 Statistical Analysis of KPIs: the Missing Link in the VDC Decision-making Process Calvin Kam (PI)- Civil and Environmental."

Similar presentations


Ads by Google