Presentation is loading. Please wait.

Presentation is loading. Please wait.

Research Measurement for Realizing Quality & Excellence in Universities 24 April 2012, Majmaah University, KSA Deanship of Quality and Skills Development.

Similar presentations


Presentation on theme: "Research Measurement for Realizing Quality & Excellence in Universities 24 April 2012, Majmaah University, KSA Deanship of Quality and Skills Development."— Presentation transcript:

1 Research Measurement for Realizing Quality & Excellence in Universities 24 April 2012, Majmaah University, KSA Deanship of Quality and Skills Development Seminar Tariq A. Aldowaisan, Ph.D., CSP, CQM/OE, CQE, CQA Associate Professor, Kuwait University (KU) MD, Gulf Lead Consultants (GLC) قياس النشاط البحثي خطوة أساسية لتحقيق الجودة والتميز في جامعاتنا

2 Kuwait University & Gulf Lead Consultants, All Rights Reserved. 1.Provide an overview of the meaning of quality; the relationship between quality and excellence; and their applications in the university context. 2.Recognize the importance of performance measurement in driving performance improvement. 3.Suggest the four-dimensional OECD performance measurement system, and give examples of its application in the research activity. Seminar Objectives 2

3 Kuwait University & Gulf Lead Consultants, All Rights Reserved. 1.The meaning of quality 2.Quality obligations of universities 3.From quality to excellence 4.Importance of performance measurement 5.Performance measurement systems: Stakeholders, BSC, & OECD. 6.Applications of the OECD Model in research activity 7.Concluding recommendations on performance measurement in research 8.Overcoming implementation challenges Agenda 3

4 Kuwait University & Gulf Lead Consultants, All Rights Reserved. Quality is customer satisfaction through: Capture of customer requirements – Quality of Design Compliance with customer requirements – Quality of Conformance What is Quality? 4 Who are our customers?

5 Kuwait University & Gulf Lead Consultants, All Rights Reserved. What is Quality? 5 Quality is to identify and encode customer requirements through STANDARD/SPECIFICATIONS, and then achieve customer satisfaction by complying with STANDARD. Diameter 12 ± 0.30 mm 11.7 mm12.3 mm Defective Payment not to exceed 30 days 30 days Defective CUSTOMERCUSTOMER REQUIREMENTS Hard Standards in Production

6 Kuwait University & Gulf Lead Consultants, All Rights Reserved. What is Quality? 6 Hard Standards in Service

7 Kuwait University & Gulf Lead Consultants, All Rights Reserved. What is Quality? 7 Continual Improvement of the QMS Input Output value adding activities info. flow CUSTOMER REQUIREMENTS CUSTOMER SATISFACTION management responsibility management responsibility product realization product realization resource management resource management meas, analysis improvement meas, analysis improvement product clause 4 clause 5 clause 6 clause 8 clause 7 value adding activities Management Systems – Soft Standards

8 Kuwait University & Gulf Lead Consultants, All Rights Reserved. Examples from ISO 9001:2008 Quality Management System Standard: – Customer satisfaction As one of the measurement of the performance of the quality management system, the organization shall monitor information relating to customer perception as to whether the organization has met customer requirement. – Monitoring and measurement of product The organization shall monitor and measure the characteristics of the product to verify that product requirements have been met. Management Standards 8

9 Kuwait University & Gulf Lead Consultants, All Rights Reserved. Hard & Soft Standards 9 Comparison HardSoft TypeTechnicalManagement LanguageSpecificationRequirements/Criteria CodingNumeralShall/Must/Is/Are CheckingInspectionAudit Difficulty of CheckingLowHigh ExampleBSI-999, CSE, ISO ISO 9001, CSE, ISO 17025, ABET

10 Kuwait University & Gulf Lead Consultants, All Rights Reserved. Basic Quality Obligation Basic Quality Obligation graduates abilities are aligned with the markets current and potential needs and expectations. National Quality Obligation National Quality Obligation contribute to the development of their nations. Quality Obligations of Universities 10 Source: Aldowaisan, T. Quality Obligations and Newly Established Universities, Quality Beat, 1 st Issue October 2011.

11 Kuwait University & Gulf Lead Consultants, All Rights Reserved. 1.Alignment of the universitys strategic plan with the national plan. 2.Selection of a robust performance measurement system. 3.Choice and development of faculty members. Quality Obligations of Universities 11 Quality Controls Source: Aldowaisan, T. Quality Obligations and Newly Established Universities, Quality Beat, 1 st Issue October 2011.

12 Kuwait University & Gulf Lead Consultants, All Rights Reserved. From Quality to Excellence 12 Customer Focus Organization Focus Meet Requirements Exceed Requirements Good operation (e.g. 3 ) Good operation (e.g. 3 ) Great operation (e.g. > 6 ) Great operation (e.g. > 6 ) Customer Results Customer Results Stakeholder Results Stakeholder Results Employers Research Community Society Faculty & Staff Students Ranking Bodies Accredit Rank Accreditation Bodies Satisfy Delight

13 Kuwait University & Gulf Lead Consultants, All Rights Reserved. BNQP – Performance Excellence Program 13 1 Leadership 3 Customer Focus 7 Results 4 Measurement, Analysis, and Knowledge Management 2 Strategic Planning 6 Operations Focus 5 Workforce Focus Organizational Profile: Environment, Relationships, and Strategic Situation Education Criteria for Performance Excellence Education Criteria for Performance Excellence Framework

14 Kuwait University & Gulf Lead Consultants, All Rights Reserved. BNQP – Performance Excellence Program 14 1 Leadership 3 Customer Focus 7 Results 4 Measurement, Analysis, and Knowledge Management 2 Strategic Planning 6 Operations Focus 5 Workforce Focus Organizational Profile: Environment, Relationships, and Strategic Situation Education Criteria for Performance Excellence Importance of Performance Measurement

15 Kuwait University & Gulf Lead Consultants, All Rights Reserved. ISO 9001 QMS 15 Continual Improvement of the QMS Input Output value adding activities info. flow CUSTOMER REQUIREMENTS CUSTOMER SATISFACTION management responsibility management responsibility product realization product realization resource management resource management meas, analysis improvement meas, analysis improvement product clause 4 clause 5 clause 6 clause 8 clause 7 value adding activities Importance of Performance Measurement

16 Kuwait University & Gulf Lead Consultants, All Rights Reserved. Excellence Drivers 16 1 Leadership 3 Customer Focus 7 Results 4 Measurement, Analysis, and Knowledge Management 2 Strategic Planning 6 Operations Focus 5 Workforce Focus Organizational Profile: Environment, Relationships, and Strategic Situation Improvement Team Education Criteria for Performance Excellence Framework

17 Kuwait University & Gulf Lead Consultants, All Rights Reserved. Leadership Leadership – a priori, – 2 nd weight (120) Results Results – clarity of target, – 1 st weight (450) Improvement Team Improvement Team – Spearhead change – Link leadership & Results Triads Determiners of Deployment Effectiveness 17

18 Kuwait University & Gulf Lead Consultants, All Rights Reserved. Results – Get it Right 18 Encompassing An Encompassing Vision Times Higher Education QS World University Ranking Academic Ranking of World Universities US News World Best Universities etc. Achieving success in our universities,

19 Kuwait University & Gulf Lead Consultants, All Rights Reserved. Results – Get it Right 19 Real Real Goals – Outcome based Research Teaching Community Development

20 Kuwait University & Gulf Lead Consultants, All Rights Reserved. Performance Measurement Systems 20 Stakeholders Perspectives PMs

21 Kuwait University & Gulf Lead Consultants, All Rights Reserved. Performance Measurement Systems 21 Balanced Scorecard (BSC) Source: SmartDraw 2010 PMs

22 Kuwait University & Gulf Lead Consultants, All Rights Reserved. Performance Measurement Systems 22 OECD Model – Activity Based Source: Measuring Government Activity, OECD 2009 Inputs (Resources) Activities Intermediate Outcomes Final Outcomes Outputs organization structure and management provisions Context

23 Kuwait University & Gulf Lead Consultants, All Rights Reserved. Performance Measurement Systems 23 OECD Model Inputs (Resources) Activities Intermediate Outcomes Final Outcomes Outputs Efficiency ProductivityQuality Customer National/ University Objectives Effectiveness

24 Kuwait University & Gulf Lead Consultants, All Rights Reserved. OECD Performance Measures Dimensions 24 Efficiency: measures how well resources are utilized to produce the desired output (resource/input-driven). Productivity: measures the capacity to produce the maximal output possible for given resources (output-driven). Quality: measures how well the produced outputs meet or exceed relevant quality dimensions standards (output-focus). Effectiveness: measures how well the outputs and outcomes agree with the planned/targeted outputs and outcomes (outcome-focus) Doing Things RightDoing Right Things Inputs (Resources) Activities Intermediate Outcomes Final Outcomes Outputs organization structure and management provisions Context

25 Kuwait University & Gulf Lead Consultants, All Rights Reserved. Teaching – Undergraduate Teaching – Graduate Research Community Services; e.g. consultation, professional development, and testing. Key University Activities 25

26 Kuwait University & Gulf Lead Consultants, All Rights Reserved. The Research Activity 26 Projects Papers Patents Spinoffs Projects Papers Patents Spinoffs Research Activity Research Activity Budget Faculty Facilities Budget Faculty Facilities Country Priorities Research Community University Ranking Business Community

27 Kuwait University & Gulf Lead Consultants, All Rights Reserved. Efficiency: the cost of funded published papers (journal & conferences). The cost may be expanded to include the administration of the research activity. Productivity: the number of published funded and non-funded papers (journal & conferences) per faculty (all faculty). Of course more specific measures of productivity could be developed for journals and conferences separately. Quality: the percentage of papers (funded and non-funded) that are published in the top quartile of the Journal Citation Reports (JCR) Effectiveness – the compounded annual growth rate (CAGR) of papers (journal & conference) over a certain period (Immediate outcome). – the percentage of papers addressing national priorities (Intermediate outcome). – the percentage of papers utilized by market (Final outcome). Suggested Performance Measures for Research Activity 27

28 Kuwait University & Gulf Lead Consultants, All Rights Reserved. Performance Measures & Objectives 28 Research Investment Faculty members 25% address Priorities At least20% published in top quartile of JCR Quality 1.5paper per faculty member productivity 2000KD per paper Efficiency 10% utilized by market Note: 1.Papers include conference and journal papers as well as funded and non-funded. 2.Faculty members include all faculty members (active and non-active). 3.ForResearch Investment Efficiency, we consider funded papers only. 25% address priorities 30% CAGR Effectiveness Research Activity Papers

29 Kuwait University & Gulf Lead Consultants, All Rights Reserved. Using the PM to Drive Improvement 29 "You can't manage what you can't measure" __ Edwards Deming

30 Kuwait University & Gulf Lead Consultants, All Rights Reserved. Process/activity based More appropriate for non- profit institutions Readily supports ratio performance metrics Simple, efficient, robust Supports doing things right and doing the right things Advantages of OECD Model 30

31 Kuwait University & Gulf Lead Consultants, All Rights Reserved. Focus more on reporting outcome-related performances and their development over the year. Include non-funded published papers as well as conference papers to calculate faculty productivity (i.e. papers per faculty); which is an important measure of immediate outcome (i.e. output). Include a measure of research cost to calculate research investment efficiency (i.e. KD/paper). Include a priority effectiveness index related to research relevance to the priority areas (i.e. KD/KD or Papers/papers). Include a marketability effectiveness index related to marketability of research output (i.e. KD/KD or Papers/papers). Recommendations on Performance Measures in Research 31

32 Kuwait University & Gulf Lead Consultants, All Rights Reserved. Include commercial effectiveness indices on spinoff businesses and patents. Include a leading indicator faculty research engagement (i.e. percentage of faculty involved in research). Include a leading indicator on faculty collaboration index (KD of researches that involves two or more faculty of total research investment). Include a leading indicator on interdepartmental collaboration index (i.e. % of collaboration research projects involving collaboration from different departments). Include a leading indicator on large-scale interdisciplinary research collaboration (i.e. % of collaboration large-scale research projects involving different disciplines). reflect on the impact of the level and quality of research on the university rank; through engaging credible ranking bodies. Recommendations on Performance Measures in Research 32

33 Kuwait University & Gulf Lead Consultants, All Rights Reserved. No.Measure OECD Perspective BSC Perspective Stakeholder Perspective 1 Faculty productivity Productivity Internal Process faculty, research community, Academic Institution 2 Investment efficiency EfficiencyFinancialFunding agencies 3 Publications quality QualityCustomer Research community, faculty, Academic Institution 4 Publications growth Immediate outcome – Effectiveness Academic Institution 5 Priority effectiveness index Intermediate outcome – Effectiveness National and local community Cross-Check for Completeness 33

34 Kuwait University & Gulf Lead Consultants, All Rights Reserved. No.Measure OECD Perspective BSC Perspective Stakeholder Perspective 6 Marketability effectiveness index Final outcome – Effectiveness National and local community, Academic Institution 7 Faculty research engagement Learning & Growth Faculty, Academic Institution 8 Faculty collaboration index Learning & Growth Faculty, Academic Institution 9 Interdisciplinary collaboration index Learning & Growth Faculty, Academic Institution Cross-Check for Completeness 34

35 Kuwait University & Gulf Lead Consultants, All Rights Reserved. Ensure selected performance measures are collectively comprehensive. Ensure that the selected performance measures are mutually exclusive. Gain the benefits of each measurement framework. Benefits of Cross-Checking 35

36 Kuwait University & Gulf Lead Consultants, All Rights Reserved. Definition of a Performance Measure 36 Base Value Assumptions Data Sources Benchmarks Targets Beneficiaries Analyses Measurement Unit Frequency Formula Dimension Interpretation ReferenceGoal/Objective

37 Kuwait University & Gulf Lead Consultants, All Rights Reserved. Leadership Commitment Culture – Fear of accountability – Not meeting a target construed as a failure Competency building – General awareness starting at top – Deep knowledge & skills of improvement team to cover both measurement and improvement methods and tools Cumbersome to monitor – Automation is a must Implementation Challenges 37

38 Kuwait University & Gulf Lead Consultants, All Rights Reserved. كل الشكر والامتنان 38 طارق عبد المحسن الدويسان


Download ppt "Research Measurement for Realizing Quality & Excellence in Universities 24 April 2012, Majmaah University, KSA Deanship of Quality and Skills Development."

Similar presentations


Ads by Google