Let Quality Guide Quality Evaluation Recent Trends Implemented in the Middle East

An Empirical Approach to Evaluation for Higher Education Institutions: Opportunities in Saudi Arabia Ms. Tahira Hoke, Director Academic Assessment and Planning Center (AAPC) Dr. Eqbal Darandari, National Commission on Academic Assessment and Accreditation (NCAAA)

Overview Introduction Quality Management Survey Results Answer Questions Highlighted in Abstract 3

Intro: Prince Sultan University

Case Study: Quality Management in Saudi Arabian Higher Education Institutions 5

Case Study Context Participants Instrument Results Discussion Implications 6

7 Participants N= 20

Participants 8

 Public/Private  Small/Large  Established /New Quality Assurance Systems  High/Low Institutional Support for Quality Assurance 9

Part I: Quality Management Structure of Quality Assurance Center Use of Quality Assurance Data Evaluation Approaches Used for Quality Assurance

Part II: Performance Indicators Type Use Availability Quality Influence Awareness (across items)

Types of Performance Indicators (N=20) Note: No Participants Selected Quantitative Only

Selected Performance Indicators Standard Management of Quality Assurance Quality of Learning and Teaching Indicator Participation of Faculty in Quality Assurance Processes Quality of program and course reports

Selected Performance Indicators Standard Learning Resources Quality of Learning and Teaching Indicator Documents describing processes for identifying and responding to course requirements Results of User Surveys (e.g., Ratings by students, graduates and employers of the quality of programs)

Availability of Performance Indicators (N=20) Note: If participants left blank = no.

Influence of Performance Indicators (N=20)

Performance Indicator Comments “I need to know the key performance indicators first then I can say my recommendations.” (New Quality Assurance Center; Did Not Know if Quality Assurance Data Used). 17

Discussion: Limitations & Solutions Low Response Rate: Surveyed at the end of two-day workshop along with a workshop evaluation. 18 LimitationsSolutions Administration of Survey: Distribute in the middle of workshop.

Discussion: Limitations & Solutions Limited Knowledge of Target Population: About 50% target population not familiar enough with performance indicators to complete survey. 19 LimitationsSolutions Experienced Target Population: Target population should be only be quality assurance managers and/or committee members (i.e., involved at least one year)

Action Plan 1. Administer a revised survey to a specific audience to further guide future benchmarking studies in KSA. 2. Revise the capacity building model based on the data collect in this survey, our previous surveys, and current literature. 20

Action Plan 3. Organize Quality Assurance Leadership Symposium during the Spring Coordinate technical assistance teams to work closely with each university on major quality assurance issues. 21

Conclusion: Opportunities Evaluators: Provide workshops Provide technical assistance Collaborate on Research Projects 22

Questions & Answers 23

Closing Remarks 24

Does Accreditation Work? Hypothesis Testing with Performance Indicators? Quasi-Experiments across universities? 25

Does Accreditation Work? Hypothesis Testing with Performance Indicators? Quasi-Experiments across universities? 26

#1. Q/A: Accreditation Question Does an accreditation intervention really improve the quality of learning and teaching higher education in KSA? Answer Depends on the institutional commitment to support quality assurance and evaluation efforts.

#2. Q/A: Indicators Question What type of indicators would measure if quality assurance measures significantly increased an institution’s effectiveness? Answer Depends on the beneficiary (i.e., student experience, society).

#3. Q/A: Quantitative Methods Question Would introducing quantitative methods in post-evaluation procedures in KSA contribute to advancing the field of evaluation? Answer Depends on the availability of data, the development level (i.e., readiness) of the institution, education ministries, and people.

#4. Q/A: Quantitative Methods Question Would introducing quantitative methods in KSA attempt to standardize something that should not be standardized? Answer If the developmental approach is with the institutions and the higher education system is used, relative benchmarking exercises will be effective.

Questions & Answers 31

Quality Assurance Center Structure (N=20) 32

Quality Assurance Structure Comment Established “Standing committee under division in college/university. Under institutional research and advancement.” New “As a new university, our quality assurance center is just developing and still a lot to be done.” 33

Use of Quality Assurance Data (N=20) 34

Use of Quality Assurance Data Comment Used “Curriculum revision, new programs approval, surveys, factbook, analysis of data centralized and presented to appropriate division within the university for approach for action [and] for change.” Not Used “It is very poor, it is not used. “ 35

Contact Information Ms. Tahira Hoke 36