Presentation on theme: "Oman Academic Accreditation Authority (OAAA)"— Presentation transcript:
1Oman Academic Accreditation Authority (OAAA) Briefing on Institutional and Program Standards Assessment: Approach to decision-making Dr Salim Radhawi 11 March 2014
2Briefing Outline Evolution of Institutional and Program Accreditation OAAAEvolution of Institutional and Program AccreditationHEI and Program QA processesProject aims and outcomesFeatures of standards and assessment processesConsultation process and feedbackOrganisation of standards, ratings and outcomesPublic Reporting and HEI comparabilityFees for Standards Assessments
3Evolution of Institutional and Program Accreditation (1) OAAAHEI and program accreditation are key parts of OAAA’s mandate (Royal Decree 54/2010)First set of institutional and program standards were published in ROSQA in 2004ROSQA standards used in 2004/2006 (two HEIs went through institutional and program accreditation).A review of ROSQA was carried out in 2006; this was followed by the development of the Quality Plan
4Evolution of Institutional and Program Accreditation (2) OAAAA new two-stage approach was proposed for HEI accreditation (Quality Audit and Standards Assessment)The first Quality Audits were carried out in 2008A separate process for program accreditation was proposedThe Institutional Standards Assessment project was originally launched in 2011The current stage of the project, which includes both Institutional and Program Standards Assessments, commenced in 2013
5HEI QA Processes OAAA HEI Licensure HEI Accreditation Stage 1 : QualityAuditFirst cyclecommenced20084 years≤4 yearsHEI AccreditationCertificateHEI AccreditationStage2:StandardsAssessmentAppealMetMetNot met1-2 years onProbationProcessDocumentStart/EndKEYHEI StandardsReassessmentHEIAccreditationTerminated5
7Projects aims and outcomes OAAAProjects aims and outcomesTo review and reviseHEI standards; program standards; and accreditation processes as set out in ROSQA to improve their relevance to the Omani context and to reflect current regional and international best practice (through benchmarking)To produceInstitutional and program Conceptual Design Frameworks (CDFs) with clear approach to decision-makinga revised set of HEI standards and a revised set of generic standards for programsInstitutional and Program Standards Assessment Manuals which include guidelines for HEIs and external reviewersTraining workshops for the sector
8Features of HEI and Program Accreditation OAAAFeatures of HEI and Program AccreditationRecognition that responsibility for quality lies with HEIsBased on ROSQA but incorporates criteria not included in ROSQA, e.g. academic integrityAlign with the nine areas of the scope of the QAM and reflect quality audit findingsInternationally benchmarked and reviewedAcknowledge the diversity of HE provision in OmanEncourage excellence – HEIs can receive Accredited with Merit overall or at standard(s) levelEncourages the use of ADRIDeveloped through a consultative process
9Features of the Institutional Standards Assessment process OAAAFeatures of the Institutional Standards Assessment processThe process looks at all areas of activity within an HEI and considers the quality systems applied to programsNational institutional accreditation is compulsory for all HEIsTakes into consideration the HEI’s response to formal conclusions in Quality Audit ReportNational schedule based on audit schedule with some flexibility
10Features of the Program Standards Assessment process OAAAFeatures of the Program Standards Assessment processThe program is the unit of analysis but considers the impact of institutional-level activitiesNational program accreditation is compulsory for all HEI programsAccreditation is for Omani programs as well as programs developed by overseas HEIs and/or with external accreditationProgram accreditation can be applied for once a cohort of students has graduated
11Consultation ProcessOAAAFormation of a national Consultative Committee to facilitate comprehensive consultation with the HE sectorFormation of an external panel of experts to provide an international perspectiveWorking in partnership with MoHEDialogue with external stakeholders (including Education Council, MoM, MoH, MoD, and MoE)Publication of drafts on OAAA website with discussion boardNational Symposium October 2013 (to which CC, stakeholders, professional body, employer and student representatives were invited)
13Response to Feedback: Organisation of the Standards OAAAResponse to Feedback: Organisation of the StandardsA considerable amount of feedback highlighted the large number of elements to be assessedThere are nine standardsThere are 75 criteriaEach criterion has a number of indicatorsIndicators no longer need to be met for a criterion to be met – indicators are guidelinesRating performance and commentary against indicators is no longer requiredHEIs may provide other evidence to show that the requirements of a criterion has been met
14Response to Feedback: Approach to Decision-making OAAAResponse to Feedback: Approach to Decision-makingSignificant feedback was provided on the rating of ‘not applicable’ (NA) with a numerical value (‘0’)No value is now given to NAFeedback suggested that the rating scheme was too subjective and difficult to applyClearer definitions have been provided for each criterion ratingDescriptions/characteristics will assist HEIs/reviewers in determining the most appropriate rating for each applicable criterion and standard
15Positive Feedback It was real OAAAPositive Feedback[The CDFs] are in many respects excellent, so congratulations to you all.Prof Malcolm Cook, Former Pro-VC, University of Exeter, UKIn my view the Conceptual Design Framework is in very good shape.Prof Ulrich Hommel, Director, EBS Business School, GermanyIt was realI think you should submit both the policies/standards and processes for international recognition.Dr Mike Hillyard, Former President of the University of St Augustine, USAIt was a pleasure for me to join your national symposium. I was really impressed with the approach OAAA has taken and the way participants got engaged in it.Prof Badr Abu Ela, Executive Director, CAA, UAE
16Previous and Current Approach to Organisation of the Standards OAAAPrevious and Current Approach to Organisation of the StandardsPrevious ApproachCurrent ApproachIndicators are essential components of each criterionIndicators are guidelines for the type of evidence HEIs/programs might submit to demonstrate good provision or practiceIndicators need to be met in order to meet the criterion, and for the criterion to meet the standardHEIs/programs can submit evidence based on other indicators to demonstrate how a criterion has been metEach indicator is given a rating which informs the overall rating of the criterionNo rating is given to an indicatorCommentary is provided by HEIs/programs regarding how each indicator has been metNo commentary is required at indicator level
17Comparison of Previous and Current Criterion Rating Scheme OAAAComparison of Previous and Current Criterion Rating Scheme#Previous RatingCurrent Rating4Worthy of MeritExcellent*3Good2Satisfactory1UnsatisfactoryNot met*Not ApplicableNA* These terms were revised following further benchmarking and in order to improve clarity of understanding
18ADRI as a Review Tool OAAA IMPROVEMENT APPROACH RESULTS DEPLOYMENT Internal ADRI ReviewFollowed by External Review
19Criterion Rating Definitions OAAACriterion Rating Definitions#RatingDefinition/characteristics4ExcellentDefinition:Provision or practice is consistently very high quality, and is underpinned by effective quality improvement arrangements.Characteristics:Provision or practice exceeds the requirements of the criterion.Provision or practice is highly effective, and is undertaken consistently; it may be a model of good practice.Provision or practice is exceptional rather than typical of other comparable HEIs/programs.Provision or practice incorporates systematic and effective quality improvement arrangements.
20Example of ‘Excellent’ OAAAExample of ‘Excellent’‘Excellent’ rating can be compared to ‘Commendation’ in Quality Audit ReportsA revised innovative academic advising system was successfully implemented and has been shown to have had a significant impact on retention and identifying ‘at risk’ students; the system has received significant positive feedback, been periodically reviewed for effectiveness, indicating continuous improvementRating against criterion 6.5
21Criterion Rating Definitions OAAACriterion Rating Definitions#RatingDefinition/characteristics3GoodDefinition:Provision or practice is consistently high quality in most areas and is underpinned by effective quality improvement arrangements.Characteristics:Overall, provision or practice exceeds the requirements of the criterion.Overall, provision or practice is high quality; is undertaken consistently; and meets the norms for good practice.Provision or practice incorporates effective quality improvement arrangements.
22OAAAExample of ‘Good’‘Good’ rating can be compared to positive text in Quality Audit ReportsAs part of its continuous improvement system, the HEI has introduced a revised academic advising system in response to feedback; there is a comprehensive handbook and training for staff and students which has been consistently implemented; the system has had a positive impact; and the HEI has clear plans for how the system will be evaluatedRating against criterion 6.5
23Criterion Rating Definitions OAAACriterion Rating Definitions#RatingDefinition/characteristics2SatisfactoryDefinition:Provision or practice is effective most of the time, and is underpinned by adequate quality improvement arrangements.Characteristics:Provision or practice meets the requirements of the criterion, and is effective most of the timeProvision or practice is underpinned by adequate quality improvement arrangements, which are effective most of the time.
24Example of ‘Satisfactory’ OAAAExample of ‘Satisfactory’‘Satisfactory’ rating means that the HEI/program has shown that it meets the requirements of the criterionThe HEI has implemented an effective formal academic advisory system which, overall, supports students in meeting their educational goals; the system has been evaluated and improvement plans have been implemented in most departmentsRating against criterion 6.5
25Criterion Rating Definitions OAAACriterion Rating Definitions#RatingDefinition/characteristics1Not metDefinition:Provision or practice does not meet the requirements of the criterion.Characteristics:Provision or practice lacks effective quality improvement arrangementsNANot ApplicableAn HEI will be expected to justify why the standard or criterion is not applicable to the institution/program.
26OAAAExample of ‘Not met’‘Not Met’ rating could be compared to an Affirmation or Recommendation in Quality Audit Reports.While the HEI has a policy for academic advising, the system has not been implemented effectively throughout the HEI and has not been monitored or evaluated for its effectivenessRating against criterion 6.5
27Examples of ‘Not applicable’ OAAAExamples of ‘Not applicable’‘Not Applicable’ rating needs to be justified by the HEI/programFrom XX HEI:Criteria 3.1 – 3.6 do not apply to XX HEI as it is classified as a College and does not run Student Learning by Research Programs(see SM001 HEI license approval)
28Criteria ratings towards meeting the standard OAAACriteria ratings towards meeting the standardDescriptionStandard RatingMost* of the criteria are rated as ‘Excellent’; no criteria are rated as ‘Not Met’.ExcellentMost of the criteria are rated as ‘Good’; no criteria are rated as ‘Not Met’.GoodMost criteria are rated satisfactory; no criteria are rated ‘Not Met’.SatisfactoryOne or more criteria are rated ‘Not Met’Not met* ‘Most’ in this context means more than 50%
29How Standards Ratings inform the Assessment Outcome OAAAHow Standards Ratings inform the Assessment OutcomeStandardAssessment OutcomeMost of the standards are rated ‘Excellent’Accredited with MeritAll standards are rated, as a minimum, ‘Satisfactory’, and one or more standards are rated as ‘Excellent’Accredited with Merit in one or more standard(s)All standards are rated either ‘Satisfactory’ or ‘GoodAccreditedOne or more of the standards are rated as ‘Not Met’On ProbationOne or more of the standards are rated as ‘Not Met’ by the Standards Reassessment PanelNot accredited
30Standards Assessment Outcome: Outcome Deferred OAAAStandards Assessment Outcome: Outcome DeferredAccreditation OutcomesDescriptionOutcome DeferredWhere action(s) by the HEI is required to meet a standard (s), and where the Standards Assessment Panel determines that the HEI can implement the action(s) is a short time period (up to three months), the OAAA may grant the HEI a period of time to demonstrate that it has addressed these outstanding issues, and that it meets the standard. An accreditation outcome decision will not be made public during this time, and this stage will be considered part of the accreditation process. Once the OAAA is satisfied that the standard(s) has been fully met, the HEI/program will be accredited, and the outcome made public.If the standard is not met within the designated timeframe, the HEI/program will be placed on probation, and the outcome made public.
31Status of HEIs and Programs on OAAA website LicensedAudited (for HEIs only)On probationAccreditedAccredited with MeritAccredited with Merit in one or more standardsNot accredited (after Standards Reassessment)
32Public Reporting and Comparability OAAAPublic Reporting and ComparabilityAccreditation outcomes will be published on OAAA website, with ratings against each standard and criterionA 4-point rating scale provides a transparent means for stakeholders to identify/compare how an HEI/program has performedStakeholders will be able to apply their own priorities in order to identify the HEI/program which meets their needsThis approach to public reporting avoids institutional/program ‘league tables’
33Stakeholder Comparison of HEIs OAAAStakeholder Comparison of HEIsOverall scoreHEIs
34Stakeholder Comparison of Programs OAAAStakeholder Comparison of ProgramsStakeholders input their own weightage for every criterionOverall scorePrograms
35Fees for Standards Assessments OAAAFees for Standards AssessmentsA proposal for fees for Institutional Standards Assessment has been submitted to the Ministry of FinanceThe fee will be calculated on the HEI’s Classification and number of FTE studentsThe fee structure is based on international benchmarkingFees for Program Standards Assessment are yet to be determined