Presentation is loading. Please wait.

Presentation is loading. Please wait.

Accreditation Visit to

Similar presentations


Presentation on theme: "Accreditation Visit to"— Presentation transcript:

1 Accreditation Visit to
* 07/16/96 Accreditation Visit to <name of HEI> <date range> Note: HEI means Higher Education Institution, which includes Universities and Colleges. This phrase may also be abbreviated as “Institution” elsewhere in this presentation. Purpose of this presentation This template presentation, intended for use by Visit Chairs to brief the Visit Team at the visit kick-off meeting, has been annotated to indicate where additional information may be found. If this template presentation can be customized by the Chair with details specific to the visit (replacing red text in the first few pages) and circulated to the team members well in advance of the visit, the team members can use it as a guide to additional information, and the Chair can use it as a framework for review at the kick-off meeting. Sources referenced The sources referenced in this PowerPoint presentation are listed below with the internet pages where they can be found: CEAB Questionnaire (specific to institution, 50+ pages) Template: Completed by institution: access supplied to Visitors by Secretariat, 6 weeks prior to visit CEAB Accreditation Criteria and Procedures 20xx (87 pages) CEAB Manual of Accreditation Procedures (23 pages) CEAB Accreditation Site Visit Calendar of Events (4 pages) CEAB Visiting Team Report (specific to institution, 8+ pages for institution plus 23+ pages per program) Visiting Team Report Template: CEAB General Visitor Manual (14 pages) Online training for accreditation visit team members (several hours) (access to Visitors by Secretariat, 6 weeks prior to visit) General Visitor Report to the Constituent Association (template 2 pages) General Visitor – typical duties (1 page) General Visitor gets this directly from Secretariat staff Team Member Responsibilities (1 page) Available on the Information Exchange Site, as part of current online training program Travel Policy (6 pages) Expense Claim (1 page) Suggested Interview Questions for CEAB Visits (15 pages) Guideline Ethics Code (9 pages) Engineers Canada Code of Conduct for Volunteers (13 pages) under “Policies” Engineers Canada Process Monitoring (Whistle-Blowing) policy (3 pages) Ten Commandments for the Site Team (1 page) Engineers Canada Syllabus (length varies by program) Tracking of Program Issues: Working document form (5 pages) Sample Visit Schedule (2 pages) Standard Statement for the exit interview with the Dean (1 page) Statement is provided to Team Chairs by Secretariat Name Title *

2 Welcome! Outline of this presentation Background
* 07/16/96 Welcome! Outline of this presentation Background General notes on accreditation Goals of the CEAB Objectives of the visit Team’s responsibilities Tasks and tools Visit schedule Accreditation criteria highlights NEW! Graduate attribute criteria Potential issues Post-Visit Activities Visit-Specific Issues More information related to this slide: In planning their travel to and from the site, Visit Team members need to refer to the following documents: “Travel Policy” “Expense Claim” *

3 Background History and current situation Purpose of this visit
* 07/16/96 Background History and current situation <when HEI opened> <current student complement> <any major contextual factors> Purpose of this visit <list programs being visited> <indicate when these programs were most recently visited and what the CEAB decisions was> More information related to this slide Information shown on this slide is a summary. Full information about the institution and the program being visited is developed by the institution and supplied to team members 6 weeks in advance of the visit, in the “Questionnaire”. *

4 General notes about accreditation
* 07/16/96 General notes about accreditation Accreditation applies only to programs, not to departments or faculties Undertaken only at the invitation of the HEI and with the consent of the appropriate regulator Accreditation constitutes: Quantitative and qualitative evaluation of the curriculum Qualitative evaluation of the program environment Accreditation is granted for a period of time up to and normally not exceeding six years Four basic references related to this presentation There are four basic general references about the Canadian Engineering Accreditation Board and the accreditation process: Accreditation Criteria and Procedures 20xx (updated annually) Manual of Accreditation Procedures (updated occasionally) 3) Calendar of Events (updated annually) 4) Online training for accreditation visit team members (updated annually) Reviewing these four references will answer most general questions that a Visit Team Member will have. “Accreditation Criteria and Procedures” should be read first. “Manual of Accreditation Procedures” provides more detail on how visits are to be conducted. Tables of Contents are shown below. “Calendar of Events” places the Site Visit in the context of the overall Accreditation cycle, and helps focus the reader on timelines, deliverables and stages where their further input or review may be required. “Online training for accreditation visit team members” provides highlights of the other references, in an online training format, with the opportunity to test understanding using quizzes. Accreditation Criteria and Procedures 20xx Table of contents Accreditation Board members The role of the Accreditation Board Policy statement 1. Terms of reference Goals Expected outcomes Functions and responsibilities Status and authority Organization 2. Purpose of accreditation 3. Accreditation criteria Continual improvement Graduate attributes Students Curriculum content Program environment Accreditation procedures and application 4. Procedures Initiation and timing of accreditation visit Selection of visiting team Preparation for accreditation visit Accreditation visit Visiting team report Accreditation decision Formal review Informal evaluation or visit Publication Accredited engineering programs by institution Accredited engineering programs by program title Substantially equivalent programs International mutual recognition agreements Chairs, members and secretaries Constituent associations of Engineers Canada 5. Appendices Advanced standing, prior studies, exchange studies, and transfer credit regulations (effective January 2012: Regulations for granting transfer credits) Interpretive statement on natural sciences Statement of interpretation on licensure expectations and requirements Statement of interpretation on curriculum content for options and dual-discipline programs Use of the K-factor Confidentiality: policies & procedures Program development advisory procedure Procedures for formal review of an Accreditation Board decision to deny accreditation Manual of Accreditation Procedures 1. Foreword 2. Introduction 2.1 The role of the Canadian Engineering Accreditation Board 2.2 Terms of reference 2.3 Purpose and criteria for accreditation 3. Procedures 3.1 The visiting team 3.2 Visit preparation 3.3 The visit 3.4 Visiting team reports 3.5 Post-visit activities Appendix A: Calendar of accreditation visit events Introduction A.1 Fall visits A.2 Summer visits A.3 Winter visits A.4 Early re-visits Appendix B: Non-standard visits *

5 * 07/16/96 Goals of the CEAB Engineering programs offered by Canadian institutions will meet or exceed minimum educational standards acceptable for professional engineering licensure in Canada The quality and relevance of engineering education will continuously improve The Engineers Canada Board of Directors will be provided with advice and recommendations on international matters relating to engineering accreditation and education More information related to this slide These goals of the CEAB are listed in “Accreditation Criteria and Procedures 20xx”. *

6 Objectives of the visiting team
* 07/16/96 Objectives of the visiting team Conduct fact-finding on behalf of the Accreditation Board Review, validate and/or add to the information provided by the host institution Review of materials, meetings, and facility tours to corroborate program strengths and weaknesses and bring forward issues to the CEAB Describe progress toward use of graduate attributes in program assessment and improvement Collaborate in preparing a report of the team’s findings The visiting team or its members do not make any recommendations ~ accreditation decisions are made by the CEAB More information related to this slide This list of objectives of the visiting Team is provided in more detail in the “Accreditation Criteria and Procedures 20xx”, including the list of accreditation criteria grouped in the following categories: Continuous Improvement Graduate Attributes Students Curriculum Content Program Environment The template for the final report, the deliverable from the Visit, is “Visiting Team Report Template”. Each Program Visitor will complete their own report (or a consensus report when a program has more than one Visitor). Observations by General Visitor, Vice-Chair and Chair will be integrated into the Program Visitors’ reports as well as forming the prefatory and concluding material of the report. In addition, the General Visitor will prepare a report to the association/ordre having jurisdiction, documenting the process of the visit, using “General Visitor Report to the Constituent Association” as a template. *

7 Team’s responsibilities
* 07/16/96 Team’s responsibilities Thoroughly read the institution’s completed questionnaire Identify issues for investigation during the visit Attend the visit Participate in team discussions Complete your report (before the exit meeting!) Be available to answer questions after the visit and before the CEAB’s decision meeting More information related to this slide The team’s responsibilities, including the role of each team member, is described in detail in “Manual of Accreditation Procedures”. An additional document, “General Visitor – Typical Duties”, is available. A basic list of distribution of duties, “Team Member Responsibilities”, is provided by the chair to the team members. *

8 Team’s responsibilities ~ cont’d.
* 07/16/96 Team’s responsibilities ~ cont’d. The confidential nature of the process Check everything you question with the program coordinator or other responsible person ~ don’t assume! Be sure to get the names of all the people that you interview. Circulate an attendance list. Visitors should not get into giving suggestions All issues that are to appear in the final report must be brought up in the exit meeting ~ i.e. there must not be any surprises in the final report. We must agree on all issues to be raised and I must have a clear understanding of them. Do not use the terms “concern”, “weakness” or “deficiency”. In the accreditation process, these terms have very specific meanings (accreditation jargon!). More information related to this slide The team’s responsibilities, including the role of each team member, is described in detail in “Manual of Accreditation Procedures”. An additional document, “General Visitor – Typical Duties”, describing the specific role of the General Visitor, is available. A basic list of distribution of duties, “Team Member Responsibilities”, is provided by the chair to the team members. More information about confidentiality is available in “Manual of Accreditation Procedures”. Further information guiding ethics, conduct of volunteers and process monitoring is provided in: “Guideline Ethics Code” “Engineers Canada Code of Conduct for Volunteers” “Engineers Canada Process Monitoring (Whistle-Blowing) policy” A resource of general interest to accreditation visitors in any field is “Ten Commandments for the Site Team”. *

9 * 07/16/96 Tasks and tools Interviews with appropriate senior administrative officers, including the president, the dean of engineering and the chairs of the departments responsible for the programs Interviews with individuals and groups of faculty members to evaluate: professional attitudes motivations morale the balance of opinions concerning theoretical and practical elements of the curriculum Interviews with individuals and groups of students. Ask open-ended questions to get them talking Examine compliance with graduate attribute criteria More information related to this slide In carrying out the tasks listed on this slide, “Suggested Interview Questions for CEAB Visits” is a helpful resource, along with the description of accreditation criteria given in “Accreditation Criteria and Procedures”. *

10 Tasks and tools ~ cont’d.
* 07/16/96 Tasks and tools ~ cont’d. Tours of physical facilities such as laboratories, libraries, and computing facilities, to evaluate their effectiveness Note that the Accreditation Board does not require any Faculty to spend money - the question is whether the equipment, supplies, etc. are adequate A review of recent examination papers, laboratory instruction sheets, student transcripts, student reports and theses, models or equipment constructed by students and other evidence of student performance Are performance expectations and grading standards appropriate? More information related to this slide For tasks related to review of program material, program visitors refer to their own experience of course materials. In addition, the Engineers Canada Syllabus is a resource. General visitor, in assessing physical facilities, can refer to “Manual of General Visitor”. Chair and Vice-Chair, in examining transcripts, can refer to “Accreditation Criteria and Procedures”, including the “Advanced standing, prior studies, exchange studies, and transfer credit regulations” (effective January 2012 “Regulations for granting transfer credits”). *

11 Tasks and tools ~ cont’d.
* 07/16/96 Tasks and tools ~ cont’d. Attendance form ~ for meetings and interviews AU re-allocation form Sample questions for faculty and staff Sample questions for students Interpretive Statements and Regulations Statement on the Evaluation of Natural Science Accreditation Units Statement of Interpretation on Licensure Expectations and Requirements Advanced Standing, Prior Studies, Exchange Studies, and Transfer Credit Regulation (effective January 2012: Regulations for granting transfer credits) More information related to this slide Attendance form: circulate a sheet that includes time, date, general name of group being met, and, for each participant, name, program, program year. AU re-allocation form: Note: for the uninitiated, a copy of the AU spreadsheets and an explanation of how the AU counts and specific AU counts are verified would be a valuable resource. Sample questions are in “Suggested Interview Questions for CEAB Visits”. Interpretive statements and regulations are Appendices in “Accreditation Criteria and Procedures”. *

12 Visit Schedule Pre-visit Activities
* 07/16/96 Visit Schedule Pre-visit Activities All team members review the written material submitted by the host institution, and spend time reviewing graduate attributes Team members prepare a list of potential issues and a preliminary draft of their contribution to the report. Initial team meeting A detailed schedule for Sunday to Tuesday has been prepared for each team member More information related to this slide Written material submitted by the institution is found in “Questionnaire”. Team members each develop list of potential issues using the “Tracking Program Issues: Working document” form. Team members each develop a preliminary draft of their contribution to the “CEAB Visiting Team Report”. Initial team meeting will review this PowerPoint. Detailed schedule is provided to visitors a few days in advance of the visit. “Sample Visit Schedule” provides a general idea of what to expect, as does this PowerPoint. *

13 Visit Schedule ~ cont’d.
* 07/16/96 Visit Schedule ~ cont’d. Visit – Day 1 (Sunday)‏ Lunch with institution officials In the afternoon, the team visits the institution to review course materials including consideration of graduate attribute compliance Team dines together Team meets in the evening ~ agenda includes discussion of: observations and findings of the day potential issues and how to investigate further previous decision issues and areas to be re-examined More information related to this slide Team members will together review the combined list of potential issues that they have separately developed using the “Tracking Program Issues: Working document” form. Team members each revise their preliminary draft of their contribution to the “CEAB Visiting Team Report”. *

14 Visit Schedule ~ cont’d.
* 07/16/96 Visit Schedule ~ cont’d. Visit – Day 2 (Monday)‏ Plenary Session with the Team, Dean, Program Chairs / Co- ordinators: Introductions and purpose (fact-finding for the CEAB) Process and time lines Conduct visit Lunch with administrators and faculty members Continuation of visit Team dinner and meeting Discussion and preliminary consensus regarding issues List areas of strength and list issues that require further investigation More information related to this slide Team members will together review the combined list of potential issues that they have separately developed using the “Tracking Program Issues: Working document” form. Team members each further revise their preliminary draft of their contribution to the “CEAB Visiting Team Report”. *

15 Visit Schedule ~ cont’d.
* 07/16/96 Visit Schedule ~ cont’d. Visit – Day 3 (Tuesday) Update information with Dean, Program Coordinators Revise visit schedule as necessary In camera Team Working Lunch: Complete draft reports Review reports, arrive at consensus on final conclusions Draft copies of each report provided to the chair More information related to this slide Team members each finalize their draft of their contribution to the “CEAB Visiting Team Report”. *

16 Visit Schedule ~ cont’d.
* 07/16/96 Visit Schedule ~ cont’d. Visit – Day 3 (Tuesday) ~ cont’d. Exit meeting with Dean, Program Chairs, Faculty (verbal presentation by the Team Chair) Restate - Visiting Team’s role is fact-finding Accreditation decisions are made only by the Accreditation Board Repeat time-line of the process to follow Emphasize confidentiality Summarize all issues and state strengths Thank the Dean and staff for visit arrangements and their hospitality More information related to this slide Team Chair will read aloud the current version of the official “Exit Statement of Team Chair” (provided to Chair prior to visit). *

17 Timeline after visit Chair submits report to CEAB Secretariat
* 07/16/96 Timeline after visit Chair submits report to CEAB Secretariat Report is edited, formatted and returned with any questions to chair Chair may contact team members with questions Report finalized, sent to institution Institution responds and sends update Accreditation decision made (June or Sept mtg) Institution and Team members notified of decision (within month) More information related to this slide You can consult the “CEAB Accreditation Site Visit Calendar of Events” (4 pages) for further information. *

18 If you see an issue with a program
* 07/16/96 If you see an issue with a program Visit Team is on a fact-finding mission Institution’s documentation will emphasize the positive but your direct observation may differ You need to verify documentation and identify discrepancies if any, to inform CEAB decision Add something about editing process. If there is an issue, the institution still has multiple opportunities to address it and improve Do not hesitate to dig for the full picture and describe it accurately in your report More information related to this slide Accurate Visit Team reports are an essential part of Canada’s overall system for improving accredited engineering education. When issues are identified during site visits and reported, institutions are motivated to carry out necessary improvements. The cycle of improvement includes opportunity for the institution to address issues and avoid losing accreditation. If you are unsure whether to draw attention to an apparent issue, seek support from your Visit Team Chair. *

19 Accreditation Decisions 2011-2012 Cycle
* 3/22/201707/16/96 Accreditation Decisions Cycle Significance of this slide From the graph on this slide, it is clear that the large majority of visits lead to a decision according the maximum term of accreditation (6V decision). In some cases, information provided related to the accreditation site visit does lead to a decision for a shorter accreditation term. There are now 271 accredited programs at 43 Higher Education Institutions in Canada And 13 substantially equivalent programs in 4 HEIs outside of Canada *

20 Accreditation Criteria Highlights
* 07/16/96 Accreditation Criteria Highlights The processes of accreditation place emphasis on the quality of the curriculum, the students, the academic staff, support staff, facilities, and resources The accreditation criteria: Reflect the need for engineers to be adaptive, creative, resourceful, and responsive Ensure that the graduates understand the role and responsibilities of professional engineers to society Reflect the need for the professional engineer to function as an effective member of a team and to communicate effectively More information related to this slide Accreditation criteria are described in “Accreditation Criteria and Procedures 20xx”. *

21 Accreditation Criteria Highlights cont’d.
* 07/16/96 Accreditation Criteria Highlights cont’d. Qualitative Evaluation – Curriculum Considerations Curriculum must include the application of computers and appropriate laboratory experience and safety procedures Students must be exposed to material dealing with professionalism, ethics, equity, public and worker safety and health considerations, concepts of sustainable development, environmental stewardship The Curriculum must prepare students to learn independently and to work as an effective member of a team More information related to this slide Accreditation criteria are described in “Accreditation Criteria and Procedures 20xx”. *

22 Accreditation Criteria Highlights cont’d.
* 07/16/96 Accreditation Criteria Highlights cont’d. Qualitative Evaluation – Curriculum Considerations Curriculum must include studies in: communication skills engineering economics impact of technology on society subject matter that deals with central issues, methodologies and thought processes of humanities and social sciences, and; must culminate in a significant design experience More information related to this slide Accreditation criteria are described in “Accreditation Criteria and Procedures 20xx”. *

23 Accreditation Criteria Highlights cont’d.
* 07/16/96 Accreditation Criteria Highlights cont’d. Qualitative Evaluation – Curriculum Considerations Engineering Design: integration of curriculum elements creative, iterative and open-ended subject to constraints imposed by legislation or standards to satisfy specification using optimization economics should be part of the design experience to be supervised by licensed engineers Every program must culminate in a significant design experience More information related to this slide Accreditation criteria are described in “Accreditation Criteria and Procedures 20xx”. *

24 Accreditation Criteria Highlights cont’d.
* 07/16/96 Accreditation Criteria Highlights cont’d. Quantitative Evaluation Accreditation units (AU) one hour of lecture (corresponding to 50 minutes of activity) = 1 AU one hour of laboratory or scheduled tutorial = 0.5 AU More information related to this slide Accreditation criteria are described in “Accreditation Criteria and Procedures 20xx”. *

25 Accreditation Criteria Highlights cont’d.
* 07/16/96 Accreditation Criteria Highlights cont’d. More information related to this slide Accreditation criteria are described in “Accreditation Criteria and Procedures 20xx”. Please note: A terminology change in the amended criteria in May 2008 has resulted in references to Basic Science being changed to Natural Science. More information about requirements for natural science is found in “Interpretive Statement on Natural Sciences”, an appendix of “Accreditation Criteria and Procedures 20xx”. *

26 Accreditation Criteria Highlights cont’d.
* 07/16/96 Accreditation Criteria Highlights cont’d. In reviewing course information and course materials, check reasonableness of AU allocations ~ not an exact science! AU re-allocations should be team decisions, after discussion We will discuss re-allocations, if any, on Sunday evening, and again on Monday evening Discuss allocations with responsible faculty member, but no need to argue Agree to disagree More information related to this slide Accreditation criteria are described in “Accreditation Criteria and Procedures 20xx”. *

27 Accreditation Criteria Highlights cont’d.
* 07/16/96 Accreditation Criteria Highlights cont’d. Professional Licensure Dean, Department Chairs, and faculty members teaching courses that are primarily engineering science and engineering design are expected to be licensed to practice engineering in Canada minimum of 225AU of ED to be instructed by P.Eng./ing. minimum of 600 AU of ES+ED to be instructed by P.Eng./ing. or 'Applied’ Interpretive Statement as guidance Curriculum development and control should be in the hands of persons licensed to practice engineering in Canada More information related to this slide Accreditation criteria are described in “Accreditation Criteria and Procedures 20xx”. Specific information associated with professional licensure of instructors in engineering schools is found in “Statement of Interpretation on Licensure Expectations and Requirements”, an appendix of “Accreditation Criteria and Procedures 20xx”. Regarding the last point on the slide, note that some individuals in some Provinces will have a license to practice (limited) but not a full-blown PEng. *

28 Accreditation Criteria Highlights cont’d. – Graduate Attributes
* 07/16/96 Graduate Attributes Accreditation Criteria Highlights cont’d. – Graduate Attributes Criterion Graduate Attributes Two components Attributes: Interpreted at time of graduation Recognized that achievement does not end there Continuous Improvement: Ongoing evolution of engineering programs Processes needed Assessment of attribute achievement Results used to improve program *

29 Accreditation Criteria Highlights cont’d. – Graduate Attributes
* 07/16/96 Graduate Attributes Accreditation Criteria Highlights cont’d. – Graduate Attributes Criterion Graduate Attributes Timeframe: “While programs are expected to provide evidence to demonstrate compliance with this criterion, a transition and development period will be allowed. Starting in June 2015, the Accreditation Board will make decisions about compliance with the Graduate Attribute criteria. Deficiencies may be assessed in cases of non-compliance.” Quote is from “Accreditation Criteria and Procedures 2011”. *

30 Accreditation Criteria Highlights cont’d. – Graduate Attributes
* 07/16/96 Graduate Attributes Accreditation Criteria Highlights cont’d. – Graduate Attributes *

31 Graduate Attributes ~ Examples
* 07/16/96 Graduate Attributes Graduate Attributes ~ Examples The institution must demonstrate that the graduates of a program possess the attributes under the following headings: 3.1.1 A Knowledge Base for Engineering: Demonstrated competence in university level mathematics, natural sciences, engineering fundamentals, and specialized engineering knowledge appropriate to the program. 3.1.8 Professionalism: An understanding of the roles and responsibilities of the professional engineer in society, especially the primary role of protection of the public and the public interest. *

32 Graduate Attributes progress ~ Evaluation by Program Visitors
* 07/16/96 Graduate Attributes Graduate Attributes progress ~ Evaluation by Program Visitors The Program Visitor must evaluate progress toward each graduate attribute to fill out the report template: Program Visitor’s Observations on Implementation Evaluate the evidence and actions (either seen on-site or in the questionnaire) proposed to demonstrate the level of achievement of each graduate attribute Evaluate the evidence and actions (either seen on-site or in the questionnaire) proposed to demonstrate the level of continual improvement achievement Note regarding evaluation: As stated on Slide 6, the team is on site to: - Conduct fact-finding on behalf of the Accreditation Board - Review, validate and/or add to the information provided by the host institution - Review of materials, meetings, and facility tours to corroborate program strengths and weaknesses and bring forward issues to the CEAB - Describe progress toward use of graduate attributes in program assessment and improvement   The last point may involve some subjectivity, at least until this component of the Accreditation Criteria review is well established. Visitors are invited to be aware of the absence of objective standards at this time, and to explain their reasoning and expectations when they comment on progress toward use of graduate attributes. *

33 Graduate Attributes progress ~ Evaluation by Program Visitors
* 07/16/96 Graduate Attributes Graduate Attributes progress ~ Evaluation by Program Visitors Things the Program Visitor will need to see: Graduate Attributes (Accreditation Criteria) Learning outcomes that support Graduate Attributes Indicators Acceptable levels Feedback mechanism *

34 Graduate Attributes: Evaluation by HEI
* 07/16/96 Graduate Attributes Graduate Attributes: Evaluation by HEI The program is assessed, not the students Continuous improvement process Not required to assess every student Not required to assess in every course Not required to assess every year *

35 Continuous Improvement – The big picture
* 07/16/96 Graduate Attributes Continuous Improvement – The big picture HEIs evaluate: Are students meeting expectations? In what areas are they successful What areas require improvement What data would help institution improve their program? *

36 What to look for: Program background
* 07/16/96 Graduate Attributes What to look for: Program background Is the program clearly described? Is there a curriculum map? Is the context of the program clear? *

37 What to look for: Curriculum Mapping
* 07/16/96 Graduate Attributes What to look for: Curriculum Mapping Information in the curriculum map is Accurate, with some depth Identifies intended outcomes from learning experiences Not simply a list of topics “covered” Map provides information for each attribute Can include curricular and other experiences *

38 Methodology: Data Collection Plan
* 07/16/96 Graduate Attributes Methodology: Data Collection Plan On what does the program propose collecting data (i.e. indicators)? What methods are proposed for collecting data? Is the data collection plan good? *

39 Terminology for Data Collection (1)
* 07/16/96 Graduate Attributes Terminology for Data Collection (1) Valid Indicators measure what they are supposed to measure Reliable Indicators the results are consistent; the measurements are the same when repeated with the same subjects under the same conditions *

40 Terminology for Data Collection (2)
* 07/16/96 Graduate Attributes Terminology for Data Collection (2) Direct measures directly observable or measurable assessments of student learning Indirect measures opinion or self-reports of student learning or educational experiences Use both direct and indirect measures if possible. *

41 Data Collection - Indicators
* 07/16/96 Graduate Attributes Data Collection - Indicators An indicator is like a sensor: what indicators has the program chosen? Where have they placed their indicators? Where are the data collection points? Does the proposed data collection plan make sense? *

42 What to look for: Overall - data collection plan
* 07/16/96 Graduate Attributes What to look for: Overall - data collection plan Integrity: How good is the quality of the data collection plan Are Indicators well chosen? Are assessment points well chosen? Is valid, reliable data collection proposed? Is plan cyclic, continuous? will results be useful for informing curriculum change? Ask the question: “why are you collecting this data?” *

43 What to look for: Indicators in data collection
* 07/16/96 Graduate Attributes What to look for: Indicators in data collection Indicators align with attributes and questions Indicators are “leading indicators”: central to attribute; indicate competency Enough indicators defined to identify strength areas and weak areas within an attribute Not too many indicators – resulting in reams of data but little deep information Indicators are clearly articulated and measurable *

44 Selecting Assessment Points
* 07/16/96 Graduate Attributes Selecting Assessment Points Learning is generally demonstrated through: Artifacts, e.g. written test, report, built project Performances, e.g. oral presentation, observed practice *

45 What to look for: Assessment Points
* 07/16/96 Graduate Attributes What to look for: Assessment Points Indicators are well aligned to the proposed assessment points Enough assessment points are utilized Expectations of performance quality are clear, i.e. the scale is defined *

46 What to look for: Triangulation in improvement process
* 07/16/96 Graduate Attributes What to look for: Triangulation in improvement process Are opportunities included for informal assessment, students’ self-reports of learning, and even unsolicited data from placement supervisors or employers? Are more than one type of assessment used when analyzing data? Are all assessments valued, not just major events? Are the data gained from assessment used to answer questions about authentic learning? Are data across time intervals looked at? *

47 Measurement Tools ~ Illustrative Examples
* 07/16/96 Graduate Attributes Measurement Tools ~ Illustrative Examples *

48 Quality Improvement Loop
* 07/16/96 Graduate Attributes Quality Improvement Loop 3.1 Graduate attributes Engineering programs are expected to continually improve There must be processes in place that demonstrate that program outcomes are being assessed in the context of the attributes And, that the results are applied to the further development of the program. Thus, If observed outcomes are not consistent with expected attributes, then system inputs and/or process must be adjusted *

49 What to look for: Evidence of Feedback Loop
* 07/16/96 Graduate Attributes What to look for: Evidence of Feedback Loop Results are consolidated for each learning outcome. Determination is made regarding whether learning outcome is met. Results of assessment are used to determine if changes need to be made in curriculum, courses, prerequisites, performance criteria or metrics. Change is implemented Assessment is repeated to determine effect of change. *

50 Continuous Improvement
* 07/16/96 Graduate Attributes Continuous Improvement Program Objectives and Expected Outcomes Curriculum Assessment Processes and Metrics Data Collection Evaluation *

51 Criterion 3.1 ~ 2009 Findings Graduate Attributes
* 07/16/96 Graduate Attributes Criterion 3.1 ~ 2009 Findings Most programs at beginning stages of developing GA measurement tools and data management systems Some examples: Mapping of GAs to curriculum Structuring activities to address specific GA aspects *

52 Criterion 3.1 ~ 2011-2012 Cycle Findings
* 07/16/96 Graduate Attributes Criterion 3.1 ~ Cycle Findings Accreditation site visits in assessed state of progress toward Graduate Attribute compliance. Out of 17 programs, based on present progress: 7 were rated “will fail to reach compliance by 2014” 3 were rated “likely to fail to reach compliance by 2014” 7 were rated “on track to comply by 2014” *

53 Potential Issues that visit can reveal
* 07/16/96 Potential Issues that visit can reveal Large classes Faculty numbers ~ “faculty who are not faculty” Long-term leaves and long sabbaticals (counted at institution, but not available to teach) Soft-funded faculty Teaching loads ~ critical dependence on a single individual Course failure rates Check the course information sheet to see if the numbers make sense (tough courses should have higher failure rates)‏ Students pass while failing Attrition rate *

54 Potential Issues cont’d.
* 07/16/96 Potential Issues cont’d. Grading of major written reports Appropriateness of AU allocations ~ especially for Engineering Design and Complementary Studies Plans for renewal of equipment Design experience Access to Dean’s office, Program Director's office, etc. Notional contact hours Admissions: Practices followed for granting advanced standing or transfer credits? Who authorizes exceptions? Is control rigorous? *

55 Report Expectations All issues on the visit
* 07/16/96 Report Expectations All issues on the visit Must be tied to criteria Finalized by Monday night Complete report to extent possible by Tuesday p.m. Will be used to make exit statement Recall ~ all issues need to be raised at the exit meeting Your reports transferred to my computer before you go *

56 Post-Visit Activities
* 07/16/96 Post-Visit Activities Chair prepares a draft team report Incorporates Program Visitors' reports Target ~ within 2 weeks of Visit Distributed by to all team members for review and comment Target ~ comments returned within 4 weeks of Visit Report edited by CEAB Executive Committee member Edited report sent to HEI for review and comment CEAB relies on report + HEI’s response to make decision *

57 Post-Visit Activities cont’d.
* 07/16/96 Post-Visit Activities cont’d. All team members will be advised of decision Opportunity to be acknowledged by CEAB for your contribution Evaluation HEI evaluates the visit process Team Chair evaluates team members Team members evaluate Team Chair and process (currently this is a pilot project. Not all teams do this) General Visitor provides report on visit process to association of jurisdiction where institution is located More information related to this slide Regarding evaluation, forms for first three items will be provided by Secretariat directly to those required to complete them. General Visitor completes “General Visitor Report to the Constituent Association”. *

58 Visit-Specific Issues
* 07/16/96 Visit-Specific Issues Insert per pre-visit summary of issues list More information related to this slide Summary of Issues is developed on the “Tracking Program Issues: Working document” form. *

59 1100-180 Elgin Street, Ottawa, Ontario K2P 2K3
* 07/16/96 At this time, I would be happy to answer any questions. Thank you. Elgin Street, Ottawa, Ontario K2P 2K3 Tel / Fax engineerscanada.ca *


Download ppt "Accreditation Visit to"

Similar presentations


Ads by Google