1. Company LOGO Understanding the Evaluators Responsibilities Karen Froslid Jones Director, Office of Institutional Research and Assessment American University

2. Overview 1. Accreditation and the Visit 2. The Role of the Chair and Evaluators 3. Preparing for the Visit 4. What to Expect During the Visit 6. Keys to a Successful Evaluation 5. What Happens After you Leave

3. Participants in the Accreditation Process Commission Institution Middle States Rep Team Chair Visiting Team Public Self-Study Committee President Faculty, administrators Staff and Students

4. Putting The Visit in Context: The Road to Re-Accreditation Before Visit During Visit After Visit Self-Study Institute Chair and Steering Committee Selected Self-Study Design Submitted Visiting Team selected Self-Study Completed Visit to campus by MS Rep Logistics decided Materials Sent to Team Additional documents requested Travel info communicated. 2004-2007 Team meetings Reception/Dinner Meetings with Steering Committee, etc. Interviews Writing time Report written Recommendation on re-accreditation Oral Report Chair sends report to institution Institution corrects mistakes of fact Chair sends final report Institution sends response Chair attends commission meeting Commission acts on accreditation Institution – and team- is notified of commission action.

5. Purpose of Visit The importance of peer review. The value of meeting face-to-face. The opportunity for institution to benefit from others expertise. The need for objective assessment.

6. The Value of an Evaluator Colleague –Similar institution? –Similar mission? –Appreciation of the challenges facing institutions Expertise –Particular Area(s) –Characteristics of Excellence/Middle States Expectations Provide Outside Perspective –Not tied to politics/personalities/history of institution –Able to look at big picture and put it into context –No conflict of interest

7. The Role of the Chair Acts as conduit between visiting team and institution. –Visits Institution before team visit –Gets early copy of Self-Study –Represents team when planning logistics –Resolve problems/conflicts between team and institution –Main communicator between team and institution during visit –Represents team at Commission meeting

8. The Role of the Chair (cont.) Organizes and Leads Work of Visiting Team –Sets team member assignments (primary and secondary) –Sets up some meetings with campus groups in advance (board of trustees, steering committee, etc.) –Sets report format –Sets work agenda/deadlines Is draft required? When will team meet? When is final report due to chair? –Leads discussions –Leads/Organizes Exit Interview

9. The Role of the Evaluator Assigned one or two primary areas (usually connected to the Characteristics of Excellence) to review. Responsible for reading the self-study, conducting interviews, and reviewing documents to determine the degree to which Characteristics are being meet. Responsible for writing portion of final report. Responsible for contributing to overall report.

10. Before the Visit: What to do Now Know Characteristics of Excellence –Be prepared on standards you consider your areas of expertise –Know all standards thoroughly –Look for ways that standards interact Familiarize yourself with self-study options – be sure you understand the type of self-study your institution is using Know the Report Process/Possible Actions Begin to familiarize yourself with institution –Check out website

11. Before the Visit: What to do When you Receive the Report Read entire report several times at least a few weeks before visit. Clarify with chair your expected areas of coverage. Decide on must interviews – inform chair Any documents you need right away? Let chair know. Provide institution/chair with travel plans.

12. Arriving on Campus Accommodations –Usually stay at same location, close to campus –Leave plenty of time to get checked in/settled before first meeting. Transportation Meeting Room(s) -Usually a work room/ meeting room set up in central location

13. Arriving: The Atmosphere on Campus Keep in mind that this is a BIG deal –Welcoming –Nervous –Worried –Excited –Agendas –Protective –Defensive –Helpful

14. The Visit Begins 1 st Team Meeting –Discuss logistics/assignments/report format. –Come prepared to discuss initial impressions. –Have preliminary interview needs ready. Hosts Program –Chance to get to know individuals/players. –Convey interest/enthusiasm for visit. –Informal inquiries/ not hard line questions. –Be aware of time

15. Setting Up Interviews Read report (and other documents) with eye towards people you may want to meet. –Who seem to be a key players in the area you are covering? –Do certain names come up in the report? Dont be afraid to ask who would know about_____? Prioritize and Coordinate Can ask interviewees if they can suggest others that may be worth meeting.

16. Conducting Interviews Have a clear objective going in. –What needs clarification? –What is missing from self-study and needs to be answered? –What has happened since self-study was written? Be open to where the interview may go. Dont assume everyone you interview knows about accreditation. Create a friendly, open conversation. Avoid overt judgments. Dont make promises/commitments. Look for evidence of a culture of continuous improvement. –Example: –What did you learn about your institution as a result of doing the self- study? Did any findings surprise you? –How do you plan to implement the recommendations of the self-study? Ask questions that give you a sense of direction/commitment/focus

17. Conducting Interviews (cont.) Keep very good notes. –Get cards, if possible, from interviewees. Consider meeting them on their home turf. Consider how combinations of individuals together may change dynamics of interviews.

18. Tips for Interactions with Institution Be careful of off-hand remarks –Everything you say/do will be interpreted Be neutral/open Listen Observe Reserve judgments/opinions Smile

19. Using the Document Room Make sure you build in time early to view documents. Come prepared with a list of must see documents. Be open to looking through a wide range of information. –Meeting notes, school newspaper stories, survey results, annual reports, etc. Put things back where you found them! Shred – or take home - sensitive notes. Dont forget to erase info from borrowed computer!)

20. Writing the Report Informal Observation Self-Study Document Analysis Individual Interviews TEXT Team Members Institution Mission Characteristics Findings Significant Accomplishments Suggestions, Recommendations, Requirements

21. Writing the Report* DO Speak directly to the accreditation standards and institution mission Respond to/recognize the approach used in the self-study Concentrate on the fundamentals Cover all areas of activity, including off-cite, certificate programs, etc. Provide rationale and evidence for the teams recommendations –(model effective institutional assessment) Make clear distinctions between suggestions, recommendations and requirements. Emphasize the importance of outcomes and assessment of both overall institutional effectiveness and student learning. DONT Be highly prescriptive. Name individuals. Cite specific formulas, solutions, educational theories, etc. Reveal accreditation recommendation. * See pp.46-49, Team Visits

22. Writing Tips Clarify which voice you will use as a team. Begin writing early – you can always revise. Keep track of who you talked to/when Keep track of sources of factual information (keep citations) Integrate report as much as possible.

23. Team Interactions A friendly, collegial atmosphere Where you stand depends on where you sit. Ask questions, seek input from others. Offer insight from your perspective. Disagreements likely but usually resolvable.

24. Completing the Visit The final draft –Be open to last minute changes The oral report Leaving the institution

25. After the Visit: What Happens Next? Team Chair submits report to institution for review. Institution corrects factual errors. Team Chair submits final team report along with a confidential memo to the Commission. Institution sends team members final team report. Institution can provide formal response to report. Chair Attends commission meeting. Commission acts on report. Logistics: expense reports, evaluations, etc.

26. Miscellaneous Tips: (What you wont find in any official publication) Come to the visit rested Leave home institution at home Where comfortable/sensible clothing/shoes Alert the institution to any food allergies, special needs, etc. Bring a flash drive. Expect the unexpected – be prepared to go with the flow Dont try to preach/perfect. Have fun!