1 Fred Trapp, Ph.D. Administrative Dean, Institutional Research/Academic Services (Retired) Long Beach City College Cambridge West Partnership, LLC Robert Pacheco, Ed.D. Dean Of Research, Development and Planning Barstow College Comprehensive Assessment Reports

2 Outcomes for the Session The participant will be able to  Describe the comprehensive assessment report concept.  Critique selected best practice examples from other colleges.  Discuss how the comprehensive report idea can be used at your college as part of the institution’s learning, planning and/or resource allocation processes. Note: Wiki, Cloud and Video

3 Curriculum Map Introduction (Purpose and Goals) What Makes A Report Good? How Do the Reports Improve Learning? Where Can I Go? 3

4 ACCJC Institutional Effectiveness Rubric (2011) Part III Student Learning Outcomes  Proficiency  Comprehensive assessment reports exist, are completed and updated on a regular basis.

5 What Did We Look At, With Whom Did We Consult? ACCJC. Institutional Effectiveness Rubric ACCJC Standards ACCJC. Themes ACCJC. Guide to Evaluating Institutions Professional literature Efforts of national groups/institutes Institutional web sites, listservs and colleagues

6 Guiding Questions How can the report writing experience:  Help faculty explore the student learning process?  Determine the extent to which the curriculum is working?  Inform the decisions as to where time, energy and/or money be allocated for continuous improvement in learning?  Stimulate dialogue about results to gain broader institutional learning experiences?  Help meet our quality assurance pledge to the community?

7 Illustration Selections Council for Higher Education Accreditation (CHEA)  Annual Award for Outstanding Institutional Practice in Student Learning Outcomes  Demonstrated commitment to & developed highly effective practice to use SLO assessment  Willingness to share the practices they developed  Selection committee  Selection criteria  Articulation & evidence of outcomes  Success with regard to outcomes  Information to the public about outcomes  Using outcomes for improvement

8 Illustration selections (continued) Cited by scholars and peers  Schools with assessment work cited in scholarly books and articles  Schools with assessment work selected for presentation at national conferences  Web presentations publicly available for you to eavesdrop upon

9 What Might Be Included? 1. Assessment focus-course, program, general ed., etc. 2. What outcomes were assessed? 3. Which, how and when were students assessed? 4. What were the results? 5. Who reviewed the results, made sense of the them and what conclusions were reached? 6. What are the implications for practice and/or policy or future assessment work?

10 CC of Baltimore County (MD) Course-level Reporting Middle States Commission on Higher Education Community College Futures Assembly, Bellwether Award, 2008  Instructional Programs & Services for High Impact Course Level Assessment CHEA award winner, 2006  Institutional Progress in Student Learning Outcomes National Council on Student Development (NCSD) Exemplary Practice Award Winner

11 CC of Baltimore County (MD) Course-level Reporting  Projects are at least three semesters long  Individual and high-impact courses (all sections) included  Project proposal by a faculty group  Measurable objectives  External review & approval in selecting methods/instrument & analyzing results. Benchmarking should be included if possible.  Controls and sample size considered.  Course improvements based on data analysis  Reassessment expected  Results/report shared across the college and web posted

12 CC of Baltimore County (MD) Example report  CHEM 108  An initial “failure” turned to success and collaboration with a four- year school Activity  Read the report  In small groups respond to the discussion questions  Questions 2 and 3  Report out to larger group

13

14 Hocking College (OH) Program-level Reporting North Central Association of Colleges & Schools, Higher Learning Commission A career & technical education institution CHEA award winner, 2008  Institutional Progress in Student Learning Outcomes

15 Hocking College (OH) Learning outcomes data collected in a student E- portfolio  Direct internal and external evidence (1 to 10 measures)  Indirect evidence (1 to 4 measures) Evidence drawn from samples of student work for faculty to apply an agreed upon holistic rubric  Eight general education outcomes (student success skills)  Discipline-specific exit competencies or outcomes

Hocking College Triangulation 16

Hocking College (OH) Program-level Report Example report and analysis  Practical Nursing Technology Activity  Read the report  In small groups respond to the discussion questions  Questions 3, 4, and 5  Report out to larger group 17

18

19

20

21 Mesa College (AZ) General-Education Reports North Central Association of Colleges & Schools, Higher Learning Commission A large comprehensive community college CHEA Award winner, 2007  Institutional progress in Student Learning Outcomes

22 Mesa Community College (AZ) General education studies completed ;  Numeracy  Scientific inquiry  Problem solving/critical thinking ( )  Global Awareness ( pilot test)  Information literacy  Workplace skills (CTE) (2007-8; ) General education studies completed ;  Arts & humanities  Cultural diversity  Oral communication  Written communication

23 Mesa College (AZ) General Education Reports Annual Report example  General Education section only Activity  Read the report  In small groups respond to the discussion questions  Report out to larger group

24

25

26 From page 14

27 From page 15

28

29 Authorship Course-level, program-level & general education  Teaching faculty study team with technical assistance from  institutional research or assessment committee  No “lone ranger” authors Institutional summary  Academic administrator with assistance from  Learning outcomes coordinator or assessment committee  Compilation of work accomplished in one or two academic years across the institution

Your Uses How do you see your school using annual comprehensive reports? 30 See last four pages in the Handout and Discussion Questions packet Northern Arizona U.- Assessment Report Evaluation Rubric U. of Illinois Urbana-Champaign- Reported Unit Improvements from Outcomes Assessments

31 Putting the Reports to “Work” To all affected participants & on at least the intranet Campus committees  Curriculum, assessment, resource allocation group, unit (department) leadership, general academic and college leadership Campus fairs, brown-bag lunches, poster sessions for information sharing Faculty professional development programs Accreditation self-study committee work groups Unit plans, program reviews, requests for resources College web site for the public

32 Reports & Institutional Learning The assessment data reflection activity  sense-making process = a faculty learning experience Linking results to future interventions = a learning experience Using results to inform an intervention, then reassess = a learning experience (accomplished one or more terms later) Reference for future assessment work and other groups on campus

33 Reports as Institutional Learning & Resource Allocation Hocking College (OH)  Annual summary  Improvements in the program in the previous year brought on by study of assessment results  Expenditures of time, money & materials for the assessment program  Requests for assistance in implementing assessment  Recommendations for altering the institution’s assessment process  Mega faculty learning insight  transition from evaluating individual students to assessing groups of students & the curriculum experience

34 Reports as Institutional Learning & Resource Allocation Community College of Baltimore County (MD)  Learning Outcomes Assessment Advisory Board  Links findings in assessment reports to other college-wide initiatives and professional development opportunities  Use of assessment processes and (findings) results  Challenged faculty to reexamine prompts used in assessment Greater clarity of written prompt See how to support program goals  Common assignment options and common rubric increases faculty understanding and buy-in  Public web page enhances communication and accessibility to information

35 Reports as Academic Quality & Resource Allocation Mesa College (AZ) Results Outreach Committee  Promotes use of outcomes data in relation to faculty development, pedagogy and academic climate  Groups of faculty offer a proposal for summer or academic year work above the course level  Resulting report placed on the web and used for campus discussion and action

36 Report as Quality Assurance Promising Vehicles for Expanding Information to the Public  Brief narrative report from annual assessment reports  Simple statistical reports on learning outcomes or surveys  Best practices stories supported by assessment Peter Ewell. Accreditation & the Provision of Additional Information to the Public about Institutional and Program Performance, CHEA, May 2004

37

38

39 Where Can I Go? Resources Cloud Wiki Video

Contacts & Questions Robert Pacheco (Barstow College)  Fred Trapp (Cambridge West Partnership)  Questions and Comments 42