1. Phil Dunwoody, Gerald Kruse Juniata College Huntingdon, PA

2. Learning Outcomes In new course proposals, faculty must provide the “objectives, skills and knowledge students are expected to gain” and answer the question, “how will you evaluate the students in the course?” A Student Learning Outcome (SLO) for a course is a brief statement of what a student will know and be able to do at the end of a course. For more info: Assessment Quickies #3, Levels of Student Learning http://www.youtube.com/watch?v=9XTpyloIuxA& feature=context-cha http://www.youtube.com/watch?v=9XTpyloIuxA& feature=context-cha

3. Syllabi Upload All faculty are required to upload electronic copies of their syllabi http://services.juniata.edu/faculty/syllabi/index.html http://services.juniata.edu/faculty/syllabi/index.html Random sample of 50 courses, during the 2010-11 and 2012-13 academic years (pre and post Middle States) In 2010-11, 15 courses (30%) and in 2012-13, 21 courses (42%) did not have a syllabus on file* * the next section or closest number course was substituted to maintain sample size of 50, and in the meantime CNS has implemented a “reminder” button

4. Rubric, existence only 2 - Explicitly listed as Student Learning Outcomes, Student Goals, Goals, etc. 1 - NOT Explicitly listed BUT CAN be inferred from the course syllabus 0 - NOT listed AND can NOT be inferred from the course syllabus

5. Results of 50 Syllabi Reviewed Juniata College Gettysburg College* SLOs in Syllabi 2010-112012-13 200820112012 None 22 %8 % 23 %16 %10 % Inferred 22 %26 % 13 %23 %12 % Clearly Stated 56 %66 % 64 %61 %78 % *source: Gettysburg Committee on Learning Assessment (COLA) newsletter

6. SLO – a “micro” view of one MA 116, Discrete Structures – first semester IT/CS Foundational math for study in IT and CS: logic, proof, set theory, combinatorics, relations, functions, graphs, and trees SLO in syllabus Previous:... develop an understanding of some of the mathematical structures fundamental to computer science including... induction... New: apply mathematical induction to prove conjectures about sequences

7. SLO – it’s not scary Direct measures of student learning Quizzes, tests, portfolios, papers, standardized tests, capstone projects, … Take advantage of existing measures! Inductive Proof on Quiz immediately after Module Inductive Proof on exam Two additions Pre-assessment proof Link to Kahn Academy video on Induction

8. SLO – Closing the loop Fall 2012, N = 14 students No one successfully completed the pre-assessment proof 5 students reported watching the Kahn Academy video (watching it wasn’t incentivized) “Mechanical Proof” Prove for all integers n > 0 that 1 + 2 +... + n = n*(n+1)/2 “Deeper Proof” As a person arrives at a meeting, they shake hands with all the other people present. If n people are at the meeting, then prove n*(n-1)/2 handshakes occurred.

9. SLO – Closing the loop “Mechanical” Proofs after Module and after Exam “Deeper” Proof on Exam

10. Psychology Department Goals In 2007 the APA published guidelines and 10 goals for undergraduate education. Voluntary No APA accreditation for undergraduate programs Our department adopted these goals Differentiated between direct goals and indirect goals Established SLO for all direct goals

11. Our Indirect Goals… Goal 4: Application of Psychology. Students will understand and apply psychological principles to personal, social, and organizational issues. Goal 5: Values in Psychology. Students will be able to weigh evidence, tolerate ambiguity, act ethically, and reflect other values that are the underpinnings of psychology as a discipline. Goal 6: Information and Technological Literacy. Students will demonstrate information competence and the ability to use computers and other technology for many purposes. Goal 8: Sociocultural and International Awareness. Students will recognize, understand, and respect the complexity of sociocultural and international diversity. Goal 9: Personal Development. Students will develop insight into their own and others’ behavior and mental processes and apply effective strategies for self-management and self-improvement. Goal 10: Career Planning and Development. Students will emerge from the major with realistic ideas about how to implement their psychological knowledge, skills, and values in occupational pursuits in a variety of settings.

12. Indirect Goal Example Goal 8: Sociocultural and International Awareness. Students will recognize, understand, and respect the complexity of sociocultural and international diversity. Goal 8 is an indirect goal for our psychology department. While it is addressed in some of our courses it is not directly assessed.

13. Our Direct Goals… Goal 1: Knowledge Base of Psychology. Students will demonstrate familiarity with the major concepts, theoretical perspectives, empirical findings, and historical trends in psychology. Goal 2: Research Methods in Psychology. Students will understand and apply basic research methods in psychology, including research design, data analysis, and interpretation. Goal 3: Critical Thinking Skills in Psychology. Students will respect and use critical and creative thinking, skeptical inquiry, and, when possible, the scientific approach to solve problems related to behavior and mental processes. Goal 7: Communication Skills. Students will be able to communicate effectively in a variety of formats.

14. Direct Goal Example Goal 1: Knowledge Base of Psychology. Students will demonstrate familiarity with the major concepts, theoretical perspectives, empirical findings, and historical trends in psychology. SLO: Students will be able to answer content questions about the major content areas within the field of psychology (Abnormal Psychology, Learning & Conditioning, Development, Cognition, Biopsychology, and Social Psychology). Teaching Methods: All psychology POEs are required to take at least four courses out of the five major content areas (Abnormal Psychology, Learning & Conditioning, Development, Cognition, Biopsychology, and Social Psychology). These courses will include readings, lectures, projects, and assessments designed to develop and test student mastery of the relevant content material. Assessment: Each senior Psychology POE will take the Psychology Area Concentration Achievement Test (PACAT). This is a nationally normed test and we receive percentile scores to compare our students with similar students nationally. The PACAT provides separate sections and scores for each of the 5 major areas listed above. All senior psychology students are required to take this assessment in a senior capstone. We expect all students to have a minimum overall score at or above the 25 th percentile. We expect our departmental average for all students to be at or above the 50 th percentile. This assessment is given by the instructor of Senior Capstone.

15. Direct Goal Example Goal 2: Research Methods in Psychology. Students will understand and apply basic research methods in psychology, including research design, data analysis, and interpretation. SLO1: Students will be able to answer conceptual and applied questions about research methods and statistics. SLO2: Students will be able to accurately identify and interpret key aspects of research methodology and statistical analysis while reading an empirical psychology article. SLO3: Students will be able to accurately create and interpret a research study with hypothetical results to test a hypothesis. Teaching Methods: All psychology POEs are required to take Statistics for the Social Sciences (NDSS214) and Research Methods (PY309). These courses will include readings, lectures, projects, and assessments designed to develop and test student mastery of research methods and statistics. Assessment SLO1: All students will take the Psychology Area Concentration Achievement Test (PACAT) in the senior capstone. Scores on the PACAT subsections of Experimental Design and Statistics will be used to assess SLO1. This assessment is administered by the instructor of Senior Capstone. Assessment SLO2 & SLO3: All students in Research Methods (PY309) will complete a graded assessment at the conclusion of PY309 designed to evaluate SLO1 and SLO2. All sections of PY309 will utilize the same assessment tool in any given year. This assessment is given by the instructors of Research Methods.

16. Common Exam Scenario Professor John is interested in the impact of color on attention. He thinks that brightly colored shop signs will have a greater effect during the day than at night. To test this hypothesis he uses an animal model, presenting rats with dishes of different colors under different lighting conditions. Each dish is filled with sand, but in each trial, only one dish has a Cheerio hidden in the sand—that dish is either dull blue or bright red, with the rest gray. The rats’ search times to find the Cheerio are graphed below in ms:

17. Common Exam Scenario 6) What type of statistical test must Professor John do to test his hypothesis? What must be significant to support his hypothesis? 7) Interpret Describe any main effects and/or interaction effects shown in the data graphed above. (Assume that any difference of more than 1ms is statistically significant.) 8) Is there support for Professor John’s hypothesis? Explain your answer. 9) What group at Professor John’s school had to approve this study? What is one thing the professor had to prove to them to obtain approval? 10) Professor John writes a clear and well-sourced article about his findings. In his discussion, he makes predictions about human behavior when seeking restaurants and shops during the day versus late at night, based on these results. What is one validity concern an editor might raise about these predictions?

18. Yearly Assessment Report Assessment Goal (Assessor)Assessment Outcome PACAT Overall: All students to have a minimum overall score at or above the 25 th percentile. We expect our departmental average for all students to be at or above the 50 th percentile. Goals achieved. Every students earned above the 25 th percentile. On average, our students scored at the 79 th percentile nationally. PACAT Experimental Design and Statistics: All students to have a minimum overall score at or above the 25 th percentile. We expect our departmental average for all students to be at or above the 50 th percentile. Goal mostly achieved. On average, our students scored at the 81 st percentile in experimental design and at the 85 th percentile in statistics. Percentile scores are NOT reported for individuals’ subsections. Based on the percentile ranges reported by PACAT, all students except 1 scored above the 25 th percentile in experimental design and all students except 1 (a different student) scored above the 25 th in statistics. Research Methods Common Exam: All students will earn at least 50% of the available points. Goal achieved. Across three sections of Research Methods, students averaged 74% of the common exam points available. Critical Thinking: Assessed in 2012. Needs to be assessed once every 5 years. Not directly assessed this year. NA Communication (PY309): Students will perform better on the final version of paper 2 than they did on paper 1. All students will have a passing grade on the final paper (>60). Goal Achieved. Across all 3 sections of Research Methods, paper 1 had an average score of 82 points while paper 2 had an average of 88 points. Every student received a passing grade on paper 2.

19. Suggestions for writing SLOs Be specific Don’t use passive voice, use active verbs Analyze Classify Design Evaluate Perform Synthesize Teach Assessment Quickies #2, Writing Student Learning Outcomes http://www.youtube.com/watch?v=KA0DeysBX7M&feature=plcp

20. Levels of Student Learning Basic Define, Label, Recognize, … Mid-range Apply, Interpret, Solve Advanced Compose, Formulate, Plan, Judge, Defend Assessment Quickies #3, Levels of Student Learning http://www.youtube.com/watch?v=9XTpyloIuxA&feature=context-cha