Frameworks for Assessment of Student Learning: Questions of Concept, Practice, and Detail Christine Siegel, Ph.D. Associate Professor of School Psychology Associate Dean, Graduate School of Education & Allied Professions Fairfield University Learning in Action: Innovative Pedagogy and Course Redesign XI Summer Teaching Conference June 2, 2011

How do we know what students are learning? C. Siegel, May 2011

What is being learned? How do we know what students are learning? C. Siegel, May 2011

What is being learned? What is learning? How does learning occur? What factors influence learning? Who are learners? How can learning be promoted? What do we expect students to learn? How do we know what students are learning? How do we know what we know? CONCEPTUALPRACTICAL DETAIL C. Siegel, May 2011

What is being learned? What is learning? How does learning occur? What factors influence learning? Who are learners? How can learning be promoted? What do we expect students to learn? How do we know what students are learning? How do we know what we know? CONCEPTUALPRACTICALDETAIL C. Siegel May 2011

What is being learned? What is learning?Who are learners? What do we expect students to learn? How do we know what students are learning? How do we know what we know? CONCEPTUALPRACTICALDETAIL C. Siegel May 2011

What is learning? C. Siegel May 2011

What is learning? Learning involves change Learning endures over time Learning occurs through experience C. Siegel May 2011

Who are learners? C. Siegel May 2011

What do we expect students to learn? C. Siegel May 2011

What do we expect students to learn? Learning Outcomes Aspirations C. Siegel May 2011

What do we expect students to learn? Learning Outcomes Aspirations Goals C. Siegel May 2011

What do we expect students to learn? Learning Outcomes Aspirations Goals Objectives C. Siegel May 2011

What is being learned? Knowledge – General – Content -specific – Procedural – Content-specific procedural Skills – Basic – Higher Order – Cross-disciplinary – Content-Specific Values – Perspectives – World views – Attitudes C. Siegel May 2011

How do we know what students are learning? Inferential C. Siegel May 2011

How do we know what students learn? What students say What students do What students make C. Siegel May 2011

How do we know what students are learning? What students say (statements) What students do (behaviors) What students make (products) Knowledge general content procedural content-specific procedures Skills basic higher-order cross-disciplinary content-specific Values attitudes world views perspectives C. Siegel May 2011

How do we know what students are learning? What students say (statements) What students do (behaviors) What students make (products) Knowledge general content procedural content-specific procedures Test/ quiz performance Homework performance Performance on essays / papers Skills basic higher-order cross-disciplinary content-specific Values attitudes world views perspectives C. Siegel May 2011

How do we know what students are learning? What students say (statements) What students do (behaviors) What students make (products) Knowledge general content procedural content-specific procedures Survey/ questionnaire Interviews Reflective writings Course Evaluations Test/ quiz performance Homework performance Performance on essays / papers Portfolios Capstone project Works of art Research projects Professional work products Skills basic higher-order cross-disciplinary content-specific Survey / questionnaire Interviews Reflective writings Think-aloud Plans / proposals Course Evaluations Theatrical/ Artistic Performance Demonstration Class presentation Observations by others Portfolios Capstone project Works of art Research projects Professional work products Values attitudes world views perspectives Survey / self-report Interviews Self-report Reflective writings Course Evaluations Anecdotal Records Engaged time Money spent Portfolios Capstone project Works of art Research project Professional work products C. Siegel May 2011

PY 538 PSYCHO-EDUCATIONAL ASSESSMENT II: COGNITION Example 1 GSEAP Psychological & Educational Consultation School Psychology PY 538 C. Siegel May 2011

What is being learned? What is learning? Gaining proficiency in the use of intelligence tests for education decision making, through exposure to and practice with a variety of measures of intelligence Who are learners? Advanced degree candidates in school psychology What do we expect students to learn? ASPIRATION: Employability as a School Psychologist GOAL: Proficiency in data-based decision making OBJECTIVES: Acquiring knowledge, skills, and ethics associated with the professional administration and interpretation of standardized intelligence tests How do we know what students learn? C. Siegel May 2011

How do I know what students are learning in PY 538? What students say (statements) What students do (behaviors) What students make (products) Knowledge Current intelligence theory Intelligence test administration Intelligence test interpretation Cross-battery testing Report writing Survey/ questionnaire “Pre-test” of familiarity with individually administered standardized intelligence tests Course evaluations Test/ quiz performance Quizzes on test development, psychometric properties, administration, and interpretation of tests Tests on current intelligence theory Professional work products Test protocols Written assessment reports Skills Intelligence test administration Intelligence test interpretation Cross-battery testing Report writing Think-aloud In class demonstrations of test administration In class exercises in test score interpretations Course evaluations Demonstration Videos of test administration Observations by others Peer observation of in- class demonstration Professional work products Test protocols Written assessment reports Values Awareness of the limitations intelligence tests in educational decision making Awareness of issues related to the use of intelligence test with special populations Survey/ questionnaire “Pre-test” of familiarity with individually administered standardized intelligence tests Course evaluations Anecdotal Records Informal observation of student comments about tests and practice subjects Engaged time Informal reports of time spent with practice subjects Professional work products Consent Forms Test protocols Written assessment reports C. Siegel May 2011

How do I know what students are learning in PY 538? What students say (statements) What students do (behaviors) What students make (products) Knowledge Current intelligence theory Intelligence test administration Intelligence test interpretation Cross-battery testing Report writing Survey/ questionnaire “Pre-test” of familiarity with individually administered standardized intelligence tests Course evaluations Test/ quiz performance Quizzes on test development, psychometric properties, administration, and interpretation of tests Quizzes on cross-battery assessment Tests on current intelligence theory Professional work products Test protocols Written assessment reports Skills Intelligence test administration Intelligence test interpretation Cross-battery testing Report writing Think-aloud In class demonstrations of test administration In class exercises in test score interpretations Course evaluations Demonstration Videos of test administration Observations by others Peer observation on in- class demonstration Professional work products Test protocols Written assessment reports Values Awareness of the limitations intelligence tests in educational decision making Awareness of issues related to the use of intelligence test with ethnic minority populations Survey/ questionnaire “Pre-test” of familiarity with individually administered standardized intelligence tests Course evaluations Anecdotal Records Informal observation of student comments about tests and practice subjects Engaged time Informal reports of time spent practicing, and administering tests Professional work products Consent Forms Test protocols Written assessment reports C. Siegel May 2011

How do we know what School Psychology candidates are learning? What students say (statements) What students do (behaviors) What students make (products) Knowledge Data-based decision making Interpersonal communication, collaboration and consultation Effective instruction & development of cognitive academic skills Socialization and development of behavioral competencies Student diversity in learning and development School structure, organization, and climate Prevention, wellness promotion, and crisis intervention Home/school community collaboration Research and program evaluation Legal issues, ethical practice and professional development Information technology Admissions Interviews Philosophy of School Psychology Course Evaluations Alumni Surveys Test/ quiz performance MA-level comprehensive exam PRAXIS II in School Psychology Employer Surveys e - portfolios Skills Data-based decision making Interpersonal communication, collaboration and consultation Effective instruction & development of cognitive academic skills Socialization and development of behavioral competencies Student diversity in learning and development School structure, organization, and climate Prevention, wellness promotion, and crisis intervention Home/school community collaboration Research and program evaluation Legal issues, ethical practice and professional development Information technology Plans / proposals Treatment Plans Research Proposals Field work Goals Internship Goals Alumni Surveys Class presentations Observations by others Field Work Evaluations Internship Evaluations Employer Surveys e -portfolios Values Legal issues, ethical practice and professional development Reflective writings across courses Course Evaluations Alumni Surveys Anecdotal Records Engaged time Employer Surveys e-portfolios C. Siegel May 2011

How do we know what School Psychology candidates are learning? What students say (statements) What students do (behaviors) What students make (products) Knowledge Data-based decision making Interpersonal communication, collaboration and consultation Effective instruction & development of cognitive academic skills Socialization and development of behavioral competencies Student diversity in learning and development School structure, organization, and climate Prevention, wellness promotion, and crisis intervention Home/school community collaboration Research and program evaluation Legal issues, ethical practice and professional development Information technology Admissions Interviews Philosophy of School Psychology Course Evaluations Alumni Surveys Test/ quiz performance MA-level comprehensive exam PRAXIS II in School Psychology Employer Surveys e - portfolios Skills Data-based decision making Interpersonal communication, collaboration and consultation Effective instruction & development of cognitive academic skills Socialization and development of behavioral competencies Student diversity in learning and development School structure, organization, and climate Prevention, wellness promotion, and crisis intervention Home/school community collaboration Research and program evaluation Legal issues, ethical practice and professional development Information technology Plans / proposals Treatment Plans Research Proposals Field work Goals Internship Goals Alumni Surveys Class presentations Observations by others Field Work Evaluations Internship Evaluations Employer Surveys e -portfolios Values Legal issues, ethical practice and professional development Reflective writings across courses Course Evaluations Alumni Surveys Anecdotal Records Engaged time Employer Surveys e-portfolios

PY – ED 534 THEORIES OF LEARNING Example 2 GSEAP PEC School Psychology Applied Psychology Special Education ESTP TESOL/ Bilingual Education PY-ED 534 Secondary Education TEFO MFT School- based MFT C. Siegel May 2011

What is being learned? What is learning? Making explicit the implicit assumptions professionals have about learning that guide their practice, through the critical examination of and reflection on a variety of learning theories across multiple fields of study Who are learners? Advanced degree candidates in variety of programs across GSEAP What do we expect students to learn? ASPIRATION: The GSEAP aims to prepare scholar practitioners who engage in reflective practice, promote human development and learning, and advocate for children and families. GOALS: Develop a personal theory of learning to guide their professional practice PROFICIENCIES / OBJECTIVES: Acquiring foundational knowledge in education, skills for designing effective instruction and interventions, dispositions of reflective practice and advocacy How do we know what students learn? C. Siegel May 2011

How do I know what students are learning in PY-ED 534? What students say (statements) What students do (behaviors) What students make (products) Knowledge Historical and contemporary theories of learning Psychological, social, and cultural influences on learning Survey/ questionnaire Pre and post group discussions around central questions Participation in Class Discussions What do you know about…? What did you think about…? Reflective Writings Course evaluations Participation in class discussions and group work Class Presentations Professional work products Personal Theory of Learning Skills Critical analysis Personal Reflection Communicating about learning across a range of perspectives (Developing effective interventions and instruction) Reflective Writings Course evaluations Observations during group work Class Presentations Professional work products Personal Theory of Learning Values Respect for a variety of opinions regarding the process of learning Respect for open dialogue across opinions Awareness of the impact of learning theory on professional practice Reflective Writings CL group evaluation form Course evaluations Anecdotal Records Informal observations during group work CL group evaluation form Engaged Time Informal observations during group work Professional work products Personal Theory of Leaning C. Siegel May 2011

What is being learned? What is learning?Who are learners? What do we expect students to learn? How do we know what students learn? How do we know what we know? CONCEPTUALPRACTICALDETAIL C. Siegel May 2011

How do we know what we know? C. Siegel May 2011

How do we know what we know? Quantitative Reliability – Stability – Equivalence – Internal consistency – Inter-rater Validity – Content – Internal – External Predictive concurrent Qualitative Credibility Transferability Dependability Confirmability C. Siegel May 2011

Special Thanks C. Siegel May 2011