Assessment of an Arts-Based Education Program: Strategies and Considerations Noelle C. Griffin Loyola Marymount University and CRESST CRESST Annual Conference September 8, 2005

History Artful Learning Program Leonard Bernstein Center for Learning The Artful Learning Program Based on the work of the late composer Implemented through the Grammy Foundation

Artful Learning Program Arts-based education model Best School Practice Components Teacher as Expert Currently implemented in 9 states, 23 schools

Evaluation Overview 3 year evaluation project Focus on implementation process, and the impact of school and teacher-level factors Quantitative data Qualitative data

Original Data Collection Plan -Student achievement data -Survey of teacher practice -Implementation interviews -Professional development evaluation

Limits of “Traditional” Approach -Standardized test data: focus on basic reading/math skills only, “trailing” indicators at best -Lack of applicable student assessment instruments -Ignore teacher transformation

Delving Deeper into Teacher Experience Teachers Teacher attitude Teacher practice Teacher collaboration Students Motivation Content Retention Led us to explore collective teacher efficacy during Y3

Collective Teacher Efficacy Teachers’ beliefs in the shared ability as a group to achieve goals/tasks (1)Analysis of the teaching task (Task Analysis) (2) Assessment of teaching competence (Group Competence) *Integrated into quantitative and qualitative data collection

Task Analysis -The process of analyzing what is required for the school to be effective in the process of teaching. Sample Items “The lack of instructional materials and supplies makes teaching very difficult.” “The quality of school facilities here really facilitates the teaching and learning process.”

Group Competency -The assessment of quality and competence of the teaching staff. Sample Items “If a child doesn’t learn something the first time, teachers will try another way.” “Teachers in this school do not have the skills to deal with student disciplinary problems.”

Dependent & Independent Variables Dependent Variables  Total Efficacy  Group Competency  Task Analysis Independent Variables  Grade Level Taught · Teaching Experience (years) · Professional Development · School-Level Collaboration · Shared Leadership · Artful Learning Program Use (years) · Extent of Artful Learning Implementation

Collective Teacher Efficacy Scale Multiple Regression Findings Shared Leadership Teaching Experience (number of years) Teaching Experience (number of years) Grade Level Taught Total Efficacy Total Efficacy Group Competency Task Analysis *All Significant relationships are negative *No significant relationship to other Artful learning implementation variables

Interview Findings Teacher experience Experiences of primary vs. secondary teachers The role of shared leadership: importance of structural support

Tailoring an Approach to Student Learning: Available Resources -Artful Learning Units: Structured archives of student work -Artful Learning teams, cadres, and other teacher-based organizational structures -Regular training opportunities -Pre-existing “rubric” approach

Example: Artful Learning Classroom Rubirc

Example: Artful Learning Student Rubric - In Progress

Alternative Assessment Approaches -Use classroom rubrics to assess teacher work/curriculum -Develop more comprehensive rubrics for student work to assess quality of student learning -Use scoring rubrics for both evaluative and formative assessment purposes

Considerations/Roadblocks -Refinement of rubrics: elaborated and usable -Selection of student data -Reliability and validity considerations -Training and on-going technical support

Links - Grammy Foundation & Leonard Bernstein Center -CRESST