1. CRESST’s Evaluation of the Artful Learning Program: “Findings,” Contexts, and Future Explorations Noelle Griffin,Ph.D UCLA Graduate School of Education & Information Studies National Center for Research on Evaluation, Standards, and Student Testing (CRESST) CRESST Conference, September 10, 2004

2. Artful Learning Program  Arts-based education model: Arts infused across the curriculum  Currently implemented in 7 states, 22 schools  Four phases: Experience, Inquire, Create, Reflect  Use of Masterworks of art

3. Where Does Artful Learning Fit?  Comprehensive school reform  Instructional, organizational, assessment components  Focused on helping kids improve academic performance  Depends upon trained teachers and administrators

4. Theory of Action for Artful Learning The underlying theory motivating the Artful Learning model is that the joy, discipline, and commitment required by the arts will provide a useful framework for the overall improvement of education and for individual growth

5. Evaluation Background  CRESST/UCLA  Beginning in 2001  Both external and internal applications  Qualitative and quantitative strategies  Emphasis: Capacity building

6. Evaluation Methods  Yearly implementation survey (participating schools, n=@400 per year)  Pre-post professional development surveys (on-going)  Teacher/administrator interviews (n=@30 per year)  School-level achievement data  Supplemental information

7. Academic Achievement Student Engagement/Motivation Quality Implementation Organization And Leadership Classroom Instruction Connections Processes Professional Development/ Support District/Schoo l Context

8. Contextual Factors  Need for both administrative and district support  Middle/high school vs. primary  Teacher/student turnover  Role of arts teachers/specialists  Based on qualitative data

9. Professional Development and Support  Overall quality/utility high (quantitative and qualitative)  Consistency across levels/experience  Differences across program components

10. Professional Development Quality: Percent “Agree” or “Strongly Agree”

11. Significant Pre/Post Development Increase in Reported Teacher Expertise (MANOVA)  Level I and Level II  Artful Learning Instructional Process  Assessment Practices  Organizational/structural Practices  General Instructional Quality

12. How Well Did Development Prepare You to Implement Classroom Component?

13. How Well Did Development Prepare You to Implement Assessment Component?

14. Implementation Survey Multiple Regression Results: Implementation of Artful Learning Classroom Process Artful Learning Process Standards Use/Understanding Assessment Use/Understanding High Quality Instruction Reported Impact Shared Mission * * * * * *=Significantly Predicts p<.05

15. Multiple Regression: No Significant Relationship to Implementation  Artful learning experience  Teaching experience/Teacher background  Parent involvement  Shared leadership

16. Implementation: Other Findings  Teacher evaluation of program after use is high  Continued growth of program/use over time  Assessment/parent involvement weakest components  Based on Qualitative and Quantitative Data

17. Additional Findings: Shared Efficacy  Teacher perceptions of school’s overall effectiveness in teaching process  Student achievement links  Group competency and task analysis components  Included in implementation survey

18. Multiple Regression Findings: Shared Efficacy Shared Leadership Years Experience Grade Level Taught Combined Shared Efficacy Combined Shared Efficacy Group Competency Task Analysis *All Significant relationships are negative *No significant relationship to other Artful learning implementation variables

19. Student Outcomes  Engagement and other teacher- reported outcomes  Achievement  Quantitative and qualitative consistency  Access to information – what teachers “don’t know” re: assessment results

20. Teacher Reported Program Impact: Student Engagement

21. Teacher Reported Program Impact: Quality of Student Work

22. Teacher Reported Program Impact: Classroom Tests/Assessments

23. School-Level Achievement Data: Limitations  Cohort group issues  Dilution of effects  Test changes/lack of comparability  Insufficient information statistically

24. Tentative Findings: Comparative Average Growth in Students Meeting/Exceeding Standards GRAMMYDISTRICTCOMP. SCHOOLS READING16109 ELA985 MATH1296

25. CRESST Project Synergy: QSP use in Artful Learning  QSP: Quality School Portfolio assessment use software  Address program difficulties collecting achievement data in multiple districts/states  Build project self-evaluation capacity

26. Potential QSP Applications for Artful Learning:  Track student-level data  Compare participating vs. non- participating students  Compare students based on “dosage”  Compare teachers based on program experience

27. Enduring Questions Related to Arts Education  What should we value as attainment of serious learning?  Can arts programs in schools be sustained as political and parental pressure increasingly focuses on traditional academic attainments?  What should count as quality education?  How scalable is change away from a compartmentalized view of learning?  How can such change be sustained and expanded ?