1. Mixing Authentic and Large-Scale Visual Arts Assessment: What might work for you? Thomas Brewer University of Central Florida NAEP Research Team Members Read Diket, William Carey University Lihua Xu, University of Central Florida Anne Grey, University of Central Florida 2012 FAEA Conference Friday, November 9 St. Petersburg, FL

2. Proposal Description This presentation will address the issue of developing authentic visual arts assessment that might be used in larger scale school, district, and state-wide applications. As a National Assessment of Educational Progress (NAEP) Visual Arts Consortium member, the presenter is charged with informing current art educators and administrator about the various curriculum and assessment resources available through the NAEP Arts website and the NAEP Data Explorer (NDE). Whether you are interested in developing assessments for your classroom, school, district, or state, this presentation will provide some foundational entry points.

3. What Might Work for You? My Role with Team Rater procedures for the NAEP assessment and implication for results and student learning. Possible teacher assessment procedures and how teachers could use NAEP testing blocks to assess their students’ art learning. (Assessment, curriculum, policy implications) Provide discernible and useful arts education policy information and interpretations to aid in the improvement of arts education.

4. What Might Work for You? Possible teacher assessment procedures and how teachers could use NAEP Testing blocks to assess their students’ art learning. (Assessment, curriculum, policy implications.) What is NAEP Arts Report Card? http://nces.ed.gov/nationsreportcard/http://nces.ed.gov/nationsreportcard/ > Sample Questions, Analyze Data, Data Explorer or Sample Questions Examples of authentic and large-scale assessment test blocks and instruments: Siegesmund and Diket (2000) Brewer (2011), Developed Bundled Assessment Brewer and Grey with Mother/Child exemplars NAEF 2010

5. What Might Work for You? Relevant Findings I Possible teacher assessment procedures and how teachers could use NAEP Testing blocks to assess their students’ art learning. (Assessment, curriculum, policy implications.) UCF Graduate Research classes use NAEP Data Explorer (NDE) to more closely look at 192 variables. One very important finding was discovered by Kathy Arndt in 2011 were students scores on a 300 point scale were significantly higher only for those who received art instruction from full time or part time art specialist and not from: Artists-in-residence (this might also be applied to teaching artists); Classroom teachers (though we love them); Volunteers (though we love them too), or other faculty (PE).

6. What Might Work for You? Relevant Findings I Possible teacher assessment procedures and how teachers could use NAEP Testing blocks to assess their students’ art learning. (Assessment, curriculum, policy implications.) Specialist in this study only meant being certified but no specifics were given about what path the art educators took to certification. None the less Arndt’s (2012) finding have enormous Teacher preparation and education policy implications and is extremely important to art education. What are some possible meanings and implications?

7. What Might Work for You? Relevant Findings II Provide discernible and useful arts education policy information and interpretations to aid in the improvement of arts education. One very important arts education policy position comes from our current Mother/Child Block analysis where we may be unearthing existing or recognizing new models of art learning and discovering new paths to teaching methodologies given the relationships between students’ art knowledge, technical knowledge, aesthetic properties, and meaning. Dr. Xu

8. What Might Work for You? Relevant Findings II These conclusions seem to have significant implication for curriculum designers, principals, decision makers in curriculum design in the field of visual arts education. At the same time, it provides statistical evidence for the significance of proper question layout to maximize test takers’ performance in arts assessment. Assessment experts and visual arts educators are advised considering presenting the visual arts assessment questions in the order of arts knowledge, technical knowledge and skills, and aesthetic properties, and interpretation of art works. 8SEM analysis

9. What Might Work for You? Relevant Findings III The 2008 Mother/Child block remained essentially the same as 1997. However, the 2008 sample of eighth grade students included a disproportionate number of Hispanic students and a likely increase in high poverty schools. Catholic Schools were over represented in the private school designation. Despite these factors and a small loss of art specialist expertise in the classrooms (65% in 2008; in 1997, 73%), and with coding changes, the basic performance did not decline between the 1997 and 2008 samples.

10. Implications/Applications So what does any of this have to do with you? What sort of assessment practices are you aware of and are using? Are you being ask to assess student learning in art? If so what resources do you use? Do you need some assessment items? 10SEM analysis

11. Some foundations for the 2008 NAEP Secondary Analysis Diket, R. M., Xu, L., and Brewer, T. (2012). Discussing Educational Policy and Implications of Efficacy in the Arts as School Subject Areas: NAEP>Teacher-Made Assessment,> Subject Area Growth. (AERA Symposium Presentation). Diket, R., Xu, L., & Brewer, T. (2012). Toward an Aspirational Learning Model gleaned from large-scale assessment. (Paper under editorial review Studies in Art Education). Diket, R. M., & Brewer, T. (2011). NAEP and policy: Chasing the tail of the assessment tiger. Arts Education Policy Review, 112(3), 35-47. Brewer, T. (2011). Lessons learned from the Bundled Visual Arts Assessment. Visual Arts Research, 37(1), 79-95. Brewer, T. (2008). Developing a Bundled Visual Arts Assessment Model. Visual Arts Research, 34(1), 63-74.

14. Thank you for attending

15. Mother and Child Portraits for NAEP 1997/08 - AG William Sergeant Kendall. Beatrice. Courtesy of Pennsylvania Academy of the Fine Arts, Philadelphia.Philadelphia. Nigerian. Figure of Woman and Child. Courtesy of National Museum of African Art, Washington, D.C.

16. Mother and Child Portraits for NAEP 1997/08 - AG Raphael. The Small Cowper Madonna. Courtesy of National Gallery of Art, Washington, D.C. Courtesy of Joan Miro. Series I. The Family. Byzantine. Madonna and Child on a Curved Throne. Courtesy of National Gallery of Art, Washington, D.C.

17. Mother/Child Constructs Path Diagram 17SEM analysis

18. Hypothesized model Two structural models were conducted with the hierarchical representation from art knowledge to technical to aesthetic properties to meaning and with or without the path from art knowledge to meaning. The hypothesized model is displayed in the path diagram. – represents constructs (latent variables) represents measured items/questions (variables) – Line connecting variables implies a hypothesize direct effect – Absence of a line connecting variables implies lack of a hypothesized direct effect. 18SEM analysis

19. Diket, R., Xu, L., & Brewer, T. (2012). A (Aspirational) Theory of How Students Progress to Meaning in Art. (Paper under editorial review NUMBERS, Online?). Informing Assessment Practices of Art Education Teachers: How Test Blocks from the National Assessment of Educational Progress Arts Report Cards (1997 and 2008) Can Serve as General Models When Designing Classroom Assessment Using Contemporary Art 2010 NAEF Research Grant Read Diket, William Carey University Thomas Brewer and Anne Grey, University of Central Florida