Portfolio Use for Learning and Assessment Dr Val Klenowski

Portfolio Use for Learning and Assessment Aims of the session are to: Analyse the formative and summative purposes of portfolios Analyse the assessment processes involved in formative and summative portfolio assessment Consider key issues related to validity and reliability.

Portfolio Use for Learning and Assessment Examine a process for using criteria to assess portfolios. Discuss the guidelines for implementation of portfolios for formative and summative purposes.

Assessment Assessment is the purposeful, systematic and ongoing collection of information as evidence for use in making judgments about student learning. (Education Queensland 2001, Years 1-10 Curriculum Framework for Education Qld Schools, Department of Education, p.13)

Assessment Purposes Promote, assist and improve students’ learning Inform teaching and learning Provide data that can be communicated to a range of people about the progress and achievements of individual students or groups of students. (Education Queensland 2001, Years 1-10 Curriculum Framework for Education Qld Schools, Department of Education, p.13)

Formative and Summative Purposes Assessment of learning equates to summative assessment. Process of summing up or checking what has been learned at the end of a particular stage of learning. Assessment for learning equates to formative assessment. Assessment that helps students learn. (Weeden, Winter and Broadfoot, 2002)

Assessment of Learning Adds procedures or tests to existing work Involves only marking and feedback of grades or marks to students (The Assessment Reform Group. http://www.assessment-reform-group.org.uk.2002)

Assessment for Learning Involves students in self-assessment Provides feedback that leads to students’ recognition of their next steps and how to take them Is underpinned by confidence that every student can improve Involves both teacher and students in reviewing and reflecting on assessment data (The Assessment Reform Group. http://www.assessment-reform-group.org.uk.2002)

Assessment for Learning Is embedded in a view of teaching and learning of which it is an essential part Involves sharing learning goals with students Aims to help students know and recognise the standards for which they are aiming (See: Department of Education and Training, Schools of Isolated and Distance Education, Culture of the Dreaming, 2003)

What is a portfolio? … a purposeful collection of student work that tells the story of the student’s efforts, progress or achievement in (a) given area(s). This collection must include student participation in selection of portfolio content; the guidelines for selection; the criteria for judging merit; and evidence of student self-reflection. Arter and Spandel, 1992: 36

What is a portfolio? “are sources of evidence for judgements of … achievement in a range of contexts, from classroom monitoring of student performance to high stakes summative assessment. … contain ‘pieces of evidence.’ The more relevant the evidence, the more useful it is for inferring a student’s level of achievement in a learning area.” Forster & Masters (1996:2)

What is a portfolio? A portfolio is a collection of work that can include a diverse record of an individual’s achievements, such as results from authentic tasks, performance assessments, conventional tests or work samples. A portfolio documents achievements over an extended period of time.

Why use portfolios? Produces an accurate and holistic portrait of the student. Involves students in decisions about the choice of inclusions and quality of work completed. Allows students to exhibit difference e.g. multiple intelligences, cultural diversity

Why use portfolios? Process oriented portfolios tell a story about the growth of the learner. They document the processes of learning and creating, including earlier drafts, reflections on the process and barriers to learning throughout the course/programme.

Why use portfolios? Product oriented portfolios require a student to document and reflect on the quality and range of accomplishments. Improve the quality of teaching by integrating assessment and teaching/learning. Facilitates the accountability of teachers and schools. (Queensland Government, Education Queensland, 2003)

Purposes and Processes What are the key assessment concepts with which you need to be familiar if you are to use portfolios for: formative and summative assessment purposes?

Summative Portfolios Focus on learning outcomes and contain evidence that shows the range and extent of students’ skills. A summative portfolio demonstrates learning outcomes rather than the process of learning. If the intention is to assess a student’s skills or knowledge then the assessment is summative.

Formative Portfolios The main role of a formative portfolio is to show the processes of learning in which a student has engaged. If the assessment is concerned with the learning process then assessment should be formative. (http://wwweddept.wa.edu.au/deo/goldfields/curriculum/assessment)

Formative and Summative When both summative and formative assessment are developed in tandem, the result is more effective assessment and more effective teaching and learning. (The State of Queensland, Department of Education, 2002)

Research Evidence Black and Wiliam’s review of research on formative assessment in 1998 revealed that much of it was concerned with issues such as: peer- and self-assessment, role of feedback in a pedagogy focused on learning. This finding refocused ‘formative assessment studies away from systems, with its emphasis on the formative-summative interface, and relocated it on classroom processes’ (Black and Wiliam, 2003:628)

Research Evidence Formative assessment raises standards and that current practices are under-developed. (Black and Wiliam, 1998) Difficulties uncovered: Teachers’ tests encourage rote and superficial learning The questions and methods used are not discussed with other teachers in the same school and they are not critically reviewed in relation to what they actually assess Primary teachers in particular emphasise quantity and presentation of work to the neglect of quality in relation to learning

Research Evidence Negative impact on learning from: The emphasis on the giving of marks and grades rather than useful advice that emphasises the learning function. The use of comparison that students perceive to be for competitive purposes rather than for personal improvement. Negative impact of assessment feedback for students with low attainments is that they believe that they lack ‘ability’ and are de-motivated thinking that they are not able to learn.

Research Evidence The managerial role of assessments such that teachers’ feedback was used for social and managerial purposes rather than focusing learning. The push to collect marks for record purposes was given priority over the analysis of students’ work to identify their learning needs. (Black, Harrison, Lee, Marshall & Wiliam, 2003)

Self-Assessments Students need to engage with the criteria prior to self-assessment and for formative purposes. Use of assessment language in own self-assessments. Not a grading but a reflection on the process.

Understanding criteria for formative and summative purposes Consider the criteria and standards for assessment. Criteria are the “the various characteristics or dimensions on which the quality of student performance is judged” Standards are the ”levels of excellence or quality applying along a developmental scale for each criterion”. (Maxwell, 1993:293)

Criteria and Standards Examine the assessment criteria for the summative assessment. Identify the criteria. Analyse the criteria. Discuss with members of your group. Reach a consensus of understanding. Be prepared to report back to the larger group your findings.

Process Consider criteria and standards for assessment. Analysis of the criteria individually. Discussion with others to integrate understanding. Application of understanding. Discussion, debate, clarification, affirmation and integration.

Self-assessments Explicit Criteria Minimum criteria: grasp of key concepts and issues, structure, coherence, logical argument, reflection on professional practice, wide range of reading; academic conventions, referencing, presentation

Self-assessments Implicit Criteria Analysis, synthesis, evaluation, critical reflection, exploration of theory-practice links

Portfolio Assessment Processes Generally, the individual identifies work from an accumulated collection to illustrate achievement and to demonstrate learning for a particular purpose, such as summative or formative assessment Careful critical self-evaluation is an integral process and involves judging the quality of one’s performance and the learning strategies involved. For examples see: http://www.ilc.edu.hk

Portfolio Processes The individual’s understanding of what constitutes quality in a particular context and the learning processes involved is facilitated by discussion and reflection with peers, teachers, lecturers or tutors during interview, substantive conversation, exhibition or presentation of learning. The development of the portfolio involves documentation of achievements, self-evaluations, process aretfacts and analyses of learning experiences, strategies and dispositions. (Klenowski, 2002)

Processes The identification of the purpose of the portfolio, together with the criteria and guidelines, can assist the student in selecting work for inclusion in the portfolio. By involving students in the assessment process and by making clear to students that self-evaluation is important for demonstrating what has been learnt, and also how this has been achieved, an important message about what is valued in assessment is being communicated.

Processes The developmental nature of learning is promoted by the integration of assessment with curriculum, teaching and learning. The development of the a portfolio of work involves key processes such as self-evaluation, substantive conversation, reflective thinking and practice.

Processes These learning processes and associated pedagogic practices, when used to develop a portfolio of work, foster metacognitve development which promotes knowing how, when, where and why one learns.

Using Portfolios for Learning & Assessment What are the key issues, major tensions, and constraints?

Summative Purposes Administrative considerations time efficient manageable inexpensive adequate level of reliability consistency of approach (guidelines) consistency of grading consistency of standards

Summative Purposes Administrative considerations To ensure consistency of standards need to provide teachers with: Exemplars Commentary to explain the grade assigned Grade descriptors Grading instructions (See: Culture of the dreaming e-learning)

Using a portfolio for summative purposes requires specification of: standards contents by external agency, local authority, academic department, teacher to achieve formal assessment and monitoring.

Ensuring Reliability Documented, field tested scoring guides Specified criteria Annotated examples of all score points Ample practice and feedback for raters Multiple raters with agreement prior to scoring (See http://www.eddept.wa.edu.au/mse/materials.html)

Ensuring Reliability Periodic reliability checks throughout Retraining if necessary Arrangements for the collection of suitable reliability data Forster & Masters (1996:43)

Using Portfolios for Learning and Assessment What have been some of the problems when you have used portfolios for learning and assessment purposes?

Problems and Pitfalls Most attempts to design, develop and implement a portfolio for assessment system have encountered problems that have resulted from: unhelpful policy decisions conceptual confusions practical problems technical problems psychological or social barriers.

Problems and Pitfalls Major problems associated with new assessment forms such as the portfolio arise from: a mismatch of purpose to paradigm curriculum design pedagogic practices.

Problems and Pitfalls The prevailing understanding of the nature and purpose of assessment can militate against the success of portfolio use by distorting the associated teaching and learning processes. Teachers need to engage in professional development that will support the integration of new assessment systems into the curriculum and their teaching practice, the policy context needs to be supportive of such change

Problems and Pitfalls Students need specific teaching and support to develop the cognitive processes of critical reflection and self-evaluation that help them to develop a portfolio of work. Technical problems encountered in the assessment of the portfolio include: validity of inferences generalisability reliability and resourcing

Problems and Pitfalls In the assessment of the portfolio the threats to validity are: Construct under-representation ie omission of important given criteria when assessing the portfolio of work Construct irrelevant variance ie use of irrelevant or idiosyncratic criteria not intended or included in the assessment guidelines

Using Portfolios for Learning & Assessment What advice should be given about portfolios in guidelines for their use?

Implementing Portfolios for Assessment and Learning Decide on the purpose of the portfolio Determine the outcomes to be achieved and how the portfolio might be used to demonstrate them Create the means by which teachers an students can collaborate in determining the contents of portfolios Allow time for students to reflect and engage in self-assessment

Portfolio Building Process Identify the areas of skills that the student is to develop From these skill areas, develop specific learning outcomes for the student to achieve. Identify performance indicators that establish that the student has achieved their learning outcomes and indicate the evidence that the student needs to collect. Collect evidence that demonstrates the student has met the performance indicators. Organise this evidence in a portfolio so assessors can easily understand how the evidence relates to each performance indicator. (http://wwweddept.wa.edu.au/deo/goldfields/curriculum/assessment)

Guidelines for Using Portfolios Include: Suggestions for introduction and implementation of portfolio process Framework for portfolio Learning outcomes and suggestions of evidence suitable to address these outcomes

Guidelines for Using Portfolios Advice regarding self-evaluation and reflective statements Criteria to be used in the assessment of the portfolio Grade descriptors Exemplars that illustrate standards

References Arter, J. A, & Spandel, V. (1992) ‘Using Portfolios of Student Work in Instruction and Assessment’, Educational Measurement: Issues and Practice, Spring, pp. 36-44. Assessment Reform Group. http://www.assessment-reform-group.org.uk.2002 Black, P. and Wiliam, D., (2003) ‘In Praise of Educational Research: Formative Assessment,’ British Educational Research Journal, 29,5, pp.623-637. Black, P, Harrison, C. Lee, C. Marshall, B. & Wiliam, D. (2003) Assessment for Learning Putting it into practice, Maidenhead: Open University Press

References Department of Education and Training, Western Australia, Schools of Isolated and Distance Education, Culture of the Dreaming, 2003 Education Queensland 2001, Years 1-10 Curriculum Framework for Education Qld Schools, Department of Education. Forster, M. & Masters, G. (1996) Portfolios, Victoria: Australian Council for Educational Research.

References Klenowski, V. (2002) Developing Portfolios for Learning and Assessment: Processes and Principles, London: RoutledgeFalmer. Maxwell, G. (1993) ‘Criteria and Standards Based Assessment in Applied Statistical Mathematics’, in J.Izard and M.Stephens (eds) Reshaping Assessment Practices in the Mathematical Sciences Under Challenge, Proceedings of the First International Conference on Assessment in the Mathematical Sciences, Geelong, October, 1992, Victoria: Australian Council for Educational Research. Weeden,P., Winter, J. and Broadfoot, P. (2002) Assessment (What’s in it for schools?), London: RoutledgeFalmer.