Presentation on theme: "Portfolio Use for Learning and Assessment Dr Val Klenowski"— Presentation transcript:
1 Portfolio Use for Learning and Assessment Dr Val Klenowski
2 Portfolio Use for Learning and Assessment Aims of the session are to:Analyse the formative and summative purposes of portfoliosAnalyse the assessment processes involved in formative and summative portfolio assessmentConsider key issues related to validity and reliability.
3 Portfolio Use for Learning and Assessment Examine a process for using criteria to assess portfolios.Discuss the guidelines for implementation of portfolios for formative and summative purposes.
4 AssessmentAssessment is the purposeful, systematic and ongoing collection of information as evidence for use in making judgments about student learning.(Education Queensland 2001, Years 1-10 Curriculum Framework for Education Qld Schools, Department of Education, p.13)
5 Assessment Purposes Promote, assist and improve students’ learning Inform teaching and learningProvide data that can be communicated to a range of people about the progress and achievements of individual students or groups of students.(Education Queensland 2001, Years 1-10 Curriculum Framework for Education Qld Schools, Department of Education, p.13)
6 Formative and Summative Purposes Assessment of learning equates to summative assessment. Process of summing up or checking what has been learned at the end of a particular stage of learning.Assessment for learning equates to formative assessment. Assessment that helps students learn.(Weeden, Winter and Broadfoot, 2002)
7 Assessment of Learning Adds procedures or tests to existing workInvolves only marking and feedback of grades or marks to students(The Assessment Reform Group.
8 Assessment for Learning Involves students in self-assessmentProvides feedback that leads to students’ recognition of their next steps and how to take themIs underpinned by confidence that every student can improveInvolves both teacher and students in reviewing and reflecting on assessment data(The Assessment Reform Group.
9 Assessment for Learning Is embedded in a view of teaching and learning of which it is an essential partInvolves sharing learning goals with studentsAims to help students know and recognise the standards for which they are aiming(See: Department of Education and Training, Schools of Isolated and Distance Education, Culture of the Dreaming, 2003)
10 What is a portfolio?… a purposeful collection of student work that tells the story of the student’s efforts, progress or achievement in (a) given area(s). This collection must include student participation in selection of portfolio content; the guidelines for selection; the criteria for judging merit; and evidence of student self-reflection.Arter and Spandel, 1992: 36
11 What is a portfolio?“are sources of evidence for judgements of … achievement in a range of contexts, from classroom monitoring of student performance to high stakes summative assessment.… contain ‘pieces of evidence.’ The more relevant the evidence, the more useful it is for inferring a student’s level of achievement in a learning area.”Forster & Masters (1996:2)
12 What is a portfolio?A portfolio is a collection of work that can include a diverse record of an individual’s achievements, such as results from authentic tasks, performance assessments, conventional tests or work samples.A portfolio documents achievements over an extended period of time.
13 Why use portfolios?Produces an accurate and holistic portrait of the student.Involves students in decisions about the choice of inclusions and quality of work completed.Allows students to exhibit difference e.g. multiple intelligences, cultural diversity
14 Why use portfolios?Process oriented portfolios tell a story about the growth of the learner. They document the processes of learning and creating, including earlier drafts, reflections on the process and barriers to learning throughout the course/programme.
15 Why use portfolios?Product oriented portfolios require a student to document and reflect on the quality and range of accomplishments.Improve the quality of teaching by integrating assessment and teaching/learning.Facilitates the accountability of teachers and schools. (Queensland Government, Education Queensland, 2003)
16 Purposes and Processes What are the key assessment concepts with which you need to be familiar if you are to use portfolios for:formative andsummative assessment purposes?
17 Summative PortfoliosFocus on learning outcomes and contain evidence that shows the range and extent of students’ skills. A summative portfolio demonstrates learning outcomes rather than the process of learning. If the intention is to assess a student’s skills or knowledge then the assessment is summative.
18 Formative PortfoliosThe main role of a formative portfolio is to show the processes of learning in which a student has engaged. If the assessment is concerned with the learning process then assessment should be formative.(http://wwweddept.wa.edu.au/deo/goldfields/curriculum/assessment)
19 Formative and Summative When both summative and formative assessment are developed in tandem, the result is more effective assessment and more effective teaching and learning.(The State of Queensland, Department of Education, 2002)
20 Research EvidenceBlack and Wiliam’s review of research on formative assessment in 1998 revealed that much of it was concerned with issues such as: peer- and self-assessment, role of feedback in a pedagogy focused on learning.This finding refocused ‘formative assessment studies away from systems, with its emphasis on the formative-summative interface, and relocated it on classroom processes’ (Black and Wiliam, 2003:628)
21 Research EvidenceFormative assessment raises standards and that current practices are under-developed. (Black and Wiliam, 1998)Difficulties uncovered:Teachers’ tests encourage rote and superficial learningThe questions and methods used are not discussed with other teachers in the same school and they are not critically reviewed in relation to what they actually assessPrimary teachers in particular emphasise quantity and presentation of work to the neglect of quality in relation to learning
22 Research Evidence Negative impact on learning from: The emphasis on the giving of marks and grades rather than useful advice that emphasises the learning function.The use of comparison that students perceive to be for competitive purposes rather than for personal improvement.Negative impact of assessment feedback for students with low attainments is that they believe that they lack ‘ability’ and are de-motivated thinking that they are not able to learn.
23 Research EvidenceThe managerial role of assessments such that teachers’ feedback was used for social and managerial purposes rather than focusing learning.The push to collect marks for record purposes was given priority over the analysis of students’ work to identify their learning needs. (Black, Harrison, Lee, Marshall & Wiliam, 2003)
24 Self-AssessmentsStudents need to engage with the criteria prior to self-assessment and for formative purposes.Use of assessment language in own self-assessments.Not a grading but a reflection on the process.
25 Understanding criteria for formative and summative purposes Consider the criteria and standards for assessment.Criteria are the “the various characteristics or dimensions on which the quality of student performance is judged”Standards are the ”levels of excellence or quality applying along a developmental scale for each criterion”. (Maxwell, 1993:293)
26 Criteria and Standards Examine the assessment criteria for the summative assessment.Identify the criteria.Analyse the criteria.Discuss with members of your group.Reach a consensus of understanding.Be prepared to report back to the larger group your findings.
27 Process Consider criteria and standards for assessment. Analysis of the criteria individually.Discussion with others to integrate understanding.Application of understanding.Discussion, debate, clarification, affirmation and integration.
28 Self-assessments Explicit Criteria Minimum criteria: grasp of key concepts and issues, structure, coherence, logical argument, reflection on professional practice, wide range of reading; academic conventions, referencing, presentation
30 Portfolio Assessment Processes Generally, the individual identifies work from an accumulated collection to illustrate achievement and to demonstrate learning for a particular purpose, such as summative or formative assessmentCareful critical self-evaluation is an integral process and involves judging the quality of one’s performance and the learning strategies involved.For examples see:
31 Portfolio ProcessesThe individual’s understanding of what constitutes quality in a particular context and the learning processes involved is facilitated by discussion and reflection with peers, teachers, lecturers or tutors during interview, substantive conversation, exhibition or presentation of learning.The development of the portfolio involves documentation of achievements, self-evaluations, process aretfacts and analyses of learning experiences, strategies and dispositions. (Klenowski, 2002)
32 ProcessesThe identification of the purpose of the portfolio, together with the criteria and guidelines, can assist the student in selecting work for inclusion in the portfolio.By involving students in the assessment process and by making clear to students that self-evaluation is important for demonstrating what has been learnt, and also how this has been achieved, an important message about what is valued in assessment is being communicated.
33 ProcessesThe developmental nature of learning is promoted by the integration of assessment with curriculum, teaching and learning.The development of the a portfolio of work involves key processes such as self-evaluation, substantive conversation, reflective thinking and practice.
34 ProcessesThese learning processes and associated pedagogic practices, when used to develop a portfolio of work, foster metacognitve development which promotes knowing how, when, where and why one learns.
35 Using Portfolios for Learning & Assessment What are the key issues, major tensions, and constraints?
36 Summative Purposes Administrative considerations time efficientmanageableinexpensiveadequate level of reliabilityconsistency of approach (guidelines)consistency of gradingconsistency of standards
37 Summative Purposes Administrative considerations To ensure consistency of standards need to provide teachers with:ExemplarsCommentary to explain the grade assignedGrade descriptorsGrading instructions(See: Culture of the dreaming e-learning)
38 Using a portfolio for summative purposes requires specification of: standardscontentsby external agency, local authority, academic department, teacher to achieve formal assessment and monitoring.
39 Ensuring Reliability Documented, field tested scoring guides Specified criteriaAnnotated examples of all score pointsAmple practice and feedback for ratersMultiple raters with agreement prior to scoring(See
40 Ensuring Reliability Periodic reliability checks throughout Retraining if necessaryArrangements for the collection of suitable reliability dataForster & Masters (1996:43)
41 Using Portfolios for Learning and Assessment What have been some of the problems when you have used portfolios for learning and assessment purposes?
42 Problems and PitfallsMost attempts to design, develop and implement a portfolio for assessment system have encountered problems that have resulted from:unhelpful policy decisionsconceptual confusionspractical problemstechnical problemspsychological or social barriers.
43 Problems and PitfallsMajor problems associated with new assessment forms such as the portfolio arise from:a mismatch of purpose to paradigmcurriculum designpedagogic practices.
44 Problems and PitfallsThe prevailing understanding of the nature and purpose of assessment can militate against the success of portfolio use by distorting the associated teaching and learning processes.Teachers need to engage in professional development that will support the integration of new assessment systems into the curriculum and their teaching practice, the policy context needs to be supportive of such change
45 Problems and PitfallsStudents need specific teaching and support to develop the cognitive processes of critical reflection and self-evaluation that help them to develop a portfolio of work.Technical problems encountered in the assessment of the portfolio include:validity of inferencesgeneralisabilityreliability andresourcing
46 Problems and PitfallsIn the assessment of the portfolio the threats to validity are:Construct under-representationie omission of important given criteria whenassessing the portfolio of workConstruct irrelevant varianceie use of irrelevant or idiosyncratic criteria notintended or included in the assessmentguidelines
47 Using Portfolios for Learning & Assessment What advice should be given about portfolios in guidelines for their use?
48 Implementing Portfolios for Assessment and Learning Decide on the purpose of the portfolioDetermine the outcomes to be achieved and how the portfolio might be used to demonstrate themCreate the means by which teachers an students can collaborate in determining the contents of portfoliosAllow time for students to reflect and engage in self-assessment
49 Portfolio Building Process Identify the areas of skills that the student is to developFrom these skill areas, develop specific learning outcomes for the student to achieve.Identify performance indicators that establish that the student has achieved their learning outcomes and indicate the evidence that the student needs to collect.Collect evidence that demonstrates the student has met the performance indicators.Organise this evidence in a portfolio so assessors can easily understand how the evidence relates to each performance indicator.(http://wwweddept.wa.edu.au/deo/goldfields/curriculum/assessment)
50 Guidelines for Using Portfolios Include:Suggestions for introduction and implementation of portfolio processFramework for portfolioLearning outcomes and suggestions of evidence suitable to address these outcomes
51 Guidelines for Using Portfolios Advice regarding self-evaluation and reflective statementsCriteria to be used in the assessment of the portfolioGrade descriptorsExemplars that illustrate standards
52 ReferencesArter, J. A, & Spandel, V. (1992) ‘Using Portfolios of Student Work in Instruction and Assessment’, Educational Measurement: Issues and Practice, Spring, ppAssessment Reform Group.Black, P. and Wiliam, D., (2003) ‘In Praise of Educational Research: Formative Assessment,’ British Educational Research Journal, 29,5, ppBlack, P, Harrison, C. Lee, C. Marshall, B. & Wiliam, D. (2003) Assessment for Learning Putting it into practice, Maidenhead: Open University Press
53 ReferencesDepartment of Education and Training, Western Australia, Schools of Isolated and Distance Education, Culture of the Dreaming, 2003Education Queensland 2001, Years 1-10 Curriculum Framework for Education Qld Schools, Department of Education.Forster, M. & Masters, G. (1996) Portfolios, Victoria: Australian Council for Educational Research.
54 ReferencesKlenowski, V. (2002) Developing Portfolios for Learning and Assessment: Processes and Principles, London: RoutledgeFalmer.Maxwell, G. (1993) ‘Criteria and Standards Based Assessment in Applied Statistical Mathematics’, in J.Izard and M.Stephens (eds) Reshaping Assessment Practices in the Mathematical Sciences Under Challenge, Proceedings of the First International Conference on Assessment in the Mathematical Sciences, Geelong, October, 1992, Victoria: Australian Council for Educational Research.Weeden,P., Winter, J. and Broadfoot, P. (2002) Assessment (What’s in it for schools?), London: RoutledgeFalmer.