1. Principles of Electronic Portfolios
Simon Cotterill, Tony McDonald
School of Medical Education Development
University of Newcastle upon Tyne, UK

2. Objectives Proposed Structure
The workshop draws on the experiences of developing, implementing and evaluating Web-based ePortfolios, at the University of Newcastle, UK.
Objectives
To discuss some of the principles of electronic portfolios
To gain familiarity with the potential educational and technical
‘value-added’ features of using electronic media.
To consider approaches to the implementation of ePortfolios
Proposed Structure
Presentation: Principles of ePortfolios
Exercise: ePortfolios in your context
Presentation: Case study: FDTL4 ePortfolio project
Exercise: Strengths and weaknesses
Plenary / discussion

3. Introduction – What are Portfolios ?
Unstructured
Formative
Sample
Best work
Factual / Quantitative
Learner Owned
Structured
Summative
All work
Representative
Reflective / Interpretive
Employer Owned
Portfolios have been around for years, mean all things to all people.

4. Introduction – What are Portfolios?
A collection of work or evidence on performance
Processes (the ‘journey’)
Learning Outcomes (the ‘destination’)
Record volume and variety of experiences
Essays, project work, logs of experiences/achievements, artwork, records of accomplished work, audits etc.
Portfolios are defined by their PURPOSE(s)
Employment: Builder gets commissions on the basis of his record of satisfied customers
Portfolio for application: The researcher includes a personal profile in a grant application to show experience
Professional requirements: Med/Law/Education -evidence of essential experience and competency that can be audited
Aims of the session
What are ePortfolios ?
Portfolios in Medicine: context and rationale

5. What might an Electronic Portfolio be?
A computer application which stores the user’s information on a local drive.
A static Web page eg. CV on the Internet
A portfolio composed using ‘simple tools’ (e.g. Word, Dreamwaever, Powerpoint etc.)
A sophisticated database-driven system, typically accessed over a network.
The term “electronic portfolio” is potentially confusing as there are many definitions…

6. 3D Model Overview (Planning / PDP) Content Discussion (Evidence)
Model presented by Jan van Tartwijk (Utrecht University)
Presented at the ALT-SURF Spring Conference & Research Seminar,
Edinburgh 2004

7. Principles of Electronic Portfolios
1. Should bring significant advantages over paper- based alternatives
2. Should be considered in the context of wider (human) processes
3. Clarity of purpose(s)
4. One size does not fit all
5. Should be learner-centric
6. Should be an integral part of the learning experience
7. Should support life-long learning
8. Research / evaluation is essential
….not an exhaustive list !

8. Principle 1. ePortfolios should bring significant advantages over paper-based alternatives

9. 10 ‘value added’ features of an online approach to portfolios
Highly customisable
Multi-purpose eg. formative & summative = reduced duplication
Multiple structures / views
Easier cross-referencing
Sharable / facilitate interaction
Transportable
Searchable
Reduced admin
Secure access from a range of locations
Not left on the bus !
My Portfolio
Easier cross-referencing
eg. linking a record for course with one ore more learning outcomes
this has a pedagogic impact as it encourages a more ‘longitudinal outlook’
rather than just simply completing a linier sequence of forms.
Shareable
potential for formative input and peer-learning
Transportable
can be output in a variety of formats
Reduced admin
monitor form completion and automate reminders

10. Potential Weaknesses of ePortfolios
There may be barriers to effective use of electronic portfolios e.g.
Limited access to computers / Internet
Varied IT experience & Technophobia !
Reliability of IT systems
Possible duplication / overlap with paper systems
Research / Evaluation is essential (Principle 8)
Optimisation will take a number of years
There are some potential drawbacks to electronic portfolios.

11. Principle 2. ePortfolios should be considered in the context of wider (human) processes

12. Humans are important too !
Need buy-in of key stakeholders:
Learners
Clarity of purpose ?
Sense of Ownership ?
Tutors & Admin staff
Extra work ?
Senior Curriculum staff
Employers / Professional bodies
Integration with existing workflow and/or change in practice and culture ?
Training and support (all)
Monitoring and evaluation

13. Principle 3. Clarity of purpose(s)
Choose / design your portfolio according to purpose
Learners and assessors also need to be clear about purpose and requirements (motivation / fairness)
ePortfolios can readily support multiple purposes
Portfolio assessment

14. Summative and Formative Purposes
Portfolios are often used to evidence the achievement of learning outcomes for summative assessment.
There may be potential problems if the portfolio serves both formative and summative processes.
Reflection is less likely to be open and honest if the learner knows that the work will be assessed (loss of authenticity).
The production of a portfolio can itself be a formative learning process ie. it is as much a ‘journey’ as an end-point for assessment.

15. Summative and Formative Purposes (2)
The use of electronic portfolios may potentially reduce the tension between formative and summative processes by supporting both assessed and private / non-assessed content.
Students have the choice to select which content is private and which is made available to assessors, appraisers, tutors, peers, and others involved in their education.

16. Supporting Multiple Purposes
Portfolio for
Presentation
Portfolio for
Assessment
Portfolio for
Application
(job / promotion)
Portfolio for
Accreditation/
Revalidation
Learner’s
‘repository’
Institutional
Data
Portfolio for
Appraisal
Central data:
Transcript
MIS/ HR data
Programme data:
Granular assessment data
Outcomes / skills sets
PDP
(shared)
PDP / Reflective
(private)

17. Portfolio Assessment Where to begin ? Key issues: Validity
Baume D. A Briefing on the Assessment of Portfolios.
LTSN Generic Centre. Assessment Series No.6 (2001)
Friedman M, Davis MH, Harden RM, Howie PW, Ker J, Pippard MJ
AMEE Medical Education Guide No. 24: Portfolios as a method of student assessment. Medical Teacher. 2001; 23:535-51
Knows
Knows how
Does
Shows how
Miller GE. Acad Med 1990:S63-7
Assessing the
development of
competencies
over time
Key issues:
Validity
Reliability
Attitudes & behaviours
Time demands

18. ePortfolios and Assessment
of assessment (results / essays etc)
for assessment
-Potential benefits of ePortfolios:
Formative input during portfolio building (not just at end)
Random sampling & ‘drilling down’
Peer Assessment (formative and summative)
Automatic inclusion of assessment results + feedback
Automated summary information (quantitative)
Reduced monitoring / admin demands
-Piloting essential before use in ‘high-stakes’ assessment
-Need for further research in this field !

19. Principle 4. One size does not fit all
Institutional Subject / context
requirements vs specific
(standardisation) requirements
ePortfolios should be highly flexible / customisable
support diversity (accessibility, learning styles etc)

20. The need for flexibility & customisation
e.g. selecting tools by course / year groups
Course Admin view
Student view
install from a set of ‘generic’ tools
create context-specific
tools via simple Web forms

21. The need for flexibility & customisation (2)
e.g. configuring learning outcomes / skills sets
Course Admin view
Student view

22. Principle 5. ePortfolios should be learner-centric
Personalisation / customisation
Sense of ownership / privacy / sharing
Constructivist theories.
is eLearning contributing to ‘Paradigm shift’ ?
Student Independent Learner
Teacher Facilitator
Didactic Interactive + Explorative
1 location Distance learning
1 provider Multiple providers

23. Emphasis on Learner Ownership / Control
Students control which parts of their portfolio can be viewed for specific purposes or by specific people
Mixed-Ownership Model
Personal
Data
Corporate
(HR) data
This may ease the tension between using portfolios for formative and summative assessment
Tutor’s
Data

24. Ability for Learners to Share their Portfolio Content
Users can control access to their portfolio

25. Adding ‘child’ objects to an artefact
Flexibility for the Learner to Add, Link & Cross-Reference ePortfolio Artefacts
Adding ‘child’ objects to an artefact
Cross-reference
An artefact with
A skill

26. Principle 6. ePortfolios should be an integral part of the learning experience
Educational (integration with the curriculum)
Technical (integration within the MLE)

27. Educational integration Technical integration
Should not be perceived as a ‘bolt-on’
Support a holistic view of curriculum / outcomes (as well as specific components)
Technical
integration
(example from
Medicine at
Newcastle)
ePortfolio integrated with the VLE
Tool properties include ‘module code’
= dynamic links to Study Guides + resources
Learning outcomes linked to curriculum databases: ‘modules’ and units
Links to other systems eg. SSC selection
2 main areas of development in the first year of the project.
Firstly a generic portfolio
This is a stand-alone application
We have developed a common portfolio framework
-this allows sharing, sorting, cross-referencing
It has been developed to be highly customisable:
-components can be customised by course and year group
-learning outcomes or skills sets can be customised by course
-terminology can be customised eg. ‘tutor’ vs. ‘educational supervisor’
Secondly we have applied this technology to our undergraduate MBBS programme
This version has been integrated into the virtual learning environment
Developments are in place for piloting in 2003/4 across the curriculum:
-phase 1 students have a reflective diary, CV and other tools
-trialled in the stage 3 essential junior rotation in Child Health
-all our stage students have to complete the portfolio for 1 of their 3 SSCs

28. Principle 7. ePortfolios should support life-long learning
Help develop life-long learning skills
Continuity in LLL (transferability)

29. Continuity in LLL (transferability)
Towards a life-long learning record
School HE Continuing Development
Interoperability
(technical standards to support the interaction and transfer of data between IT systems)
IMS specifications /standards:
Learner Information Package (LIP)
Enterprise / Content Packaging
ePortfolio (early stage of development)
CETIS UK Learner Profile (based on IMS-LIP)
ePortfolio interaction with VLEs / MLEs

30. The ePortfolio process should promote the develop life-long learning skills
Preparation for professional requirements:
Independent learning skills
Personal development planning
Time-efficient evidencing of performance / development for:
- Appraisals
- Assessment
- Revalidation
Aiming to promote a reflective
/ deep-learning approach ….research

31. Principle 8. Research / Evaluation is Essential
Are the purpose(s) being fulfilled ?

32. Evaluation and Refinement
‘A good fit happens over time with lots of use’
Spandel, 1997
cited by Jan van Tartwijk (ALT-SURF Conference, 2004)
Action Research approaches
Incremental development and refinement
Qualitative research methodologies
(+some quantitative - not mutually exclusive!)
Ethical and practical considerations

33. Research and Evaluation
Does the ePortfolio improve learning ? How so ?
Some evidence that PDP improves learning (more needed)
A systematic map and synthesis review of the effectiveness of personal Development Planning for improving student learning. EPI Centre
Can ePortfolios foster a reflective approach to learning ?
How defined ? How measured ?
How do particular groups of learners approach reflective portfolios ?
Learning Styles
Gender and Age
Cultural Background
John Mole. The geography of thinking. Clin Med 2002; 2:343-5
Does the technology change the learning process ?
What are learner and staff perceptions / attitudes towards ePortfolios ?
There are potential drawbacks to electronic portfolios.
Demographic factors
People in different cultures are taught to think differently. How we gather information, process, rationalise, justify and communicate our ideas is culturally determined……”

34. Principles of Electronic Portfolios
1. Should bring significant advantages over paper- based alternatives
2. Should be considered in the context of wider (human) processes
3. Clarity of purpose(s)
4. One size does not fit all
5. Should be learner-centric
6. Should be an integral part of the learning experience
7. Should support life-long learning
8. Research / evaluation is essential
….the challenge is putting the principles into practice !

35. Further information: S.J.Cotterill@ncl.ac.uk