1. ePortfolio research: Select, Reflect, Connect On-line tracking/monitoring Questionnaires: sampling of learner perceptions Interviews: pre-, during and post- a phase of ePF use Focus groups: real interaction / dialogue with learner Exemplar material A baseline of past practice Identify parallel behaviours in different cultures Action research Case study Linking to attainment, retention, career success Observation over time Longitudinal studies of ePortfolio users Presentation Gallery Discussions with employers Sampling of learner perceptions by questionnaires + interviews Use of control groups, not using ePF? Analyse something like 1 st destination data, but better Need to release the evidence embedded in established practice in existing effective areas Individual thoughts Additional ideas after discussion Stop researching method of implementation of Portfolio Research the depth of learning through research on Learning Journals generally What is the ePortfolio helping learners to achieve? What difference does it make? Presentation gallery Identify the problems people have to which ePF is the answer e.g. How can ePF help make institutions systems work more easily? What would motivate learners at a higher level? Does ePF use by learners change pedagogy? Does semi-automation, pre-population of fields, make a real difference to student engagement, by providing a high starting point? Select What methods will help answer your questions?

2. ePortfolio research: Select, Reflect, Connect Impact on curriculum, practice, quality reflections / usage In terms of logical services, breaking down PDP In terms of how discrete services form flows Against predictive outcomes Cultural / human aspects Technology implications Quantitatively - % of users etc Qualitatively – feedback from those in categories 1. and 3. above Collate and compare comments. Identify specific themes and common issues. Follow up with deeper research e.g. interview or with wider research e.g. questionnaire Individual thoughts Additional ideas after discussion Cultural / human aspects Changes in practice Sufficient notion practice Kinds of service that technology needs to offer Educationalists vs Technologists ePF aggregates existing domains of practice The value of ePF is in integration of previously separate activities, e.g. ILP and IAG Literature reviews on Journals / use of electronic processes, etc Changes in practice Reflect How could the data be analysed?

3. ePortfolio research: Select, Reflect, Connect Exemplar material on-line for student reference; articles? To the domains of practice for logical services Through implementation of example systems Workshops Networks At different levels: - strategists - managers - potential users as well as academics To inform development and usage Need to disseminate to all who have responsibility for front-line implementation – need to demonstrate relevance – face to face best Disseminate through reports, presentations, film, discussion, conferences, blogging, etc. Individual thoughts Additional ideas after discussion Avoid self-serving academic roundabout Audience? Policy makers in government? Managers in School ECM agenda Exemplars Do/Should ePortfolios need implementers? Inform future development audience Connect How should the findings be disseminated?

4. ePortfolio research: Select, Reflect, Connect General comments If is easy to provide answers to the wrong questions, in particular to demonstrate the benefit of learning a technique rather than the principles behind a technique. Peter Rees Jones If the question is: How far does the use of ePortfolios by learners (this category needs breaking down into major groups /types) enhance: a. Integrative Learning; b. EmployabilityAngela Smallwood

5. ePortfolio research: Connect Project reports, vendors claims Own institution case studies Conference presentation Discussion forums, standard body documents HESA stats, QA assumptions Number of ePF holders – by type, institution, demography, etc. Number of institutions using ePF Type of organisation using ePF Demographic data of ePF holders Purposes of ePF Types of ePF hosts Size of ePF storage Software products used to support ePF Frequency of use of ePF Age of ePF General topics / sections within ePF Financial data Any that is interoperable A lot?A little? Student reactionswhy they feel Student usagethat way Staff attitudes Personal details use UCAS CETL reward projects (+ELP?) –Application of these projects to next cohort and response to methodology –Student feedback from reward project –Industry feedback from organisations – who have employed ePortfolio students vs non-ePortfolio students. –Human-computer interaction research (particularly where it relates to users who work paper-based) ease of transition –Soft-skills methodology? Individual thoughts Additional ideas after discussion Unexpected outcomes Make researchers assumptions explicit UCAS / HESA Number of holders / users / institutions (Demographics; Where are they coming from; etc.) Purposes of ePF Technology – types, hosts, storage, size Frequency of use – maintainability Age of ePF Cost Different subject areas? Tool to complete or purpose usage stats from non vocational courses Vendor claims … Access to personal data for research purposes What data are already available?

6. ePortfolio research: Connect Self-conception of Identity Usage stats Qualitative data for forming categories Problem of access personal data Prof competencies: what profs want to be collected Take up rates of generically offered ePF, PDP within depts Employment skills required Users of social space sites – numbers grouped by institution, type of study Reasoning and priorities of institution, implementing PDP& eP Usability testing and feedback from implemented systems Transition, progression (achievement and impact) and retention data on users and non-users, or different type of ePFs How it has(nt) aided employability How it has(nt) aided confidence, identity How theyre used and who used by What role do they play? [Feedback] Can they be used in assessments? [Feedback] Interviews, focus groups, tracking data? How students come to fill them in What they really do with them Long-term usage and impact Evidence CETL reward projects (+ELP?) Individual thoughts Additional ideas after discussion Transition, progression, retention, achievement - Users and non-users - Impact on employability - Impact on users Depends on purpose of research Discipline dependencies Views of learners and teachers Project reports Case studies UCAS Is e relevant aside from just portfolio? User groups What subject areas are using it? Which and how many institutions are using it? Behaviour tracking, output analysis Observations Financial data (cost) Usage across the UK? What data need to be collected?

7. ePortfolio research: Connect School / uni data on marks, destinations Questionnaires and interviews Protocol analysis? Behaviour tracking (observation and analysis of output) Usable STD format (electronic) Quantitative Individual thoughts Additional ideas after discussion Qualitative, quantitative assumptions of implementers Guides, intros, include explanations Reactions Any formats that are interoperable Survey, portfolios, interviews In what format?

8. ePortfolio research: Connect an ePortfolio Time Taxonomies of Portfolio practice Interpersonal / ethnographic skills Usage and take up rates Usage patterns – just in time or developmental non-users Survey tools Co-operation Researchers / Framework – Linking / usable across a network of all institutions Case study built-up - Learners need CURRENT PERSPECTIVE/OPINION vs. IDEAL? - Portfolio vs. ePortfolio How are paper-based portfolios more/less effective? - Subject areas? Same approach for each area vs. differentiation? Individual thoughts Additional ideas after discussion Permission Access External -School / college / uni data -Questionnaires and interviews Survey tools Co-operation Reflection: how is this identified, recognised, measured? Researchers? Across Universities What resources are needed for collection?

9. ePortfolio research: Connect General comments Purpose: Recipient Design; Policy formation; Evaluation; Social Scientific research; Institutional; Practitioner

10. ePortfolio research: Purpose and Audience To inform the design of effective ePortfolios systems To understand the scope and purpose of ePortfolios To understand how ePortfolios fit into the space of learner support systems To share good practice To add to body of research To enable development of ePortfolios To give concrete, quantifiable evidence and avoid folksonomy and demonstrate what purpose / use is actually most helpful and useful. Individual thoughts Additional ideas after discussion Social Network The importance of social interaction what is it around the take up of some social software that engenders success The importance of the human interface Traffic ·|· formal and informal learning processes What is the purpose?

11. ePortfolio research: Purpose and Audience Systems designers Policy makers Researchers Teachers and lecturers Developers (open source and commercial) Government bodies Institutions Learners Policy makers – where does the government need to intervene and where not Educational practitioners and institutions Policy makers, developers, practitioners (and those who train them) Individual thoughts Additional ideas after discussion Social Network The importance of social interaction what is it around the take up of some social software that engenders success The importance of the human interface Traffic ·|· formal and informal learning processes Who must the research inform (audience)?

12. ePortfolio research: Purpose and Audience How can ePortfolio assist a learner to cross the boundary between formal and informal learning? How can ePortfolios condition, frame and assist a lifetime of learning? How can ePortfolios be appreciated by the learner as a valued tool? How ePortfolios aid transition Impact on employment / admission to appropriate courses How will ePortfolio processes drive, support or hinder the modernisation of current industrial revolution style education? Does collection / showing of evidence actually support transition? Individual thoughts Additional ideas after discussion Evidence from employers perspective www.h2g2.com What makes social software successful? (H2G2 model vs Wikipedia) What specific research questions interest you?

13. ePortfolio research: Purpose and Audience A continuing set of interlinked projects, forming a community of researchers and practitioners 2 years + - long term so can track Why would learners want to use eP processes if they were not required by education / employment etc. to do so? In other words, what is in it for them outside the general requirements of education and employment More than 3yrs – for evidence of lifelong use, therefore need reasonably sized cohort to be able to collect results. >50? Individual thoughts Additional ideas after discussion Human / system interfaces What should be the scale of the research (scope and timeline)?

14. ePortfolio research: Purpose and Audience Researchers Practitioners / universities (experience + know- how) CETIS and other pertinent bodies What happens with learners who do not like articulating themselves through ePortfolios in: Required eP situations Voluntary use of eP X – sectoral groups – balance of pedagogy and users including lay people. Individual thoughts Additional ideas after discussion Trying to make it personalised – why is it worth doing anything at all Need to show benefits, need evidence based Who should do research into ePortfolios? Digital (literacy), articulacy, divide