1. RAMS Overview James Dalziel Professor of Learning Technology, and Director, Macquarie E-Learning Centre Of Excellence (MELCOE) Macquarie University james@melcoe.mq.edu.au www.melcoe.mq.edu.au Presentation for Cambridge University workshop, October 9 th, 2006

2. Overview RAMP Project More on LAMS Introducing RAMS Sample eResearch Activityflow Use Cases Early demonstrations Why? Summary of RAMS activities Progress to date Challenges of RQF assessment

3. RAMP project RAMP (Research Activityflow and Middleware Priorities) is a new Aust Gov funded eResearch project ($2.9M) Part 1: Authorisation –Development of a generalised authorisation library using XACML, focus on implementation for repositories (esp Fedora) –Builds on MAMS work: SAML + XACML = Alternative to DRM Part 2: Activityflow –“People-based workflow” for eResearch Especially concurrent multi-actor multi-step workflows –Demonstrator of re-usable activityflows (builds on LAMS 2 core) –Theoretical review of workflow standards and concepts Part 3: Authorisation/Activityflow “fusion” exploration

4. More on LAMS LAMS project over 4 years old, widespread adoption for next generation e-learning –Fundamental driver: Any teacher could create and run –Web application, J2EE + Flash, open source (GPL) –Based on new field of “Learning Design” –IMS Learning Design specification as a starting point Core concepts of LAMS are not e-learning specific –Concurrent multi-actor, multi-step workflow system –BPEL expert: “Who are you guys?” LAMS Community for faculty sharing LAMS sequences and discussing their use (1600 members) –100 end-user created, shared and adapted eLearning workflows

5. Introducing RAMS The “Research Activity Management System” (RAMS) builds on the LAMS V2 workflow core (+ new eResearch features) A new suite of activity tools appropriate for people- based eResearch activityflows –Plus multi-purpose tools that apply across eLearning and eResearch The result is two different domain-specific applications (LAMS for eLearning; RAMS for eResearch) that draw on a common workflow core Everything is open source

6. Introducing RAMS LAMS ApplicationRAMS Application “Education Workflow Engine” (LAMS core + new RAMS development) AdminAuthorMonitorParticipant eLearning specific toolsMulti-purpose toolseResearch specific tools TeachersResearchers

7. High level use cases from RAMP proposal: 1.Managing the research enterprise lifecycle (from grant planning to grant submission, to project initiation, to project lifecycle management, to research outcome dissemination), 2.Implementing auditable evaluation processes for assessing research quality (RQF assessor workflows, journal/conference peer review management, etc), 3.Designing and tracking article submission processes for Institutional Repositories, 4.Flexibly configuring and running online research collaboration processes (such as staged collaborative analysis and discussion for PhD/Postdocs around raw data, leading to interpretation, visualisation, and ultimately publications), and 5.Process-oriented research data collection from human subjects (such as in the humanities, and social and cognitive sciences). Sample eResearch Activityflow Use Cases

8. Use case 3: Institutional Repository submission workflow

9. Use case 2: RQF assessor evaluation process

10. Use case 4: Example of weekly research group meeting

11. Use case 4a: Alternative example of weekly research group meeting

12. Why? Greater standardisation of common or repeatable research processes, leading to higher quality outcomes and improved efficiency; The ability to share descriptions of common research processes both within institutions, and between institutions – including the ability to adapt and localise shared research processes; Greatly improved accountability and audit for processes involving multiple actors across multiple steps – such as for research assessment (eg, RQF assessor workflows), as well as for research itself (eg, as a deterrent to academic fraud); and Providing a process-oriented checklist to ensure the ordered completion of relevant research tasks.

13. Summary of RAMS activities Development of RAMS: activity tools + core additions –Eg, branching, conditionality, grouping, tool data in/out, “edit on the fly” Demonstrators for iterative requirements gathering Sharing of “good practice” activityflows for review, re- use, adaptation –Community website for discussion + sharing activityflows (based on experiences from the LAMS Community) Workflow theory review (Dr Yoichi Takayama) Initial exploration of activityflow/authorisation fusion

14. Progress to date (1) Mid-way through workflow theory review –Looks like LAMS/RAMS breaks significant new group, no really comparable system/specification found to date –Key difference is that in LAMS/RAMS *people* travel through the workflow, not simple data/processes Already contributed to eResearch workflow modelling discussions in Australia (eg RAE equivalent process – RQF) Ramscommunity.org website ready to launch as basis for sharing RAMS designs and discussion of issues Soon to release re-skinned LAMS V2 as initial RAMS prototype (no new features) Begun work on RAMS features for V2.1 (eg, edit on the fly; branching and conditionality, data in/out, etc)

15. Progress to date (2) New work for RAMS –New “Welcome” page based on researcher workspace for all eResearch workflows –Includes “current status” information for all workflows –Allow system-launched sequences (eg, repository submission workflow) –Investigating sequence aggregation, hierarchies and linking –Investigating (actionable) roles for RAMS tools, including multiple roles across multiple actors with differential impact on different tools (ugly) –Investigating challenging “what constitutes task completion” issues (easy for single user, hard for groups) (ugly)

16. Challenges of RQF assessment Consider the following version of the RAE/RQF assessment workflow: –Step 1: Academic submits articles for assessment; assessors (including assessor manager) can then view articles –Step 2: Assessors (including assessor manager) discuss quality of articles (eg, chat, forum or “offline”) –Step 3: Assessors (including assessor manager) provide overall rating of academic’s quality and impact; assessor manager then finalises an overall score for quality and impact based on prior discussion and review of ratings from all assessors; at a later stage, the scores can be made viewable by the academic

17. Challenges of RQF assessment Step 1 Step 2Step 3 SubmitDiscussRate Simple?

18. View Challenges of RQF assessment Step 1 Step 2Step 3 Academic Role Assessor Role Assessor Manager Submit View No task Discuss View Rate & Finalise View Discuss View Rate ??? ?? (System)

19. Challenges of RQF assessment Some problems to solve –How to design tools to allow for actionable roles without the system becoming unworkably complex for non-technical users? –How does the system handle multiple actors within and across different roles? –What constitutes task completion in group workflows? How does the system know to notify assessors that articles have been submitted (not too hard)? How does the system know that the discussion is finished and the rating has begun (harder)? How does the system know to notify the academic that their rating is now viewable (quite hard)?