1. Dr John Willison School of Education University of Adelaide Research Skill Development across undergraduate degree-programs and the benefits for graduates An Interactive Seminar for the University of South Australia

2. Session Outline 1.Why Explicit Research Skill Development?  Graduates’ perspectives 2.What is Research Skill Development?  What academics have done across degree programs 3.How is Research Skill Development carried out?  Your questions and morning tea 4.What if you were to use the RSD?  Time for you to consider possibilities, plan and question

3. “I know that research is important, not only from an educational perspective, but if I’m in a work situation it’s just basically understanding what I want to achieve in my role with my customer... Where we started… a summary of the first RSD project

4. …and how I actually go about breaking that down into manageable easy steps. So, yes, it’s got a practical application in my world in what I do. -Monash Business Ethics Student Summer 07-08 Cohort, interviewed in April 2009. 89% of students (First Year to Masters) indicated the research skills they developed would be useful in employment Where we started… a summary of the first RSD project

5. Why Research Skill Development? Course-by-Course Research Skill Development Variety, nuances Coherent thread for students to generalise the learning experience as a whole

6. Using Research Skills in Oral Health “Before I left for Cambodia, I actually took a silver fluoride which is a product that we didn’t even actually come into contact with in the Bachelor of Oral Health… so I was doing a lot of research with that… I ended up purchasing some and taking it over with me and I was using it a lot when I was over there.” (2010 B. Oral Health graduate interviewed in Nov. 2011)

7. Using Research Skills in Media The rubric was basically a guide that I followed to finish the assignment. It was really good in the sense because I could go step by step, and not just kind of flick from one bit to another bit. In my work now, I find it is also a step by step process, and that [RSD]rubric from Media Research Methods and also from Global Media kind of set the fundamentals of what to do when you’re actually in the workplace as well. B Media Graduate 1

8. Using Research Skills in Engineering …to search through the work databases and … I'm doing some information on some assets that we have here and trying to find information that's been dated back from 20 or 30 years ago; I'm definitely using those skills that I learnt at uni to find that. Electrical Engineer Graduate 4

9. Research Skills for Subsequent Studies? I'm currently using it for inspecting basically what other people are doing within my current field and things that people are doing outside my current field that could potentially relate to my actual field at the moment. Then obviously documentation skills and also planning skills. Facilitator:What do you mean by planning skills? Interviewee:Being able to meet deadlines, being able to basically break up a task into small enough parts that you can effectively deal with them as one lot, rather than having it as one big problem that you'll never solve. Electrical Engineering Graduate 5. doing Masters by Research

10. A common framing across a degree program Since the beginning [of First Year], they have given us assignments based on this criteria. You might not have liked the assignments, but because they have been consistently applying this structure to all of our assignments, we have come to think that way for science, in the perspective of science and writing … You might not know that you’re following their guidelines, but you are. (Medical Science Honours Student 4)

11. Why Research Skill Development? Without appropriate structure and guidance, by third year students tend to perform open-inquiry projects in each of their biology courses at ‘the same level of sophistication as in their introductory core course’. Chaplin, S. (2003). Guided development of independent inquiry in an anatomy/physiology laboratory. Advances in Physiology Education 27(4), 230-40.

12. Without a common framing … Looking back at my undergrad, I sort of didn’t really have much to do with this framework in the large scheme of things. It was more sort of focus on whatever assignments you had at the time and not really looking at the big picture. Animal Science Honours Student 6 ‘… given the growth of ever more detailed marking schemes for assessments, does feedback become something which is too specific to a single episode of assessment rather than generalisable to the learning experience as a whole’ (Adcroft, 2011 p. 417).

13. Reinforcing across a degree program It [the RSD rubric] tells you a lot, because I remember one of the criteria was to be specific … in my [First Year ] Human Biology course… so I think it’s very relevant to Honours, because when we were doing our first seminar, some of the topics were really broad. Automatically you think: I have to be specific otherwise it’s too broad… (Medical Science Honours Student 7)

14. RSD… a frozen conversation I was assessed to be in the 2B range, and I was of course not happy about that, which is why I spoke to my coordinator, and he explained it, because we didn’t know if you are following your recipe, like, the supervisor’s recipe, or is this your idea. We wouldn’t know, so we naturally assumed this is what your supervisor gave you and you simply follow instructions. So that to me all okay, I need to make it very clear in my writing, and from now on in my theses, that this is my idea. (Student 14) (Medical Science Honours Student 14)

15. Researching develops Graduate Attributes Facilitator:In general, in terms of research skills - what do you think you are pretty good at? Interviewee:I think I'm good at solving complex problems and coming up with new ideas to solve complex problems. That's my main strength… I think I'm good at doing something and then critically analysing it, finding out what's bad about it or what we can do, or what ours and the differences and things like that. I can critically analyse it. EEE Graduate 3 (Doing PhD)

16. Interviewed one year after completing required part of degree 27 Graduates/ 23 Honours students Benefits of current RSD use  43- Yes-  2 – No disadvantages of current RSD use  17-Yes  18- No RSD should be used throughout the undergraduate degree programs  43-Yes  5- No

17. Please indicate your level of awareness of the Researcher Skill development (RSD7) framework a) never heard of it b) have heard of it c) know of the 5-level RSD d) quite familiar e) have used it f) other

18.  Frames  Literature/published data research  laboratory research  clinical research  field research  combined forms Research Skill Development framework

19. Disciplines Involved in using the RSD Accounting, Business, Management, and Tourism Computing Science, Electronic Engineering, Petroleum Engineering. Dentistry/ Oral Health, Dietetics, Medical Science, Medicine Animal, Vet Science, Honours Stream Science English, History, Media Studies Introduction to Tertiary Learning, Introduction to Academic Learning for International Students, Graduate Certificate in Higher Education Pathology, Pharmacy, Pharmacology, Physiology, Work Integrated Learning

20. Uni Adelaide South Aust. Maquarie Melbourne Monash Latrobe James Cook W’gong Canberra Alberta Trin Col Dublin Wisconsin South Pacific

21. 21

22. Organize &manage Analyse& synthesis Embark & clarify Find & generate Evaluate & reflect Communicate & apply Facets of research

23. 23 Facets of research (cont) Progressive revisiting the same skills:  in varying contexts  with increasing degrees of rigour, conceptual demand

24. 24

25. Degrees of Autonomy

26. Prescribed Research (Level 1) Highly structured directions and modelling from educator prompt student research ‘Some lecturers have got different standards, different requirements. Usually when they don’t make those requirements clear, that’s what hinders my assignment work and sort of research and so on.’ This and the following 4 quotes from Monash University Students, 16 Months after completing Business Ethics Unit.

27. First Years researching with a high degree of structure/guidance

28. Degrees of Autonomy

29. Bounded Research (Level 2) Boundaries set by and limited directions from educator channel student research So you get a hypothesis or something and you basically try to prove it or disprove it, by trying it on other people’s work.

30. First Years researching with some structure/guidance

31. Degrees of Autonomy

32. Scaffolded Research (Level 3) Scaffolds placed by educator shape student independent research ‘If you have a tool to help you to get started with something, yes, I think you will go a long way in terms of motivation. I think people half the time would be more discouraged when they just don’t know how to start something, or they don’t know how to go about something.’

33. Degrees of Autonomy

34. ‘I looked at the movement of customers, so people that were churning, people that were disconnecting, people that were switching on, and basically I sort of plotted them all across a timeline and the way they picked them was I looked at what happens prior to competitors activating their D-slams…I looked at it basically a couple of months before they had it and a couple of months after, and then you could sort of see how they did it….It was not like a lengthy word paper, but it was a bit of research and findings.’ Self-actuated Research (Level 4) Students initiate the research and this is guided by the educator

35. I think the group one was the best research one we’d done, because we actually did independent research…. We had to come up with hypotheses and things like that… … so we’ve clearly articulated research questions, whereas before this year, we probably wouldn’t have used that. Self-actuated Research (Level 4 Continued) Students initiate the research and this is guided by the educator

36. 36 First years initiating research with guidance

37. Degrees of Autonomy

38. Open Research (Level 5) Students research within self-determined guidelines that are in accord with discipline or context. ‘Oh, you’ve got to determine the object of your need.’ So, yes, I may have skipped a few steps, but of course I would have to determine a need for why I should go and decide what research I’m doing.’ With the project I did at work, I could sort of see why I was doing it. I was doing it specifically to help me with my campaigns.

39. Degrees of Autonomy

40. Level of Autonomy in a Course? There is no rule Raises teaching questions E.g. the move from first year to second year... Rigour Conceptual Demand Depth of background knowledge Student Autonomy

41. Autonomy vs Time Semester at Uni 1 st 2 nd 3 rd 4 th 5 th 6 th 7 th 8 th Extent of Autonomy L 5 L 4 L 3 L 2 L 1

42. Autonomy vs Time Semester at Uni 1 st 2 nd 3 rd 4 th 5 th 6 th 7 th 8 th Extent of Autonomy L 5 L 4 L 3 L 2 L 1

43. Autonomy vs Time in 2 nd Year Physiology Lab (Luckie, 2004) Week of Laboratory 1 wk 2 wk 3 wk 4 wk 5 wk 6 wk 7 wk 8 wk Extent of Autonomy L 5 L 4 L 3 L 2 L 1

44. Why RSD framework? 3. A framework to provide a common conceptualisation. ‘I see the framework being what I’ll call nebulous enough for everybody to be able to accommodate it. Because it’s not very prescriptive. It just sets a framework, and everybody can work within a framework … [yet] it’s relatively comprehensive in what it describes over all.’ [Medical Science Academic]

45. 45 Revisiting Facets of research progressive revisiting same skills in varying contexts

46. Two Kinds of Questions 1.What questions do you have for me?  Clarifications  Challenges  Speculative 2.What questions does the RSD ask you?  Do you give students too much or too little structure and guidance in assessment tasks?  Do you address the facets of the RSD and provide feedback on them?  Raise any new possibilities?

47. Discerning Determined Curious 47 Harmonising Creative Constructive

48. Workshop Approaches Specific Assessment reframed with RSD Reframing a whole course Program Level thinking Other possibilities

49. Degree Program Development Synthesis… that’s the kind of thing I struggle with, because I would look and, oh, I think I’m doing that, but I’m not really sure what that is, so okay, I hope I’m doing it, whereas by third year you understand the meaning of it, you’ve read more papers that really do those things and you understand what they’re looking for. I think maybe in first year if they – the idea is good; it just needs to be, for a better term, dumbed down to first-year level. Medical Science Student thinking back to earlier years

50. Fourth Year Student ‘… even if they don’t actually provide me that framework on an actual piece of paper, I sort of try and do that, try and get into that habit anyway, because I realise that when you’re out there on your own as a researcher, that’s essentially what you do. No one is going to hand you a piece of paper and say, this is what you’ve got to do for a publication.’ (Honours student, Medical Science interviewed 2012)

51. 1. Assessment rubric scaffolding 51

52. Source: Brian Ng and Said Al- Sarawi, University of Adelaide

54. www.rsd.edu.au

55. A Priori Mapping

56. 56 Monash Pharmacy Degree: Post Hoc Mapping

57. First year units – teaching facets of enquiry Course mapping of skills with the RSD 57

58. First year units – assessing facets of inquiry Course mapping of skills with the RSD 58 Fourth year units – assessing facets of inquiry

59. 59

60. Evaluating Sources Should I click on ‘Complete Formular’ to get my tax return? Each table has 2 minutes to list as many indicators of source credibility, positive and negative, as possible. Group with the highest number of indicators reads them out. 60

61. 61

62. What are some skills that you employed in this task? Embark and clarify Find Information, generate data Evaluate sources & process Organise information, manage process Analyse & Synthesise Apply and Communicate

63. 63

64. The facets of student research In researching, students: 1.embark on an inquiry and so clarify a need for knowledge/understanding 2.find/generate needed information using appropriate methodology 3.critically evaluate information/data and reflect on all processes employed 4.organise information collected/generated and manage research processes 5.Analyse and synthesise new knowledge 6.apply and communicate knowledge and the processes used to generate it, … with an awareness of ethical, social and cultural issues. (Willison & O’Regan, 2007)

65. Research may be closed to open in terms of Beginning How to proceed End points Level 1: highly prescribed by lecturer, low student autonomy. Level 5: high degree of student autonomy (in line with the discipline) Levels of Autonomy

66. Framing Research Questions to Illustrate Levels 1 to 5 Level 1 Research require a high degree of structure and guidance eg modelling the process of asking questions Facet A) Embark on Inquiry Respond to questions / tasks arising explicitly from a closed inquiry. Facet E) Synthesis, analysis, application Ask questions of clarification or curiosity. 66

67. Hill o’ Slain What was the belief system of the people who built this structure? List all the evidence you can find in the photo, organise it, evaluate it and draw a conclusion. Then pose several questions relating to your conclusion that cannot be answered by the evidence you have found.

68. Framing Research Questions (cont) Level 2 ‘Requires some structure and guidance’ eg scaffolding the process of asking questions Facet A) Embark on Inquiry Respond to questions / tasks required by and implicit in a closed inquiry. Facet E) Synthesis, analysis, application Ask relevant, researchable questions. 68

69. 69 Explain why the ‘passage tomb’ of Newgrange had its passage aligned with sunrise of the winter solstace. Pose several questions about the society that built Newgrange.

70. Framing Research Questions (cont) Level 3 Research independently within a closed inquiry Facet A) Embark on Inquiry Respond to questions / tasks generated from a closed inquiry. Facet E) Synthesis, analysis, application Ask rigorous, researchable questions based on new understandings. 70

71. List and explain the most likely geological processes that formed this arch. Formulate a research question about one of these processes.

72. Framing Research Questions (cont) Level 4 Research at the level of an open inquiry with structured guidelines Facet A) Embark on Inquiry Generate questions / aims / hypotheses framed within structured guidelines. 72

73. 73 In First Year Human Biology, student pose their own research question or hypothesis as the beginning of their research.

74. Framing Research Questions (cont) Level 5 Research with self-determined guidelines Facet A) Embark on Inquiry Generate questions / aims / hypotheses based on experience, expertise and literature. 74

75. ‘… almost inadvertently, they found themselves thinking differently about the nature of research within their disciplines, the necessity for students at all levels to appreciate how research is conducted and to be initiated into the process themselves, and about how to inject these new perspectives into their teaching strategies.’ (Nightingale, 2009). ‘Some Measures of Success’ Academics cant fully understand potential gains until they have engaged in the process. Testimony from other academics is crucial. First RSD run is vital.

76. [A faculty member] said that she suddenly realised that students did not know where the material in her lectures came from, that they seemed to think it was just there for her to tell them about; they did not recognise the years of research in the field, the teacher’s own research to enable her to present that knowledge, or that they themselves are engaged in a research process when they attempt to learn (External Review of the RSD project, October, 2009). Academics’ perspective of explicit RSD

77. Why explicitly develop students’ research skill? … instead of just reading and just taking stuff out…[it] forced us to analyse the research much more closely than we would have normally. I have used these skills for my other subjects as well, that I learned from Hollywood or Bust…I still use the same techniques to evaluate the research that I do. (Student interviewed in 2009 about experiences in ‘Hollywood or Bust’ in 2008) 77

78. Why develop Students’ Research Skill? I suppose when you get that skill of being able to research, I think it sort of inspires something in you that makes you want to find out things... … I probably see myself getting a job in some sort of NGO or department, and then collecting data and doing surveys... and finding things that other people have done, and then either being able to support that or prove it wrong. (2008 interview of 2007 First Year Human Biology Student) 78

79. Why develop Students’ Research Skill? “I know that research is important, not only from an educational perspective, but if I’m in a work situation... it’s just basically understanding what I want to achieve in my role with my customer,...... and then understanding what I want to achieve and how I actually go about breaking that down into manageable easy steps. So, yes, it’s got a practical application in my world in what I do. - Monash Business Ethics Student Summer 07-08 Cohort, interviewed in April 2009.

80. Session Outline 1.A general overview to learn more background information about the project. 2.Measures of success 3.The practical application of the framework  Examples of how academics have applied the tool. 4.Drivers and motivations to research

81. 81

82. Look at the descriptors for ‘embark on inquiry’ on the following pages of the RSD Handbook: Page 16, 19, 22, 28, 41 What are the similarities and differences?

83. 2. Analyse existing curricula 83... at our academic staff meeting this morning we agreed to review the way we teach research skills from first year to PHD using the RSD...... with a view to developing a coherent School based approach for nursing and midwifery - again based on RSD. Trinity College Dublin, School of Nursing and Midwifery

84. Discussion

85. 85 The drivers & emotions of research

86. Increasing autonomy has implications concerning motivation. Levels of Autonomy

87. Affective Domain: Emotions & Motivations In small groups, consider single-word or short phrase descriptions of affect (motivations and emotions) for each facet. Facet A: ‘students embark on an inquiry and so clarify a need for knowledge/ understanding Phrase as ‘ ………………. (adjective) students’ e.g. Intense Collaborative Capable

88. Affective Domain Facet A: Students embark on an inquiry and so determine a need for knowledge/ understanding Curious ‘I am neither especially clever nor especially gifted. I am only very, very curious.’ Albert Einstein

89. Affective Domain (continued) Facet B: Students find/generate needed information using appropriate methodology Determined ‘It's not that I am so smart. It's just that I stay with problems longer.’ Albert Einstein

90. Affective Domain (continued) Facet C: students critically evaluate information/data and the process to find/generate Discerning "I think and think for months and years. Ninety-nine times, the conclusion is false. The hundredth time I am right. " Albert Einstein

91. Affective Domain (continued) Facet D: students organise information collected/generated and manage research processes Systematic ‘Out of clutter, find simplicity.’ Albert Einstein

92. Affective Domain (continued) Facet E: Students synthesise and analyse and apply new knowledge Creative Knowledge is limited. Imagination encircles the world- Albert Einstein

93. Affective Domain (continued) Facet F: students communicate knowledge and the processes used to generate it, with an awareness of ethical, social and cultural issues. Constructive ‘Go to where the silence is and say something’ – Amy Goodman "the greatest talent is the ability to strip a theory until the simple basic idea emerges with clarity." -- Albert Einstein

94. Work Skill Development framework for WIL 4. Inspire new framework Available at http://www.jcu.edu.au/teaching/idc/groups/public/documents/advice/jcuprd_047283 Design Skill Development 94

95. 5. PhD Supervision International Bridging Program for International Students. Students Self-Assess on rubric and supervisors assess with same rubric Tool for conversation about expectations of students and supervisors and gaps between these Sometimes ased on the extended version of the RSD, the Research Skill Development framework (RSD7). Faculty place themselves on the same continuum as students. HumSS Postgrad Induction. 95

97. What is the academic value of using Google and Wikipedia? 97

98. 98

99. 99

100. 100 Chad Habel, Academic Language and Learning Coordinator CLPD

101. 101

102. Many Dimensions in addition to Autonomy Degree of Academic Rigour Degree of Conceptual Demand Degree of disciplinary knowledge required Status of knowledge being pursued

103. Degrees of Autonomy