Presentation is loading. Please wait.

Presentation is loading. Please wait.

Computing Education Research Computer Science Education Research Conference (CSERC ‘11) 7 th April 2011, Heelen, The Netherlands Sally Fincher.

Similar presentations


Presentation on theme: "Computing Education Research Computer Science Education Research Conference (CSERC ‘11) 7 th April 2011, Heelen, The Netherlands Sally Fincher."— Presentation transcript:

1 Computing Education Research Computer Science Education Research Conference (CSERC ‘11) 7 th April 2011, Heelen, The Netherlands Sally Fincher

2 About me I run the Computing Education Research Group in the Computing Laboratory at the University of Kent (have done since 1997)  So know about trying to do disciplinary-specific education research I edit the Journal Computer Science Education  So have seen what others think are appropriate outputs of disciplinary-specific education reearch I was Secretary of ACM Special Interest Group on Computer Science Education (SIGCSE) for 6 years  So have some sense of the scale of the interest in this area

3 Bootstrapping & friends From 2002/3 I devised (together with Marian Petre from the Open University and Josh Tenenberg from the University of Washington, Tacoma) a series of workshops, aimed at helping people find a “way in” to CS Education research.

4 Bootstrapping & friends We collected some of that material into a book I’ll revisit some of that today

5 Topics and Areas One of the ways we sliced things in 2004 was by topic, by what people were interested in researching. We listed 10 areas then:  Student Understanding  Animation, visualization & simulation  Teaching methods  Assessment  Educational technology  Transferring professional practice to the classroom  Incorporating new developments & technologies  Transferring from f2f to distance education  Recruitment and retention (incl. diversity & gender)  Construction of the discipline

6 Topics and Areas Student Understanding Animation, visualization & simulation Teaching methods Assessment Educational technology Transferring professional practice to the classroom Incorporating new developments & technologies Transferring from f2f to distance education Recruitment and retention (incl. diversity & gender) Construction of the discipline Developing a Computer Science Curriculum in the South African Context

7 Topics and Areas Student Understanding Animation, visualization & simulation Teaching methods Assessment Educational technology Transferring professional practice to the classroom Incorporating new developments & technologies Transferring from f2f to distance education Recruitment and retention (incl. diversity & gender) Construction of the discipline A Lab-based Approach for Introductory Computing that Emphasizes Collaboration

8 Topics and Areas Student Understanding Animation, visualization & simulation Teaching methods Assessment Educational technology Transferring professional practice to the classroom Incorporating new developments & technologies Transferring from f2f to distance education Recruitment and retention (incl. diversity & gender) Construction of the discipline Plagiarism detection for Java: a tool comparison

9 Topics and Areas Student Understanding Animation, visualization & simulation Teaching methods Assessment Educational technology Transferring professional practice to the classroom Incorporating new developments & technologies Transferring from f2f to distance education Recruitment and retention (incl. diversity & gender) Construction of the discipline Peer Production & Peer Support at the Free Technology Academy

10 Topics and Areas Student Understanding Animation, visualization & simulation Teaching methods Assessment Educational technology Transferring professional practice to the classroom Incorporating new developments & technologies Transferring from f2f to distance education Recruitment and retention (incl. diversity & gender) Construction of the discipline Student discussion forums: What is in it for them?

11 Topics and Areas Student Understanding Animation, visualization & simulation Teaching methods Assessment Educational technology Transferring professional practice to the classroom Incorporating new developments & technologies Transferring from f2f to distance education Recruitment and retention (incl. diversity & gender) Construction of the discipline Sciences, Computing, Informatics: who is the keeper of the Real Faith?

12 Topics and Areas Student Understanding Animation, visualization & simulation Teaching methods Assessment Educational technology Transferring professional practice to the classroom Incorporating new developments & technologies Transferring from f2f to distance education Recruitment and retention (incl. diversity & gender) Construction of the discipline Game Based Learning for Computer Science Education

13 Topics and Areas Student Understanding Animation, visualization & simulation Teaching methods Assessment Educational technology Transferring professional practice to the classroom Incorporating new developments & technologies Transferring from f2f to distance education Recruitment and retention (incl. diversity & gender) Construction of the discipline A Distributed Virtual Computer Security Lab with Central Authority

14 Topics and Areas Student Understanding Animation, visualization & simulation Teaching methods Assessment Educational technology Transferring professional practice to the classroom Incorporating new developments & technologies Transferring from f2f to distance education Recruitment and retention (incl. diversity & gender) Construction of the discipline Some challenges for Computer Science Education in a Knowledge Society

15 Topics and Areas Today, these don’t seem to be wrong. They’re still areas that motivate people to work in the area. The still give a response to the question “Why are you interested in Computing Education Research?” But they are not a complete answer. They don’t speak to what people want as a result of a piece of Computing Education Research And they don’t begin to address the “How do you do Computing Education Research?” question

16 CSEd Research: Three lenses Discipline Classroom Community

17 CSEd Research: Three lenses Discipline Classroom Community

18 Discipline The methods of CS education research are not the methods of CS. You cannot study classrooms, people and their interactions with algorithms and proofs. As CSEd researchers, we have to use other methods – other epistemologies, “ways of knowing” ̶ from other disciplines  Education...  Sociology...  Psychology...  Anthropology...  Can be methodologically isolating

19 Cognitive Psychological tradition Many early CS Ed investigations (1980’s) were from the psychological tradition – conducted by psychologists for the most part. Programming was a new and interesting way to investigate how people think, and was conducive to laboratory-based investigation. There was a very influential series of meetings/publications Empirical Studies of Programmers

20 Cognitive Psychological: what people want The sort of positivist, quantitative knowledge that cognitive psychology creates is very attractive – and persistently so. PPIG posting, 20 th March 2011 To clarify my test … I'm going to compare miniC (a minimal C implementation built on BYOB) vs regular C environments. I want to test if students that learned C by using miniC: - do less syntactic mistakes - remember to declare their variables more often - use sequence/loop/conditionals in a more consistent way

21 PPIG mailing list: 20 th March 2011 What I'm not sure is if they are using the correct instruments to get correct, unbiased data from their tests. … After all, from these tests they should claim that their work is successful. Are they doing it in the right way? Shouldn't we have a common, clearly-understood test-bed on which this kind of experimentation should be performed? This doesn't mean that the test- bed should be unupdateable, but at least important part of it should. Otherwise our tests wouldn't be comparable as they should.

22 PPIG mailing list: 20 th March 2011 What I'm not sure is if they are using the correct instruments to get correct, unbiased data from their tests. … After all, from these tests they should claim that their work is successful. Are they doing it in the right way? Shouldn't we have a common, clearly-understood test-bed on which this kind of experimentation should be performed? This doesn't mean that the test- bed should be unupdateable, but at least important part of it should. Otherwise our tests wouldn't be comparable as they should.

23 PPIG mailing list: 20 th March 2011 What I'm not sure is if they are using the correct instruments to get correct, unbiased data from their tests. … After all, from these tests they should claim that their work is successful. Are they doing it in the right way? Shouldn't we have a common, clearly-understood test-bed on which this kind of experimentation should be performed? This doesn't mean that the test- bed should be unupdateable, but at least important part of it should. Otherwise our tests wouldn't be comparable as they should.

24 PPIG mailing list: 20 th March 2011 What I'm not sure is if they are using the correct instruments to get correct, unbiased data from their tests. … After all, from these tests they should claim that their work is successful. Are they doing it in the right way? Shouldn't we have a common, clearly-understood test-bed on which this kind of experimentation should be performed? This doesn't mean that the test- bed should be unupdateable, but at least important part of it should. Otherwise our tests wouldn't be comparable as they should.

25 Epistemology We can see in this plea the assumptions – and expectations – of the experimental sciences.  A fixed natural world, with knowable data  The notion of predictive theory  The expectation that my classroom, my students and their abilities are in some sense the same as yours, are directly comparable with yours.

26 Cognitive Psychological, today Developing a validated assessment of fundamental CS1 concepts. Proceedings of the 41st SIGCSE Technical Symposium on Computer Science Education, (Milwaukee, WI), , 2010 The FCS1: A Language Independent Assessment of CS1 Knowledge. Proceedings of the 42 nd SIGCSE Technical Symposium on Computer Science Education, (Dallas, TX), 2011 Allison Elliot Tew & Mark Guzdial

27 Other questions, other needs I was trying to understand real learning in real classrooms, but I was using the conventional pencil- and-paper things and massaging the data with multivariate statistics. Collecting the data directly from students... and then trying to make sense of it just led me into a huge shift in my thinking I went away and started to think about it – opening up the whole business of how experience is interpreted. I don’t think before that I’d thought much about how people construct their own meaning for experiences and events (quotes reported in Fencham, p. 42 & 45)

28 Sociological tradition Sociology asks different questions, and uses different methods to answer them. It is (usually) more group-based than psychological investigations, and more interested in the nature of processes. Its theories (characteristic of the social sciences) are explanatory rather than predictive. Key sociological questions are:  What role does education play in the life chances of different groups?  How can we best explain why some groups systematically win and others lose?  Is education a means of liberating individuals or is a means of social control?

29 Sociological tradition Lecia Jane Barker, Kathy Garvin-Doxas & Michele Jackson Defensive Climate in the Computer Science Classroom Proceedings of the 33rd SIGCSE technical symposium on Computer science education (Covington, KY), 43-47, 2002 A learning environment comprises “all of the physical surroundings, psychosocial or emotional conditions and social or cultural influences” present in a learning situation. Over the course of an academic year they “ethnographically” observed 10 courses for a total of 254 hours.

30 Sociological tradition (ii) Categories emerging from data analysis included 1) impersonal environment and guarded behaviour; and 2) the creation and maintenance of informal heirarchy resulting in competetive behaviours. These communication patterns lead to a defensive climate characterised by competitiiveness rather than cooperation, judgments about others, superiority and neutrality rather than empathy.

31 Educational tradition Although “education” as a field is itself is a hybrid disciplinary influence and method. The Case for Case Studies Communications of the ACM (CACM) Volume 35 Issue 3, March 1992 Marcia Linn & Michael Clancy

32 Educational tradition Another way of working – teaming up. “Marcia Linn showed up at my door... she said, we want to research people learning to program and we've heard you teach a lot of those.”

33 Teaming up “There was a vast set of things I got from this. I was a clueless teacher, trying things as a whim and flying by the seat of my pants. What I got from Marcia is that she could see the big picture and say “Oh yeah—what you're doing is—and here's how it is in math, physics and chemistry”... then I could see how what they were doing in other disciplines related to what I was trying.”

34 Tested Mixed methods – “balanced groups” (some with case studies, some without), small-scale observations, statistical conclusions. “I still think that—I have not found a better way to address complex concepts but to use a case study to make concepts concrete”

35 Computer Science Compared to other areas of disciplinary-specific education research, we—uniquely—have additional disciplinary influences but they’re not academic influences They come from the practice of the discipline. And they come in two ways... Industrial practice  (e.g. pair programming) Practice of our craft  (e.g. tool building – Alice, Greenfoot, Scratch)

36 Discipline Partly a matter of temperament—what methods and approaches are you comfortable with? Partly a matter of epistemology—what questions do you want to ask and what evidence will satisfy you that they’ve been answered? Computing Education Psychology AnthropologyStatistics

37 CSEd Research: Three lenses Discipline Classroom Community

38 CSEd Research: Three lenses Discipline Classroom Community

39 Classroom Most CS Education researchers are not motivated by generalised results – at least not at first. Most are motivated to understand what happens in their classroom, how to describe the learning that takes place there and what to do to change/improve it. There are some obvious problems with this

40 Fincher (2001): A Fictitious Paper A single author, presenting results for a single institution. A Study of Assessment of Programming Skills of First-Year CS Students Sally Fincher University of Kent UK

41 Fincher (2001): A Fictitious Paper Explanations? 1.Sally can’t teach. 2.The students are British. 3.Sally teaches at an atypically poor institution 4.If Sally changed: a) From Pascal to C to C++ to Java to Python b) Objects early ← → Procedural c) Used closed labs ← → Used open labs d) More assignments ← → Less assignments

42 McCracken, et al. (2001) 10 authors; data collected at 4 universities in 2 countries: They all can’t teach? They can’t all be British! They are all atypically poor institutions? Fincher (2001): A Fictitious Paper

43 ITiCSE (Innovation and Technology in Computer Science Education). ACM SIGCSE European conference. 16 th year (27-29 June, Darmstadt) Has associated “working groups” The “ITiCSE Working Group Model”: i

44 a)topics are proposed, and peer-reviewed b)one or more topics are selected for presentation c)the topic is posted with an invitation for others to join in the work specified d)the resulting group(s) work electronically before the conference, then work at the conference (and often for a day or more in advance) e)the group(s) write a paper detailing their results. This is peer-reviewed and, if accepted, published in the SIGCSE Bulletin f)the group disbands The “ITiCSE Working Group Model”: ii

45 Usually these groups produce a report that: Distils collected resources and experiences on an issue of direct relevance to practicing teachers.  For example Resources, Tools, and Techniques for Problem Based Learning in Computing, 1998 & A Road Map for Teaching Introductory Programming Using LEGO Mindstorms Robots, 2003 Addresses common problems that benefit from the application of collective intellectual and analytical effort.  For example: How shall we assess this?, 2004 & Evaluation: turning technology from toy to tool 1996 The “ITiCSE Working Group Model”: iii

46 McCracken (2001) was the first to adapt this to an empirical study Subsequently replicated, notably: 2004: The “Leeds Group”  A study of novice program comprehension  Data from 12 universities in 6 countries The “ITiCSE Working Group Model”: iv

47 Lightweight (Relatively) modest commitment of time/effort The “ITiCSE Working Group Model”: v

48 “Bootstrapping” model Series of interventions to give practitioners a “way in” to CS Education research

49 Year OneYear Two Four day workshop “Input” – methods, presentations Work on their own studies Introduce the Experiment Kit Intervening Execution of the Experiment Kit Four day workshop Analyse data in aggregate. Write paper Work on their own studies The Bootstrapping Model

50 Experiment Kit Structure (Bootstrapping) 1.Question formulation 2.Protocol a. Data collection specification b. Human Subjects materials c. Background questionnaire d. Discriminator question e. Specification of set-up f. Experimenters script (including guidance on notes/diagramming) g. Participant design brief h. Design criteria elicitation Stimuli set i. Design criteria elicitation Recording Sheet 3. Analysis protocol 4. Background 5. Literature

51 Compared The “ITiCSE Working Group Model” Lightweight (Relatively) modest commitment of time/effort Effectively free The “Bootstrapping” Model Heavyweight commitment on organisers (writing & piloting Experiment Kit) Two-year commitment for participants Requires funding Payback A (modest) paper publication The “Bootstrapping” Model Many publications (31 to date) Spin-off studies Ongoing community Much more detail about this in Fincher & Tenenberg, JEE 2006

52 From: Fincher et al. ICER 2005

53 Classroom Authentic, situated, embodied. Need to overcome – or celebrate – limitations of single class: size, sample, generalisation, validity

54 CSEd Research: Three lenses Discipline Classroom Community

55 CSEd Research: Three lenses Discipline Classroom Community

56 There is no such thing as private research. Indeed, the concept makes little sense. For there to be ongoing research there has to be:  people doing work  places for people to publish their work  people reading that work, and building on it  (other) people using that work  places for people to meet other researchers

57 Peter Fensham There are issues with CS Education Research as an activity, as I’ve explored. There are also issues with CS Education Research as a research area. Peter Fensham undertook an interesting piece of work in He looked at the construction of “Science Education” (in regard to Secondary Schools) as a distinctive field of research. He defined three sets of criteria that would have to be populated to be able to claim that a research area was distinctively separate.

58 Peter Fensham’s criteria Research criteria Outcome criteria Structural criteria

59 Peter Fensham’s criteria Research criteria Outcome criteria Structural criteria

60 Research criteria Scientific knowledge Asking questions Conceptual & theoretical development Research methodologies Progression Model publications Seminal publications

61 Research criteria Scientific knowledge  Knowledge required to conduct the research Asking questions  Asking distinctive questions not addressed by other fields Conceptual & theoretical development  Theoretical models with predictive or explanatory power Research methodologies  Invention, development or adaptation Progression  Researchers informed by, and build upon, previous studies Model publications  Held up as models of conduct & presentation in this field Seminal publications  Recognised as important or definitive studies; new directions or new insights

62 Peter Fensham’s criteria Research criteria Outcome criteria Structural criteria

63 Peter Fensham’s criteria Research criteria Outcome criteria Structural criteria

64 Outcome criteria Implications for practice  Outcomes from research that are applicable to the practice of [computer] science education

65 Peter Fensham’s criteria Research criteria Outcome criteria Structural criteria

66 Peter Fensham’s criteria Research criteria Outcome criteria Structural criteria

67 Peter Fensham’s criteria Research criteria Outcome criteria Structural criteria  Professional Associations  Research Conferences  Research Journals  Academic Recognition

68 Professional Associations For Computer Science Education Research, there really are none explicitly. But it does occur as part of other activities: SIGCSE SEFI Healthy national and international professional associations

69 Research Conferences SIGCSE: Symposium ITiCSE: Innovation and Technology in Computer Science Education FiE: Frontiers in Education Regular conferences for the direct exchange of research that enable researchers to meet in person These are practitioner, not research, conferences. Nevertheless some research studies are reported in them. Characterised by small-scale investigations on a single aspect (of discipline or practice). Most often reflect a single classroom. Frequently take an individual “action research” approach. Reflect their practitioner focus.

70 Research Conferences PPIG: Psychology of Programming Interest Group 24 years Mostly UK ASEE: American Society of Engineering Education Primarily Engineering Education, some CS element Regular conferences for the direct exchange of research that enable researchers to meet in person Subject-area focussed Methodologically focussed

71 Research Conferences Koli Calling  11 th year. Mostly - but not exclusively - Scandinavian ICER workshop: International Computing Educational Research  7 th year: International in name and International in scope  2011, 8 th & 9 th August Providence, Rhode Island  2012, Auckland, New Zealand  2013, San Diego, California  2014, Glasgow, Scotland Both Koli & ICER have associated Doctoral Consortia Regular conferences for the direct exchange of research that enable researchers to meet in person

72 Research Journals Successful journals for reporting quality research

73 Academic Recognition Well, there’s me... Full faculty appointments in the area of research

74 Computing Education Research So what is there? A common object of attention – through whatever lens we choose to focus – on the teaching and learning of computing. Who it is for, how it is successful, why it works when it works and why it doesn’t work when it doesn’t. A recognition of a community of colleagues.

75 Computing Education Research We have many (although not all) of the characteristics that Peter Fensham describes There is a community of researchers There is a body of work There is also much work to be done  Curriculum gravity  Interpretation for practice CSERC is happening at an exciting time Welcome!

76 Acknowledgements Part of this material is based upon work supported by the National Science Foundation (NSF) under grants numbered DUE and DUE Any opinions, findings, conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the NSF. The “McCracken” slides of this talk were adapted from those prepared by Raymond Lister for ICER Thanks, Raymond.

77 References Peter J. Fensham The Evolution of Science Education as a Field of Research: Defining an Identity, 2004, Springer Sally Fincher & Marian Petre (eds) Computer Science Education Research, Routledge Falmer, 2004 Sally Fincher, Raymond Lister, Tony Clear, Anthony Robins, Josh Tenenberg, Marian Petre Multi-Institutional, Multi- National Studies in CSEd Research: Some design considerations and trade-offs, 2005 ICER Sally Fincher & Josh Tenenberg Using theory to inform capacity-building: Bootstrapping communities of practice in computer science education research. Journal of Engineering Education, 95(4): , October 2006 Sally Fincher & Ian Utting (eds) Special Issue on Initial Learning Environments ACM Transactions on Computing Education (TOCE) Volume 10 Issue 4, November 2010

78 References Raymond Lister, Elizabeth S. Adams, Sue Fitzgerald, William Fone, John Hamer, Morten Lindholm, Robert McCartney, Jan Erik Moström, Kate Sanders, Otto Seppälä, Beth Simon, Lynda Thomas, A multi-national study of reading and tracing skills in novice programmers ITiCSE working group reports, ACM SIGCSE Bulletin pp Michael McCracken, Vicki Almstrum, Danny Diaz, Mark Guzdial, Dianne Hagan, Yifat Ben-David Kolikant, Cary Laxer, Lynda Thomas, Ian Utting, Tadeusz Wilusz, A multi-national, multi-institutional study of assessment of programming skills of first-year CS students 2001 ITiCSE working group reports, ACM SIGCSE Bulletin pp Project pages for the series of “Bootstrapping …” workshops can be found from: kits/index.html

79 This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 2.5 License.Creative Commons Attribution-NonCommercial-ShareAlike 2.5 License


Download ppt "Computing Education Research Computer Science Education Research Conference (CSERC ‘11) 7 th April 2011, Heelen, The Netherlands Sally Fincher."

Similar presentations


Ads by Google