1. Insight research, design research and impact on practice Engineering research in education Hugh Burkhardt Shell Centre School of Education, University of Nottingham

2. Outline Paradigms for research and development Delivering effective tools and processes Mathematical literacy – a case study Cost effectiveness Building a design community

3. The Shell Centre Team Malcolm Swan, Daniel Pead, Rita Crust, Alan Bell, HB, with many associates Does engineering research – design and development of: teaching materials and processes assessment tasks professional development materials and processes Tools and strategies for system change with some associated insight research Based in the University of Nottingham School of Education Work with many others, notably Michigan State, Berkeley and school systems in UK and US Contact: Hugh.Burkhardt@nottingham.ac.uk www.mathshell.com

4. Research in Education Three research traditions, and products: Humanities: insight focus > papers Science: insight focus > papers Engineering: impact focus >new tools, processes + papers R P Theme: for research-based practice we need a better balance of research effort – more engineering (cf also medicine,…)

5. Influence on policy and practice Humanities dominates policy making (all can play) plausibility ~ common sense – often fails Science dominates research, informs policy, but no body of agreed results > little influence Engineering still marginal, except for tests* some systematic work in some places eg here

6. What is design research? Design Research has emerged as an accepted part of educational research, with a strong input from Cognitive Science. Key features include: insight focus > products and papers realistic classroom situations exploring teaching and learning theory building but with atypical teachers exceptional support no claim to wider usability >no direct impact Engineering research sees the products as first drafts

7. What is design research? For more, see e.g. Educational Design Research eds Jan van den Akker, Koeno Gravemeier, Susan McKenney, Nienke Nieveen Routledge 2006 pragmatic, grounded, interactive, iterative and flexible, integrative, and contextual

8. R P process – key ingredients body of reliable research exemplars of tools and processes funded development programs stable design teams systematic iterative development clear range of effectiveness comparative evaluation-in-depth funded by clients who understand the process see HB+AHS Educ Researcher 32 (2003)

9. …but the academic value system Favours new ideas over reliable research new results over replication and extension trustworthiness over generalizability small studies over major programs personal research over team research first author over team member disputation over consensus building papers over products and processes

10. What is good design? Good design: makes users more powerful via tools and processes that fit their purposes gives users, and others, pleasure does these in a cost-effective way

11. Types of contribution Good design can: make something more widely available printing press, Model-T Ford, www,.. slates, paper, lesson study, ….. lead to continuous improvement cars, TV, medical treatments,… teaching materials,….. enlarge possibilities for practice telephones, search engines, arthroscopes computers, geometric software, microworlds provide existence proofs

12. What do good designers do? They know how to use research results and design skills to improve best practice tackle new challenges effectively pass on their knowledge to other practitioners novices through their materials.

13. Approaches to Innovation Craft-based innovation Builds on authors successful practice Simple and inexpensive process Research-based innovation Builds on best practice, research, design skills Slow, complex and expensive R&D process that uses trialling to build robust innovations ie ambitious innovations that work well

14. Craft-based development model The normal process of authorship Write draft materials Circulate to an expert group Discuss and revise Publish

15. Craft-based development model Questions: How reliable is the extrapolation of author's professional experience to others needs in the new situation? How effective is the communication of the author's intention to the target community of users? OK for small changes Extrapolation and communication unreliable for real innovation

16. Research-based development model Input from: earlier research and development worldwide craft skills of best practice Design skill and creativity led by designers of exceptional achievement well-defined locus of design control Systematic development iterative rounds of developmental trials with: rich, detailed feedback guiding revision of the materials in each round so that intentions and outcomes converge Continuing customer feedback

17. Educational design principles Heuristic, phenomenological theory: Some based on insight research, eg active learning constructive build multiple connections Others design-based, eg role shifting cognitive conflict student ownership Design theory is not often discussed in enough detail to be useful

18. Design beyond just principles Design brilliance is more than these: Surprises that are clearly right Handling complexity simply Controlled innovation Balance in all aspects We know it when we see it – iPod,…

19. The design process Seek excellence in design. What is it? a coherent plan, influenced by research lessons that work well with target S materials that communicate to target T robustness across circumstances of use enjoyment in use by S and T Develop in realistic circumstances Cover all the main goals Sell to many users

20. Mathematical Literacy exemplar development need ML enables non-specialist adults, if they are taught how, to benefit from using mathematics in their everyday lives to better understand the world they live in, and to make better decisions. The sophisticated use of, often elementary, mathematics also called functional mathematics, quantitative literacy, numeracy … Post-11 mathematics is non-functional for most people

21. Mathematics is useless

22. Who needs it? Plan Potential secretaries asked to critique and complete the spreadsheet for planning a conference budget

23. Sudden Infant Deaths = Murder? In the population as a whole, about 1 baby in 8,000 dies in an unexplained "cot death". The cause or causes are at present unknown. Three successive babies in one family have died. The mother is on trial. An expert witness says: " One cot death is a family tragedy; two is suspicious; three is murder. The odds on just two deaths in one family are 64 million to 1" Discuss the reasoning behind the expert witness' statement, noting any errors, and write an improved version to present to the jury.

24. Validate Solve Formulate Interpret Problem Report The modelling process

25. Validate Formulate Interpret Problem Report Solve Dysfunctional math curriculum

26. Oft-neglected Design Aspects a view Is this outward-looking mathematics? few students will become mathematicians math can give them power in their lives does this curriculum do that? for all? (cf ELA) or is it just math (RPF) symptoms: all topic focus, no modelling, tasks What dimensions of engagement? many students lack interest in math itself is make the math interesting all this does? symptoms: variety of activities, of tasks (cf ELA)

27. Oft-neglected Design Aspects a few more Does this develop student autonomy? reliable imitation is not enough to do math what transfer distances do the tasks cover? how long are the chains of reasoning? …involving, which problem solving phases? symptoms: no linked phases, similar tasks together Does this give teachers enough support? student centered teaching is difficult it is easy to overload the teacher what design tactics are used to avoid this? symptoms: teacher in hot seat, centre-stage; no support tactics; too much innovation at once; ……

28. Modelling Joe buys a six-pack of coke for $3 to share among his friends. How much should he charge for each bottle? If it takes 40 minutes to bake 5 potatoes in the oven, how long will it take to bake one potato? If King Henry 8th had 6 wives, how many wives had King Henry 4th?

29. Plan a trip: fault finding and fixing Alison and two friends has planned a cycling trip around Derbyshire on Saturday. Here is their plan for the day. Read through the plan and the information sheets (next page). If you find a mistake, or realise something has been forgotten, write it down and say how they should change the plan. Meet at Loughborough station at 7.23 am. Buy tickets and then catch the train to Derby. This arrives at 7.51 am. At Derby, catch the 8.20 am train to Cromford. This arrives at 8.41 am. Here are the instructions for getting to the Cycle Hire centre: Turn left as you come out of Cromford station, walk along by the river and down Mill road. Cross over the A6. Walk up Cromford hill for about 1/2 mile and you will see..

30. Authentic information sheets

31. Design a tent - open task

32. Design a tent - scaffolded task It must be big enough for two adults to sleep in. The sloping sides and the two ends will be made from a single, large sheet of material. 1.Write down the height and breadth (shoulder to shoulder) of a typical adult. 2.Write down suitable measurements for CD, DE and AP. Remember to leave space for baggage. 3.Draw a sketch to show how you will cut the material from a single piece. Label your drawing with letters A to G. 4.Use Pythagoras theorem to calculate the length of AD. 5.Calculate angles <ADP and <ACD using trigonometry.

33. Design a tent - final version 1.Estimate the relevant dimensions of a typical adult. 2.Estimate the dimensions of the base of your tent. Estimate the length of the vertical tent poles you will need. 3.Show how you can make the top and sides of the tent from a single piece of material. Show all the measurements clearly. Calculate any lengths or angles you don't know. Explain how you figured these out. Your task is to design a tent like the one in the picture. It must be big enough for two adults to sleep in (with baggage). The tent should be big enough so that someone can move around while kneeling down. Two vertical tent poles will hold the whole tent up.

35. Bowland Maths ~20 case studies, including: Reducing road accidents How risky is life? You reckon? Alien invaders Professional development 5 module package, activity based Delivered on-screen on web June 17 www.bowlandmaths.org.uk

36. Body of reliable research needed RF: universities, journals, funders encourage : team research with long timescales on available treatments as well as new coherent in-depth work to give diverse data Mathematical Literacy Extend current research, which shows students need: knowledge of concepts and skills strategies for problem solving and modelling metacognitive control of their reasoning, and the disposition to view situations mathematically

37. What development skills are needed? The team needs: Systematic methods of observation Interview skills Protocols related to the design goals Methods for analysing observation reports, student work, interviews Design skill in using this rich feedback systematically to improve the materials. ie as products are developed in other fields

38. Stable design teams need long term coherent programs in each field developing a community that works towards higher standards of design and development training and career progression for designers Wider recognition of its importance recognising the intellectual value of design as in medicine, engineering, fine arts Mathematical Literacy needs all of these.

39. Clear range of effectiveness Trials will give an indication of conditions for successful use, including: Prior learning of students Teacher skills and support Time and support for absorbing the change Feedback from widespread use will refine this Most research fails to explore boundaries of validity – essential for design

40. Comparative evaluation-in-depth Guides choice of materials and approaches Informs improvement and future development To do this well, evaluation needs to: look at widely available treatments across variables: users, use, outcomes compare alternatives – many dimensions Still rare in all countries Mathematical Literacy will need all these in due time – typical time from agreeing goals to stable curriculum implementation ~ 10-15 years

41. Math Literacy ~ proof of concept For Mathematical Literacy units so far, it seems: All students succeed and enjoy the work FM narrows the range of performance Many, but not all, teachers can handle this work with just the materials – more with live PD training 1 or 2 new three-week units per year is digestible More research needed, across more exemplar units to warrant such general statements

42. Long term growth of design skill Freudenthal Institute 40 years, 40 fte people, coherent support US: EDC, TERC, LHS,.….. 40 years, ~ 100 people, one-off projects no coherent support Shell Centre 40 years, ~ 5 fte people, strands of work, no coherent support Such groups are rare worldwide currently, you need luck and cunning to survive

43. How much does engineering cost? Engineering research is more expensive. NSF curriculum projects ~$1,000,000 for ~160 hours >> $6,000/ hour Shell Centre curriculum units 10-15 hours ~ $30,000 per classroom hour. Is this good value?

44. How much does engineering cost? Analysis (US context) whole school curriculum ~15,000 hours 14 years*200 days*5 hours 3 parallel developments ~ $1.5 billion 15,000 hours*3*$30,000/hr Over 10 years ~0.05% of running cost $1.5 Bn/($300Bn/yr*10 years) Other changing fields? R&D typically 5-15% 80% development, 20% basic research

45. Why is design flair ignored? Tradition that, for most, math and science are necessary evils – and boring Focus on design principles but details matter (as in all art) No comparative evaluation in depth of learning or attitudes (beliefs and motivation) The classics of math and science are not digestible K-12 (cf music, literature), so Major burden on educational design, but No profession of educational designer some systematic work in some places eg here

46. Building a Design Community Improve standards of design by: sharing methods and experience developing theory of design providing training Increase recognition in: government via payoff >> funding academia via argument, funding.. Better career paths – will follow ISDDE, a growing group, is making a start International Society for Design and Development in Education www.ISDDE.org

47. ISDDE Conference 2008 FI @ Egmond aan See, NL June 30 – July 2 Linking design research to professional educational design Working groups: curriculum design designing professional development assessment design educational software Contact Peter Boon, Conference Chair p.boon@fi.uu.nl http://www.fi.uu.nl/isdde/welcome.html

48. Summary Design and development – society needs to develop the engineering research approach Functional Mathematics – we need to make good our claims of power for all Build a design community to get higher standards, and improved recognition for designers