Aspects of the Art of Assessment Design: ‘Educational Design in general is largely underestimated and unexplored area of design, and its relationship with educational research can be characterized as somewhat less than satisfying’. ‘What we need is a tight linkage between research and the creation of mathematically rich and revealing tasks for productive classroom use’. Jan de Lange (2007)
‘In an attempt to bridge the cultures of educational research and educational design we recommend more research focused on the design process so that educational designers can sharpen their professional vision’. Alan Schoenfeld (2009) The Question: Does Educational Design Research enhance and improve the world of designers, does it help designers to sharpen their professional interventions?
To find an answer we follow Lageman (2002): ‘Educational Design Research is particularly concerned with developing usable knowledge; the products of research should be relevant for practice’. Or:
‘Educational Design Research describes a family of approaches that strive toward the dual goal of developing theoretical understanding that can be of use to others while also designing and implementing interventions to address problems in practice’. Susan McKenney & Thomas Reeves (2012)
‘(Educational) Design Research is not defined by its methods but by the goals of those who pursue it. Design research is constituted within communities of practice that have certain characteristics of innovativeness, responsiveness to evidence, connectivity to basic science, and dedication to continual improvement’. Carl Bereiter (2002)
My first try at an answer: No, EDR does not sharpen my professional vision; it hardly results in usable knowledge for designers.
Example 1: ‘Formula for conceptualizing design principles: If you want to design intervention X [for purpose/function Y in context Z]; the you are best advised to give that intervention the characteristics C1, C2,..Cm [substantive emphasis]; and do that via procedures P1, P2, Pn [procedural emphasis]; because of theoretical arguments T1, T2,…Tp; and empirical arguments E1, E2, …Eq’. Jan van den Akker (1999,2010)
Example 2: “Design & Construction” Chapter in McKenney and Reeves (2012) Offers practical suggestions for design; but does not give a theoretical underpinning for educational design on the micro level. Nor does it sharpen the vision of the designer.
Kernel of Design What is missing: the Art of Design & Construction The three components of Educational Design are: -Educational Context & Community -The Science and Research -The Art of Designing
‘Since mathematics incorporates a systematic body of knowledge and involves cumulative reasoning and understanding, it is to that extend a science. And since applied mathematics involves choices which must be made on the basis of experience, intuition, and even inspiration, it partakes the quality of art. Thus in (theoretical)mathematics there are certainly to be found both art and science, and there is science in both pure and applied mathematics, as there is art’. Peter Hilton (1976)
So we are looking for something like: Since educational design theory incorporates a systemic body of knowledge and involves cumulative reasoning and understanding, it is to that extend a science. And since design and construction involves choices which must be made on the basis of experience, intuition, and even inspiration, it partakes the quality of art. Thus in design theory there are certainly to be found both art and science, and there is science in both the design theory and in design and construction, as there is art.
It’s good news for design when intuition is recognized as the important and effective skill that it is — a skill that takes years of experience and practice to develop. Designers have always relied, to some degree, on intuition and personal style. There are times when a designer is right just because he/she is right. Nathan Sinsabaugh 2009
In a knowledge-intensive world, design thinking is critical to overcoming the biggest block: overcoming analytical thinking and fear of intuitive thinking. Roger Martin 2009
‘In the swampy lowlands, problems are messy and confusing and incapable of technical solution. The irony of this situation is that the problems of the high ground tend to be relatively unimportant to individuals or to the society at large, however great their technical interest may be, while in the swamp lie the greatest problems of human concern. The practitioner [researcher] is confronted with a choice. Shall he remain on the high ground where he can solve relatively unimportant problems according to his standards of rigor, or shall he descend to the swamp of important problems where he cannot be rigorous in any way he knows how to describe?’ Donald Schön (1995)
2. Art The role of experience, intuition and inspiration is missing 3. The Science ‘Educational Research’ is to much ‘high ground’ 4. Developmental Research (Leen Streefland, 1980) & Didactical Engineering (Hans Freudenthal, 1979) are losing their academic status; just like research based design. 5. Designers fail to making their mode of operations explicit
6. Political Elements such as standards, high-stakes accountability and pre-selection of textbooks can reduce curriculum innovation (by schools). (OECD, 2012). (Note: don’t pay to much attention to OECD reports, they do not necessarily reflect reality, but perceived reality by politicians).
Standards: Mathematically proficient students understand and use stated assumptions, definitions, and previously established results in constructing arguments. They make conjectures and build a logical progression of statements to explore the truth of their conjectures. They are able to analyze situations by breaking them into cases, and can recognize and use counterexamples. They justify their conclusions, communicate them to others, and respond to the arguments of others. They reason inductively about data, making plausible arguments that take into account the context from which the data arose. (Etc).(CCSS, 2011)
7. In practice these aspirations are rarely reflected in high stakes assessment. This fact makes live of designers even more complex: teachers teach to the test, not according to the standards.
8. Technology: trend is clear: It throws us, conceptually, always backwards (Example Dutch Examinations, 2012; many more: Sept 4, 2012: ‘Replace all textbooks by iPads with textbooks on them; Ed Nijpels, former politician in Volkskrant).
9.Technology: only a useful supplement. Existing tools lack possibilities that designers would love to have. 10. Technology: Designing Electronic Learning Environment (Peter Boon 2012) is complex and should be done with content experts; designers are reluctant. 11. Technology: The ‘Lore’ of Facebook: Generic ‘share’ tools independent of content; And Echo360 (Forbes, August 30, 2012)
12. Technology: Incidental or Ad Hoc Apps (Motion Math, 2011).
13. No time, no money for ‘slow’ design (Jan de Lange, 2008) 14. Very different models for Design (& role of designers) 15. It’s for the eyes, not for learning (Looks vs. Brains: Form over Function)
The Future of Educational Design and Research in Educational Design
1. There needs to be two way traffic between Educational Design & Research in Ed. Design. 2. There need to be more collaboration with Design; more out-of-our box action 3. There need to be more attention to the ‘Art’ of design; intuition; inspiration; etc
4. Design is a coherent discipline in its own right; there exists a designerly way of thinking and communication that is different from scientific and scholarly methods of enquiry when applied to its own kinds of problems (Bruce Archer,…) and we need to establish an epistemology of practice implicit in the artistic, intuitive processes which designers bring to situations of uncertainty, instability, uniqueness and value conflict (Donald Schön, 1990)
5. Design is teamwork; but the team needs to be interdisciplinary 6. Design of content materials needs to go hand in hand with tools, new media, etc. They cannot be developed independently. 7. Designers need to contribute to new knowledge, especially for researchers
8. Innovation needs to get more room: less standards, less high-stakes accountability, less pre-selection of textbooks; here lies a task for educational designers. 9. Designers can contribute to closing the gap between aspirations (Standards) and reality (Assessments); they need to provoke political discussions about this ‘fraud’. But only in close cooperation with other communities. As long as high-stakes test are commercially produced little hope remains.
10. How and where can we help people to become good designers? 11. More attention for the communities of practice.
12. All kind of educational institutions in many different countries fail to innovate, are part of the system, are in there for the money, to please politicians, etc. There is a need for independent thinkers that can be taken somewhat seriously, and take children and students seriously, as well as what is really taught at schools, and at home. Maybe local ISDDE’s?