DIALANG Plus INTUIT Closing conference 30 January 2008 Kevin Haines Hans Veenkamp Language Centre, University of Groningen

Putting the Plus into The beginning... -DIALANG is the first major language assessment system to be based on the Council of Europe's "Common European Framework of reference", which has become established throughout Europe as the most widely recognized frame of reference in the field of language learning. -Curriculum: “no immediate connection of piloting with local pedagogy/syllabus” (Alderson 2005: 60). -B-Beta version: “accept its limitations in order to appreciate its beauty…?” (KH, Utrecht, January 2006)

Initial evaluation: recommendations Pre-test training or acclimatization is needed Follow-up is needed to make advice “real” and relevant to context Practical exercises are needed to deal with the deficiencies defined in the test The test should take place within the context of a course rather than preceding it – a systematic lead from DIALANG into the tasks on the course The link made to the Common European Framework should be consolidated

Putting the Plus into The first pilot: June 2006 –Group: 18 x 2nd Year Law students –Purpose: screening for 3rd Year –Materials: DIALANG + ‘Finland’* materials: scales and self-assessment material –Exercise: DIALANG Reading and Writing and follow-up group discussion * Thanks to Ari Huhta of Jyvaskyla University, Finland

Putting the Plus into The first pilot: Conclusions: DIALANG technically sound and user-friendly; occasional in-built errors are a minor irritation rather than a major problem Feedback generally good; students liked reviewing questions; some students would like more detailed explanation of mistakes Students liked hints and tips for moving to the next level of CEFR Some students: awareness: confirms information already known Some students: encouraging advice: practice more, practice often Follow-up as catalyst for wider discussion of learning and evaluative discussion of course content

Putting the Plus into Embedding DIALANG in Blackboard The University of Groningen Nestor matrix –Make effective link between the interfaces through test instructions in Blackboard –Create preparation and follow-up procedures –Create instructions for students AND teachers –Make link to the test through the Course Organizer matrix

Putting the Plus into The second pilot: The Set-up 350 students 14 teachers Non-teaching weeks Supervision

Putting the Plus into The second pilot: Preparation Students Teachers Evaluation

Putting the Plus into The second pilot: The Test Students Teachers Evaluation

Putting the Plus into Having now supervised my second DIALANG Plus test group, I must say I'm rather pleased with the way things are going. It's also clear, however, that, human nature being what it is, students tend to rely on the supervisor telling them what to do rather than just following the written instructions laid down in DIALANG Plus. Still, as a step-by-step guide to working with DIALANG, it's very useful. Weblog Hans Veenkamp, 5th October 2007

Putting the Plus into We discover that students find it quite complicated to follow the instructions until they get the idea that they need to use the Blackboard screen as a backdrop behind the DIALANG screen. Partly the problems lie in the connections between our information and partly in the human tendency not to read instructions.” Weblog Kevin Haines, 2nd October 2007

Putting the Plus into It was easy to refer back to the DIALANG Plus pages during the DIALANG test.

Putting the Plus into The second pilot: Follow-up Students Teachers Evaluation

Putting the Plus into Personally, I think the DIALANG level indication may very well be right, but despite this, there are some parts on which I have to disagree with the DIALANG score sheet. One of these things is that if I were to believe DIALANG, my punctuation would be consistent. Unfortunately, my punctuation is not flawless, as I see it. Another point which differs from the DIALANG outcome is the fact that I hardly ever use a dictionary but instead try to find the meaning of a word by looking at its context. Nice as a C1 score might be, there is still room for improvement. To reach the next level I think I need to expand my vocabulary, work on my punctuation and improve my pronunciation.

Putting the Plus into I was able to keep a good record of the feedback given during the DIALANG Test in order to write my reflective report later.

Putting the Plus into Recommendations –Structure DIALANG into the wider learning context to encourage meaningful interpretations: Preparation» Test » Follow-up –Make full use of the formative function of DIALANG to turn feedback into learning goals: Encourage students to make their own judgments about the meaning of the DIALANG feedback and advice Encourage teachers to engage with students to discuss the relevance of DIALANG feedback in context –Use DIALANG to increase awareness of the CEFR amongst students and teachers, providing a catalyst for the discussion of language performance

Putting the Plus into The future –e–extend use to a variety of settings, such as other large-scale courses individual formative assessments indicative screening procedures –m–make use of an institutional version will increase test security will improve reporting will promote autonomous use –f–fine-tune through continual evaluation so that DIALANG Plus remains fit for purpose