We think you have liked this presentation. If you wish to download it, please recommend it to your friends in any social system. Share buttons are a little bit lower. Thank you!
Presentation is loading. Please wait.
Published byLillian Worrel
Modified about 1 year ago
© UCLES 2013 Assessing students with disabilities: Voices from the stakeholder community Lynda Taylor and Hanan Khalifa
© UCLES 2013 Acknowledgement This presentation is based upon a chapter by Taylor & Khalifa in: Tsagari, D & Spanoudis, G (eds) (forthcoming) “Assessing Second Language Students with Learning and Other Disabilities”
© UCLES 2013 Context Test accommodations (TA) typically involve: modifications to testing materials adjustments to test administration conditions
© UCLES 2013 Some common provisions Extended time Additional breaks Enlarged print Braille Magnifiers Amplifiers Adjustable workstation Wheel chair access Switching computer mouse from right-hand to left-hand operation Adjustment to the brightness of computer screen Lip reading for listening Writer Reader
© UCLES 2013 Policy and practice Decisions about the nature and extent of test accommodations tend to be based upon expert judgment of test developers informed by insights from the educational and health professional bodies working in a given disability area underpinned by a limited amount of empirical research
© UCLES 2013 The role of research Undertaking large-scale empirical research into accommodated tests can involve significant ethical, logistical and measurement challenges. Smaller scale empirical studies can still generate insights for the development and validation of accommodated tests.
© UCLES 2013 Investigating test impact There is greater understanding today of the ways in which a test can impact upon a range of test stakeholders. But there has been relatively little work in this area so far in relation to accommodated tests.
© UCLES 2013 Our study Purpose: to investigate stakeholders’ perspectives on accommodations offered by one test provider. The stakeholder groups identified: –test developers who prepare modified materials –exam centres responsible for requesting a modified test and arranging for its administration –examiners who conduct + rate the modified test –test takers themselves
© UCLES 2013 Methodology A documentary review of current exam board policy and practice was triangulated with survey and interview data from stakeholder groups to: investigate the match between policy and practice identify any issues meriting attention and further development Data collection period: 2012
© UCLES 2013 TA requests from exam centres in 2012 Test Accommodation % Extra time 86.0% Separate invigilation 62.5% Use of word processor/copier/typewriter/assistive technology 46.3% Special listening test 44.9% A3 enlarged print papers 40.4% Single candidate versions of the speaking test 37.5% Use of amanuensis/reader 30.9% A4 modified enlarged print papers 25.7% Braille papers 26.5% Supervised breaks as necessary 26.5% Lip reading version of the listening test 24.3% Braille and enlarged print versions of the speaking test 22.8% Exemptions from listening/speaking test 19.9% Electronic version of paper – PDF 2.2% Other 6.6%
© UCLES 2013 Data collection: (1) document review Test writer guidelines for producing test materials Administrative documentation for test centres Speaking Examiner training and support documentation Internal exam board annual reports/reviews Chapters on test-taker characteristics in Studies on Language Testing volumes 26, 29, 30 and 35 (Chap 2) Guidance notes for teachers Desk research of provisions offered by other test providers
© UCLES 2013 Data collection: (2) surveys Survey for Exam centres to elicit perspectives on: –extent of the testing centre’s applications for TA –accessibility of information provided by the board on TA provision –user-friendliness of the board’s TA application process –nature of TA support offered to the centre –impact of the TA on the testing centre
© UCLES 2013 Data collection: (2) surveys Survey for Test takers inviting candidates to comment on: –whether the TA had met their needs –whether any improvements to the TA were needed –whether they had felt at a disadvantage at any time
© UCLES 2013 Data collection: (3) interviews Interviews were arranged with those responsible for –developing modified test materials for all skills –delivering the face-to-face speaking tests and providing test scores (i.e. speaking examiners). Interview protocols were developed to guide the semi- structured interviews and sent to participants beforehand. Interviews were recorded and transcribed. Field notes were also taken.
© UCLES 2013 Results Perspectives of four stakeholder groups: 1.Test developers/writers 2.Exam centres 3.Speaking test examiners/raters 4.Test takers
© UCLES 2013 1.Test developers’ perspective Policy and practice evolves over time: test modification on a request by request basis a systematic policy of pre-modifying tests for an “on-the- shelf” solution Growing professional collaboration: between test providers and relevant health professional bodies: sharing good practice in and accessing specialist expertise regarding a given area of disability
© UCLES 2013 1. Test developers’ perspective Some specific challenges Providing modified test materials where there is a potential combination of : high visual content, range of colours, candidates with limited braille skills. In speaking tests, providing a stimulus in non- standard test which is equivalent to visual input for standard test Managing candidates’ expectations due to varying regulations from one country to another with regard to provisions made.
© UCLES 2013 2. Exam centres’ perspective Some numbers to report 249 centres applied for TA in 2012 (all invited to take part) Responses received from 144 centres based in 34 countries in Europe, Asia and the Americas Roles of respondents varied from Centre Exams Manager (72%) to other staff members (28%) e.g. Exam Administrators, Exam Assistants, Exam Officers, Exam Coordinator. All respondents had experience of dealing with TA management, with 78% of the respondents having more than two years of experience TA requests in the 144 centres were made for all of the Cambridge English exams including IELTS.
© UCLES 2013 Accessibility of information on TA provision (according to exam centres) AgreeDisagree No experience Information on how to apply is easily accessible 94%5%1% The application procedure is explained clearly 89%9%2% The Special Requirements Booklet is helpful and easy to read 89%9%2% The application procedure is straightforward 87%11%2% There is plenty of information on website 68%23%9% Using in-house portal to find information and download forms is straightforward 73%24%3% The application form is easy to complete 92%6%2%
© UCLES 2013 Test provider’s service to exam centres 88% agreed that the Special Requirements unit at Cambridge was efficient and helpful 89% thought the Helpdesk in Cambridge was helpful. 80% felt that test provider staff explained procedures clearly
© UCLES 2013 Exam centres’ own service to TA applicants 96%: felt that candidates sitting for exams at their centres found the arrangements helpful. 24%: reported encountering difficulties in making arrangements (e.g. organising extra rooms, extra invigilators, a reader)
© UCLES 2013 “…when students need extra time and, therefore, separate invigilation, that means arranging for a special room just for them, an invigilator just for them, and setting up extra speakers just for them. All of this, beside bringing more hassle, is more expense. We completely agree with the special arrangement system but it does put extra strain on the centre…”
© UCLES 2013 Exam centres – some specific challenges encountered “A blind candidate who is not familiar with braille and requested specialized computer software for all the four skills.” “A YLE Starters student with Down’s Syndrome. We thought it would be a problem because of the age (Starters is for 5-7 while the candidate was 17) and because we weren’t sure what Special Arrangements were needed”
© UCLES 2013 Exam centres – some specific challenges encountered “We had a case where the candidate has limited motor skills and it takes an extremely long time for the candidate to type the answers onto the computer. The 4 parts of the test on one day is extremely taxing and we accordingly requested that the candidate take the examination over 2 days. The candidate must lie down while working.”
© UCLES 2013 3. Examiners/raters’ perspective Difficulty for speaking test examiners of building- up comprehensive and sustained expertise in this area: –opportunities for examining test takers receiving accommodations are relatively infrequent –types of disability encountered can be very varied
© UCLES 2013 3. Examiners/raters’ perspective A conflicting sense of wanting to be “kind and supportive” but at the same time being “fair”, of “getting warmth across and making candidates feel at ease” while still conducting a reliable and valid assessment. The “intense concentration” needed (on the part of the Assessor) to evaluate the linguistic production of a test taker with a speech impediment, i.e. “disentangling lack of fluency due to a stammer from the rest of the test taker’s skill in speaking English”.
© UCLES 2013 3. Examiners/raters’ perspective A sense of feeling less well prepared to examine test takers with specific learning difficulties, e.g. attention deficit hyperactivity disorder. The need to sometimes “think on your feet” in unexpected and demanding situations. The potentially unhelpful (interfering?) presence of a relative/friend who is accompanying the test taker for support (requiring firm but sensitive handling by the test centre).
© UCLES 2013 4. Test takers’ perspective In general, comments invited from test takers were positive with few requests for improvement. The following quotes are representative of the views expressed. “I am really glad to have done the exam with the special arrangements provided. I have Writer’s Camp, a disease which affects my handwriting and causes involuntary spasms affecting certain muscles of my hand and fingers. As I am no longer able to write, I need to use the computer as a means of written communication”.
© UCLES 2013 4. Test takers’ perspective “I felt well because I could sit for the exam just like my classmates.” “I’m afraid the listening paper arrangements were not much of help, as I had someone sitting very close to me. I did not feel comfortable. I would suggest you provide the recording with longer pauses or the possibility of listening three times.”
© UCLES 2013 Conclusions and recommendations Increase the extent and nature of information provided to stakeholders: provide more online exemplar materials awareness-raising of inconsistencies in national and international disability legislation provision of clearer instructions on how to deal with the mismatch between UK practices and elsewhere e.g. under Dutch Law medical practitioners are not permitted to provide medical statements; in Italy all dyslexic students are exempted from taking writing examinations
© UCLES 2013 Conclusions and recommendations Offering more comprehensive training providing a range of exemplar recordings showing test takers receiving different test accommodations accompanied by in-depth guidance for examiners, readers adequate briefing in advance and appropriate support on the day addressing the challenge of achieving a consistent and professional approach in face to face speaking assessment
© UCLES 2013 Conclusions and recommendations The need for further research in the area of assistive technology “Skill levels in using Braille can vary among blind candidates, with younger learners not necessarily learning Braille nowadays but instead using a computer and screen reading program for their reading and writing”. matters of equipment compatibility and technical support observational studies of test takers completing a modified task exploring other stakeholder perspectives, e.g. score users, professional disability agencies
© UCLES 2013 The personal story [Photo taken from the website of the Down’s Syndrome Association, UK] “A YLE Starters student with Down’s syndrome. We thought it would be a problem because of the age (Starters is for 5-7 while the candidate was 17) and because we weren’t sure what Special Arrangements were needed. In the end, it was a very positive experience - for us, for the school she came from, and for the candidate.”
© UCLES 2013 Thank you for your attention. Questions/comments?
Functional Skills Support Programme OfQual Functional Skills Qualifications Criteria – Issued November 2009.
Martha Thurlow and Laurene Christensen National Center on Educational Outcomes CEC Preconvention Workshop #4 April 21, 2010.
IAEA International Atomic Energy Agency. IAEA Outline LEARNING OBJECTIVES REVIEW TEAM AND COUNTERPARTS Team Composition Qualification PREPARATORY PHASE.
IAEA International Atomic Energy Agency. IAEA Outline LEARNING OBJECTIVES REVIEW TEAM AMD COUNTERPARTS Team Composition Qualification PREPARATORY PHASE.
Assistant Instructor Nian K. Ghafoor Feb Definition of Proposal Proposal is a plan for master’s thesis or doctoral dissertation which provides the.
Toolkit for Mainstreaming HIV and AIDS in the Education Sector Guidelines for Development Cooperation Agencies.
Supporting Deaf Students at the University of Sheffield Claire Shanks, Disability Coordinator Nora McClelland, University Teacher, Sociological Studies.
Needs Analysis Instructor: Dr. Mavis Shang Presenter: Sophia Yar-ling Tsai 27 March 2008 I-Shou University, Taiwan.
Minnesota Manual of Accommodations for Students with Disabilities Training Guide
Exploring the use of QSR Software for understanding quality - from a research funder’s perspective Janice Fong Research Officer Strategies in Qualitative.
Inclusive Education Defining Inclusive Education Inclusive Education(IE): “ …Means that all students in a school regardless of their differences are.
Introduction & Background Laurene Christensen National Center on Educational Outcomes National Center on Educational Outcomes (NCEO)
Interviews with Committee Chairs and Vice-Chairs - One way to get quality time with Members Susan Swift APLIC Conference Regina, September 9, 2014.
International English Language Testing System. IELTS Immigration International English Language Testing System Why IELTS? Further Studies Organised by.
Unit 1 – Preparation for Assessment LO 1.1&1.2&1.3.
Consistency of Assessment Languages What is Assessment? Assessment is the process of identifying, gathering and interpreting information about students'
SAN Distance Learning Project Teacher Survey 2002 – 2003 School Year... BOCES Distance Learning Program Quality Access Support.
Importance of Questioning and Feedback Technique in developing 3 Cs
How IEP Teams Make Assessment Accommodation Decisions: Rhode Island’s Research Findings Paul V. Sherlock Center on Disabilities at Rhode Island College.
A Student’s Guide to Methodology Justifying Enquiry 3 rd edition P ETER C LOUGH AND C ATHY N UTBROWN.
INACOL National Standards for Quality Online Teaching, Version 2.
PROJECT WORK. The project work at schools is used across the curriculum. We can use projects in a school subject separately or we can join more school.
Michigan Educational Assessment Program MEAP. Fall Purpose The Michigan Educational Assessment Program (MEAP) is Michigan’s general assessment.
Presenter: Alice Instructor: Dr. Kate, Chen Date: January 17, 2011 Web 2.0 in learning English: The student perspective 1.
Collecting data about service user satisfaction November 4th 2013 Workshop in London.
ask in writing for evaluation; keep a copy of the request explain child’s problems and why evaluation is needed share important information with.
Mohammad Alipour Islamic Azad University, Ahvaz Branch.
Development of Assessments Laura Mason Consultant.
Kerry Cleary An evaluation of the impact of Values Based Interviewing at the OUH Values Based Conversations and wider engagement strategies.
Specific Learning Difficulties: What administrators need to know Kirsty Wayland Helen Duncan
Assistive technology: A delivery model for college and university student services Robert Perkins School of Education University of Charleston, SC Charleston,
Dr. Rob Danin Senior English Language Fellow
2015/8/30 Ch3 Needs analysis M Venus M Carl.
Gulana Hajiyeva Environmental Specialist World Bank Moscow Safeguards Training, May 30 – June 1, 2012.
Service Delivery Models and Inclusive Practices in Speech-Language Pathology: Challenges and Solutions Connecticut Speech-Language-Hearing Association.
The Inclusive Classroom: Strategies for Effective Differentiated Instruction, 4th Edition © 2010 Pearson Education, Inc. All rights reserved.
McLendon and Polis1 An Administrator’s Guide to Assessment: A Menu of Assessment Options for MAERS and Instructional Guidance.
External examiner induction Alison Coates QA Manager (Validation & Review)
DEVELOPMENT OF ENGLISH LANGUAGE SPEAKING SKILLS AT THE UPPER- SECONDARY SCHOOL LEVEL IN LARGE CLASSES: CHALLENGES AND FUNCTIONAL ACTIVITIES AS WAYS TO.
An Introduction to IMPACT Sunnybrook Health Sciences Centre GiiC Knowledge-to-Practice Workshop Presenters: Susan Riddle & Jasmine Arellano.
BMH CLINICAL GUIDELINES IN EUROPE. OUTLINE Background to the project Objectives The AGREE Instrument: validation process and results Outcomes.
Purpose of the test: The IELTS Listening test is designed to assess a wide range of listening skills, including how well you understand main ideas and.
SACS Governance & Leadership Committee September 26, 2012.
Assessment: Reliability, Validity, and Absence of bias Lecture 2:
Guidance for AONB Partnership Members Welsh Member Training January 26/
Accessibility and Accommodations Alabama Student Assessment Program Nannette Pence Education Specialist Student Assessment Alabama State Department of.
ACE TESOL Diploma Program – London Language Institute OBJECTIVES You will understand: 1. What standardized tests are 2. Why we use standardized tests 3.
Americans with Disabilities Act (ADA) 1990 What Teachers Should know about Title II – Public Educational Institutions. Presented by Janie Beverley.
AUDIT STAFF TRAINING WORKSHOP 13 TH – 14 TH NOVEMBER 2014, HILTON HOTEL NAIROBI AUDIT PLANNING 1.
© 2017 SlidePlayer.com Inc. All rights reserved.