Assessing Reading

Ensuring valid and reliable reading assessment Covering a range of reading skills: gist comprehension, scanning, inferencing… Different text topics, types and passages per test A range of task types Encouraging critical-thinking skills Recognizing the purposes and audiences of texts Testing discourse level

What is reading? Following are needed to become efficient readers: Mastering bottom-up (letters, words, phrases) and top-down (conceptual) strategies Understanding both linear passages and non-linear sources of info (maps, pie charts, etc.) Developing appropriate content and formal schemata

What to assess in reading? The place to start is the target skills students should develop: what kind of reading, what sub-skills and strategies are important? We define the construct of reading before we start preparing the test. Construct of reading: translating abstract theories of reading ability into assessment tasks What do we mean with this?

1. You can eat here in the morning! 2. You should not swim here. 3. You must not drive fast here. A) B) C) D) E) SLOW! Dangerous crossroads SWIMMING POOL OPEN AFTERNOONS Adults- €2.50 Children-€1 BREAKFAST SERVED DANGER ! DO NOT GO INTO THE WATHER POLICE CARS ONLY 7. Kısa yazılı mesajların anlamını kavrar.

1. You can eat here in the morning! 2. You should not swim here. 3. You must not drive fast here. A) B) C) D) E) SLOW! Dangerous crossroads SWIMMING POOL OPEN AFTERNOONS Adults- €2.50 Children-€1 BREAKFAST SERVED DANGER ! DO NOT GO INTO THE WATHER POLICE CARS ONLY 3.Gerekli bilgiyi bulmak amacıyla göz gezdirerek okur.

Reading Sub-skills Major Reading Skills Skimming for gist, scan for specific details, establishing overall organization Reading for main ideas, supporting details, author’s argument and purpose, relationship of paragraphs, facts vs opinions Info transfer from non-linear texts Drawing inferences from stated and implied content

Minor Reading Skills Understanding at the sentence level Vocab, syntax, cohesive devices Understanding at inter-sentence level Identifying what pronouns refer to, recognizing discourse markers Understanding components of non-linear texts Meaning of graph/chart/tables, ability to find and interpret intersection points

Grammar and Vocabulary Generally grammar and vocabulary are contextualized as part of reading passages. However, they might also be assessed separately if you teach them separately to your students (washback effect)

Designing Tasks Many formats are possible Selective-response format: multiple-choice, T-F, matching, cloze Limited-production format: Gap-fill/cloze or short answer, filling in a form In some international exams (TOEFL, FCE), test takers are expected to deal with the text as a whole (inserting missing sentences in the text)

Specifications Helps teachers establish a link between the objectives and the assessment. Some general features of specifications: Content Conditions Grading criteria

Content Materials to be covered: will it cover topics similar to the ones Ss have encountered? Will you test some unknown vocab in context?... Any pre-reading skills? Type and number of texts, etc

Conditions Structure and component parts (how many subsections?) What format? Layout (e.g. for skimming layout is important so that Ss can read quickly) Timing Grading Criteria How will you mark it? Sample specification (p. 49)

Texts Many sources Can be purpose written Directly taken from authentic materials Adapted Can be prose passages or non-linear texts. The most important thing is to make it appropriate Be sure all texts are clear and legible Check the language and format of the texts If needed, raise or lower the level of language If needed, use line numbering, especially if there are questions related to referring or vocabulary

Questions Questions are generally at a slightly lower level than the reading text. Questions should be in the same order as the text. In case of multiple choice format, make all statements positive A negative statement and a negative option brings the problem of double negative Rephrase material using synonyms to avoid students scanning for the same word Paraphrasing has a positive washback effect as it encourages vocabulary growth

Formats The focus should be on the content, not on the complexities of the task For young learners: simplest format that involves minimal instructions to read (using pictures) Many different formats: the most common ones are MCQ, T/F, matching (effective if constructed carefully) Embedding items in rich context makes these questions effective Cloze test Even while assessing structure and vocab, we can use context (e.g. Cloze test) See an example multiple cloze task on p. 53

Multiple-choice format in reading comprehension A standard technique: JR (just right): the correct or the best answer TG (too general): this distractor is related to an option that is too broad TS (too specific): this distractor is about only one detail within the text OT (off topic): this distractor reflects an idea that is not developed in the text. For advanced learners, it is related in some way, but does not represent the main idea.

True/False format in reading comprehension Keep in mind the NG options Phrase all the statements affirmatively Avoid using extremes like always, never (they tip off that the answer is false) Keep in mind the format (let them mark, not write T/F) See sample tasks on pp

Short-answer format in reading comprehension Consider the type of task when making decisions about the degree of accuracy, spelling, punctuation, etc. Short-answer items test details, but can also be used to assess other skills. E.g. Completing a chart or a diagram Or expecting them to read the charts or diagrams See sample tasks on p. 59

Sequencing format in reading comprehension Checks students’ understanding of the organization of a text (coherence) Especially useful for narratives, biographical events, recipes, etc. While designing such tasks, create logical linkages between components by repeating key words sequencing chronologically connecting logically See a sample task on p

Combination format in reading comprehension Cloze tests can be good samples for this format Gap-fill to assess grammar in context (grammar and reading) Open cloze (reading and writing) Cloze summary (reading and summarizing) See a sample task on p