1. Review course concepts
Module 8: Review

2. Course Review
This PPT will review concepts that we have covered this session. We will review:
Course objectives
Course modules
Course concepts
Future directions

3. Course Objectives

4. Course Objectives
This course had 8 course objectives. Review the objectives on the next slide and try to remember a few things that you learned for each objective. Consider whether you feel better prepared as a teacher in each of these assessment areas.

5. 8 Course Objectives Understand and apply key concepts of assessment
Understand bottom-up and top-down subskills involved in reading, listening, speaking, and writing
Identify text features of reading and listening passages
Identify key aspects of rubrics for speaking and writing tasks
Understand how questions relate to test design
Understand features of good questions and prompts
Identify online assessment-related resources
Understand how to review English language tests

6. Course Modules

7. Course Modules
There were 8 modules in this course. Review the modules on the next slide and try to remember a few things that you learned within each module. Consider whether you feel better prepared as a teacher in each of these assessment areas.

8. Course Modules Introduction Reading Listening Speaking Writing
Test Design and Statistics
Task-based Assessment and Professional Development
Review of Key Concepts from the Class

9. Course Concepts

10. Assessment Design
The first three steps in creating an assessment are: Defining your test construct Defining the test purpose Defining the target language use

11. Assessment Design
The cycle of creating an assessment consists of the following steps:
Think about the purpose of the test
Define the construct you want to measure
Decide what content/student learning outcomes to assess
Decide what items to use to assess the content
Create a table of specifications
Write the items
Revise items

12. Reading
In bottom-up processing, readers start with the smallest units of language. They understand each unit and then fit them together into a larger text. Readers use bottom-up processing with: Word recognition Phonological processing Orthographic processing Semantic and syntactic processing

13. Reading
In top-down processing, readers understand the big picture of what they are reading. Top-down processing involves activation of schemata. (Schemata is a reader’s prior knowledge.)
Readers use top-down processes when they
Understand the main idea of a text,
Understand the purpose of a text,
Relate the text to their own knowledge, and
Make inferences that are probable.

14. Reading Item Formats Common item types to use with reading:
Multiple choice
Sentence Completion
Fill-in-the-blank
True/false/not stated
Short answer
Matching

15. Reading passages
When choosing reading passages, it is important to consider the
following things:
Students’ purpose
Type of text
Similarity of the texts
Number of texts

16. Reading passages
Features that contribute to the difficulty of a reading passage:
Background knowledge vs. text knowledge
Presence of text while answering questions
Text length
Item dependence

17. Reading passages
To understand the complexity of a reading passage, it may be useful to use one of these three readability tools:
Flesch-Kincaid (Embedded in Microsoft Word)
Lexile Analyzer
Compleat Lexical Tutor

18. Vocabulary
There are many aspects of vocabulary that can be assessed. Here are a few:
Receptive vs. Productive vocabulary knowledge
Selective vs. Comprehensive vocabulary knowledge
Discrete vs. Embedded
Context independent vs. Context dependent
Size vs. depth
High frequency vs. low frequency
Specialized / technical
Concrete vs. abstract
Grammatical class
Collocations

19. Vocabulary
A few common item formats to test vocabulary: Multiple choice Matching Sentence completion Fill-in-the-blank Short answer

20. Listening
Below are tasks that illustrate bottom-up processing: 1. Recognize word divisions 2. Recognize key words 3. Recognize key transitions in a discourse 4. Use knowledge of word-order patterns to identify discrete parts of spoken language 5. Recognize grammatical relations between elements in sentences 6. Recognize the function of word stress in a sentence 7. Recognize the function of intonation in a sentence

21. Listening
Below are tasks that illustrate top-down processing: 1. Ability to identify the gist or main idea of a discourse 2. Ability to identify the purpose of a discourse 3. Ability to deduce meaning of words from context 4. Ability to make textually-based inferences 5. Ability to distinguish between literal and figurative meanings 6. Ability to distinguish between facts and opinions

22. Listening passages
Features that impact the difficulty of listening are: Content Form Grammar Phonology Discourse Type Register Context

23. Listening item formats
Gap-fill
Note-taking
Multiple choice
Completing diagrams or tables
Multiple choice with pictures
Sorting information
Short answer questions
Spoken response
True / false / not stated
Written response
Matching

24. Speaking
Let’s quickly review a few common functions of speech: Asking for information Giving information Expressing a reaction Negotiating Problem solving Small talk Politeness Making excuses

25. Speaking
Here are a few tasks you can use to assess speaking: Description task Narrative task Instruction task Decision task Comparing and Contrasting task Explaining and Predicting task Structured Speaking task Role-play Interview

26. Speaking task
A speaking task should include: -Purpose of the task -Time limit -Approximate length of response -Directions for responding -How responses will be scored/graded

27. Scoring speaking Steps to follow when scoring speaking:
Choose the type of rubric
Find rubric or design rubric
Share rubric with students
Choose how many raters (single rater or double rater)
Train the rater(s)
Rate live speaking performance or rate delayed speaking performance (a recording)

28. Writing
Knowing how to write involves many subskills, including: Knowledge of writing (orthography, spelling, punctuation) Knowledge of phonology (sound/letter relationships) Vocabulary knowledge (multiple definitions for the same word) Syntactic knowledge (grammar, structure) Knowledge of written organizational patterns Knowledge of register (formal/informal)

29. Writing
Three ways to assess writing are: Indirect Direct Portfolio

30. Writing
We can identify some characteristics of writing tasks that will vary from task to task. These variations can make tasks easier or more difficult. These characteristics include: Topic Stimulus Genre Audience Time allowed Length

31. Writing
A writing task should include: -Purpose of the task -Time limit -Approximate length of response -Directions for responding -How responses will be scored/graded

32. Rating scales
Remember, rating scales (or rubrics) consist of the following three elements: Sub-Construct Score Scale definition / Scale descriptor

33. Holistic rubrics Strengths Weaknesses
Emphasizes what student does well
One score does not provide diagnostic information
Reduces risk of the rater focusing on one aspect of writing (e.g., grammar)
It is difficult to interpret a single score
Students may have uneven abilities across subskills
Reduced reliability

34. Analytic rubrics Strengths Weaknesses
Training raters is easier when there are different categories
Writing is more than the sum of its parts
Analytic rubrics keep raters from collapsing categories
Information from analytic rubrics may not be easy for students to interpret
There is no assurance that analytic rubrics will be used the way that they are intended.

35. Grammar
Grammar knowledge consists of several components, including: Phonological form (e.g., segmental forms, minimal pairs) Lexical form (e.g., spelling, word formation) Morphosyntactic form (e.g., affixes and suffixes, word order)

36. Grammar item formats
Grammar can be assessed using the following item types: Multiple choice Multiple choice error identification Discrimination task Noticing task Fill-in-the-blank Matching Short answer Role play tasks

37. Reliability
Inter-rater reliability: the amount of agreement between two raters Intra-rater reliability: how consistent a rater is with him or herself Test reliability: how consistent the measure is. This refers to the results we get from a test or measurement instrument, not to the measurement instrument itself.

38. Reading
Reading is an active process. It is the result of a combination of bottom-up and top-down processes. Bottom-up and top-down processing refers to the way that people internalize and understand information. This knowledge has implications for assessment. Reading comprehension tests often include questions to assess both bottom-up and top-down processes.

39. Future Directions

40. Future directions
We hope that you will use the information you have learned from this course to: Use your checklist to evaluate tests. Share what you have learned with other teachers through articles and presentations. Be an assessment resource for teachers and students at your school. Continue to use language testing resources on the internet to refresh and refine your knowledge of language assessment.

41. Next Steps
Now, you should do assignment #1. In this assignment, you will create a personalized language test checklist for your context.