Chapter 1: Testing, Assessing and Teaching Language assessment: Principles and classroom practices H. Douglas Brown ESOL Tests & Measurements (ENGL 4-528) Spring 2011

TEST- “To fear or not to fear?” Feelings evoked by tests are generally negative Goal is to help create authentic, intrinsically motivating, confidence building assessments that are learning experiences

What is a test? Definition with 5 main parts: “A method of measuring a person’s ability, knowledge, or performance in a given domain.” (Brown, p.3) Purpose: To accurately measure the ability of the test-taker in a certain domain. Test ≠ Assessment. Tests are a subset of Assessment. Teachers should constantly observe and assess students without the pressure of taking a test. Assessment & evaluation are used to drive instruction.

Types of Assessment Informal ex) coaching, unplanned comments Formal ex) planned exercises to show achievement, journals, portfolios, tests Formative ex) ongoing evaluation with feedback for future learning Summative ex) unit, final, proficiency exams, “summary” to assess what was learned

Types of Tests Norm-Referenced ex) SAT, TOEFL, numerical score & percentile rank Criterion-Referenced ex) classroom tests to assess objectives, teacher provides feedback Discrete-Point ex) skills & units of language broken down, vocabulary or grammar tests Integrative ex) cloze test & dictation, shows competency Unitary trait hypothesis→ said parts of language can’t be separated– later proved wrong. This means reading, listening, grammar, vocabulary, etc. can be tested separately.

Issue #1- How to test performance ability Problems with tests in the 1990’s: Artificial tasks that don’t use language relevant to real-life situations Attempted solutions: Communicative Language Tests- use of strategic competence Performance-Based Assessment- interactive tasks

Communicative Language Tests Relates to foreign language proficiency test in New York State (Is there anything similar in Maryland or a state where any of you are from?) Taken in 8th grade or after 1 year of foreign language study Test: Speaking, Listening, Reading, Writing Speaking *Performed 1:1 with teacher. *4 Tasks are picked randomly and pertain to real-life situations. *Student makes 4 dialogue exchanges with 2 attempts for each exchange. *Opportunity to utilize strategic competence. *Up to 10 points awarded for in-class participation throughout entire year (→pertains to alternative assessment).

Communicative Language Tests continued Listening/Dictation *Teacher reads passage aloud in target language *Students answers multiple choice based on what they heard. 10 questions & answers are written in English, 5 in target language, 5 with questions in English and choices are pictures. Reading *Multiple choice questions based on a reading passage (ads, signs, letters, passages from internet pages). Half of questions in English, half in target language. Writing *3 choices, students choose 2 to write at least 30 words for each Link to previous exams: http://www.nysedregents.org/loteslp/spanish/home.html

Pros and Cons to Proficiency exam Pros: Good attempt at communicative language test and performance-based assessment, alternative assessment included, used as summative assessment Cons: Time consuming for teacher (individual speaking tasks with all students, dictation must be read aloud- if students have accommodations then it must be read more times), expensive to create & grade (teachers, i.e. real people, grade the writing portions). *NYS cut proficiency exams from budget this school year. I feel this shows the government’s lack of importance for languages, especially Spanish because it is the 2nd language of the U.S. We need to teach & learn more languages to become better citizens of the world, not just citizens of the U.S.

Proficiency connection with ESL Assessment Even though the NYS proficiency exams are for foreign languages (other than English), similarities exist with ESL assessments: → Need for authentic assessment → Want to demonstrate use of real-life language → Should assess reading, writing, listening, speaking → Time consuming → Expensive → Must be carefully designed

Traditional & Alternative Assessment Both forms are very useful and one is not better than the other!! Teachers must find the right balance of each type depending on the students. Examples (from Table 1.1 on page 13) Traditional Alternative -standardized (SAT, TOEFL) -long-term (portfolio) -multiple choice, timed -free-response, untimed (journal) -right vs. wrong answers -open-ended, creative answers -summative (end) -formative (ongoing) -focus on product (outcome) -focus on process (how it’s done) -individual, non-interactive -interactive with others -produces extrinsic motivation -produces intrinsic motivation

Computer-Based Testing Personally, I dislike taking a test on a computer. Do you? Why or why not? I am 23 years old and grew up going to school as computers and technology use in school was emerging. Before graduating from high school, I may have taken a handful of tests on the computer. For example, a career aptitude test. No computer-based tests during college. The last computer-based test I took was the GRE about 6 months ago.

Why I dislike Computer-Based Tests Unfamiliar test format- I am used to paper and pencil tests where I can underline, mark, write on the actual exam. Cannot look at entire exam to predict difficulty, length, or skip questions and return later (I believe the tests in this online course allow you to see the questions, skip them and return later. That’s a plus for me!) Receive a numerical score- I prefer to see what I got wrong, find the right answer, or read comments about my answers. I want to learn from my tests! You are often alone in the computer testing room, or if there are other people, they probably aren’t taking the same test as you.

Advantages of Computer-Based Tests Technology-based- People who love computers may be more interested in computer-based tests than written tests Individual privacy- not with hundreds of people in testing room, computers often have dividers CAT- answers from previous question determines next question- this helps narrow in on an accurate score Self-directed- You study and prepare how you wish and can often repeat the tests numerous times (but with a fee or paying the full amount again in the case of the TOEFL, GRE). Mostly right/wrong answers- good for people who prefer this testing method Easily administered- to thousands of people in different places and scores are done electronically

Application of computer-based tests for ESL classrooms Specific or short teacher created tests can be especially useful for ELLs to help practice: Vocabulary Listening Speaking & pronunciation Reading comprehension Useful assessment tool for teachers to find out via computer results what students know, have learned and still need to practice or learn= Drive instruction!

Overall Chapter Reaction Excellent overview of different types of assessments and tests Author explains the difference between assessment and test well. Extremely important: TEACHERS MUST ASSESS STUDENTS AND USE ASSESSMENTS TO DRIVE INSTRUCTION!

Material on assessment applies to all teachers and not simply ESL teachers. I benefited from the reading because, as an experienced teacher, it was a constructive review of ways I have assessed previous students and ways that I know I will use in the future with ELLs. Leaves me wanting more specific assessment materials which I know will come. The 3 minute vocabulary quiz on pages 1-2 was a strong reminder of feelings test-takers experience- **Teachers must always keep this in mind when creating and administering assessments.**

Application of “Testing, Assessing and Teaching” Affects on my thinking: There are so many ways to assess and with each student we have, certain assessments will work better than others. Must stay open-minded and be flexible in the type and use of assessments. Cannot rely on one form of assessment. Continues on next page…

Cont. Affects on my Thinking Personal dislike of particular assessments pushes me to consider my students’ background knowledge, interests, strengths and weaknesses when creating assessments. Also encourages me to critique my own assessments as well as those commonly used (ex: TOEFL) One goal I have is to help my students use assessments as motivational tools to want to learn more and improve their English (not the other way around).

“Ready to use” Assessment “Creating and Performing a Dialogue” Assignment might look like this: 1. You and a partner will be creating and performing a dialogue in English. 2. In your dialogue you must: Introduce yourselves, and ask about the other person’s school day. Ask what classes the other person takes. Ask what the other person does in one or more classes. Ask any other relevant questions. 3. Each partner must speak 5 times (or more) and each time you speak, it needs to be more than 3 words. Each person will write down the dialogue and turn it in after they have performed. You are allowed to use your sheet when you perform. (Teacher will determine if students should perform with or without memorizing the dialogue as to whether students can read from their sheet or not) *Note: #2(content) is changed based on topic being studied/evaluated

What Students Need: 1. Dialogue outline- Teacher can provide an outline for students to fill in or students can write dialogue on blank sheet of paper 2. Writing & Rehearsing Time: provide students with adequate time to write the dialogue, practice it for correct pronunciation with help from teacher if needed, and time to memorize it (if required) 3. Class notes, texts, dictionary, any materials they wish to use as a reference to aid in writing the dialogue. This means spelling and punctuation is expected to be accurate or students need to pay better attention while writing.

What Teacher Needs: Assessment Criteria Grading Rubric See next slide for an example of a rubric on a 1-4 scale.

Conversation flows logically and everything makes sense. 1 2 3 4 Content (What is said) Conversation does not make sense. There is not a logical flow to the dialogue. Conversation makes little sense; some interchanges flow logically and are somewhat connected. Conversation makes sense; there may be 1 or 2 times when it does not flow logically. Conversation flows logically and everything makes sense. Pronunciation Audience cannot understand speaker at all or speaker is too quiet to be heard. Audience can only understand a few words or phrases. Speaker needs to talk louder or pronounce words more clearly. Audience can understand and hear the speaker most of the time. Speaker talks clearly, pronouncing most words accurately. Audience can understand and hear the speaker very clearly. Speaker pronounces almost all words accurately. Written Dialogue Student does not turn in a written dialogue or it is too sloppy to read. Dialogue is not written clearly and difficult to read. There are more than 10 spelling errors and more than 10 punctuation errors. Dialogue is written fairly clearly. There are between 5 and 10 spelling errors and/or between 5 and 10 punctuation errors. Dialogue is written clearly with 4 or less spelling errors and/or 4 or less punctuation errors.

Online Resource-Check it out! brainpop.com brainpopjr.com (grades K-3) brainpopesl.com -From homepage of brainpop.com, you can click on BrainPop ESL in the bottom right hand column, or simply go to brainpopesl.com

BrainPOP- What is it? BrainPOP can be used with students in any grade, even through high school. Mostly all subjects are available, except foreign languages (other than English). However, they do have BrainPOP Español (in Spanish) ESL BrainPOP available too!

What’s Available: There are lessons for specific topics which usually begin with a short, concise video that students watch. Then there are various activities that can be completed that provide immediate feedback to assess what students learned from the video. Word lists, quizzes, tests, and games are available that also provide feedback. Excellent learning and assessment tool! There a some free lessons that you can try out from the brainpopesl.com, otherwise you need a subscription to utilize all the resources available. Teacher resources and tools available that provide lesson plans, standards, objectives, printable worksheets, time frame for each lesson, ect. The previous school where I taught had a subscription, so all teachers could access the entire website and students really enjoyed doing BrainPOP! NOW….GO CHECK OUT THE WEBSITE! IT’S COOL!

Conclusion Assessment is essential in every classroom. The more you know, the better prepared you will be to create and utilize valid, authentic assessments that will successfully measure your objectives. There is a plethora of resources in print, online, and from other educators that are out there waiting to be used. Don’t be afraid to go find what’s out there and make it work in your classroom! Tests- “To fear or not to fear?” You shouldn’t fear them and we need to teach our students how to not fear them, but help them understand the purpose of tests and assessments as well as how these measurements can help them become more successful learners, not only in school, but throughout their entire lives. 