1. Measuring Complex Achievement: Essay Questions
Chapter 10
Jessica Rigsby

2. Objectives for Chapter 10
Discuss the use of essay questions
Compare two types of essay questions
Hypothesize the advantages and disadvantages of using essay questions
Compare two types of scoring rubrics
Evaluate sample essays using a scoring rubric
Create examples of two types of essay questions

3. What’s So Great About Essay Questions?
They allow assessment of students’ abilities to:
Formulate problems
Organize, integrate, and evaluate ideas and information
Apply knowledge and skills

4. Forms and Uses of Essay Questions
May be used to measure knowledge of factual information
Full potential realized with more higher-order thinking problems

5. Useful in meeting learning outcomes concerned with the abilities to conceptualize, construct, organize, integrate, relate, and evaluate ideas in content areas such as: history, civics, literature, science, math, etc.
Most widely used when the main focus is on student writing in any subject
Examples: distinguishing between narrative essays, expository essays, and persuasive essays or focusing on writing process

6. Confucius Say…
The freedom of response provided by essay questions is not an all-or-nothing affair but, rather, a matter of degrees.
Restricted Response
Extended- Response

7. Restricted Response Essay Questions
Usually limits both the content and the response by restricting the scope of the topic to be discussed, generally indicated in the question
Useful for measuring learning outcomes requiring interpretation and application of data in a specific area
Any outcomes measured by an objective interpretive exercise can be measured by a restricted response essay question

8. Examples of Restricted Response Essay Questions
Describe two situations that demonstrate the application of the law of supply and demand. Do not use those examples discussed in class.
State the main differences between the Vietnam War and previous wars in which the United States has participated.
Why is the barometer one of the most useful instruments for forecasting weather? Answer in a brief paragraph.

9. Is there a better method for evaluating complex achievement…
The Restricted Response essay question provides for more ease of assessment, but it restricts the scope of the topic to be discussed and indicates the nature of the desired response to the student which limits his or her opportunity to demonstrate these behaviors on their own.
Is there a better method for evaluating complex achievement…

10. Extended-Response Essay Questions
Freedom of response allows student to select information that they think is pertinent, to organize the answer in accordance with their best judgment, and to integrate and evaluate ideas as they deem appropriate
Places value on higher-order thinking skills

11. Examples of Extended-Response Essay Questions
Imagine that you and a friend found a magic wand. Write a story about an adventure that you and your friend had with the magic wand.
Compare developments in international relations in the administrations of President William Clinton and President George W. Bush. Cite examples when possible.

12. To Keep Things Simple… Objective Interpretive- select
Restricted Response Essay- supply
Extended-Response Essay- write

13. Pros & Cons of Essay Questions
Apply your knowledge
They give student the options to chose what they want to write about
reasoning
The teacher can tell if the student studies or not
Can often prepare in advance what to write
Time consuming
Grading: grammar usage vs. content
Legibility
Hard to formulate ideas in time
Takes longer to grade

14. How Can We Change Those Cons to Pros?
Unreliability- clearly defining learning outcomes to be measured, properly framing questions, carefully following scoring rules, and obtaining practice in scoring
Amount of time- reserve use of extended-response questions for learning outcomes that cannot be measured well objectively

15. Limited Sampling- try to obtain as representative a sample of learning outcomes as possible (accumulate a series of essays throughout the school year to be included in a writing portfolio)

16. Suggestions for Constructing Essay Questions
Restrict use of essays to learning outcomes that cannot be measured well objectively
Construct questions that call forth skills specified in learning standards
Example box on page 235

17. Phrase the question so that the students’ task is clearly indicated
Example:
Poor: Compare the Democratic and Republican parties.

18. Better: Compare the current policies of the Democratic and Republican parties with regard to the role of government in private business. Support your statements with examples when possible. (Your answer should be confined to two pages. It will be evaluated in terms of the appropriateness of the facts and examples presented and the skill with which it is organized.)

19. Now that we have essay questions how do we score them?
Indicate an approximate time limit for each question
Avoid the use of optional questions
Example Box page 239
Now that we have essay questions how do we score them?

20. Scoring Essay Questions
Tips to remember…
Use clear specifications of scoring criteria
Inform students of scoring criteria
Use an initial review to find “anchor” responses for comparison
Use descriptive rather than judgmental scores or levels (“writing is clear and thoughts are complete” vs. “excellent”)

21. Scoring for Restricted Response Essay Questions
In most instances, the teacher should write an example of an expected response
For example, if the student is asked to describe three factors that contributed to the start of the Civil War, the teacher would construct a list of acceptable reasons and give the student 1 point for each of up to three reasons given from the list

22. Scoring for Extended-Response Essay Questions
Analytic Scoring Rubrics
Consist of a rubric broken down into key dimensions that will be evaluated
Enables teacher to focus on one characteristic of a response at a time
Provides maximum feedback for students

23. Holistic Scoring Rubrics
Yield a single overall score taking into account the entire response
Can be used to grade essays more quickly
Does not provide as much specific feedback as analytic rubric
Should not consist of scores alone, but rather contain scores accompanied by statements of the characteristics of the response
Example Table 10.3 and 10.4

24. Suggestions for Scoring Essay Questions
Prepare an outline of the expected answer in advance and use a clear scoring rubric
Use the scoring rubric that is most appropriate
Decide how to handle factors that are irrelevant to the learning outcomes being measured

25. Evaluate all responses to one question before going on to the next one
When possible, evaluate answers without looking at the student’s name
If especially important decisions are to be based on the results, obtain two or more independent ratings
Look out for bluffing! Page 247