1. For science courses, exams determine the majority of the student grades. Testing students on their comprehension and application of the course material is usually considered an appropriate way of assessing the students’ performance in class. When it comes to exams, there are various formats including short essay questions, matching, true or false, multiple-choice, etc. Different assessment formats can be applied depending on the material to be tested. For instance, anatomy exams can find matching questions useful in testing whether the students memorized each anatomical terms. However, when trying to assess students’ understanding or misconceptions of complex concepts, open-ended questions where students get to explain their thought process may provide more information. Extensive studies have been done on different exam formats in assessing student performances in class. The major two response formats utilized by instructors are open- ended and multiple-choice. Often, due to time limitations required in grading the open- ended responses, many classes, especially science courses, use multiple-choice formats. In this poster, results from different research studies on exam formats and their efficacies and consistencies are discussed. Introduction Research by Birenbaum examined the effect of response format-- open ended (OE) versus multiple-choice (MC) on the diagnosis of eighth grade students’ misconceptions in fraction addition arithmetic exams. Out of total 285 eighth grade students from Midwestern town in the U.S., 148 took the OE format while 137 took the MC format. The two data sets from these students’ responses were compared to examine differences in the number of different error types and diagnosed sources of misconception or “bugs” indicated in the response patterns. Research by Bennett tested the effect of constrained free response formats versus Multiple Choice and Open ended formats from 737 high school seniors. This group of high school students was split into two groups between open ended and multiple choice versions. This was a computer science class where students were forced to write programs to correct certain bugs. In the study for the open ended students were randomly given 1 or 3 bugs to correct in the programs code. The three bugs represented completeness, arrangement, and detail of novice programmers. The entire test was 35 multiple choice questions with 3 free response questions that would be graded by human. Another research by Bennett tested the relationship between multiple choice and free response questions on the 1988 Advanced Placement Computer Science exam given by the College Board. There were two groups of 1,000 randomly selected students used in their analysis of the 7,372 high school students taking the exam. The majority of students were seniors (69% in Sample 1 and 67% in Sample 2) while the remainder were mostly juniors. The exam was designed to assess the mastery of various topics covered throughout the computer science course. The format used was 50 multiple choice and 5 free response questions. To determine the relationship between the skills measured by the different formats, a two-factor model was used and random exam questions were analyzed. Methods Table 4 from Birenbaum’s study represents the cross-tabulation between the two categories “rationally interpretable” and “rationally uninterpretable” bugs or misconceptions and the two test formats (OE and MC). It shows that 84.5% of OE response patterns and 63.5% of the MC response patterns are classified as rationally interpretable bugs. This may indicate that OE response formats reveal more regarding the students’ misconceptions. Results Discussion Birenbaum’s study showed a higher percentage of OE response categorized into the “rationally interpretable” than the MC response. Lower percentage of OE responses were classified as “rationally uninterpretable,” meaning less of the students’ responses were random guesses but rather incorrect responses resulting from their misconceptions. This may be indicating that OE response formats are better at diagnosing sources of misconceptions underlying the response patterns. Also, it shows that OE formats can provide more information on the students’ thought process during examination. Bennett’s Study showed a lower mean score for free response than multiple choice. The difference in mean scores was not large enough to be significant however. The free response questions covered more topics and problem solving techniques than the multiple choice did which means that although the free response questions yielded a lower score, they also tested a wider range of knowledge on the students solution skills. Overall, it was concluded in Bennett’s study that the free response ended questions proved to be more beneficial in testing students because they were forced to practice a wider range of skills and techniques in order to be able to solve the problem unlike the multiple choice questions failed to do. Bennett’s other study came to the conclusion that there was not a great difference in the material tested by multiple choice and free response questions. There are a number of reasons why this could be the case. While the free response questions do test material on a deeper level, in it certainly plausible that 50 multiple choice questions could cover a similar depth of knowledge. The exam was for an AP class, with only a year to learn the material, so there is not enough time to allow for significant specialization in skill sets. Had the exam been taken by graduate students, there could have been a noticeable difference in the testing formats due to the greater specialized knowledge. Free response questions do offer one clear advantage in that they allow for the tracing of a student’s thought process. This can highlight the various ways to approach a question, or partial knowledge and misconceptions held by students. Even though these particular questions tested similar material, the free response questions can provide instructors with slightly more useful information. Conclusion Employing free-response questions for science exams may be time-consuming for instructors to grade. However, research studies illustrate that open-ended responses can cover more topics and provide more insightful information on the comprehension, thought process, and misconceptions students may have. Thus, it is recommended that instructors integrate free-response questions addressing more complex conceptions in the exam. References 1.Birenbaum, M., Tatsuoka, K. K. (1987). Open-ended versus multiple-choice response format—it does make a difference. Applied Psychological Measurement, 11, 385-395. 2. Bennett E., Randy, Rock A. Donald, et. al. (1990). The Relationship of Expert-System Scored Constrained Free-Response Items to Multiple-Choice and Open-Ended Items. Applied Psychological Measurement, 14, 151-162. 3. Bennett, Randy Elliot, Donald A. Rock, and Minhwei Wang. "Equivalence of Free-Response and Multiple-Choice Items." Journal of Educational Measurement 28.1 (1991): 77-92. Web. 03 Apr. 2015. Grace Lee, Luke Hess, Keith Peterson, Brandon Short SC 297C Learning Assistant Pedagogy, Pennsylvania State University, PA COMPARISON OF ASSESSMENT FORMATS IN SCIENCE EXAMS Table 4 from Bennet’s Study represents the mean scores of different multiple choice test versions and free response versions. Each multiple choice question was said to be parallel and similar in difficulty. This contrasts with the free response which have lower overall scores because the free response tested three different topic and had different ranges of difficulty for identifying computer bugs. Overall, the free response questions were said to have a bigger impact on learning because it provoked more thought and problem solving skills than the multiple choice questions did. Table 2 of this Bennett study shows the loading values of the examined multiple choice and free response questions. The loading represent how much that factor explains the variable, essentially the significance of the question. The multiple choice questions had larger loading values due to their parallel construction, the difficulties were similar. The free response questions each dealt with a different topic and were therefore more difficult. Overall however, it was determined that the two question types measured similar constructs.