Oleh: Beni Setiawan, Wahyu Budi Sabtiawan Fostering a Student’s Skill for Analyzing Test Items Through an Authentic Task Oleh: Beni Setiawan, Wahyu Budi Sabtiawan Science Education Department, Faculty of Mathematics and Natural Science Universitas Negeri Surabaya, Indonesia

INTRODUCTION Expectations: Educators: ability to collect, summarizing, and applying information from the students’ answers to make a judgment in each assessment (Nitko). Test items analysis is an essential skill to be mastered by teachers. They involve: (1) Is the function of test items appropriate? (2) Is the level of difficulty appropriate? (3) are there irrelevance contents in the test items? (4) Is the answer choices effective? (Linn and Gronlund)

what they need in professional situation INTRODUCTION Facts: Students tend to focus on developing teaching materials and implementing learning models in undergraduate level. The lecturers have less information in terms of prospective teachers’ skills for analyzing test items involving validity, reliability, item discrimination index, level of difficulty, and distractor functioning . The students got lack experiences to analysis test items. There was a gap between what the students learn and what they need in professional situation

INTRODUCTION Solution: Fostering a Student’s Skill for Analyzing Test Items Through an Authentic Task The task of particular relevance to the professional task in the real world, so that it was able to stimulate the students to develop their competencies (Duffy and Jonassen; Gulikers et al.; Shrum and Glisan; Woo et al.) the environmental influence on behavior and individual learners (Keller).

EXPERIMENTAL Research Design dan Procedures: Research design: One-group Posttest Design Participant: the students enrolled in assessment class of Science Department, Faculty of Mathematics and Natural Science, Universitas Negeri Surabaya. Student’s task: “The students are required to do analysis of test items consists of 10 multiple choice and 5 essays questions for each selected basic competence. Moreover, they had to consult to the experts for reviewing and validating the test. Furthermore, they revised the test items based on the suggestions from the experts. The next step was that the students analysis the quality of the test using Excel and Anates Software. Each item was analysed involving validity, reliability, item discrimination index, level of difficulty, and distractor functioning. The last stage was displaying the results of test analysis in the form of project reports.”

Authentic Task Characteristics Manifestations in Learning EXPERIMENTAL (Con’t) Research Design dan Procedures: Authentic Task Characteristics Manifestations in Learning Authentic tasks have real-world relevance. The task required the students to do the work of a teacher, assessor, and researcher in education, that were to create questions, then analysis each item. It was conducted to evaluate the theoretical validity of the constructed test. Authentic tasks are ill-defined. They were provided only limited information without given detail instructions, such as student’s worksheet. Therefore, they had an opportunity to determine how to resolve or design solutions for their tasks independently. Authentic tasks need over a sustained period of time. The task had to be completed within two weeks (not in the class immediately) because it was quite complex.

Authentic Task Characteristics Manifestations in Learning EXPERIMENTAL (Con’t) Research Design dan Procedures: Authentic Task Characteristics Manifestations in Learning Authentic tasks provide the opportunity for students to examine the task from different perspectives. They were given the opportunity to explore learning resources from books, journal articles, and websites. Through the demand, the students gained diverse perspectives in solving or designing a solution for the task. Authentic tasks provide the opportunity to collaborate. The students were required to go to school for collaborating with the teachers and doing the analysis of test items together. Authentic tasks provide the opportunity to reflect. The students were required to show the performances as a teacher, assessor, and researcher in educational area. In addition, they each reflected one another in terms of what they have acquired during the process of finding a solution for the task.

Authentic Task Characteristics Manifestations in Learning EXPERIMENTAL (Con’t) Research Design dan Procedures: Authentic Task Characteristics Manifestations in Learning Authentic tasks can be integrated and applied across different subject areas and lead beyond domain-specific outcomes. The task encouraged the students to integrate the understanding of assessment, writing skills, and communication skills. Authentic tasks are integrated with the assessment. The task will be assessed using a rubric for the project report. Authentic tasks create a holistic product. The student’s task to be completed included the analysis of test items and the creation of project report. Authentic tasks allow competing solutions and diversity of outcome. The students were given the opportunity to explore learning resources from books, journal articles, and websites. In addition, they were allowed to find out several solutions for the completion of their duties.

EXPERIMENTAL Data Analysis Technique: The results obtained were analyzed descriptively: 1. displaying the student’s skill in terms of analysing test items. 2. associating with theories or studies from experts. 3. comparing with previous relevant research.

RESULTS AND DISCUSSION Both of assessment item score of data processing using Excel and Anates Software were in a maximum score of 12 and 18 respectively. In other words, the students achieved the best score for analysing test items using the two software.

RESULTS AND DISCUSSION The amount of 36 students got the maximum score of 18 and 4 students got a score of 12 for describing the results skill. The student's ability to conclude the analysis results was categorized in high, middle, and low level. There were 35 students getting a maximum score of 10, 6 students getting a middle score of 8, and 1 student was in a score of 0 as the lowest mark.

RESULTS AND DISCUSSION Assessing the student’s skill was determined using six assessment parameters, namely the tested sample, type and the number of question, data processing using Ms. Excel, data processing using Anates Software, description of the result, and conclusion. It can be seen that the thirty-one students got a perfect score for the analyzing, five students achieved 97% mastery, two students had 92% mastery, and another two students got 89% and 79% of mastery.

Factors affecting student’s skills RESULTS AND DISCUSSION the task of particular relevance to the real world or in accordance with what a professional do in the world of work. Factors affecting student’s skills relevance to the real world social interaction ill-­defined the students were required to construct a solution design for the task completion by themselves rather than following the steps like a recipe. One of the authentic task characters is to give students an opportunity to discuss or interact with others.

RESULTS AND DISCUSSION Rationale: First Factor The task of particular relevance to the professional task in the real world, so that it was able to stimulate the students to develop their competencies. Authentic tasks provided experience to students and to motivate them. Second Factor The discussion (language expressed) provided information on what students already know and to form new knowledge so that it was able to encourage cognitive development. The social interactions improved the achievement of learners through three different ways, that were improving attention, mutual help, and responsibility. Second Factor The transfer of knowledge was very difficult to do by teachers. Giving the task unstructured or requiring students to create their own designs are able to stimulate the improvement of metacognitive skills. Mastery of metacognitive skills has a positive impact directly on the process and outcomes of the student’s learning

CONCLUSION The students were able to analyse test items well using Microsoft Excel and Anates Software represented by the maximum score obtained the students. The skills of students to describe and conclude the results of the analysis involving validity, reliability, item discrimination index, level of difficulty, and distractor functioning were in the maximum score. The students' scores on the whole skills were that thirty-one students got a perfect score for the analyzing, five students achieved 97% mastery, two students had 92% mastery, and another two students got 89% and 79% of mastery.

ACKNOWLEDGMENT “Thanks to the Assessment Tim of Science Department, Faculty of Mathematics and Science, Universitas Negeri Surabaya (Unesa), due to the cooperation during the implementation of the research.”

THANK YOU