Information Technology Design & Multimedia. A Comparison of an Innovative Web-based Assessment Tool Utilizing Confidence Measurement to the Traditional.

Slides:

Advertisements

Similar presentations

Evaluation Overview - Basics. Purpose of Testing Diagnostic Formative Summative.

Advertisements

Advancing Communication Competencies: Using Technology for Core Instruction and Resources Trudy Bayer Slippery Rock University of Pennsylvania Karen Curto.

The Journey – Improving Writing Through Formative Assessment Presented By: Sarah McManus, Section Chief, Testing Policy & Operations Phyllis Blue, Middle.

CONCEPTUAL WEB-BASED FRAMEWORK IN AN INTERACTIVE VIRTUAL ENVIRONMENT FOR DISTANCE LEARNING Amal Oraifige, Graham Oakes, Anthony Felton, David Heesom, Kevin.

Computer Aided Assessment using QuestionMark Perception by Catherine Ogilvie Educational Development Officer.

An Introduction to Computer- assisted Assessment Joanna Bull and Ian Hesketh CAA Centre Teaching and Learning Directorate.

How to Take Tests 5 How to Take a Multiple Choice Test

Learning and Technology Services Online Assessments at Sauder Not your parents’ multiple choice exams...

Assessment & Evaluation adapted from a presentation by Som Mony

CCE GUIDELINES FOR CLASSES 6 TO 8

ISTE Standards for Teachers

(IN)FORMATIVE ASSESSMENT August Are You… ASSESSMENT SAVVY? Skilled in gathering accurate information about students learning? Using it effectively.

Emergent Technology GRIT 685. What is it?  A set of handheld devices  “Clickers”  Allow all students to answer an instructor’s questions  Allows the.

Assignment Marking via Online Self Assess Margot Schuhmacher, Lecturer Higher Education Development Unit, Centre for Learning and Teaching Support, Monash.

Student Assessment CERRA National Board Candidate Support Workshop Toolkit WS

Instructional Plan and Presentation Bernard Q Mallada CUR/516 Dr

TABLE 1: Comparison of scores from the year the immediate feedback format was used to the previous year when multi-format exams were used. * p < 0.05 compared.

© AJC /18 Extended Matching Sets Questions for Numeracy Assessments: A Case Study Alan J. Cann Department of Microbiology & Immunology University.

Miguel Martins, Senior Lecturer – Marketing Department eLearning Celebration 22 July 2007 Improving performance on a Marketing module through the use of.

INACOL National Standards for Quality Online Teaching, Version 2.

Using Assessment Within Virtual Learning Environments Colin milligan.

Overview of Tests in Blackboard. Benefits of Blackboard Testing.

ASSESSMENT& EVALUATION Assessment is an integral part of teaching. Observation is your key assessment tool in the primary and junior grades.

Developing Evaluation Instruments

MASTERS THESIS DEFENSE QBANK A Web-Based Dynamic Problem Authoring Tool BY ANN PAUL ADVISOR: PROFESSOR CLIFF SHAFFER JUNE 2013 Computer Science Department.

ASSESSMENT Formative, Summative, and Performance-Based

Introducing small-group workshops as formative assessment in large first year psychology modules Suzanne Guerin School of Psychology, University College.

Assessment Tools. Contents Overview Objectives What makes for good assessment? Assessment methods/Tools Conclusions.

Implication of Gender and Perception of Self- Competence on Educational Aspiration among Graduates in Taiwan Wan-Chen Hsu and Chia- Hsun Chiang Presenter.

E-Learning Services. e-Learning is transforming the way we learn and teach e-Learning can be broadly defined as technology assisted learning. It is all.

Assessment 101 Center for Analytics, Research and Data (CARD) United Church of Christ.

Software Evaluation Criteria Automated Assignment Applications RSCoyner 10/8/04.

Chapter 5 Building Assessment into Instruction Misti Foster

Student Assessment Workshop #5 CERRA National Board Candidate Support Workshop Toolkit WS

Context & Problem “Biotechnology Laboratory Skills” class preparing students for an internship in life science research (lecture/lab – 4h total) Bact.

Using Learning Technology to Design and Administer Student Assessments Catherine Kane Centre For Learning Technology, CAPSL Ref:

Measuring What Matters: Technology & the Assessment of all Students Jim Pellegrino.

1 The Design of Multimedia Assessment Objects Gavin Sim, Stephanie Strong and Phil Holifield.

Evelyn Wassel, Ed.D. Summer  Skilled in gathering accurate information about students learning?  Using it effectively to promote further learning?

Assessment Tools.

Assessment and Testing

Learning through Practice S.Sathiakumar School of Electrical and Information Engineering University of Sydney Problem solving should be a major part of.

Learning Assessment Loop Closing Activity in the Biology Department Vivian Navas Biology Department Assessment Coordinator May 2006.

Class Meeting 4: Putting it All Together!

“The End Justifies the Means” When and how to use Computerised Assessment to Promote Learning Phil Davies School of Computing.

Nicola Billam, Guinevere Glasfurd-Brown & Martin Sellens CAA Conference 2006 Engaging Students in Formative Assessment: Strategies and Outcomes.

Assessing Learners The Teaching Center Department of Pediatrics UNC School of Medicine The Teaching Center.

CREATING AN ACTIVE LEARNING ENVIRONMENT Using Inquiry and Primary Sources.

The Use of Formative Evaluations in the Online Course Setting JENNIFER PETERSON, MS, RHIA, CTR DEPARTMENT OF HEALTH SCIENCES.

Adventures in CPS Aaron Bruck Towns Group Meeting September 25, 2007.

Improving Results with Positive Directed Feedback in Summative Assessments Wayne Ellis

Self-assessing with adaptive exercises Chye-Foong Yong & Colin A Higgins University of Nottingham.

E-Assessment: Removing the Boundaries of C.A.A. Phil Davies School of Computing University of Glamorgan South Wales UK.

Presentation by Shahla Peterman. Assumptions About Testing Multiple Choice Format Hand Graded Format.

SCIENCE Assessment Amanda Cantafio.

Approaches to learning and self study materials in quantitative subjects Gary Simpson ABS270.

Assessment and the Institutional Environment Context Institutiona l Mission vision and values Intended learning and Educational Experiences Impact Educational.

The Exam. Three Hours I. 100 multiple choice questions ( 90 min.) II. 4 Free Response Questions ( 90 min.) Breadth and knowledge of content (60%) Critical-thinking.

SMARTER BALANCED ASSESSMENT PARA LOS NIÑOS APRIL 30, 2013 Transitioning to the Common Core.

INTE 290 Summer 2015.

Competencies and consequences … choices to make April

Collaborative Learning Community: Student Reflective Assessment: Part 1 Team Orange Tarica Miller Christina Mullikin,Michael Paekukui.

Assessment and Feedback using IT Types of assessment Formative Summative Norm-referenced Criterion-referenced.

BCPS One Tests and Quizzes

Online Driver Education and Virtual Classroom

Multi-faceted Approach to Improve Learning in Pharmacokinetics

Multi-faceted Approach to Improve Learning in Pharmacokinetics

Revised Higher Course Event

Exploring Assessment Options NC Teaching Standard 4

Online Driver Education and Virtual Classroom

Presentation transcript:

Information Technology Design & Multimedia

A Comparison of an Innovative Web-based Assessment Tool Utilizing Confidence Measurement to the Traditional Multiple Choice, Short Answer and Problem Solving Questions: Graham Farrell: Swinburne University

Content: MCQCM’s Demonstration. Basis of Development. Benefits of MCQCM. Previous Work Design Issues Comparative Analysis Conclusion and Discussion

Multiple Choice Questions with Confidence Measurement Can be accessed via the internet with student login Requires students registration of confidence for each answer, after commitment to an answer Ability to capture Data and produce feedback to both the instructor and the student immediately Offers graphic capabilities. Lets have a look!

Is there a need for it? Formative Assessment, integrated, not a disturbance to learning. Feedback, immediate and comprehensive Control of Learning into the hands of the students Recognition of Partial Learning: Medical Field (Davidoff 1995) Asking the Meta-question: “How sure are you of your knowledge?” (Diamond and Forrestor 1973) Supports Competency Based Learning

MCQCM Benefits MCQCM for the Students encourages : Honesty! Modification of their learning path (self-regulation) Reflection

MCQCM Benefits The MCQCM benefits the Instructors by supplying feedback that encourages: Honesty! Evaluation of teaching effectiveness. Modification or the learning path when required. Diagnostics at group and individual levels when most needed

Previous Work Scoring Systems that eliminate guessing by identifying correct and incorrect options (Pollard 1989,1985, 1983) Interactive Systems designed to permit the registration of confidence. (Paul 1997, Klinger 1997,Gardner-Medwin, Gahan 2003) Scoring Systems that disproportionably penalized for incorrect answers to encourage honesty

Design Issues Design MCQCM to include the following Fully Web based utilizing Server side technology (HTML,PHP graphics) Permit the students to commit initially as True or False before registering their confidence. (Self- efficacy Bandura 1977) Produce enriched feedback with direction for further study Identify and investigate further applications for the tool

Comparative Analysis Cohort of 43 students( Database Design) All students access to BB revision questions and MCQCM Exam structure: 8 MCQ(20% marks) 8 MCQCM(20% marks) 8 Short Answers(33% marks) 2 Problem Solving(27% marks)

Comparative Analysis SectionAverage GradeStandard Deviation MCQ73%17.7% MCQCM67%21.0% SA85%9.8% Problem Solving75%14.5%

Comparative Analysis Section Comparison Correlation Factor R Defined MCQ vs MCQCM.436Medium MCQ vs SA.447Medium MCQ vs PS.235Small MCQCM vs SA.554Strong MCQCM vs PS.302Medium SA vs PS.442Medium

Conclusion and Discussion Directly concerning MCQCM: Strongest correlation between MCQCM and to Short Answers (.544) Medium correlation of MCQCM to PS (.302) -compared to the small MCQ to PS (.235) Medium correlation for MCQCM to the traditional MCQ (.436)

Conclusion and Discussion Is MCQCM a valid summative assessment option? MCQCM appears to be “as good as” the others -valid assessment option MCQCM does not appear to offer any great advantage -So why bother? MCQCM as a formative tool gives rich feedback to direct the learning of the individual student and assist the instructor.

Further Research The following will be investigated as part of the ongoing research: Gauge the students perception of the MCQCM tool for summative assessment Utilize MCQCM in other areas Star my write up!