Should I Decide or Let the Machine Decide for Me? Explorations in the Use, Misuse and Disuse of Automation The Soldier Study Prancers Skip, Mary, Linda,

Slides:

Advertisements

Similar presentations

Research Methodology Chapter 1.

Advertisements

Chapter 10: The t Test For Two Independent Samples

Chapter 7 Hypothesis Testing

Chapter Ten Making Decisions. Chapter Ten Making Decisions.

During this workshop you will learn why using competency behavioral based questions is best practice. You will learn how to write and use such questions.

Automation Usage Decisions: Controlling Intent and Appraisal Erros in a Target Detection Task Hall P. Beck Appalachian State University Mary T. Dzindolet.

The Use And Misuse Of Machines That Simulate Human Cognition Hall Beck, PhD Office: 215 Smith-Wright Hall.

Quasi-Continuous Decision States in the Leaky Competing Accumulator Model Jay McClelland Stanford University With Joel Lachter, Greg Corrado, and Jim Johnston.

Realism in Assessment of Effort Estimation Uncertainty: It Matters How You Ask By Magne Jorgensen IEEE Transactions on Software Engineering Vol. 30, No.

© 2010 Pearson Prentice Hall. All rights reserved The Chi-Square Goodness-of-Fit Test.

8 Thinking Critically, Making Decisions, Solving Problems.

8. Statistical tests 8.1 Hypotheses K. Desch – Statistical methods of data analysis SS10 Frequent problem: Decision making based on statistical information.

MISUNDERSTOOD AND MISUSED

INTERPRET MARKETING INFORMATION TO TEST HYPOTHESES AND/OR TO RESOLVE ISSUES. INDICATOR 3.05.

The Problem of Transposition Behavioral Versus Cognitive Views of Learning Key Issue in Learning Theory: What is Learned? Behavioral View: An association.

ICS 463, Intro to Human Computer Interaction Design: 9. Experiments Dan Suthers.

11 Populations and Samples.

Lecture 14-2 Multinomial logit (Maddala Ch 12.2)

Sampling Designs and Techniques

Slide 1 Detecting Outliers Outliers are cases that have an atypical score either for a single variable (univariate outliers) or for a combination of variables.

* Problem solving: active efforts to discover what must be done to achieve a goal that is not readily attainable.

Copyright © 2007 Pearson Education Canada 1 Chapter 12: Audit Sampling Concepts.

Statistical hypothesis testing – Inferential statistics I.

Fundamentals of Corporate Finance, 2/e ROBERT PARRINO, PH.D. DAVID S. KIDWELL, PH.D. THOMAS W. BATES, PH.D.

McGraw-Hill/IrwinCopyright © 2009 by The McGraw-Hill Companies, Inc. All Rights Reserved. Chapter 9 Hypothesis Testing.

Effects of Warning Validity and Proximity on Responses to Warnings Joachim Meyer, Israel HUMAN FACTORS, Vol. 43, No. 4 (2001)

Overview of Statistical Hypothesis Testing: The z-Test

1 Dr. Jerrell T. Stracener EMIS 7370 STAT 5340 Probability and Statistics for Scientists and Engineers Department of Engineering Management, Information.

Jump to first page HYPOTHESIS TESTING The use of sample data to make a decision either to accept or to reject a statement about a parameter value or about.

Comparing Means From Two Sets of Data

Testing and Cost / Benefit Tor Stålhane. Why cost / benefit – 1 For most “real” software systems, the number of possible inputs is large. Thus, we can.

Programming Logic and Design Fourth Edition, Introductory

1 1 Slide © 2008 Thomson South-Western. All Rights Reserved Slides by JOHN LOUCKS St. Edward’s University.

Chapter 8 Introduction to Hypothesis Testing

Community Health Assessment: Primary Data Collection LHD TA Project – Learning Collaborative 1 Community Health Assessment Second Learning Session Sheena.

Experimental Evaluation of Learning Algorithms Part 1.

Audit Sampling: An Overview and Application to Tests of Controls

Exploration Strategies for Learned Probabilities in Smart Terrain Dr. John R. Sullins Youngstown State University.

Chapter 14: Using the Scalable Decision Process on Large Projects The process outlined is meant to be scaleable. Individual steps can be removed, changed,

1 MARKETING RESEARCH Week 5 Session A IBMS Term 2,

CHECKERS: TD(Λ) LEARNING APPLIED FOR DETERMINISTIC GAME Presented By: Presented To: Amna Khan Mis Saleha Raza.

Lecture 16 Section 8.1 Objectives: Testing Statistical Hypotheses − Stating hypotheses statements − Type I and II errors − Conducting a hypothesis test.

Reliance on Decision Aids Results from 15 Studies Using a Soldier- Detection Task April 12, 2006.

To Shoot Or Not To Shoot. Why Haven’t We Got What We Want? The world is filled with insensitive SILLY people. Our publication rate is excessively modest.

Learning Objectives Explain the role of sampling in the research process Distinguish between probability and nonprobability sampling Understand the factors.

Advanced Decision Architectures Collaborative Technology Alliance An Interactive Decision Support Architecture for Visualizing Robust Solutions in High-Risk.

The Use And Misuse Of Machines That Simulate Human Cognition Hall Beck, PhD Office: 215 Smith-Wright Hall.

Chapter 9 Hypothesis Testing 9.1 The Language of Hypothesis Testing.

Copyright © 2006 by The McGraw-Hill Companies, Inc. All rights reserved. McGraw-Hill/Irwin 8-1 Chapter Eight Audit Sampling: An Overview and Application.

© 2006 The McGraw-Hill Companies, Inc. All rights reserved. McGraw-Hill Sampling Chapter Six.

1  The Problem: Consider a two class task with ω 1, ω 2   LINEAR CLASSIFIERS.

Decision-Making. Decision Making ▪Decision Making - is choosing among two or more alternatives (choices) ▪Begins with identification of a problem and.

WERST – Methodology Group

Human and Optimal Exploration and Exploitation in Bandit Problems Department of Cognitive Sciences, University of California. A Bayesian analysis of human.

REFERENCES Bargh, J. A., Gollwitzer, P. M., Lee-Chai, A., Barndollar, K., & Troetschel, R. (2001). The automated will: Nonconscious activation and pursuit.

Donna B. Konradi, DNS, RN, CNE GERO 586 Populations and Samples.

Brian Lukoff Stanford University October 13, 2006.

Copyright © 2007 Pearson Education Canada 1 Chapter 8: Materiality and Risk.

Introduction to Machine Learning © Roni Rosenfeld,

Introduction to Biostatistics, Harvard Extension School, Fall, 2005 © Scott Evans, Ph.D.1 Sample Size and Power Considerations.

Chapter 9 Hypothesis Testing Understanding Basic Statistics Fifth Edition By Brase and Brase Prepared by Jon Booze.

Introduction to Machine Learning

Cost Benefit Analysis, Marginal Benefits, and Marginal Costs

Hypothesis Testing: Hypotheses

Inference Concerning a Proportion

Chapter 9 Hypothesis Testing.

Three Basic Economic Questions

Madison Elementary / Middle School and the New Accountability System

Indicator 3.05 Interpret marketing information to test hypotheses and/or to resolve issues.

Presentation transcript:

Should I Decide or Let the Machine Decide for Me? Explorations in the Use, Misuse and Disuse of Automation The Soldier Study Prancers Skip, Mary, Linda, Teresa, Chris, Jamie, Kelly, Marielle, Tommy, Andy, Becca, Leslie, Julie, Nick and Greg

Examples of Automation Versus Nonautomated Options Pilot uses automated or manual guidance systems to fire missiles. Accountant uses software program or personal knowledge to compute taxes. Battlefield Combat Identification System Power Mower or Push Mower

Goals of Automation Use Save time, energy or money Make an impression Practice Achieve a performance objective

Optimal Decision Requires Assessing the likelihood of success using automation. Assessing the likelihood of success using a nonautomated alternative. Select the option with the highest probability of success.

Objectives of Study 1 To determine the extent that persons would deviate from an optimal (rational) decision making strategy. To discover if automation misuse or automation disuse is the greater problem on this task.

Design 3 (Relative Performance: Inferior, Equal, Superior) x 2 (Feedback: Yes, No) between- subjects. Choice to base credit on self or computer as the dependent variable.

Trial by Trail Procedure Present photo for.75 seconds. Participant responds via mouse indicating if the target was in the photo. Contrast detector then scans photo for human form. It attempts to determine if the target is in the photo.

Still More Procedure After 200 trials, participants in feedback condition are told how many errors they and the machine made. Participants in the No Feedback condition do not receive this information. Participants are either inferior, equal or superior to the contrast detector.

High Noon All participants are told 10 more trials will be conducted which will determine the extra credit that they receive. At this point, the participant must choose to base extra credit on self or machine.

Persons Basing Extra Credit on Self

Rational Use of Automation

Objectives of Study 2 To determine the results of the first study were due to hopes of a “heroic turnabout.” To find if the outcomes of Study 1 indicate a bias against using automation.

Procedure and Design Same basic task as first study. 3(Relative Performance: Inferior, Equal, Superior) x 2 (Feedback: Yes, No) between- subjects Ten trials randomly selected. Choice to base credit on self or contrast detector.

Persons Basing Extra Credit on Self

Rational Use of Automation

Objectives of Study 3 To determine if feedback mitigates the bias against using automation. To discover what type of feedback is most effective in reducing this bias.

Forms of Feedback Trial by Trial: After each trial participant told if target was in the preceding photo. Cumulative: After 200 trials, participant told total errors made by self and contrast detector. Prior Results: After 200 trials, participants informed that persons who base extra credit on detector usually obtain more points.

Procedure and Design Same basic task as first two studies. Detector superior to all participants. 2 (Trial by Trial: Yes, No) x 2 (Cumulative: Yes, No) x 2 (Prior Results: Yes, No) between- subjects. Ten trials randomly selected. Choice to base credit on self or contrast detector.

Rational Use of Automation

Objectives of Study 4 To determine if a combination of training participants to use the “automation algorithm” decreases the bias. To discover if a combination of training and feedback if more effective than either training or feedback alone.

Rational Use of Automation