Presentation on theme: "Human Supervisory Control"— Presentation transcript:
1 16.422 Human Supervisory Control Naturalistic Decision MakingMassachusetts Institute of Technology
2 Naturalistic Decision Making How experts make decisions in the real worldDescriptive methodStress, time-pressure, dynamic conditions, ambiguous information, and ill-defined goalsHeuristics may introduce bias but can be very powerful.Other related research areas:Behavioral decision theoryJudgment decision makingOrganizational decision makingGroup of behavioral scientists who shared common theme of “the importance of (1) time pressure, (2) uncertainty, (3) ill-defined goals, (4) high personal stakes, and (5) other complexities thatcharacterize decision making in real-world settings.”2nd theme was to study people with expertise during the decisionmaking process3rd theme was that people appeared to size up situations with more importance than they would select between courses of action
3 NDM vs. Classic Decision Theory People are not always “rational” decision makersBut people are proficientSituation-action matching decision rulesAs opposed to choosing an alternateContext-bound informal modeling as opposed to context-free formal modelingProcess orientation as opposed to prediction of outcomesEmpirical-based prescriptions
4 Forms of NDM Recognition primed decision making Related theories/methodsCritical decision methodExplanation based theoryImage theory
5 Recognition Primed Decision Making (RPD) Intuitive form of diagnosis and predictionPattern matchingMultiple cues, highly correlatedKey observationsRequires learning and expertiseExperts choose feasible course of action without analyzing all or even many optionsDecision making under uncertainty, time-pressure, & stressMilitary commanders & firefightersWhat biases might we see in RPD?
7 Coping With Uncertainty Inadequate understanding, lack of information, conflicted alternativesRAWFS heuristicReducing uncertaintyGathering more informationAssumption-based reasoningFilling in gapsWeighing pros and consForestallingAnticipate undesirable consequencesSuppressing uncertaintyRationalization16.422
8 Recognition/Meta-cognition Model Another model (framework?) for how decision makers cope with uncertaintyPattern matching criticalWhat happens when recognition fails?Decision makers revert to assumption-based reasoning and meta-cognitive processesAttempt to find flaws & weaknesses in evaluationSTEP: Construct a story, test, evaluate, & planA prescriptive approach
9 NDM and Teams How does team decision making differ from individual? Team SA and shared mental modelsStudying teams in their natural environmentReal teams performing real tasks in real settingsContextual focus as opposed to more general and abstractTeam research is not easy
10 CTA Methods for NDM Research COGNITIVE TASK ANALYSISExploring the Current WorldExploring the Envisioned WorldUnderstanding the way people operatein their worldDiscovering support for how peoplewill operate in their worldGoal :Understand/model expertise se, knowledge,strategies, s, and errorTechniques :Semantic MappingEthnographic / Observational investigationsCritical Incident TechniqueCritical Decision MethodStructured Interview T w TechniquesPractitioner(s)CTAModelDesignBasisScratchField of Pr PracticeGrowth of UnderstandingCTA RepresentationPrototype RepresentationThe DomainGoal :Understand/model complexities, s,demands, s, variability, y, and complicatingfactorsTechniques :Functional/Means- s-ends AnalysisEthnographic / Observational investigationsFunctional Task and Workflow ModelingStructured Interview TechniquesDiscovering how to support the way the worldwill workUnderstanding the way the world worksTimeTimeAdapted from Carnegie Group, Inc.critical decision method in which participants are asked to describe a specific decision-making incident in detail and then to respond to probes seeking elaboration of important aspectsof the decision sequences.Semantic mapping (a.k.a., mind-mapping, idea mapping, word webbing, etc.) is a term which describes a variety of strategies designed to show how key words or concepts are related to one another through graphic representations.Mapping is an effectivetechnique for teaching vocabulary and textual patterns of organization; and it is also effectivefor improving note taking and creative thinking skills.
11 Evaluating NDM Studies Laboratory vs. real-world settingsCredibility determined by:Quality of research questionsData collection methodsSuitability of methods for research questionsPlausibility of answersReasonableness of assumptionsTransferabilityCognition in the wild
12 Forms of NDM Recognition primed decision making Related theories/methodsCritical decision methodExplanation based theoryImage theory
13 Critical Decision Method Retrospective interview techniqueCognitive probes used to elicit knowledge and strategies used in expert decision-makingNon-routine eventsBased on critical incident technique (1954)What led up to the situation?What action was effective or ineffective?What was the outcome or result of this action?CDM focuses on identifying critical cues, judgments, & decisionsWhat-if queriesJohn Flanagan is the originator if the CIT method. Read more athttp://www.air.org/overview/flanagan.htmMore CIT:
14 Explanation Based Model Need to generate a coherent story and match that story against possible choicesFormulating a full story based on incomplete factsFilling in the gapsExpectanciesRecall long term memory discussionMatch hypothesized story with possible outcomesModifying story to achieve a desired outcome is possibleWhere might we see this in aviation accidents?
15 Image TheoryDecisions are not discrete events but a more complicated processProgress Decisions: Are past decisions are being adequately carried out?Adoption Decisions: Consideration of new goals, plans, principles or actions,ProblemsUnclear underlying goalsPoor planning or implementation of strategyValues insufficient or incorrectly definedValueImageTrajectoryImageStrategicImage
16 Cognitive Continuum of Decision Making Social JudgmentTheories (Lens Model)Analytic DecisionsAnalytic strategiesExample: SEUConcurrent optionsUnbounded rationalityOptimization with constraintsIntuitive DecisionsNaturalistic Decision MakingExample: RPDSerial optionsBounded rationalityFast & frugal heuristics
17 ReferencesCooksey, R. W. (1996). Judgment Analysis: Theory, Methods, and Applications. San Diego, Academic Press.Brunswik, E. (1952). The conceptual framework of psychology. Chicago, University of Chicago Press.Klein, G. (1989). "Recognition-Primed Decisions." Advances in Man-Machine Research5: 47-92Decision Making in Action: Models and Methods. G. Klein, J. Orasanu, R. Calderwoodand C. E. Zsambok. Norwood, N.J., AblexPublishing.Beach, L.R., & Mitchell, T.R.(1990).Image theory:A behavioral theory of decision making in organizations. Research in Organizational Behavior, 12, 1-41.