Presentation is loading. Please wait.

Presentation is loading. Please wait.

Eric Horvitz Microsoft Research & University of Washington Toward Models of Surprise.

Similar presentations


Presentation on theme: "Eric Horvitz Microsoft Research & University of Washington Toward Models of Surprise."— Presentation transcript:

1 Eric Horvitz Microsoft Research & University of Washington Toward Models of Surprise

2 Surprise Unexpected event: We thought about it, but considered it would occur with probability < x Unexpected event: We thought about it, but considered it would occur with probability < x Unmodeled event: We never even thought about that… Unmodeled event: We never even thought about that… Significant consequence: Significant positive or negative swing in utility of outcome for observer or group Significant consequence: Significant positive or negative swing in utility of outcome for observer or group Events can be personal, local, national, world Events can be personal, local, national, world Unexpected or umodeled future event or outcome of significant consequence.

3 Surprise Surprise defined in terms of an observers expectations Surprise defined in terms of an observers expectations Deliberation and modeling can influence degree of or nature of surprise Deliberation and modeling can influence degree of or nature of surprise Moving from surprise to plausible expectation Moving from surprise to plausible expectation Expectation of classes or properties of surprise should given that a surprise occurs. Expectation of classes or properties of surprise should given that a surprise occurs. Unexpected or umodeled future event or outcome of significant consequence.

4 Forecasting Surprise Qualitative brainstorming about likelihood and nature of surprises over future time periods from knowledge, trends, and historical data Qualitative brainstorming about likelihood and nature of surprises over future time periods from knowledge, trends, and historical data Opportunity: Statistical models of surprise Opportunity: Statistical models of surprise Learn from databases of past surprise to predict future surprises Learn from databases of past surprise to predict future surprises Opportunity: Statistical models of predictive competency Opportunity: Statistical models of predictive competency Context- and content-centric failure to predict with accuracy Context- and content-centric failure to predict with accuracy Qualitative and quantative approaches

5 Last 12 months Top n Properties SurpriseCasting: Abstract and Project Example: Example: Identify top n most surprising things in prior 1 year (world events or personal life) Identify top n most surprising things in prior 1 year (world events or personal life) Repeat for 5 years of surprises Repeat for 5 years of surprises Characterize nature and rate of significant surprises Characterize nature and rate of significant surprises Consider top n over longer-term periods in same manner Consider top n over longer-term periods in same manner200420032002 Top n Properties Properties Properties

6 Surprise Forecasting: 2005-2020 Surprise Types Examples Attention & Information Actions SurpriseCasting: Abstract and Project Top n surprise types, by class or properties Assume type occurs; give top m examples per type Recommended triage of focus of attention: additional study, info. gathering, monitoring per type Recommended actions to react, contain, disrupt negative suprises and/or exploit positive surprises Turn to the future with statistics about types, examples. Turn to the future with statistics about types, examples. Consider triaging of attention and recommending actions. Consider triaging of attention and recommending actions. Surprises in technical sophistication of weaponry available inexpensively and universally Largescale simultaneous & continuous disabling of communication and sigint satellites available to multiple states.

7 Surprise Forecasting: 2005-2020 Surprise Types Examples Attention & Information Actions SurpriseCasting: Abstract and Project Top n surprise types, by class or properties Assume type occurs; give top m examples per type Recommended triage of focus of attention: additional study, info. gathering, monitoring per type Recommended actions to react, contain, disrupt negative suprises and/or exploit positive surprises Deliberation about potential surprises: Backcasting Deliberation about potential surprises: Backcasting Chaining from surprising outcome to explore feasibility Chaining from surprising outcome to explore feasibility Exotic transnational actors with access to increasing technical prowess. Terrorist organization establishes secret alliance with seemingly friendly former adversary state, intolerant of US empire; gains access to nuclear arms with deniability for former adversary.

8 Expecting Surprises Technical developments Technical developments International political terrain International political terrain

9 Technical Surprises from Past Big surprises over 50 years Big surprises over 50 years Flight Flight From likely unachievable dream, to major pillar of force projection and deterrence in just a few summers… From likely unachievable dream, to major pillar of force projection and deterrence in just a few summers… Quickly melted into background of obviousness Quickly melted into background of obviousness In a few years: From awe of a canvas flying contraption to concerns about low-salt meal while streaking across ocean in widebody jet. In a few years: From awe of a canvas flying contraption to concerns about low-salt meal while streaking across ocean in widebody jet. Nuclear fission and fusion Nuclear fission and fusion Theory of Relativity and implications out of the blue Theory of Relativity and implications out of the blue Shifts in balance of power Shifts in balance of power Risk to nation, human civilization Risk to nation, human civilization

10 Classes of Future Technical Surprises? Computation Computation Intelligent systems change world (consumer, defense, communications, etc.) in a fundamental way? Intelligent systems change world (consumer, defense, communications, etc.) in a fundamental way? Energy Energy New surprise source of energy changes economics of world New surprise source of energy changes economics of world Changes nature of powerplants in our and adversaries weapon systems Changes nature of powerplants in our and adversaries weapon systems Shift in balance of power hinge on the new source Shift in balance of power hinge on the new source Asymmetric force have new tool on their side for unleashing chaos and disruption Asymmetric force have new tool on their side for unleashing chaos and disruption Biology Biology Assembly of Polio virus via internet and mail order in 2002 heralds era of bio-hacking Assembly of Polio virus via internet and mail order in 2002 heralds era of bio-hacking Major disruptions via pandemics Major disruptions via pandemics

11

12 Futurist Community: Wildcards A wild card is a future development or event with a relatively low probability of occurrence but a likely high impact on the conduct of business BIPE Conseil, Copenhagen Institute for Futures Studies, Institute for the Future: Wild Cards: A Multinational Perspective, Institute for the Future 1992

13 Futurist Community: Wildcards Futurist discussion of wildcards, futurequakes Futurist discussion of wildcards, futurequakes Are natural, human-caused, or combinations Are natural, human-caused, or combinations Are negative and positive Are negative and positive Multiple wildcards can interact Multiple wildcards can interact Breaks, discontinuities vs. trends Breaks, discontinuities vs. trends Peterson (2000), Out of the Blue: How to Anticipate Big Future Surprises Peterson (2000), Out of the Blue: How to Anticipate Big Future Surprises Cornish (2003), The Wild Cards in Our Future Cornish (2003), The Wild Cards in Our Future Steinmüller (2003), The Future as Wild Card: A Short Introduction to a New Concept

14 On Expected Surprises… Earth & Sky Earth & Sky Biomedical Developments Biomedical Developments Geopolitical & Sociological changes Geopolitical & Sociological changes Technology & Infrastructure Upheaval Technology & Infrastructure Upheaval Surprise Attack Surprise Attack Spiritual & Paranormal Spiritual & Paranormal J. Petersen, Out of the Blue: How to Anticipate Big Future Surprises, 2000.

15 On Expected Surprises… We are now living in another period of significant transition--a foreshortened span of time, during which our surroundings and experiences will change more than during any era in history. Humanity has never lived through the convergence--and, in some cases, the collision--of global forces of such magnitude and diversity. … One foreseeable outcome might be global instability; another, a planetary renaissance. J. Petersen, Out of the Blue: How to Anticipate Big Future Surprises, 2000.

16 On Expected Surprises… … In any case, during the next two decades, almost every aspect of life will be fundamentally reshaped.. J. Petersen, Out of the Blue: How to Anticipate Big Future Surprises, 2000.

17 Technological Prowess and Wildcards Rapidly growing technological prowess of humans has, for the first time in history, produced new classes of wild cards that could potentially destroy the whole human race. The new kinds of wild cards are simply too destructive to allow to happen and therefore require active attempts to anticipate them and be proactive in dealing with their early indicators. " Peterson, Out of the Blue: How to Anticipate Big Future Surprises, 2000

18 Preparing for Wildcards Wild cards can be anticipated and prepared for. Thinking about a wild card before it happens is important and valuable. The more that is known about a potential event, the less threatening it becomes and the more obvious the solutions seem. " Peterson, Out of the Blue: How to Anticipate Big Future Surprises, 2000

19 Backcasting Cornish: Multipronged Aerial Terrorist Attack was identified, Cornish: Multipronged Aerial Terrorist Attack was identified, e.g.,1987 article (B. Jenkins); 1994 article (M. Cetron): e.g.,1987 article (B. Jenkins); 1994 article (M. Cetron): "Targets such as the World Trade Center not only provide the requisite casualties but, because of their symbolic nature, provide more bang for the buck. "Targets such as the World Trade Center not only provide the requisite casualties but, because of their symbolic nature, provide more bang for the buck. "In order to maximize their odds for success, terrorist groups will likely consider mounting multiple, simultaneous operations with the aim of overtaxing a government's ability to respond, as well as demonstrating their professionalism and reach." "In order to maximize their odds for success, terrorist groups will likely consider mounting multiple, simultaneous operations with the aim of overtaxing a government's ability to respond, as well as demonstrating their professionalism and reach." Despite all this, terrorism will remain a back-burner issue for Western leaders as long as the violence strikes in distant lands and has little impact on their fortunes or those of their constituents. Despite all this, terrorism will remain a back-burner issue for Western leaders as long as the violence strikes in distant lands and has little impact on their fortunes or those of their constituents. E. Cornish, The Wild Cards in Our Future. The Futurist. Washington: Jul/Aug 2003.Vol. 37, Iss. 4.

20 Backcasting How might we proceed to evaluate this warning? Poll experts on likelihood and targets of aerial suicide attacks. Assume aerial attacks plausible. Poll experts on likelihood and targets of aerial suicide attacks. Assume aerial attacks plausible. Backcasting: Reason about the challenges and solutions. Backcasting: Reason about the challenges and solutions. Terrorists are unlikely to have their own aircraft. Could they buy one? Perhaps, but the cost would be enormous and authorities might ask too many questions. Could they hijack a military bomber and load it with bombs? They would have to be awfully clever and lucky to get onto an air base, seize a plane, load it with bombs, and fly it to a target. Success would be unlikely, and terrorists would know it. Could they hijack a military bomber and load it with bombs? Could they steal a plane? An individual might have difficulty intimidating passengers and crew, but a group could probably do it even with modest weaponry. Getting into the pilot's cabin seems possible. So taking command of the plane seems feasible. But would the pilot follow instructions? Some instructions, maybe, but perhaps not all-especially if asked to do something suicidal. Could a terrorist be trained as a pilot to take over? Yes; a number of men in terrorist organizations have been trained as pilots. But how can an aircraft cause destruction if it has no bombs to drop? It can simply crash into its target. Such a crash should wreck a building; a skyscraper would be especially vulnerable. Of course, everybody on the plane would be killed, but if the terrorists are ready to die themselves, the deaths of the passengers serve as an added benefit. Scores of terrorists have gone on similar suicide missions by strapping bombs to themselves. We can now envision a group of terrorists hijacking an airliner and crashing it into an enemy target. But what target might it be? If we had read Cetron's article, we would already have one target to think about-the World Trade Center. In fact, Islamic terrorists did attack the Center in 1993, but their car bomb failed to destroy the building. However, Cetron suggested that multiple targets might be selected for an attack. So we might stress the possibility of a larger, wider attack involving more terrorists and multiple planes and targets.

21 Research on Statistical Models of Surprise Most reasoning about surprise qualitative Most reasoning about surprise qualitative Challenge: Data sparcity Challenge: Data sparcity Pushing on theoretical foundations of surprise Pushing on theoretical foundations of surprise Example: Research traffic prediction service, Seattle Example: Research traffic prediction service, Seattle

22 Streaming Intelligence Learning & Learning & reasoning reasoning Web service Sensing Sensing Rich sensing Portabledevice Continual sensing, learning, and reasoning always available in stream of daily life

23 Predictions in Your Pocket Time until jam Times until jams clear

24 Event store Learning Reasoning Multiple views on traffic Operator ID: Nick Heading: INCIDENT Message: INCIDENT INFORMATION Cleared 1637: I-405 SB JS I-90 ACC BLK RL CCTV 1623 – WSP, FIR ON SCENE Incident reports Weather Major events Traffic Prediction Challenge

25 From Incident Reports to Salience e.g., Accidents e.g., Accidents How many lanes? How many lanes? Emergency vehicles? Emergency vehicles? Operator ID: Nick Heading: INCIDENT Message:INCIDENT INFORMATION Cleared 1637: I-405 SB JS I-90 ACC BLK RL CCTV 1623 – WSP, FIR ON SCENE

26 Data Set: Data Abstraction Lane Station Bottleneck

27

28

29

30

31

32 Building Predictive Models from Data Max likely time until bottleneck evaporates Database of sensed events Data store System-wide status & dynamics Incident reports Sporting events Weather Time of day Day of week Season Holiday status Base-level predictions Machine Learning Bayesian network forecasting model

33 Predict Future Surprises Surprise at time t in the future: Infer surprise with model built from training set Evidence Evidence Time of day, day of week Time of day, day of week Bottlenecks, flows and their durations Bottlenecks, flows and their durations Incident reports Incident reports Weather Weather Games Games Accidents Accidents

34 Learn models that predict likelihood that base- level models will fail and annotate real-time inferences. Failures (Bottleneck) System-wide status & dynamics Incident reports Sporting events Weather Time of day Day of week Season Holiday status Learning Context-Sensitive Competency BN1 BN2. BNn Base-level predictions

35 Traffic forecasting service Data store Inference User logs System-wide status & dynamics Incident reports Sporting events Weather Time of day Day of week Season Holiday status Base-level predictions Context-sensitive error models Sharing Context-Sensitive Competency Competency annotation

36 Building Models of Surprise Learning predictive model of surprise Learning predictive model of surprise Model of user expectation and suprise Model of user expectation and suprise Models of future surprises Models of future surprises Expectations Human Forecaster Major events Weather Time of day Day of week Holiday status Real World Outcome System-wide status & dynamics Incident reports Sporting events Weather Time of day Day of week Season Holiday status Database of surprising events Data store Future traffic Probability ! ! Events at t

37 Building Models of Future Surprises Learning predictive model of surprise Learning predictive model of surprise Model of user expectation and suprise Model of user expectation and suprise Models of future surprises Models of future surprises Expectations Human Forecaster Major events Weather Time of day Day of week Holiday status Real World Outcome Database of surprising events Data store System-wide status & dynamics Incident reports Sporting events Weather Time of day Day of week Season Holiday status Future traffic Probability ! ! Events at t Machine learning System-wide status & dynamics Incident reports Sporting events Weather Time of day Day of week Season Holiday status Events at t-T

38 Traffic forecasting service Data store Inference User logs System-wide status & dynamics Incident reports Sporting events Weather Time of day Day of week Season Holiday status Base-level predictions Context-sensitive error models Adding Surprise Modeling Surprise & surprise forecasting Surprise forecasting models Competency annotation

39 Forecasting Surprise Forecasting Surprise Traffic surprise in e.g., 30 minutes Traffic surprise in e.g., 30 minutes

40 Where will there be surprises in time t?

41 Sample Models and Exploration Influence of Seattle sporting events Influence of Seattle sporting events

42 Sample Models and Exploration Influence of weather Influence of weather

43 Sample Models and Exploration Influence of an accident at Bottleneck 15 Influence of an accident at Bottleneck 15

44 Influence of accidents at B15 and B3 on anomaly at B4 at 30 minutes Influence of accidents at B15 and B3 on anomaly at B4 at 30 minutes Examining Details Anomaly at B4 Acc 15.5 surprise -low -low Acc 3.5 surprise -.25 low -.25 high

45

46 Alerting on Device and Desktop Specify preferences and download to device Specify preferences and download to device Time- and route-specific preferences Time- and route-specific preferences Routes composed from bottlenecks Routes composed from bottlenecks Multiple alert types Multiple alert types

47

48

49

50 Models of Surprise Definitions of surprise Definitions of surprise Qualitative and quantitative Qualitative and quantitative Abstraction about past surprises to expect classes of surprise Abstraction about past surprises to expect classes of surprise Wild card notion in world of futurists Wild card notion in world of futurists Directions: Statistical models of surprise Directions: Statistical models of surprise

51 Surprise Forecasting: 2005-2020 Surprise Types Examples Attention & Information Actions SurpriseCast Top n surprise types, by class or properties Assume type occurs; give top m examples per type Recommended triage of focus of attention: additional study, info. gathering, monitoring per type Recommended actions to react, contain, disrupt negative suprises and/or exploit positive surprises

52


Download ppt "Eric Horvitz Microsoft Research & University of Washington Toward Models of Surprise."

Similar presentations


Ads by Google