Presentation on theme: "Watson Systems By- Team 7 : Pallav Dhobley09005012 Vihang Gosavi 09005016 Ashish Yadav09005018."— Presentation transcript:
Watson Systems By- Team 7 : Pallav Dhobley Vihang Gosavi Ashish Yadav
Motivation: Deep-Blue’s Triumph over Kasparov in In search of new challenge. 2
Jeopardy! 2004 – Search ends! One of the most popular Quiz show in U.S.A. Broad/Open Domain. Complex Language. High Speed. High precision. Accurate Confidence. 3
Jeopardy! 2004 – Search ends! One of the most popular Quiz show in U.S.A. Broad/Open Domain. Complex Language. High Speed. High precision. Accurate Confidence. 4 *le IBM
Easier than playing Chess? 5 Chess: Finite moves and states. Mathematically well defined search space Symbols have mathematical meaning Natural Language: Implicit Highly Contextual Ambiguous Imprecise
Easier than playing Chess? 6 Chess: Finite moves and states. Mathematically well defined search space Symbols have mathematical meaning Natural Language: Implicit Highly Contextual Ambiguous Imprecise NO!!
Easy Question (LN(1,25,46,798*π))^3 / 34, = ? 7
Easy Question: (LN(1,25,46,798*π))^3 / 34, =
Hard Question: Where was our “father of nation” born? - contextual. - imprecise. Easy for us Indians to relate term “father of nation” with M.K. Gandhi. Not the same with computers. Need of learning from As-Is content. 9
Learning the As-Is text (NLP): 10
What is Watson? 11 Advanced Search Engine? × Some fancy Database Retrieval System? × Beginning of Sky-Net?× Science behind an Answer?√
Principles of DeepQA: Massive Parallelism - Each hypothesis and interpretation is analyzed independently in parallel to generate candidate answers. Many experts - Facilitate the integration and contextual evaluation of a wide range of analytics generated by several algorithms running in parallel. 13
Principles of DeepQA (ctd.) Pervasive Confidence Estimation - No component commits to an answer Integrate shallow and deep knowledge - Using shallow and deep semantics for better precision e.g. Shallow semantics:Keyword matching Deep semantics:Logical Relationships 14
Shallow Semantics: 15
Deep Semantics: 16
How does Watson Learn? 17
Step 0 : Content Acquisition Identifying and gathering the content to be used for answering and evidence supporting. Involves analyzing example questions from the problem space which consists of Q-A from previous games. Encyclopedias, dictionaries, wiki pages etc. are use to make up the evidence sources. Extract, verify and merge the most informative nuggets as a part of content acquisition. 18
Step 1 : Question Analysis The initial analysis that determines how the question will be processed by the rest of the system. Question Classification e.g. puzzle/math Focus and (Lexical Answer Type)LAT e.g. “On this day” LAT – date/day Relation Detection e.g. sea(India, x, west) Decomposition - divide and conquer. 19
Step 2 : Hypothesis Generation 1.Primary search : – Keyword based search – Top 250 results are considered for Candidate Answer generation. – Empirical statistics : 85% time answer is within top 250 results. 2.CA generation : above results are further processed for CA generation. 3.Soft Filtering – It reduces set of candidate answers using superficial analysis (machine learning). – Reduction in number of CA to approx. 100 – Answers are not fully discarded, may be reconsidered at final stage. 20
Step 2: Hypothesis Generation (ctd.) 4.Each CA plugged back into the question is considered a hypothesis which the system has to prove correct with some threshold of confidence. 5.If failed at this state, system has no hope of answering the question whatsoever. – Noise tolerance. 21
Step 3 : Hypothesis & evidence scoring Evidence retrieval : – Further evidences are gathered to support the Hypothesis formed in last step. e.g. Passage search: gathering passages by adding CA to primary search query. Scoring: – Deep content analysis – Determines degree of certainty that retrieved evidence supports the CA. 22
Step 4 : Final Merging and Ranking Merging: – Merging all the hypothesis which give you the same answer. – Using an ensemble of matching, normalization and co- reference resolution algorithms, Watson identifies equivalent and related hypothesis. Ranking and confidence estimation: – The final set of hypothesis after merging are ran over set of training questions with known answers. 23
Example : Q : “Who is the antagonist of Stevenson's Treasure Island?” Step 1 : Parse and generate a logical structure to describe the question. -antagonist(X) -antagonist_of(X, Stevenson’s TI) -adj_possesive(Stevenson, TI) 24
Example (ctd.): Step 2: Generating semantic assumptions - island (TI) -book(TI) - movie(TI) -author(Stevenson) -director(Stevenson) Step 3 : Builds different semantic queries based on phrases, keywords and semantic assumptions. Step 4 : Generates 100s of answers based on passages, documents and facts returned from 3. Long-John Silver is likely to be one of them. 25
Example (ctd.): Step 5: Formulate evidence in support or refutation. (+VE) evidence : 1. Long-John Silver the main character in TI. 2. The antagonist in Treasure Island is Long-John Silver 3. Treasure Island, by Stevenson was a great book. (-VE) evidence : Stevenson = Richard Lewis Stevenson antagonist = Wolverine 26
Example (ctd.): Step 6: -Combine all the evidence and their scores. -Analyze evidences to compute confidence and return the most confident answer. Long-John Silver in this case ! 27
Watson- Performance: 28
Watson’s Brain (Software): 29 Languages used : Java, C++, prolog. Apache Hadoop framework for distributed computing. Apache UIMA framework. – Helps in DeepQA’s demand for Massive Parallelism. – Facilitated rapid component integration, testing, evaluation SUSE Linux Enterprise Server 11
Watson’s Brain(Hardware): 30 One Jeopardy! Question takes 2 hours on normal desktop computer! The real task - Confidence determination before buzzing. High Time need of faster Hardware support.
Watson’s Brain: (ctd.) 31 Total Ninety POWER-750 servers. Total 2880 POWER7 processor cores. Total 16 Terabytes of R.A.M. Each POWER-750 server uses a 3.5 GHz POWER7 eight core processor, with 4 Threads per core. Size of total 8 refrigerators. Can process data up-to the speed of 500 GB/s.
Watson’s Brain: (ctd.) 32
Watson – Runtime Stack 33
The Final Blow! 34 3 rounds of Jeopardy! Between Watson, Rutter & Jennings. Watson comprehensively defeats it’s competitors with net score of $77,147 Jennings managed $24,000. Rutter ended third with $21,600.
The Final Blow! (ctd.) 35 “I for one welcome our new computer overlords” - Jennings
High performance analytics Non-cognitive Smart Learner Not invincible Conclusion: 36
Watson & Suits 37 Tech support Knowledge management Business Intelligence Improvised Information sharing
Watson for society- Health Care 38 Symptoms Patient Records Tests Medications Notes/Hypothesis Texts, Journals Diagnosis Models Finding appropriate “Disease”, As per Asked by adjoining “Symptoms” and “Records”
References: 39 Watson Systems: Wiki Page Research Papers:
References: 40 Jeopardy! IBM Watson Day 1 (Feb 14, 2011) Science Behind an Answer- watson/science-behind-an-answer.html The AI magzine view/2303
References: 41 Philip Resnik Semantic similarity in a taxonomy: An information-based measure and its application to problems of ambiguity in natural language. Journal of Artificial Intelligence Research. Tom M. Mitchell Machine Learning. Computer Science Series. McGraw-Hill.