1. ©2012 Paula Matuszek CSC 9010: Information Extraction Overview Dr. Paula Matuszek Paula.Matuszek@gmail.com (610) 647-9789 Spring, 2012

2. ©2012 Paula Matuszek Information Extraction Overview l Given a body of text: extract from it some well-defined set of information l Typically draws heavily on NLP l Three main components: –Domain knowledge base –Knowledge model –Extraction Engine

3. ©2012 Paula Matuszek Information Extraction Domain Knowledge Base l Terms: enumerated list of strings which are all members of some class. –“January”, “February” –“Smith”, “Wong”, “Martinez”, “Matuszek” –“”lysine”, “alanine”, “cysteine” l Classes: general categories of terms –Month names, Last Names, Amino acids –Capitalized nouns –Verb Phrases Often organized into ontologies

4. ©2012 Paula Matuszek Domain Knowledge Base l Rules: LHS, RHS, salience l Left Hand Side (LHS): a pattern to be matched, written as relationships among terms and classes l Right Hand Side (RHS): an action to be taken when the pattern is found l Salience: priority of this rule (weight, strength, confidence)

5. ©2012 Paula Matuszek Some Rule Examples: l => l => print “Birthdate”,, l => create address database record l “/” “/” => create date database record (50) l “/” “/” => create date database record (60) l “.” => l => create “relationship” database record

6. ©2012 Paula Matuszek Generic KB l Generic KB: KB likely to be useful in many domains –names –dates –places –organizations l Almost all systems have one l Limited by cost of development: it takes about 200 rules to define dates reasonably well, for instance.

7. ©2012 Paula Matuszek Domain-specific KB l We mostly can’t afford to build a KB for the entire world. l However, most applications are fairly domain-specific. l Therefore we build domain-specific KBs which identify the kind of information we are interested in. –Protein-protein interactions –airline flights –terrorist activities

8. ©2012 Paula Matuszek Domain-specific KBs l Typically start with the generic KBs l Add terminology l Figure out what kinds of information you want to extract l Add rules to identify it l Test against documents which have been human-scored to determine precision and recall for individual items.

9. ©2012 Paula Matuszek Knowledge Model We aren’t looking for documents, we are looking for information. What information? l A knowledge model or schema identifies the information components we want and their relationship l Typically looks very much like a DB schema or object definition

10. ©2012 Paula Matuszek Knowledge Model Examples l Personal records –Name –First name –Middle Initial –Last Name –Birthdate –Month –Day –Year –Address

11. ©2012 Paula Matuszek Knowledge Model Examples l Protein Inhibitors –Protein name (class?) –Compound name (class?) –Pointer to source –Cache of text –Offset into text

12. ©2012 Paula Matuszek Knowledge Model Examples l Airline Flight Record –Airline –Flight l Number l Origin l Destination l Date »Status »departure time »arrival time

13. ©2012 Paula Matuszek Extraction Engine l Tool which applies rules to text and extracts matches –Tokenizer –Part of Speech (POS) Tagger –Term and class tagger –Rule engine: match LHS, execute RHS l Rule engine is iterative l May include an interactive component which is essentially a query engine against already extracted information

14. ©2012 Paula Matuszek Extraction Example: Birthdates l Problem: create a database of birthdays from text with birth information l Sample sentences: –George Washington was born in 1725. –Washington was born on Feb. 12, 1725. –Feb. 12 is Washington's birthday. –Washington's birth date is Feb. 12, 1725. –George Washington was born in America. –Washington's standard was born by his troops in 1778. Examples Negative Examples

15. ©2012 Paula Matuszek Birthdates: Knowledge Model l Simple birthdate model: –Name –Birthdate l Complex birthdate model: –Name– Date –First Name – Month –Middle Name – Day –Last Name – Year

16. ©2012 Paula Matuszek Birthdates Knowledge Base l Generic KB: Name, Date l Domain specific KB: Rules –1. "was born" {"in"|"on"} =>Insert (Name, Date) into database –2. "is" "birthday" =>Insert (Name, Date) into database –3. "birth" "date" "is" =>Insert (Name, Date) into database

17. ©2012 Paula Matuszek Birthdays: Extraction Process Washington was born in 1725 l Tokenize: –"Washington" –"was" –"born" –"in" –"1725" –"."

18. ©2012 Paula Matuszek Extraction, POS Tagging l "Washington", noun, proper noun, subject l "was": auxiliary verb, past tense, third person singular (3PS) l "born": verb, past tense, 3PS l "was born": verb phrase, passive l "in": preposition l "1725": prepositional object l "in 1725" prepositional phrase

19. ©2012 Paula Matuszek Extraction, Class Tagging l "Washington": Last Name l "was": nothing additional l "born": nothing additional l "in": nothing additional l "1725": Year

20. ©2012 Paula Matuszek Extraction: Rules l Name Rules: –"Washington": Name l Date Rules: –"1725": Date l Birthday Rule # 1: –Insert (Washington, 1725) into database

21. ©2012 Paula Matuszek Other Approaches l Classifiers trained using machine learning approaches. Our text discusses some of these. l Hybrid systems using both approaches

22. ©2012 Paula Matuszek Tools l Tools we will cover: –I2E. Commercial product from Linguamatics (http://www.linguamatics.com/) with strong linguistics background and sophisticated interfaces.http://www.linguamatics.com/ –GATE. Open source product (http://gate.ac.uk/) with extensive text processing capabilities.http://gate.ac.uk/ l Some other well-known tools –NLTK has some limited entity and relation extraction tools – Calais. Commercial product from Reuters, based on ClearForest. Limited free functionality available (http://www.opencalais.com/)http://www.opencalais.com/ –ReVerb. (formerly TextRunner). U of Washington program for extracting binary relationships (http://reverb.cs.washington.edu/)http://reverb.cs.washington.edu/ –AeroText. Commercial product originally developed by Lockheed, now owned by Rocket Software. (http://www.rocketsoftware.com/products/aerotext/overview)http://www.rocketsoftware.com/products/aerotext/overview l Find more at http://www.kdnuggets.com/software/text.htmlhttp://www.kdnuggets.com/software/text.html

23. ©2012 Paula Matuszek Summary l Text mining below the document level l NOT typically interactive, because it’s slow l Typically builds up a DB of information which can then be queried l Uses a combination of term- and rule- driven analysis and Natural Language Processing parsing.