1. Recent research at CLAIR Dragomir Radev University of Michigan, Ann Arbor radev@umich.edu Fall 2004

2. WWW as a textual database Large: 10 10 pages, 200 TB [Lyman&Varian 03] cf. brain (10 11 neurons) Multilingual: English 56.4% of sites, German 7.7%, French 5.6%, Japanese 4.9%, Chinese 2.4% Evolving: 22% of sites change every day, another 31% change every month [Cho&Garcia-Molina 00] Uneven importance: at different levels Adequate representations are needed for user-friendly access

3. Outline Introduction Random walks and social networks LexRank Projects in language modeling and machine learning

4. Outline Introduction Random walks and social networks LexRank Projects in language modeling and machine learning

5. Natural Language Processing (NLP) Typical NLP problems Entity extraction Relation extraction Text classification Summarization Information retrieval Machine translation Question answering Text understanding Parsing Word sense disambiguation Lexical acquisition Paraphrasing NLP is very hard! –The pen is in the box. –Every American has a mother. –Boston called. –I saw Zoe. The poor girl looked tired. –Mary and Sue bought each other a book. –The spirit is willing but the flesh is weak. –Children make delicious snacks. –Army head seeks arms. –Czech President and playwright Havel to receive honors

6. Recent trends in NLP Multidisciplinary Statistical Well founded Scaleable NLP Sociology Linguistics Lin. Algebra Graph theory Bioinformatics Stat. Mechanics E-commerce Bioinformatics Info. Retrieval Intelligence User interfaces Translation

7. Finding structure Language doesn’t have a regular structure (like a database) Sentences are very unlike each other Linguistic analysis: parse trees Hard to generalize Finding structure –Across sentences –Across sites/sources/documents –Over time Representations –Graphs everywhere!

8. NewsInEssence MEAD: salience-based extractive summarization Centroid-based summarization (single and multi document) Vector space model Additional features: position, length, lexrank (1000+ downloads) Cross-document structure theory (CST) NIE: first robust news summarization system (2001)

18. CST relationships 18 CST relationships in total

19. CST-based summarization (1) - + - + C = -1

20. CST-based summarization (2) - + - + removed added C = +1

21. Outline Introduction Random walks and social networks LexRank Projects in language modeling and machine learning

22. Social networks Induced by a relation Symmetric or not Examples: –Friendship networks –Board membership –Citations –Power grid of the US –WWW

23. Krebs 2004

24. Graph-based representations 1 2 3 4 5 7 68 12345678 111 21 311 41 51111 611 7 8 Square connectivity (incidence) matrix P Graph G (V,E)

25. Markov chains A homogeneous Markov chain is defined by an initial distribution x and a Markov kernel P. Path = sequence (x 0, x 1, …, x n ). The probability of a path can be computed as a product of probabilities for each step i.

26. Random walks Access time H ij = expected number of steps to go from i to j. Example [Lovász 1993]. What is H ij on a path with nodes 0, 1, n- 1? H(k-1,k) = 2k-1 H(i,k) = H(i,k-1) + 2k-1 H(i,k) = (2i+1) + (2i+3) + … + (2k-1) = k 2 – i 2 H(0,k) = k 2 (Brownian motion: travel distance sqrt(t) in time t) Electrical networks –R st is the resistance between two nodes s and t. The round-trip travel time between s and t is exactly 2mR st, where m is the number of edges.

27. Stationary solutions The fundamental Ergodic Theorem for Markov chains [Grimmett and Stirzaker 1989] says that the Markov chain with kernel E has a stationary distribution p under three conditions: –E is stochastic –E is irreducible –E is aperiodic To make these conditions true: –All rows of E add up to 1 (and no value is negative) –Make sure that E is strongly connected –Make sure that E is not bipartite Example: PageRank [Brin and Page 1998]: use “teleportation”

28. 1 2 3 4 5 7 68 Example t=10 This graph E has a second graph E’ superimposed on it: E’ is the uniform transition graph.

29. Eigenvectors An eigenvector is an implicit “direction” for a matrix. Ev = λv, where v is non-zero, though λ can be any complex number in principle. The largest eigenvalue of a stochastic matrix E is λ 1 = 1. For λ 1, the left (principal) eigenvector is p, the right eigenvector = 1 In other words, E T p = p.

30. Prestige and centrality Degree centrality: how many neighbors each node has. Closeness centrality: how close an actor is to all of the other nodes Betweenness centrality: based on the role that a node plays by virtue of being on the path between two other nodes Eigenvector centrality: the paths in the random walk are weighted by the centrality of the nodes that the path connects. Prestige = same as centrality but for directed graphs.

31. Computing the stationary distribution function PowerStatDist (E): begin p (0) = u; i=1; repeat p (i) = E T p (i-1) L = ||p (i) -p (i-1 )|| 1 ; i = i + 1; until L <  end Solution for the stationary distribution

32. 1 2 3 4 5 7 68 Example t=0 t=1 t=10

33. Outline Introduction Random walks and social networks LexRank

34. Centrality in summarization Motivation: capture the most central words in a document or cluster Centroid score [Radev & al. 2000, 2004a] Alternative methods for computing centrality?

35. Sample multidocument cluster 1 (d1s1) Iraqi Vice President Taha Yassin Ramadan announced today, Sunday, that Iraq refuses to back down from its decision to stop cooperating with disarmament inspectors before its demands are met. 2 (d2s1) Iraqi Vice president Taha Yassin Ramadan announced today, Thursday, that Iraq rejects cooperating with the United Nations except on the issue of lifting the blockade imposed upon it since the year 1990. 3 (d2s2) Ramadan told reporters in Baghdad that "Iraq cannot deal positively with whoever represents the Security Council unless there was a clear stance on the issue of lifting the blockade off of it. 4 (d2s3) Baghdad had decided late last October to completely cease cooperating with the inspectors of the United Nations Special Commission (UNSCOM), in charge of disarming Iraq's weapons, and whose work became very limited since the fifth of August, and announced it will not resume its cooperation with the Commission even if it were subjected to a military operation. 5 (d3s1) The Russian Foreign Minister, Igor Ivanov, warned today, Wednesday against using force against Iraq, which will destroy, according to him, seven years of difficult diplomatic work and will complicate the regional situation in the area. 6 (d3s2) Ivanov contended that carrying out air strikes against Iraq, who refuses to cooperate with the United Nations inspectors, ``will end the tremendous work achieved by the international group during the past seven years and will complicate the situation in the region.'' 7 (d3s3) Nevertheless, Ivanov stressed that Baghdad must resume working with the Special Commission in charge of disarming the Iraqi weapons of mass destruction (UNSCOM). 8 (d4s1) The Special Representative of the United Nations Secretary-General in Baghdad, Prakash Shah, announced today, Wednesday, after meeting with the Iraqi Deputy Prime Minister Tariq Aziz, that Iraq refuses to back down from its decision to cut off cooperation with the disarmament inspectors. 9 (d5s1) British Prime Minister Tony Blair said today, Sunday, that the crisis between the international community and Iraq ``did not end'' and that Britain is still ``ready, prepared, and able to strike Iraq.'' 10 (d5s2) In a gathering with the press held at the Prime Minister's office, Blair contended that the crisis with Iraq ``will not end until Iraq has absolutely and unconditionally respected its commitments'' towards the United Nations. 11 (d5s3) A spokesman for Tony Blair had indicated that the British Prime Minister gave permission to British Air Force Tornado planes stationed in Kuwait to join the aerial bombardment against Iraq. (DUC cluster d1003t)

36. Cosine between sentences Let s 1 and s 2 be two sentences. Let x and y be their representations in an n- dimensional vector space The cosine between is then computed based on the inner product of the two. The cosine ranges from 0 to 1.

37. LexRank (Cosine centrality) 1234567891011 11.000.450.020.170.030.220.030.280.06 0.00 20.451.000.160.270.030.190.030.210.030.150.00 30.020.161.000.030.000.010.030.040.000.010.00 40.170.270.031.000.010.160.280.170.000.090.01 50.03 0.000.011.000.290.050.150.200.040.18 60.220.190.010.160.291.000.050.290.040.200.03 7 0.280.05 1.000.060.00 0.01 80.280.210.040.170.150.290.061.000.250.200.17 90.060.030.00 0.200.040.000.251.000.260.38 100.060.150.010.090.040.200.000.200.261.000.12 110.00 0.010.180.030.010.170.380.121.00

38. d4s1 d1s1 d3s2 d3s1 d2s3 d2s1 d2s2 d5s2 d5s3 d5s1 d3s3 Cosine centrality (t=0.3)

39. d4s1 d1s1 d3s2 d3s1 d2s3 d2s1 d2s2 d5s2 d5s3 d5s1 d3s3 Cosine centrality (t=0.2)

40. d4s1 d1s1 d3s2 d3s1 d2s3d3s3 d2s1 d2s2 d5s2 d5s3 d5s1 Cosine centrality (t=0.1) Sentences vote for the most central sentence!

41. LexRank T 1 …T n are pages that link to A, c(T i ) is the outdegree of pageT i, and N is the total number of pages. d is the “damping factor”, or the probability that we “jump” to a far-away node during the random walk. It accounts for disconnected components or periodic graphs. When d = 0, we have a strict uniform distribution. When d = 1, the method is not guaranteed to converge to a unique solution. Typical value for d is between [0.1,0.2] (Brin and Page, 1998).

42. Cosine centrality vs. centroid centrality ID LPR (0.1) LPR (0.2) LPR (0.3) Centroid d1s1 0.6007 0.6944 1.0000 0.7209 d2s1 0.8466 0.7317 1.0000 0.7249 d2s2 0.3491 0.6773 1.0000 0.1356 d2s3 0.7520 0.6550 1.0000 0.5694 d3s1 0.5907 0.4344 1.0000 0.6331 d3s2 0.7993 0.8718 1.0000 0.7972 d3s3 0.3548 0.4993 1.0000 0.3328 d4s1 1.0000 1.0000 1.0000 0.9414 d5s1 0.5921 0.7399 1.0000 0.9580 d5s2 0.6910 0.6967 1.0000 1.0000 d5s3 0.5921 0.4501 1.0000 0.7902

43. Evaluation metrics Difficult to evaluate summaries –Intrinsic vs. extrinsic evaluations –Extractive vs. non-extractive evaluations –Manual vs. automatic evaluations ROUGE = mixture of n-gram recall for different values of n. Example: –Reference = “The cat in the hat” –System = “The cat wears a top hat” –1-gram recall = 3/5; 2-gram recall = 1/4; 3,4-gram recall = 0 ROUGE-W = longest common subsequence Example above: 3/5

44. CODEROUGE-1ROUGE-2ROUGE-W C0.50.390130.104590.12202 C100.385390.101250.11870 C1.50.380740.099220.11804 C10.381810.100230.11909 C2.50.379850.101540.11917 C20.380010.099010.11772 Degree0.5T0.10.390160.108310.12292 Degree0.5T0.20.390760.110260.12236 Degree0.5T0.30.385680.108180.12088 Degree1.5T0.10.386340.108820.12136 Degree1.5T0.20.393950.113600.12329 Degree1.5T0.30.385530.106830.12064 Degree1T0.10.388820.108120.12286 Degree1T0.20.392410.112980.12277 Degree1T0.30.384120.105680.11961 Lpr0.5T0.10.393690.106650.12287 Lpr0.5T0.20.388990.108910.12200 Lpr0.5t0.30.386670.102550.12244 Lpr1.5t0.10.399970.110300.12427 Lpr1.5t0.20.399700.115080.12422 Lpr1.5t0.30.382510.106100.12039 Lpr1T0.10.393120.107300.12274 Lpr1T0.20.396140.112660.12350 Lpr1T0.30.387770.105860.12157 Centroid Degree LexPageRank

45. Evaluation results Centroid: C0.5, C10, C1.5, C1, C2.5, C2 Degree: D0.5T0.1, D0.5T0.2, D0.5T0.3, D1.5T0.1, D1.5T0.2, D1.5T0.3, D1T0.1, D1T0.2, D1T0.3 LexRank: Lr0.5T0.1, Lr0.5T0.2, Lr0.5t0.3, Lr1.5t0.1, Lr1.5t0.2, Lr1.5t0.3, Lr1T0.1, Lr1T0.2, Lr1T0.3 Rouge-2 Lr1.5t0.20.115 D1.5T0.20.114 D1T0.20.113 … C1.5 0.099 Rouge-1 Lr1.5t0.10.400 Lr1.5t0.20.400 Lr1T0.20.396 … C10.382 Rouge-4 Lr1.5t0.10.124 Lr1.5t0.20.124 Lr1T0.20.124 … C20.118

46. DUC results Peer codeTaskROUGE-1ROUGE-2ROUGE-3ROUGE-4ROUGE-LROUGE-W 1413521122 1423511143 1434121166 1444311177 1454122244 Recall LCS

47. Results and applications DUC results (MU recall, ROUGE): –1 st place 2003 (duc.nist.gov) –1-2 place 2004 applications: –Web page summarization (WIE) –Topical crawling –Answer focused –wireless access –Cross-lingual –IR-based evaluation –Knowledge based Beyond summarization: –Classification –WSD –Spam recognition

48. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28

49. Outline Introduction Random walks and social networks LexRank Projects in language modeling and machine learning

50. Syntactic Alignment Sequence alignment for (near) paraphrasing [Barzilay&Lee 03] No syntax used Dynamic programming Different penalties for alignment depending on the syntactic similarity John talked Mary had with achat

51. Syntactic Alignment A police official said it was a Piper tourist plane and that the crash had set the top floors on fire. According to ABCNEWS aviation expert John Nance, Piper planes have no history of mechanical troubles or other problems that would lead a pilot to lose control. April 18, 2002 8212; A small Piper aircraft crashes into the 417-foot-tall Pirelli skyscraper in Milan, setting the top floors of the 32- story building on fire. Authorities said the pilot of a small Piper plane called in a problem with the landing gear to the Milan's Linate airport at 5:54 p.m., the smaller airport that has a landing strip for private planes. Initial reports described the plane as a Piper, but did not note the specific model. Italian rescue officials reported that at least two people were killed after the Piper aircraft struck the 32-story Pirelli building, which is in the heart of the city s financial district. A small piper plane with only the pilot on board crashed Thursday into a 30-story landmark skyscraper, killing at least two people and injuring at least 30. Police officer Celerissimo De Simone said the pilot of the Piper Air Commander plane had sent out a distress call at 5:50 p.m. just before the crash near Milan's main train station. Police officer Celerissimo De Simone said the pilot of the Piper aircraft had sent out a distress call at 5:50 p.m. (11:50 a.m.) Police officer Celerissimo De Simone said the pilot of the Piper aircraft had sent out a distress call at 5:50 p.m. just before the crash near Milan's main train station. Police officer Celerissimo De Simone said the pilot of the Piper aircraft sent out a distress call at 5:50 p.m. just before the crash near Milan's main train station. Police officer Celerissimo De Simone told The AP the pilot of the Piper aircraft had sent out a distress call at 5:50 p.m. just before crashing. Police say the aircraft was a Piper tourism plane with only the pilot on board. Police say the plane was an Air Commando 8212; a small plane similar to a Piper. Rescue officials said that at least three people were killed, including the pilot, while dozens were injured after the Piper aircraft struck the Pirelli high-rise in the heart of the city s financial district. The crash by the Piper tourist plane into the 26th floor occurred at 5:50 p.m. (1450 GMT) on Thursday, said journalist Desideria Cavina. Police officer Celerissimo De Simone said the pilot of the Piper aircraft, en route from Switzerland, sent out a distress call at 5:54 p.m. just before the crash near Milan's main train station.

52. Algorithm and results Three lexical methods Two syntactic methods Generate new sentences method 4 (syntactic alignment except for stop words): –Grammaticality 3.74 –Fidelity 3.77 –on a scale from 1 to 4 Best lexical method: –Grammaticality 3.12 –Fidelity 3.07

53. Chronology recovery S1: Italian TV says the crash put a hole in the 25th floor of the Pirelli building, and that smoke is pouring from the opening. (04/18/02 12:22) S2: Italian TV showed a hole in the side of the Pirelli building with smoke pouring from the opening. (04/18/02 12:32) S3: Italian state television said the crash put a hole in the 25th floor of the Pirelli building. (04/18/02 12:42) S4: Italian state television said the crash put a hole in the 25th floor of the 30-story building. (04/18/02 12:44) S1 S2 S3 S4 S1 0 10 12 13 S2 10 0 15 16 S3 12 15 0 1 S4 13 16 1 0

54. S 2 (d=10) S 1 (d=0) S 3 (d=12) S 4 (d=13) 2 (d=12) 1 (d=3.5) time t S1S1 S2S2 S3S3 S4S4 Best representation: stop words removed

55. 1. Other Party, governmental and law enforcement authorities must take similar actions beginning from the start of next year. 2. Other Party and government agencies and judicial departments must also take similar actions early next year. 3. All other Party, Government and Judicial Departments must start similar actions at the beginning of next year. 4. Other Party, government, and judicatory departments must take similar action at the beginning of next year. 5. Other party and government departments as well as judicial departments must take similar action from the beginning of next year. 6. All other party government and judicial departments must also take similar measures from the beginning of next year. 7. Other party and judicial authorities should take similar actions from the beginning of next year. 8. Other departments of the Party, the government and the judicial departments must also take similar actions early next year. 9. Other Party and Government departments as well as judicial departments must also take similar measures from the beginning of next year. 10. The other law enforcement agencies and departments will also take part in similar proceedings from the beginning of next year. 11. Other party, governmental and judicial departments will have to take similar action from the beginning of next year. 12. Other party politics and judicial department also will have to start from next year beginning of the year to adopt similar motion. 13. Other party and judicial section must start from the beginning of year of next year taking similar action also 14. The beginning of a year for and res judiciaria as welling must from next year of other party commences assuming is similar toing the proceeding. 15. At the beginning of next year politics and judicial department other parties must also start to pick to take similar action. 16. Other party politics and the judicial department also will have to start from at the beginning of next year to take the similar action. 17. Other party policies and judicial department must also begin from early next year to take similar action. 其他党政及司法部门也必须从明年年初开始采取类似行动。 Phylogenetic Text Modeling Machine translation identification

56. t-test: p<0.05 Chinese: Levenshtein 50/50, BLEU 50/50 Arabic: Levenshtein 50/50, BLEU 48/50

58. A small plane has hit a skyscraper in central Milan, setting the top floors of the 30-story building on fire, an Italian journalist told CNN. The crash by the Piper tourist plane into the 26th floor occurred at 5:50 p.m. (1450 GMT) on Thursday, said journalist Desideria Cavina. The building houses government offices and is next to the city's central train station. Several storeys of the building were engulfed in fire, she said. Italian TV says the crash put a hole in the 25th floor of the Pirelli building, and that smoke is pouring from the opening. Police and ambulances are at the scene. Many people were on the streets as they left work for the evening at the time of the crash. Police were trying to keep people away, and many ambulances were on the scene. There is no word yet on casualties. CNN 4/18/02 12:22pm; CNN 4/18/02 12:32pm; ABCNews 4/18/02 1:00pm; MSNBC 4/18/02 1:00pm; La Stampa 4/18/02 12:45pm A small plane has hit a skyscraper in central Milan, setting the top floors of the 30-story building on fire, an Italian journalist told CNN. The crash by the Piper tourist plane into the 26th floor occurred at 5:50 p.m. (1450 GMT) on Thursday, said journalist Desideria Cavina. The building houses government offices and is next to the city's central train station. Several storeys of the building were engulfed in fire, she said. Italian TV showed a hole in the side of the Pirelli building with smoke pouring from the opening. RAI state TV reported that the plane had apparently radioed an SOS because of engine trouble. Earlier though, in Rome, the senate's president, Marcello Pera, said it "very probably" appeared to be a terrorist attack. Police and ambulances are at the scene. Many people were on the streets as they left work for the evening at the time of the crash. Police were trying to keep people away, and many ambulances were on the scene. There is no word yet on casualties. TV pictures from the scene evoked horrific memories of the September 11 attacks on the World Trade Center in New York and the collapse of the building's twin towers. "I heard a strange bang so I went to the window and outside I saw the windows of the Pirelli building blown out and then I saw smoke coming from them," said Gianluca Liberto, an engineer who was working in the area told Reuters. The building is known as the Pirelli skyscraper but the Italian tyre and cable company does not operate out of the building. It is one of the symbols of Italy's financial capital and is one of the world's tallest concrete buildings, designed between 1955 and 1960. A small plane crashed into a skyscraper in downtown Milan today, setting several floors of the 30-story building on fire. The plane crashed into the 25th floor of the Pirelli building in downtown Milan. The weather was clear at the time of the crash. Smoke poured from the opening as police and ambulances rushed to the area. The president of the Italian Senate, Marcello Pera, told Italian television it "very probably" appeared to be a terrorist attack but soon afterwards his spokesman said it was probably an accident. A transport official told Reuters the plane had reported problems with its undercarriage and was circling the city ahead of trying to land at a local airport. The Pirelli building houses the administrative offices of the local Lombardy region and sits next to the city's central train station. It is constructed of concrete and glass. The crash happened just before rush hour, as office workers were closing their day. A small airplane crashed into a government building in heart of Milan, setting the top floors on fire, Italian police reported. There were no immediate reports on casualties as rescue workers attempted to clear the area in the city’s financial district. Few details of the crash were available, but news reports about it immediately set off fears that it might be a terrorist act akin to the Sept. 11 attacks in the United States. Those fears sent U.S. stocks tumbling to session lows in late morning trading. Witnesses reported hearing a loud explosion from the 30-story office building, which houses the administrative off ices of the local Lombardy region and sits next to the city s central train station. Italian state television said the crash put a hole in the 25th floor of the Pirelli building. News reports said smoke poured from the opening. Police and ambulances rushed to the building in downtown Milan. No further details were immediately available. Un aereo da turismo, un Piper si è schiantato questo pomeriggio a Milano, poco prima delle 18, contro il grattacielo Pirelli, sede anche della Regione Lombardia (il presidente della Regione, Roberto Formigoni, è in missione ufficiale in India con una delegazione della regione). Lo si è appreso in ambienti investigativi. L' impatto sarebbe avvenuto attorno al 25/o piano dei 30 del grattacielo. Almeno sei piani alla vista risultano sventrati. I detriti sono stati lanciati dal'esplosione a una quarantina di metri intorno all'edificio. In tutta l'area attorno al grattacielo Pirelli lecomunicazioni telefoniche anche via cellulare sono interrotte o quasi impossibili. La Borsa ha sospeso la seduta serale a Piazza Affari dopo lo schianto dell'aereo da turismo, anche il presidente Bush è stato subito avvertito dell'espolosione al Pirellone.«Con molta probabilità si tratta di un attentato». Lo ha detto Marcello Pera aprendo la seduta a Palazzo Madama. Ma secondo quanto si è appreso, l'aereo da turismo era probabilmente in avaria: il pilota, infatti, avrebbe lanciato l'SOS, raccolto dalla torre di controllo di Linate.

59. Fact tracking 04/18/02 13:17 (CNN) The plane, en route from Locarno in Switzerland, to Rome, Italy, smashed into the Pirelli building's 26th floor at 5:50 p.m. (1450 GMT) on Thursday. 04/18/02 13:42 (ABCNews) The plane was destined for Italy's capital Rome, but there were conflicting reports as to whether it had come from Locarno, Switzerland or Sofia, Bulgaria. 04/18/02 13:42 (CNN) The plane, en route from Locarno in Switzerland, to Rome, Italy, smashed into the Pirelli building's 26th floor at 5:50 p.m. (1450 GMT) on Thursday. 04/18/02 13:42 (FoxNews) The plane had taken off from Locarno, Switzerland, and was heading to Milan's Linate airport, De Simone said.

60. Questions from Milan corpus 1. How many people were injured? 2. How many people were killed? (age, number, gender, description) 3. Was the pilot killed? 4. Where was the plane coming from? 5. Was it an accident (technical problem, illness, terrorist act)? 6. Who was the pilot? (age, number, gender, description) 7. When did the plane crash? 8. How tall is the Pirelli building? 9. Who was on the plane with the pilot? 10. Did the plane catch fire before hitting the building? 11. What was the weather like at the time of the crash? 12. When was the building built? 13. What direction was the plane flying? 14. How many people work in the building? 15. How many people were in the building at the time of the crash? 16. How many people were taken to the hospital? 17. What kind of aircraft was used?

61. Relative order, time to stabilize and number of incorrect or partially correct answers before stabilization Changing answers: –How many people were injured?: 40 different answers! ``no word yet on casualties/injuries'', ``20 people were taken to a nearby hospital'', ``20 to 30 people were hospitalized with iinjuries'', ``many people were injured'', ``there was no official word on the number of people injured in the building'', ``at least 20 injured were taken to hospital from the scene dozens of people had been taken to the hospital'', ``injuring dozens'', ``injuring at least 30'', ``injuring 60'', ``dozens were injured'', ``60 others were injured'', ``the number of injured, originally at 60, was revised downward Friday to 36''. Only 24 hours after the crash do agencies settle on the accurate number, namely ``36 people''.

62. Time(EST) ABCNews Source CNN FoxNews MSNBC USAToday Next Day 9:5012:4712:4912:51 13:0113:1713:4213:4614:1314:2114:2914:3214:5215:0215:2215:3115:3617:5218:1318:3518:40 9:3118:02 one deadat least two four peoplefour dead no word yet on casualties two deathsat least threeat least four no immediate reports two deaths three people killed three people dead at least twoat least three five people killed at least five at least three five reported dead Fasulo and two others killed incorrect partial correct

63. Web-based QA TREC questions –Where is Inoco based? –When was London's Docklands Light Railway constructed? –Who followed Willy Brandt as chancellor of the Federal Republic of Germany? –What is Grenada's main commodity export? TREC evaluation –Earliest conference papers (Radev & al. ANLP’2000, Prager & al. SIGIR’2000) Reranking models

64. Question Modulation TREC question set Start with initial formulation TRDR = Total Reciprocal Document Rank (range: 0 to 2.92) Evolutionary operators: mutation, permutation, crossover, drop, insert, phrase What country is the biggest producer of tungsten? 0.44 What country “biggest producer” of tungsten? 1.11 country “biggest producer of tungsten”? 1.98 Web results using Google as the backend search engine –0.4 MRR (mean reciprocal rank) Query modulation results –42% increase in TRDR (from 0.79 to 1.12)

65. Models of the Web A B a b Erdös/Rényi 59, 60 Barabási/Albert 99 Watts/Strogatz 98 Kleinberg 98 Menczer 02 Radev 03 Evolving networks: fundamental object of statistical physics, social networks, mathematical biology, and epidemiology

66. Self-triggerability across hyperlinks Document closures for information retrieval Self-triggerability [Mosteller&Wallace 84]  Poisson distribution Two-Poisson [Bookstein&Swanson 74] Negative Binomial, K-mixture [Church&Gale 95] Triggerability across hyperlinks? pjpj pipi p p’ by with from p p’ photo dream path

67. Evolving Word-based Web Observations: –Links are made based on topics –Topics are expressed with words –Words are distributed very unevenly (Zipf, Benford, self- triggerability laws) Model –Pick n –Generate n lengths according to a power-law distribution –Generate n documents using a trigram model Model (cont’d) –Pick words in decreasing order of r. –Generate hyperlinks with random directionality Outcome –Generates power-law degree distributions –Generates topical communities –Natural variation of PageRank: LexRank PageRank Hits

68. Tripartite updating Modeling classification problems using bipartite graphs Weakly supervised learning – why? –bootstrapping, co-training, active learning Spectral partitioning –Fiedler vector Singular value decomposition Random walks Tripartite updating Matrix representation Iterative power method L U F T1T1 T2T2

69. Tripartite updating Tasks: –Spam detection –Named entity classification –PP attachment –Number classification Features: –Number classification: 5 classes based on context and hobbs class Four-way or three-way classification For the same accuracy of SP and TU, TU handles twice as many labeled examples with ten times as many unlabeled examples L U F T1T1 T2T2

70. Relation extraction User gives examples of entity E1 and entity E2. Example: song = “Let it Be”, singer = “the Beatles”. System finds other songs and singers with a very minimal number of training examples. The relation may be quite different, e.g., protein-protein, organization- leader, book-author, drug-disease. Weakly supervised learning based on graphs is used.

71. Protein Regulatory Network Recognition Wnt signaling Glycogen synthase kinase-3 (GSK- 3) and CK1 (casein kinase 1) alpha phosphorylate Arm (Armadillo,  - catenin) and cause it to degrade. Axin also binds to the phosphatase PP2A PP2A activity inhibits Wnt signaling Hsu 1999, Li 2001, Yanagawa 2002, Liu 2002, Nusse 2003

72. Method and Results Medline: –“signal transduction” as MeSH major topic and “Wnt” or “AKT” or “Beta-catenin” as words 3300 papers extracted by Carlos Santos 441 putative proteins (“X is a protein”, “the X protein” “X verbs”) Verbs: Bind associate interact activate repress inhibit upregulate regulate downregulate complex dimerize localize bound regulate stabilize control translocate antagonize amplify transduce trigger

73. Syntax in Statistical Machine Translation Noisy channel model: assume that a source sentence has to be translated into a target language sentence Goal: find Obvious problems can be fixed with syntax (?) JHU 02 and 03 projects (Franz Och, Jan Hajic, Dan Gildea + others) Solution using log-linear combination of features

74. Setup Given: a Chinese sentence+ The top 1000 candidate translations in English Parse all of these Compute features: monolingual, bilingual, syntax-free, and syntactic Evaluation using BLEU (BiLingual Evaluation Understudy) Example: –Is the number of constituents across languages the same? –Is the english tree grammatical? –Are the two sentences of comparable length? Feature combination –Use a greedy maxbleu algorithm

75. 中国十四个边境开放城市经济建设成就显著 NRCDMNN VV China 14 border open cities economic achievements marked CLP QP NP VPNP IP Chinese parse tree

76. 1. fourteen chinese open border cities make significant achievements in economic construction 2. xinhua news agency report of february 12 from beijing - the fourteen chinese border cities that have been opened to foreigners achieved satisfactory results in their economic construction in 1995. 3. according to statistics, the cities achieved a combined gross domestic product of rmb 19 billion last year, an increase of more than 90 % over 1991 before their opening. 4. the state council successively approved the opening of fourteen border cities to foreigners in 1992, including heihe, pingxiang, hunchun, yining and ruili, and permitted them to set up 14 border economic cooperation zones. 1. significant accomplishment achieved in the economic construction of the fourteen open border cities in china 2. xinhua news agency, beijing, feb. 12 - exciting accomplishment has been achieved in 1995 in the economic construction of china 's fourteen border cities open to foreigners. 3. statistics have indicated that these cities produced a combined gdp of over 19 billion yuan last year, an increase of more than 90 %, compared with that in 1991 before the cities were open to foreigners. 4. in 1992, the state council successively opened fourteen border cities to foreigners. these included heihe, pingxiang, huichun, yining, and ruili. meanwhile, the state council also gave its approval to these cities to establish fourteen border zones for economic cooperation. 1. in china, fourteen cities along the border opened to foreigners achieved remarkable economic development 2. xinhua news agency, beijing, february 12 - the economic development in china 's fourteen cities along the border opened to foreigners achieved gratifying results in 1995. 3. according to statistics, these cities completed a gross domestic product in excess of rmb 19 billion in last year, an increase of more than 90 % over 1991 ( the year before they were opened ). 4. in 1992, the state council successively approved fourteen cities along the border to be opened to foreigners, which included hei he, pingxiang, hunchun, yining and ruili etc. at the same time, these cities were also given approvals to set up fourteen border @-@ economic @-@ cooperation zones. 1. economic construction achievement is prominent in china 's fourteen border opening up cities. 2. xinhua news agency, beijing, february 12 - delightful economic construction result was achieved in china 's fourteen border opening up cities in 1995. 3. according to statistics, gdp registered over 19 billion yuan last year in those cities, over 90 % higher than those of year 1991 before opening up. 4. fourteen border cities like heihe, pingxiang, huichun, yinin, and ruili etc were approved successively by the state council in 1992 as the cities opening to the outside world, setting up of fourteen border economic cooperation zones in these cities were also approved simultaneously. 1. china 's 14 open border cities marked economic achievements 2. xinhua news agency, beijing, february 12 chinese 14 border an open city 1995 economic development to achieve good results 3. according to statistics, the city last year 's gross domestic product ( gdp ) over 19 billion yuan, and opening up of more than 90 % growth in 1991. 4. the state council in 1992 has approved the heihe, pingxiang, huichun, yining and ruili, 14 border cities as an open city, and the city also approved a total of 14 border economic cooperation. Multiple references

77. Syntactic features (S1 (S (NP (NP (NNP china) (POS 's)) (CD 14) (ADJP (JJ open)) (NN border) (NNS cities)) (VP (VBD marked) (NP (JJ economic) (NNS achievements))))) (S1 (S (NP (CD fourteen) (ADJP (JJ chinese) (JJ open)) (NN border) (NNS cities)) (VP (VBP make) (NP (JJ significant) (NNS achievements)) (PP (IN in) (NP (JJ economic) (NN construction)))))) (S1 (NP (NP (JJ significant) (NN accomplishment)) (VP (VBN achieved) (PP (IN in) (NP (NP (DT the) (JJ economic) (NN construction)) (PP (IN of) (NP (NP (DT the) (CD fourteen) (JJ open) (NN border) (NNS cities)) (PP (IN in) (NP (NNP china)))))))))) (S1 (S (PP (IN in) (NP (NNP china))) (,,) (NP (NP (CD fourteen) (NNS cities)) (PP (IN along) (NP (DT the) (NN border)))) (VP (VBN opened) (PP (TO to) (NP (NP (NNS foreigners)) (VP (VBN achieved) (NP (JJ remarkable) (JJ economic) (NN development)))))))) (S1 (S (NP (JJ economic) (NN construction) (NN achievement)) (VP (AUX is) (ADJP (JJ prominent) (PP (IN in) (S (NP (NP (NNP china) (POS 's)) (NP (CD fourteen) (NN border))) (VP (VBG opening) (PRT (RP up)) (NP (NNS cities)))))))))

78. Flipdeps

79. PRED say APPS, PAT increase ACT rate EXT pct ACT name TWHEN January ACT Spoon RSTR Alan TWHEN recently PAT president APP Newsweek RSTR ad RSTR 5 ACT &Gen; RSTR Newsweek TR FUF CAT pp PREP LEX in NP LEX January DETERMINER none CIRCUM PARTIC PROCESS AFFECTEDAGENT CAT clause PROCESS PARTIC CREATEDAGENT CATHEADCLASSIFIERPOSSESSOR CAT LEX say TENSE past OBJECT-CLAUSE that LEX Newsweek

80. Results BLEU baseline: –31.6% Most features: –30.0%-31.8% Flipdeps: –31.8% Best single feature: –32.5% Best combination –32.9% (statistically significant improvement) Results in [Och&al.04]

81. Number classification XXXXX Summarization MEAD/CST/NIE XXXXXXXXX Lexical Web models XXXXX Statistical MT XXXXX Protein networks XXXXXX Relation extraction XXXXXXX Phylogenetic alignment XXXXXXXX QA/NSIR XXXXX Topical crawling XXXXXX XML retrieval XXX Fact tracking XXXXX multilingual multisourceUneven importanceredundantGraph structureevolvingunstructuredHard to train Manual evaluation

82. A grabbag of research problems Finding adequate representations for dynamic texts Integrating user models Using self-triggering for information retrieval Weakly supervised and active learning Robust semantic analysis Adequate models of the Web Relation extraction Syntax-based machine translation and summarization Automatic knowledge acquisition from the Web

83. Conclusion New approaches to natural language processing and information retrieval using graph-based techniques such as random walks Applications beyond NLP Highest ranked system at DUC Promising results in semi-supervised machine learning Acknowledgments: –CLAIR (Güne ş Erkan, Jahna Otterbacher, Siwei Shen, Zhu Zhang) –UROP program –NSF and NIH –Mark Newman To read more: –http://tangra.si.umich.edu/clair –http://www.summarization.com –http://www.newsinessence.com Papers: CACM 2005; JAIR 2004; EMNLP 2004; IP&M 2004; JASIST 2002, 2004, 2005; WWW 2002; AAAI 2002; SIGIR 1995, 2000; ACL 1998, 2003; HLT 2001; HLT-NAACL 2004; CIKM 2001, 2003; ANLP 1997, 2000; LREC 2002, 2004; IJCNLP 2004; CL 1998, 2002; COLING 2000, 2004

84. 05 20 ACL 2005 www.aclweb.org June 25-30, 2005 Ann Arbor, MI General chair: Kevin Knight, ISI Program co-chairs: Kemal Öflazer, Sabanci U.; Hwee Tou Ng, NUS Local chair: Dragomir Radev, U. Michigan Submission deadline: January 14

85. S VP NPVBPP INPRPNP PRP$NN Thank you for your attention ! tangra.si.umich.edu/clair