We think you have liked this presentation. If you wish to download it, please recommend it to your friends in any social system. Share buttons are a little bit lower. Thank you!
Presentation is loading. Please wait.
Modified over 7 years ago
MITRE © 2001 The MITRE Corporation. ALL RIGHTS RESERVED. What Works, What Doesn’t -- And What Needs to Work Lynette Hirschman Information Technology Center The MITRE Corporation USA
MITRE © 2001 The MITRE Corporation. ALL RIGHTS RESERVED. What Works... 0 If we believe Eric Brill, we should just collect and annotate data... -Since data collection seems to work better than looking for new algorithms 0 What this really means is that data collection is more cost-effective than funding research -Similarly, we might conclude that waiting for the next chip is more cost-effective than creating faster algorithms So we should all stop doing research and look for data and wait...
MITRE © 2001 The MITRE Corporation. ALL RIGHTS RESERVED. Conversational Interaction: A Case Study 0 Speech researchers have always said, there’s no data like more data 0 Many speech problems are, by definition, data- constrained: -Conversational interfaces require real(istic) data on what people will say to machines in the context of a specific application -Such application-specific data tends to be difficult (expensive) to collect -It requires simulation of interaction with a system, or a running system to collect data to build the system… 0 How do we collect millions of sentences of application specific data?
MITRE © 2001 The MITRE Corporation. ALL RIGHTS RESERVED. How to Collect Real Data Cheaply 0 Lesson from Victor Zue’s MIT Jupiter system: -Put something out there that people want to use: on-line weather information -This can be done by bootstrapping from a primitive system, using the collected data -MIT has been very successful in collecting data from real users; methodology now used by the DARPA Communicator program So to collect data to build a system, we need a system that works well enough for people to use it
MITRE © 2001 The MITRE Corporation. ALL RIGHTS RESERVED. Real Systems To Collect Real Data... 0 Building a usable system requires integration of multiple technologies: -We need ways to interface to real data sources -We need language understanding -We need intelligible generation and synthesis -We need dialogue management -We need ways to apply the techniques to a different problem domain (application portability) because otherwise, we have to do all this again for the next application 0 So collection of real data raises basic research issues
MITRE © 2001 The MITRE Corporation. ALL RIGHTS RESERVED. Error Rate Over Time in ATIS (Air Travel Info System) Sentence Transcription SL Error NL Error Word Error Error rate log scale Time (months) Understanding easier than transcription Limiting factor: understanding, not word error
MITRE © 2001 The MITRE Corporation. ALL RIGHTS RESERVED. Conclusion: What Needs to Work 0 So we can’t just wait for data -- we need to collect it 0 And to collect data, we need systems that work so that real users will use them; they must be: -Scalable to handle large amounts of data -Robust so they keep working -Fast, so people can stand to use them -Interactive and engaging, so people want to use them 0 And while we are at it, it would be nice if the systems not only supported data collection, but were able to learn interactively…
Consumer-Centric Knowledge Web A Vision of Consumer Applications of Software Agent Technology - Enabling Consumer-Centric Knowledge-Based Computing Jack.
Silvia Mosso 1 Research Connection 2009: the LUNA project 1 Prague, 7 May 2009 Research Connection 2009 Silvia Mosso LUNA: the Power of Understanding.
© 2007 Eaton Corporation. All rights reserved. LabVIEW State Machine Architectures Presented By Scott Sirrine Eaton Corporation.
The Chinese Room: Understanding and Correcting Machine Translation This work has been supported by NSF Grants IIS Solution: The Chinese Room Conclusions.
Breakout session B questions. Research directions/areas Multi-modal perception cognition and interaction Learning, adaptation and imitation Design and.
“To Build or Not to Build” by Rev. Dr. Lane R. Seitz, President Our Savior Lutheran Church Eyota, MN Tuesday, September 27, 2011.
Lecture 10: Heap Management CS 540 GMU Spring 2009.
1 Chapter 1 Why Parallel Computing? An Introduction to Parallel Programming Peter Pacheco.
Ways to Deal with Annoying People. Introduction Unfortunately, we can’t get along with everyone. And it’s a fact that throughout our life, we’ll be in.
Copyright 1999 all rights reserved Why Conduct a Task Analysis? n Uncovered Information may lead to abandoning a bad idea –The world is ready for video.
Environmental Adaptation Proposal for September 2006 CMU-Boeing Experiments.
Languages & The Media, 5 Nov 2004, Berlin 1 New Markets, New Trends The technology side Stelios Piperidis
ICANN workshop, September 14, Athens, Greece Intelligent Affective Interaction technologies and applications.
Internet Marketing The Digital World. Digital Benefits Moore’s Law – bits become cheaper Digital environments Convergence Separation between products.
Chapter 1 Introduction to Object- Oriented Programming and Problem Solving.
CS 300 – Lecture 22 Intro to Computer Architecture / Assembly Language Virtual Memory.
Logic Design Outline –Logic Design –Schematic Capture –Logic Simulation –Logic Synthesis –Technology Mapping –Logic Verification Goal –Understand logic.
User studies. Why user studies? How do we know security and privacy solutions are really usable? Have to observe users! –you may be surprised by what.
Resources Primary resources – Lexicons, structured vocabularies – Grammars (in widest sense) – Corpora – Treebanks Secondary resources – Designed for a.
1 Chapter 4. 2 Measure, Report, and Summarize Make intelligent choices See through the marketing hype Key to understanding underlying organizational motivation.
© 2022 SlidePlayer.com Inc. All rights reserved.