1. Nathan Eagle Sandy Pentland
Learning by Listening
The OverHear project is one instance of attempting to move computers beyond simply speech recognition and towards speech understanding - specifically, it's geared toward teaching the computer an awareness of a conversation's context.
Now instead of using a wearable with all the sensors that Brian had, this project simply used a device that digitally recorded all of a user’s conversations.
Nathan Eagle Sandy Pentland

2. Overview
Omni-present “familiar” listens in a user’s conversations in a variety of contexts (working in office, chatting with girlfriend, at home, etc)
The conversations are transcribed by a commercial voice recognition engine
The agent learns the word patterns unique to a given context and creates a classification algorithm
This algorithm can be leveraged to figure out a user’s current context given new conversation data
The goal of the project is to get a computer to distinguish whether a users is in a bar chatting with friends, in an argument with his girlfriend back at his apartment, or even giving a demo for sponsors here at the lab. By wearing something that always is listening in on everyday conversations, the computer can gradually start learning models of my behavior and ultimately, after a lot of training, it can give a good guess as to current location, people involved, and even the topic of the conversation.

3. Data Collection 2 months / 30 hours of labeled conversations Labels
location
ie: home, lab, bar
persons
ie: girlfriend, officemate, advisor
type/topic
ie: argument, meeting, chit-chat

4. Initial Results 90+% accurate using only location information
Demonstrated some speaker independence (many people have common priors – esp at the media lab)
Information from correlations among classes is necessary to establish the people in the conversation (typically my officemate is not in my apartment)

5. This is a short chip to get an idea of the training phase: Over 30 hours of daily conversations were recorded and input into the algorithm over the course of a month.