1. The Perfect Search Engine Is Not Enough
Jaime Teevan, MIT
with Christine Alvarado, Mark Ackerman and David Karger

2. Let Me Interview You! Web: Email: Files:
What’s the last Web page you visited? How did you get there?
Have you looked for anything on the Web?
What’s the last you read? What did you do with it?
Have you gone back to an you’ve read before?
Files:
What’s the last file you looked at? How did you get to it?
Have you looked for a file?

3. Overview: Understanding
Search
Directed
Introduction
Related work
Methodology
What we learned
How?
Why?
Who?
So what?
Prefer to search in steps
Because it’s easier
Step size varies by person

4. Haystack: Personal Information Storage
Web pages
Haystack
Lots of separate info, Haystack stores in central repository.
Easy to separate info from its form, easy to connect related info.
A lot of people in our group are working on getting information into the Haystack.
Talk will be about a study we did to understand how information should be gotten out of the Haystack.
Perhaps worth noting: What we found is not what we were expecting. Does not just reinforce the direction we were going with Haystack but has modified the direction.
Files
Calendar
Contacts

5. Directed Search in Haystack
What was that paper I read last week about Information Retrieval?
Haystack

6. Directed Search in Haystack
Ah yes!
Thank you.
Haystack

7. …Or Elsewhere
Ah yes!
Thank you.
“Perfect Search Engine”

8. Related Work Directed search Observational studies [Malone83]
Lab studies [Capra03, Maglio97]
Log analysis [Broder02, Spink01]
Observational studies [Malone83]
Information Seeking
Marchionini, O’Day and Jeffries, Bates, Belkin, …
Evolving information need

9. Modified Diary Study Subjects: 15 CS graduate students
Ten interviews each (2/day x 5 days)
Two question types
Last /file/Web page looked at
Last /file/Web page looked for
Supplemented with direct observation and an hour-long semi-structured interview

10. Overview: Understanding
Directed
Search
Introduction
Related work
Methodology
What we learned
How?
Why?
Who?
So what?

11. Directed Search Today Target: Connie Monroe’s office number
 Type into a search engine:
“Connie Monroe, office number”

12. What We Observed
Interviewer: Have you looked for anything on the Web today?
Jim: I had to look for the office number of the Harvard professor.
I: So how did you go about doing that?
J: I went to the homepage of the Math department at Harvard

13. What We Observed
I: So you went to the Math department, and then what did you do over there?
J: It had a place where you can find people and I went to that page and they had a dropdown list of visiting faculty, and so I went to that link and I looked for her name and there it was.

14. What We Observed
J: I knew that she had a very small Web page saying, “I’m here at Harvard. Here’s my contact information.”

15. Strategies Looking for Information
Teleporting
Orienteering

16. Why Do People Orienteer?
The tools don’t work
Easier than saying what you want
You know where you are
You know what you find

17. Easier Than Saying What You Want
Describing the target is hard
Can’t
Prefer not to
Habit
“Whichever way I remember first.”
Search for source
E.g., Your last search

18. Stay in known space Backtracking You Know Where You Are
URL manipulation
Bookmarks
History
Backtracking
Following an information scent
Never end up at a dead end

19. You Know What You Find Context gives understanding of answer
“I was looking for a specific file. But even when I saw its name, I wouldn’t have known that that was the file I wanted until I saw all of the other names in the same directory…”
Understanding negative results
“I basically clicked on every single button until I was convinced… I don’t think that it exists…”

20. Individual Strategies
Search strategies varied by individual
People who pile information take small steps
People who file information take big steps
Where was the last you found?
Inbox?
Elsewhere?

21. File or Pile
Filer
Piler

22. How Individuals Search For Files
Filers
Big steps
Irony: Those people who had their information well classified don’t take advantage of their classification to get to their information
While people who had their information all in a jumble wanted to sort through that jumble.
Similar on the Web.
Pilers were more likely to navigate to a specific site and do site search, an orienteering behavior
While filers used a general purpose search engine like Google more often.
Pilers
Small steps

23. Applying What We Learned
 Support orienteering
Advantages to orienteering
Easier than saying what you want
You know where you are
You know what you find
Individual differences in step size
Meta-info, source, flag sources with info
URL manipulation, paths apparent, all steps
Answer context, trusted sources, exhaustive
Allow for different step sizes

24. Structural Consistency Important
All must be the same to re-find the information!

25. Preserve What User Remembers
Supports orienteering for re-finding
Allows access to new information

26. More to Learn from the Data
Differences in finding v. re-finding
How organization relates to search
Importance of type ( , files and Web)
Looked at v. looked for
 Keep in mind population

27. Questions?
Teevan, J., Alvarado, C., Ackerman, M. S. and Karger, D. R. (2004). The Perfect Search Engine is Not Enough: A Study of Orienteering Behavior in Directed Search. To appear in Proceedings of CHI 2004.
(Linked from
Questions.

28. Relating How and What People only keyword search 39% of the time
Specific
General
Document
Other 47 19 41
Keyword 34 23 17
People only keyword search 39% of the time
What people look for related to how they look
Now we understand how people search and what they search for
Even looking at a subset of the orienteering behavior, we see people orienteer a lot
For example, we see people orienteer to specific documents all the time
Not surprising – think how you would get to, for example, a specific or a specific file
We see it’s a little more likely that a person would teleport when looking for general information
But while you might expect that trend to continue for specific information, instead we see that people orienteer a lot, too!
Weird. Why?
Surprise:
Orienteer to specific information

29. Relating How and Corpus
Files
Web
Other 59 42 19
Keyword 06 10 64
and files: Almost never keyword searched
Easy to associate information with document
Web: Used keyword search much more often
Additional information I don’t have time to talk about …
Available if there are questions.
We saw that how people searched relates to what they were searching for
This table shows how the corpus people were searching in related to how they searched
and files almost always orienteered
Most searches were for specific information
Most file searches were for documents
Also, easy to associate information with the document in these cases, because they knew the information
On the Web we saw more teleporting
Although keyword search is easy on Web, so people may still be orienteering a lot, it’s just getting classified as orienteering

30. Relating What and Corpus
Files
Web
Specific 39 7 33
General 10 30
Document 08 35 14
Additional information I don’t have time to talk about …
Available if there are questions.
searches were primarily for specific information
File searches were primarily for documents
Web searches were more evenly distributed