1. Struggling or Exploring? Disambiguating Long Search Sessions
Ahmed Hassan, Ryen White, Susan Dumais and Yi-Min Wang

2. Web Search Query Taxonomy (Broder, 2002)
Informational queries: The purpose of such queries is to find information assumed to be available on the web in a static form (Broder, 2002) Users query search engines in order to accomplish tasks issuing multiple queries as they attempt to accomplish tasks (Jones and Klinkner, 2009)
Navigational
Transactional
Informational

3. Moving from Queries to Sessions
At the session/task level, informational search can be:
Directed Search
Exploratory Search
Closed-ended
Open-ended
Single-faceted
Multi-Faceted
In exploratory search, users generally combine querying and browsing strategies to foster learning and investigation (White and Roth, 2009)

4. Long Sessions: Exploring or Struggling?
Users are engaged in an open-ended and multi-faceted information-seeking task to foster learning and discovery.
Struggling
Users are experiencing difficulty locating the required information. Note that struggling may not necessarily result in failure

5. Long Sessions: Exploring or Struggling?

6. Long Sessions: Exploring or Struggling?

7. Characterizing Exploring vs. Struggling Behavior Query Similarity
Queries are more different from the first query in struggling sessions.

8. Characterizing Exploring vs
Characterizing Exploring vs. Struggling Behavior Queries Transition Strategies
When exploring, addition & removal are more popular and substitution is less popular.
Adding Keyword, Removing Keywords, and Substituting Keywords(morphological variations, spelling corrections and Semantic variations).

9. Characterizing Exploring vs. Struggling Behavior Clicks
Clicks are more when exploring. Difference gets larger as the session progresses.

10. Characterizing Exploring vs. Struggling Behavior Dwell Time
Dwell time is longer when exploring. Last query accounts for a large proportion of dwell time when struggling.

11. Characterizing Exploring vs. Struggling Behavior Topics
The likelihood of exploring vs. struggling varies significantly depending on the topic.

12. Predicting Session Type
Query Features
Number of queries issued in session
Query length in number of characters
Query length in number of words
Time between queries
Number of manually typed queries
Number of clicked queries
Click Features
Number of clicks in session
Number of clicks per query
Number of queries with no clicks
Total dwell time in session
Dwell time per click
Dwell time per query
Time to first click
Number of unique clicked URLs
Number of unique clicked domains
Query Transition Features
Similarity between queries
Number of terms that exactly match the previous query
Number of added terms
Number of removed terms
Number of substituted terms
Number of query generalizations
Number of query specifications
Search History Features
Number of query impressions
Query clickthrough rate
Query success clickthrough rate
Query quickback clickthrough rate
Entropy of click distribution
Topic Features
Visited URLs topic
Number of unique topics per session
Topic distribution entropy

13. Exploring vs. Struggling
Prediction Accuracy improves as more behavioral information is available.

14. Feature Importance
Feature contribution varies depending on the point where we make the prediction.
10 points if feature ranked first, 9 if ranked second, etc. 0 points if ranked beyond the first 10 features

15. Implication on Success
At a high level, user behavior in exploring and struggling is similar:
Multiple consecutive related queries
Multiple queries is a good thing when exploring (engagement)
Multiple queries is a bad thing when struggling (effort)

16. Success Prediction
Integrating the session type into search success models significantly improves performance.

17. Thanks!
Ahmed Hassan