Presentation on theme: "Struggling or Exploring? Disambiguating Long Search Sessions Ahmed Hassan, Ryen White, Susan Dumais and Yi-Min Wang."— Presentation transcript:
Struggling or Exploring? Disambiguating Long Search Sessions Ahmed Hassan, Ryen White, Susan Dumais and Yi-Min Wang
Web Search Query Taxonomy (Broder, 2002) Navigational Informational queries: The purpose of such queries is to find information assumed to be available on the web in a static form (Broder, 2002) Users query search engines in order to accomplish tasks issuing multiple queries as they attempt to accomplish tasks (Jones and Klinkner, 2009) Transactional Informational
Moving from Queries to Sessions At the session/task level, informational search can be: Directed SearchExploratory Search Closed-endedOpen-ended Single-facetedMulti-Faceted In exploratory search, users generally combine querying and browsing strategies to foster learning and investigation (White and Roth, 2009)
Long Sessions: Exploring or Struggling? Exploring –Users are engaged in an open-ended and multi-faceted information-seeking task to foster learning and discovery. Struggling –Users are experiencing difficulty locating the required information. Note that struggling may not necessarily result in failure
Struggling Long Sessions: Exploring or Struggling?
Exploring Long Sessions: Exploring or Struggling?
Characterizing Exploring vs. Struggling Behavior Query Similarity Queries are more different from the first query in struggling sessions.
Characterizing Exploring vs. Struggling Behavior Queries Transition Strategies Adding Keyword, Removing Keywords, and Substituting Keywords(morphological variations, spelling corrections and Semantic variations). When exploring, addition & removal are more popular and substitution is less popular.
Characterizing Exploring vs. Struggling Behavior Clicks Clicks are more when exploring. Difference gets larger as the session progresses.
Characterizing Exploring vs. Struggling Behavior Dwell Time Dwell time is longer when exploring. Last query accounts for a large proportion of dwell time when struggling.
Topics Characterizing Exploring vs. Struggling Behavior Topics The likelihood of exploring vs. struggling varies significantly depending on the topic.
Predicting Session Type Click Features Number of clicks in session Number of clicks per query Number of queries with no clicks Total dwell time in session Dwell time per click Dwell time per query Time to first click Number of unique clicked URLs Number of unique clicked domains Search History Features Number of query impressions Query clickthrough rate Query success clickthrough rate Query quickback clickthrough rate Entropy of click distribution Topic Features Visited URLs topic Number of unique topics per session Topic distribution entropy Query Transition Features Similarity between queries Number of terms that exactly match the previous query Number of added terms Number of removed terms Number of substituted terms Number of query generalizations Number of query specifications Query Features Number of queries issued in session Query length in number of characters Query length in number of words Time between queries Number of manually typed queries Number of clicked queries
Exploring vs. Struggling Prediction Accuracy improves as more behavioral information is available.
Feature Importance 10 points if feature ranked first, 9 if ranked second, etc. 0 points if ranked beyond the first 10 features Feature contribution varies depending on the point where we make the prediction.
Implication on Success At a high level, user behavior in exploring and struggling is similar: –Multiple consecutive related queries Multiple queries is a good thing when exploring (engagement) Multiple queries is a bad thing when struggling (effort)
Success Prediction Integrating the session type into search success models significantly improves performance.