Presentation on theme: "Struggling or Exploring? Disambiguating Long Search Sessions"— Presentation transcript:
1 Struggling or Exploring? Disambiguating Long Search Sessions Ahmed Hassan, Ryen White, Susan Dumais and Yi-Min Wang
2 Web Search Query Taxonomy (Broder, 2002) Informational queries: The purpose of such queries is to find information assumed to be available on the web in a static form (Broder, 2002) Users query search engines in order to accomplish tasks issuing multiple queries as they attempt to accomplish tasks (Jones and Klinkner, 2009)NavigationalTransactionalInformational
3 Moving from Queries to Sessions At the session/task level, informational search can be:Directed SearchExploratory SearchClosed-endedOpen-endedSingle-facetedMulti-FacetedIn exploratory search, users generally combine querying and browsing strategies to foster learning and investigation (White and Roth, 2009)
4 Long Sessions: Exploring or Struggling? Users are engaged in an open-ended and multi-faceted information-seeking task to foster learning and discovery.StrugglingUsers are experiencing difficulty locating the required information. Note that struggling may not necessarily result in failure
7 Characterizing Exploring vs. Struggling Behavior Query Similarity Queries are more different from the first query in struggling sessions.
8 Characterizing Exploring vs Characterizing Exploring vs. Struggling Behavior Queries Transition StrategiesWhen exploring, addition & removal are more popular and substitution is less popular.Adding Keyword, Removing Keywords, and Substituting Keywords(morphological variations, spelling corrections and Semantic variations).
9 Characterizing Exploring vs. Struggling Behavior Clicks Clicks are more when exploring. Difference gets larger as the session progresses.
10 Characterizing Exploring vs. Struggling Behavior Dwell Time Dwell time is longer when exploring. Last query accounts for a large proportion of dwell time when struggling.
11 Characterizing Exploring vs. Struggling Behavior Topics The likelihood of exploring vs. struggling varies significantly depending on the topic.
12 Predicting Session Type Query FeaturesNumber of queries issued in sessionQuery length in number of charactersQuery length in number of wordsTime between queriesNumber of manually typed queriesNumber of clicked queriesClick FeaturesNumber of clicks in sessionNumber of clicks per queryNumber of queries with no clicksTotal dwell time in sessionDwell time per clickDwell time per queryTime to first clickNumber of unique clicked URLsNumber of unique clicked domainsQuery Transition FeaturesSimilarity between queriesNumber of terms that exactly match the previous queryNumber of added termsNumber of removed termsNumber of substituted termsNumber of query generalizationsNumber of query specificationsSearch History FeaturesNumber of query impressionsQuery clickthrough rateQuery success clickthrough rateQuery quickback clickthrough rateEntropy of click distributionTopic FeaturesVisited URLs topicNumber of unique topics per sessionTopic distribution entropy
13 Exploring vs. Struggling Prediction Accuracy improves as more behavioral information is available.
14 Feature ImportanceFeature contribution varies depending on the point where we make the prediction.10 points if feature ranked first, 9 if ranked second, etc. 0 points if ranked beyond the first 10 features
15 Implication on Success At a high level, user behavior in exploring and struggling is similar:Multiple consecutive related queriesMultiple queries is a good thing when exploring (engagement)Multiple queries is a bad thing when struggling (effort)
16 Success PredictionIntegrating the session type into search success models significantly improves performance.