Web Search Results Visualization: Evaluation of Two Semantic Search Engines Kalliopi Kontiza, Antonis Bikakis, University College London, Department of Information Studies

Overview ● Semantic search engines improve the accuracy of search results: - by understanding the meaning and context of terms as they appear in web documents, - by using semantics to represent and process the user’s queries and the web data. ● Other parameters that define the quality of a search engine: - its performance, -its usability, -the presentation of the search results

Overview “Whether and how do semantic search engines improve the visualization of search results, enhancing the search experience? “

Structure of the Presentation ●Methodology ●Background information (InfoVis) ○1. Analytical Inspection ●Experiment ○2. The User Evaluation ●Results of the User Evaluation ●Discussion

Methodology ●An analytical Inspection area of heuristic evaluation ‘the Visual Information-Seeking Mantra’, Shneiderman (1996) ● A user-oriented evaluation study ●Interactive Information Retrieval (IIR) systems: semantic search engines Sig.ma and KngineSig.maKngine

1. Background Information Information Visualization -Works as umbrella for all kinds of visualizations -Best applied for exploratory tasks -Ultimate purpose : amplify cognition -Requires well formed data

2. Analytical Inspection 1.OverviewTask-domain 2.Details on demandinformation 3.Filter out/Highlightactions supported 4.Relateby an information 5.Historyvisualization system, that 6.Export users wish to perform ➔ Layout of the SUIs: Control (ie more), Input (ie search box)Features of SERP Personalised (ie move content) interface Informational (ie result item)

2. Analytical Inspection ( Questionnaire videos) Questionnaire videos

3. Design & Set up of the User Evaluation A.Variables a. Dependent 1. Overview i. Task domain information actions 2. Details on demand 3. Filtering out ii. User Satisfaction4. Relate 5. History b. Independent6. Export Predefined queries: a) Web Transactional Navigational b) Informational Factual Source

3. Design & Set up of the User Evaluation B.Questionnaire - Online, closed-type questions, 5 point Likert, - Pre assigned queries presented in playlist of videos - Sections: A. Introduction B. Evaluation C. About C.Sample - 83 participants

4. Results of the User Evaluation ●55% male, 45% female ●34% age group, 51% age group, 11% age group ●67% had used more than one search engines ●86% rated their search skills with 4 and 5 on 5 point Likert scale

The comparative presentation of user ratings for the visualization of the task domain information action criteria

4. Results of the User Evaluation ● Informational tasks received 71%, 61% ● Visualization was ranked 4th, 73% graded it with 4 and 5 on 5 point Likert scale ● User-satisfaction perceived acceptance: o good satisfaction for history and export but o more expectations from overview and details on demand

5. Discussion -Different perspective to view data -Non linear and dynamic visualization favoured Q1. The visualization of the search results in semantic search engines improves the understanding of data and supports the user in assessing search results.

5. Discussion - Visualization was ranked important in tasks as Overview, Details on demand, Filter out, Relate -Careful consideration regarding additional visual representations Q2. Semantic search engines make more effective use of visualization in displaying search results providing a better user experience.

5. Discussion Q3. Semantics improve the visualization of search results. -User can filter out due to the semantically organised data in properties and values -The visualization of that task receives high preference amongst users

5. Discussion Q4. The visualization of search results in semantic search engines provides a better search and thus user’s satisfaction. -Users satisfied in general with the visualizations of the semantic search engines -Visualization of search results plays a significant role in shifting users searching behaviour

Conclusions ● While visualization methods used by semantic search engines improves user understanding of the results, the extent to which visualization methods are used in such search engines can be improved even more. ● User experience rated positively but user satisfaction not accomplished in all cases

Further questions to investigate A more in-depth analysis needs to be performed on the collected data: ● Are there any differences in the results of the user evaluation for the different types of queries, considering the type of data that is searched or the complexity of the query? ● Is there any correlation, for example, between the user characteristics and the obtained data? ● Could a standardized cognitive and ability test help us further investigate the relationship between information visualization in semantic search engines and knowledge visualization ?

Thank you for your attention