1. Tefko Saracevic This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 United States License Creative Commons Attribution-NonCommercial-ShareAlike 3.0 United States License Why is relevance still the basic notion in information science? (Despite great advances in information technology & applications) Tefko Saracevic, Ph.D. Rutgers University, USA tefkos@rutgers.edu 1

2. Tefko Saracevic Fundamental concepts Relevance is a fundamental concept or notion in information science Every scholarly field has a fundamental, basic notion, concept, idea... 2 It was, but is it still?

3. Tefko Saracevic Definition relevance 1 a : relation to the matter at hand 2: the ability (as of an information retrieval system) to retrieve material that satisfies the needs of the user 3

4. Tefko Saracevic What is “matter at hand”? Context in relation to which – a problem is addressed – an information need is expressed as a question – a query for searching is formulated interaction is taking place No such thing as relevance without a context Axiom: One cannot not have a context in information interaction. 4 Relevance is ALWAYS contextual

5. Tefko Saracevic Context has to be there 5

6. Tefko Saracevic Relevance – by any other name...  Many names connote relevance e.g.: pertinent; useful; applicable; significant; germane; material; bearing; proper; related; important; fitting; suited; apropos;... & nowadays even truthful  Connotations may differ but the concept is still relevance "A rose by any other name would smell as sweet“ Shakespeare, Romeo and Juliet 6

7. Tefko Saracevic 7

8. Two worlds in information science Information retrieval (IR) systems offer as answers their version of what may be relevant – by ever improving algorithms People go their way & asses relevance – by their problem at hand, context & criteria The two worlds interact 8 Considered here: human world of relevance NOT covered: how IR deals with relevance

9. Tefko Saracevic Two large questions Why? (Part I) Why did relevance become a central notion of information science? Why still? (Part II) Why did relevance still remain a central notion? - despite advances in technology 9

10. Tefko Saracevic WHY RELEVANCE? Part I 10

11. Tefko Saracevic Bit of history Vannevar Bush : Article “As we may think” 1945Vannevar Bush :As we may think – Defined the problem as “... the massive task of making more accessible of a bewildering store of knowledge.” problem still with us & growing – Suggested a solution, a machine: “Memex... association of ideas... duplicate mental processes artificially.” Technological fix to problem 11 1890-1974

12. Tefko Saracevic Information Retrieval (IR) – definition 12 Term “information retrieval” coined & defined by Calvin Mooers, 1951 “ IR:... intellectual aspects of description of information,... and its specification for search... and systems, technique, or machines... [to provide information] useful to user ” 1919-1994

13. Tefko Saracevic Technological determinant In IR emphasis was not only on organization but even more on searching – information technology was eminently suitable for searching particularly computers Technological fix to the problem of information explosion 13

14. Tefko Saracevic Two important pioneers at IBM pioneered many IR computer applications – first to describe searching using Venn diagrams at Documentation Inc. pioneered coordinate indexing – first to describe searching as Boolean algebra 14 Mortimer Taube1910-1965 Hans Peter Luhn 1896-1964

15. Tefko Saracevic Searching & relevance Searching became a key component of information retrieval – extensive theoretical & practical concern with searching – technology uniquely suitable for searching And searching is about retrieval of relevant answers 15 Thus RELEVANCE emerged as a key notion

16. Tefko Saracevic Basic 16

17. Tefko Saracevic Why relevance? Aboutness A fundamental notion related to organization of information Relates to subject & in a broader sense to epistemology Relevance A fundamental notion related to searching for information Relates to problem-at-hand and context & in a broader sense to pragmatism 17 Relevance emerged as a central notion in information science because of practical & theoretical concerns with searching

18. Tefko Saracevic Aboutness vs. relevance 18

19. Tefko Saracevic Claims & counterclaims in IR Historically & from the outset: “My system is better than your system!” Well, which one is it? OK: Lets test it. But: – what criterion to use? – what measure(s) based on the criterion? Things got settled by the end of 1950’s and remain mostly the same to this day 19

20. Tefko Saracevic Relevance & IR testing In 1955 Allen Kent & James W. Perry were first to propose two measures for test of IR systems: – “relevance” later renamed “precision” & “recall” A scientific & engineering approach to testing 20 Allen Kent 1921 - 2014 James W. Perry 1907-1971

21. Tefko Saracevic 21

22. Tefko Saracevic Relevance as criterion for measures Precision Probability that what is retrieved is relevant – conversely: how much junk is retrieved? Recall Probability that what is relevant in a file is retrieved – conversely: how much relevant stuff is missed? 22 Probability of agreement between what the system retrieved/not retrieved as relevant (systems relevance) & what the user assessed as relevant (user relevance) where user relevance is the gold standard for comparison

23. Tefko Saracevic User relevance still... 23

24. Tefko Saracevic 24

25. Tefko Saracevic WHY STILL RELEVANCE? Part II 25

26. Tefko Saracevic changing dramatically, globally 26 Many new applications Transformations Impacts Connections New, newer, newest

27. Tefko Saracevic Social media... Twitter Facebook Instagram Linkedin Tumbrl Youtube Google+ Pinterest 27

28. Tefko Saracevic Search engines... Discovery tools 28 hamster

29. Tefko Saracevic And of course... 29 tefko

30. Tefko Saracevic Societies, Journals, Conferences... 30

31. Tefko Saracevic Users Up to the time of the Web After the Web took over the world Everybody is a user Everybody searches for everything And everything reflects their needs, fashion, behavior People... all over the globe 31 Primary & almost exclusive users were – scientists – professionals – businesses – policy makers And everything reflected their needs, behavior Professionals searched

32. Tefko Saracevic 32 Everyone...

33. Tefko Saracevic As the word is changing - so is research 33

34. Tefko Saracevic Relevance experiments – then First experiments reported in 1960 & 61 – by an IBM group – compared effects on relevance judgements of various representations In the next 50 years some 300 or so experiments conducted A variety of factors in human judgments of relevance addressed 34

35. Tefko Saracevic Relevance experiments move on Eye-tracking studies in information science first reported in 2003 Continued with studies of web and online searching Moved to include relevance in 2012 35

36. Tefko Saracevic Cognitive aspects of human information interaction make information science a good field for application of neuroscience theories & tools 36 Jacek Gwizdka, Gmunden Retreat on NeuroIS 2012 (Neuro Information Systems) Hypothesis for relevance experiments: – fundamental neural processes are associated with relevance decisions & these processes can be detected by EEG or fMRI. Marrying neuro-cognitive methods & information science

37. Tefko Saracevic Symbolically 37

38. Tefko Saracevic Types of techniques Eye-tracking – measurement of eye activity. Where do we look? What do we ignore? Functional magnetic resonance imaging (fMRI) – measures brain activity by detecting associated changes in blood flow 38 Electro-encephalography (EEG) – detects electrical activity in the brain using small, flat metal discs (electrodes) attached to the scalp

39. Tefko Saracevic Experiment detecting brain activity related to information relevance judgments – 10 subjects given news stories; looked for factual relevant information Provides experimental design & conduct but results is in next paper Won the Dr. Hermann Zemlicka Award (“most visionary paper”) – among 23 papers 39 Jacek Gwizdka, Gmunden Retreat on NeuroIS 2013

40. Tefko Saracevic Does the degree of relevance of a text document affect how it is read? YES “relevant documents tend to be read more coherently, whereas irrelevant documents tend to be scanned.” 40

41. Tefko Saracevic 41 EEG = Electro-encephalography

42. Tefko Saracevic A study from Finland – in SIGIR Forty participants viewed six terms: which is relevant for given topics? (relevant and irrelevant terms defined by “experts”) Findings: “... showed improvement up to 17% in relevance prediction based on brain signals alone.” 42

43. Tefko Saracevic Another study from Finland Nine subjects viewed images – which is relevant for a task? Findings: “the relevance of an image a subject looks at can be decoded from MEG signals with performance significantly better than chance...” 43 MEG= magnetoencephalographic

44. Tefko Saracevic “There is one more thing...”There is one more thing “I think the biggest innovation of the twenty first century will be the intersection of biology and technology. A new era is beginning... “ 44

45. Tefko Saracevic 45 But also a reality in numbers No. of people using the Internet: 2.9 billion No. of people NOT connected to the Internet: 4.4 billion (60%) of these, 3 billion live in only 20 countries No. of people in the world: 7.3 billion

46. Tefko Saracevic...... different technology... 46

47. Tefko Saracevic and relevance in its use 47

48. Tefko Saracevic 48

49. Tefko Saracevic 49 Thank you for inviting me! Franjo Christian

50. Tefko Saracevic FYI – For Your Information Presentation and paper at: http://comminfo.rutgers.edu/~tefko/articles.htm 50 URLs and references are in PowerPoint Notes – accessible after download