1. Making Search Personal with NEPOMUK and Refinder
Dr Leo Sauermann, CEO getrefinder.com

2. Making Search Personal with NEPOMUK and Refinder
Hello, my Name is
Can’t remember things
Realizing the Semantic Desktop since 2002
Making Search Personal with NEPOMUK and Refinder

3. Brigitte
Secretary of Prof. Andreas Dengel, DFKI KM Lab.
Loves Skiing.

4. Making Search Personal with NEPOMUK and Refinder
personal search…. ok
Making Search Personal with NEPOMUK and Refinder

5. Making Search Personal with NEPOMUK and Refinderok
Brigitte
Secretary of Prof. Andreas Dengel, DFKI KM Lab.
Loves Skiing.
Making Search Personal with NEPOMUK and Refinder

6. Secretary of Prof. Andreas Dengel, DFKI KM Lab.
Loves Skiing.

8. How are they related?

11. Alps
ski
Secretary of Prof. Andreas Dengel, DFKI KM Lab.
Loves Skiing.

12. Alps
ski

13. Alps
ski

14. Leo Sauermann prototype: “Gnowsis”, 2003 http://leobard.net 28.11.2012
Making Search Personal with NEPOMUK and Refinder

15. Prof Stefan Decker Name: “Semantic Desktop” Idea: EU project
Name: “Semantic Desktop” Idea: EU project
Making Search Personal with NEPOMUK and Refinder

16. Making Search Personal with NEPOMUK and Refinder
DFKI
Ansgar Bernardi Andreas Dengel Team of artificial intelligence scientists
Making Search Personal with NEPOMUK and Refinder

17. Making Search Personal with NEPOMUK and Refinder
Mandriva Linux
Sebastian Trüg Stéphane Laurière
KDE-Linux operating system
Making Search Personal with NEPOMUK and Refinder

18. Making Search Personal with NEPOMUK and Refinder
Making Search Personal with NEPOMUK and Refinder

19. …to conduct research about the future of knowledge work, how to empower an individual knowledge worker to better exploit their personal information space and to maintain fruitful communication with others.
Making Search Personal with NEPOMUK and Refinder

20. Making Search Personal with NEPOMUK and Refinder
Making Search Personal with NEPOMUK and Refinder

21. Desktop includes
documents, images, music, videos, address book, personal calendar, todo lists, note-taking tools, s, …
+ future stuff people may need
Making Search Personal with NEPOMUK and Refinder

22. Design Goals Make information reusable, Detached from the application
Link items, organize items
Compare items, gather them in lists
Use information in project/task context
Automatic, based on artificial intelligence
Cross-plattform on KDE-Linux, Windows, MacOsX establish open source semantic desktop standard
Making Search Personal with NEPOMUK and Refinder

23. organize things using tags and stars
Making Search Personal with NEPOMUK and Refinder

24. Making Search Personal with NEPOMUK and Refinder
Example: Dolphin
Assign tags, search for tags
Making Search Personal with NEPOMUK and Refinder

25. Making Search Personal with NEPOMUK and Refinder
Example: Dolphin
Assign tags, search for tags
Making Search Personal with NEPOMUK and Refinder

26. Gwenview

27. plugin app file explorer browser extension api data base service data
Making Search Personal with NEPOMUK and Refinder

28. NEPOMUK Dazzler Blueprint
Making Search Personal with NEPOMUK and Refinder

29. Making Search Personal with NEPOMUK and Refinder
Making Search Personal with NEPOMUK and Refinder

30. plugin app file explorer browser extension api data base service data
Making Search Personal with NEPOMUK and Refinder

31. Works to integrate all data on the web.
Making Search Personal with NEPOMUK and Refinder

32. NIE Ontologies for rich data index
Create a rich index to build flexible applications on top
Integrated & related
“Contact” is used in s, im messages, author of documents, …
Reusable
Representation in RDF, XML, KEY/VALUE possible
Standards-Based
adaption of EXIF, vcard, vcal, MPEG
332 class types:
contacts, calendaring, task management, downloads, file types, generic multimedia stuff, exif information for images, ID3 Tags for MP3s, messages and , file sharing, … personal information management: projects, tasks, organizations, places, ….
866 properties of these classes:
structured data, a lot for calendar and multimedia formats. …
Making Search Personal with NEPOMUK and Refinder

33. RDF Database: Virtuoso
Stores all structured data and fulltext in a searchable index
Fulltext Database
Data Database
RDF Graph Database
Lorem ipsum… C S P O A
file://x
size
512
author
nepomuk://hh-45
Lorem ipsum dolor…
Making Search Personal with NEPOMUK and Refinder

34. Index the Data: crawlers
Initial indexing of the data into the store
Strigi C++, Linux strigi.sourceforge.net Aperture Java aperture.sourceforge.net
Making Search Personal with NEPOMUK and Refinder

35. Making Search Personal with NEPOMUK and Refinder
Making Search Personal with NEPOMUK and Refinder

36. Making Search Personal with NEPOMUK and Refinder
Improving the Indexer
Initial indexing took 100% CPU time, little additional benefit over predecessor
Alan Grimes, annoyed by 100% CPU time taken by indexer: ml
Indexer takes less CPU and memory now.
…What the fuck is this semantic bullshit? Computers most notoriously cannot think. They are purely storage and processing systems...
Making Search Personal with NEPOMUK and Refinder

37. it’s the user experience, stupid
Multimedia and PIM use.
More end-user applications.
Making Search Personal with NEPOMUK and Refinder

38. Filter in the File Explorer
Multiple facets can be used
Making Search Personal with NEPOMUK and Refinder

39. Semnotes http://smile.deri.ie/projects/semn/details 28.11.2012
Making Search Personal with NEPOMUK and Refinder

40. Semnotes
Making Search Personal with NEPOMUK and Refinder

41. Making Search Personal with NEPOMUK and Refinder
Making Search Personal with NEPOMUK and Refinder

42. Making Search Personal with NEPOMUK and Refinder
Making Search Personal with NEPOMUK and Refinder

43. establish a semantic desktop standard community!
Making Search Personal with NEPOMUK and Refinder

45. Filter Big Bang Theory Episodes
Season 3, The Vengeance Formulation
Making Search Personal with NEPOMUK and Refinder

46. Making Search Personal with NEPOMUK and Refinder
Filter Music
ATB Albums, No Silence
Making Search Personal with NEPOMUK and Refinder

47. Open files by tag
Making Search Personal with NEPOMUK and Refinder

48. Save files with tag&description
Making Search Personal with NEPOMUK and Refinder

49. Making Search Personal with NEPOMUK and Refinder
Multimedia Use Cases
Beyond TV Shows
I'm using Nepomuk extensively everyday, …, to manage all my media, with the help of Nepoogle, Drop2TagIcon and Neposideckick, and results are amazing. Things like locate duplicate files or manage TVShows are only some samples of the capabilities of Nepomuk at desktop level. …I have 2133 video files … 35 are related to the actress 竹内結子 (Takeuchi Yūko), 11 files released in year 2012, and 3 have in common actor 阿部寛 (Abe Hiroshi). t=98468&sid=f90bf929b2f1faae2540e053 bab6f904&start=15#p221521
Synonyms, multi-linguality
Relations between files and actors
Actors and their relations are copied from web-databases and stored on semantic desktop to enhance local queries
Making Search Personal with NEPOMUK and Refinder

50. Making Search Personal with NEPOMUK and Refinder
Zeitgeist
Plans to integrate Zeitgeist and NEPOMUK for KDE
Making Search Personal with NEPOMUK and Refinder

51. Finding duplicate files
select distinct ?u1 ?u2 where { ?f1 a nexif:Photo . ?f2 a nexif:Photo . ?f1 nfo:hasHash ?h . ?f2 nfo:hasHash ?h . ?f1 nie:url ?u1 . ?f2 nie:url ?u2 . filter(?f1!=?f2) . }
Making Search Personal with NEPOMUK and Refinder

52. Filter & Drill down on data
Virtuoso RDF Store
SELECT * WHERE { ?s nco:fullname ?name . ?name bif:contains "'brigitte*'". }
Brigitte
Structured queries
Fulltext queries
Inference Engine: rules, ontologies, generates new facts
Making Search Personal with NEPOMUK and Refinder

53. Answer complex questions
Through natural language processing, full-text analysis and the index of relations
“Whats the phone number of the KM Group Secretary?”
 Not possible to answer using fulltext search. Only SIRI or Watson have the superpowers to…
Making Search Personal with NEPOMUK and Refinder

54. Map the query to the graph
The answer is found by spreading activation and partial queries
Property <phone_number>
Class <secretary>
Instance <KM-Group>
(no result)
<KM-Group><unit><SecretaryAD>
<SecretaryAD><phone_number> „ “
[1]
[2]
[3]
[4]
instances:
<SecretaryAD>, …
“phone number of the KM-Group secretary”
[5]
Combining Metadata and Document Search with Spreading Activation for Semantic Desktop Search, Schuhmacher, K.; Sintek, M. & Sauermann, L. ESWC 2008
Making Search Personal with NEPOMUK and Refinder

55. PIMO to build a personal network
Project
Meeting
Location
Concetps & Associations to organize
Connected with documents
Inspired by basic human concepts
Kaiserslautern
Andreas
Kickoff
nepomuk
Brigitte
Documents
Papers
Projects
nepomuk
Research
Files s
Inbox
Todo
SAP
nepo
Karlsruhe
Contacts
Andreas Dengel
Brigitte Selzer
Sven Schwarz
Associations are like a human person thinks
Semantics give meaning to the elements in a computer
Lets focus on a detail here.
Relations with documents
Individual s can be categorized to a project
PIMO - a Framework for Representing Personal Information Models

56. Supports Orienteering Behavior
“users go somewhere, look around, know where they are, go on”
Filtering
keeping orientation
Long-term users stopped using web-search interface
spatial
Teevan, J.; Alvarado, C.; Ackerman, M.S. & Karger, D.R. The perfect search engine is not enough: a study of orienteering behavior in directed search CHI 2004, pp
Semantic Desktop for PIM

57. Making Search Personal with NEPOMUK and Refinder
NOKIA
Search, on the phone
„Brings the power of computers to mobile devices“
N900 and N810 are Maemo Linux-powered
Nokia M9 Phone, millions shipped
Tracker indexing framework
maemo.nokia.com
Making Search Personal with NEPOMUK and Refinder

58. Making Search Personal with NEPOMUK and Refinder
Philip Van Hoof,
Last week
inter-app information sharing on a high level using a inter-app defined and shared ontology with a query language suitable for the purpose, SPARQL, combined with a efficient file system meta data indexer. We delivered that and proof that its high quality software exists in the fact that a few million N9 users, yes a few million, are using it right now and yet there are no massive protests on forums about corrupted data: Tracker's SPARQL endpoint is today tested and robust high quality software. I salute Jürg [Billeter] for that.
mail.gnome.org/archives/tracker-list/2012-November/msg00009.html
Making Search Personal with NEPOMUK and Refinder

59. Making Search Personal with NEPOMUK and Refinder
Nepomuk, in the cloud
Making Search Personal with NEPOMUK and Refinder

60. Connect and Crawl + Connectors: Scanning of any cloud app.
Dropbox
Evernote
Contacts
RSS
Twitter
Docs
Basecamp
Box.net
SkyDrive
Campfire
MindMeister
Calendar +
November 2012

61. Refinder Cloud Search
Scalable, Full-text, Sorting, Faceting, Filtering, Highlighting, customizable Ranking.
Scalable full-text and phrase search and/or term conjunction
Sorting and faceting on any item attribute
Customizable relevancy ranking
Results including item attributes, result highlighting, previews, and filter options
November 2012

62. Refinder Cloud Search
More cloud apps, growing data, small screenspace
Desktop Data  Personal Data Desktop Apps  Cloud Apps
November 2012

63. Making Search Personal with NEPOMUK and Refinder
Timeline of NEPOMUK
and adoption rates
developers
users
4 guys 3k
Refinder
~100s
0.6 mio
Nokia maemo & MeeGo
~10
25k
Aperture
~10
million
NEPOMUK & Gnome
~10
millions
NEPOMUK-KDE 4.0
~60
100
EU-IST Research
Making Search Personal with NEPOMUK and Refinder

64. Making Search Personal with NEPOMUK and Refinder
Join! nepomuk-kde mailinglist
Making Search Personal with NEPOMUK and Refinder

65. Making Search Personal with NEPOMUK and Refinder
I am looking for a partner to scale Refinder up nepomuk-kde mailinglist
Making Search Personal with NEPOMUK and Refinder