1. Danica Damljanović, Milan Agatonović, Hamish Cunningham contact: danica@dcs.shef.ac.uk FREyA: an Interactive Way of Querying Linked Data using Natural Language

2. N ATURAL L ANGUAGE I NTERFACES AND L INKED O PEN D ATA  What are NLIs?  Challenges:  NL understanding/grammar  ambiguity/expressiveness  knowledge structure/portability small ontologies, large ontologies, multiple ontologies, Linked Open Data? 30 MAY 2011 QALD-1 2

3. S HIFT IN C HALLENGES 30 MAY 2011 QALD-1 3 Closed-domainOpen-domain Find the answerDisambiguate the answer Language ambiguityData ambiguity Increase recall (Wordnet, etc.)Increase precision  Portability 5 years ago vs. today?  Heterogeneity, incompleteness, redundancy  Can one system support both?

4. FRE Y A WORKFLOW 30 MAY 2011 QALD-1 4

5. F INDING POC S 03 J UNE 2010 ESWC 2010 5

6. F INDING OC S 03 J UNE 2010 ESWC 2010 6

7. geo:City geo:State new york POC population geo:cityPopulation M APPING POC TO OC S 03 J UNE 2010 ESWC 2010 7 geo:State

8. N EW Y ORK IS A CITY 03 J UNE 2010 ESWC 2010 8

9. N EW Y ORK IS A STATE 03 J UNE 2010 ESWC 2010 9

10. L EARNING 10 POCOC (context)candidate OCfunction new yorkgeo:State- new yorkgeo:City- populationgeo:Stategeo:statePopulation- populationgeo:Citygeo:cityPopulation- IFTHEN

11. L EARNING 11

12. Q UERYING L INKED D ATA WITH FRE Y A: THE USUAL CYCLE 30 MAY 2011 QALD-1 12 for i=1 to n { Initialise the system using dataset A i (forceDialog) Train the system by asking questions Save learningModel i } Intialise the system (automatic mode) by loading learningModel i, i=1,n connect to the repository containing all A i datasets, where i=1, n

13. EVALUATION  Initialisation  System performance  Precision, recall, f-measure using MusicBrainz and DBPedia datasets  Mean Reciprocal Rank to assess the effect of learning mechanism  Analysis of failures 30 MAY 2011 QALD-1 13

14. I NITIALISATION AND THE DATASETS SIZE MusicBrainzDBpedia explicit statements14 926 841328 318 709 statements19 202 664372 110 845 entities5 490 23796 515 478 SPARQL queries30361623 initialisation time1380s (0.38h)182779s (50.77h) 30 MAY 2011 QALD-1 14

15. R ESULTS : F- MEASURE STATISTICS MusicBrainzDBPedia Training Testing Precision0.75/0.77 0.66/0.80.74/0.85 0.49/0.63 Recall0.66/0.68 0.54/0.660.58/0.66 0.42/0.54 F-measure0.70/0.74 0.59/0.710.67/0.72 0.45/0.58 not supported questions 6 911 7 reformulated questions 1 64 6 avg #dialogs per question 3.4 3.652.7 2.85 partially correct questions 1 3 12 30 MAY 2011 QALD-1 15

16. L EARNING MusicBrainzDBPedia untrained/trained MRR0.63/0.680.52/0.54 30 MAY 2011 QALD-1 16

17. C ONCLUSION  Output:  Correct answer OR  Identifying the flaws in the data?  Ranking/disambiguation algorithms to improve MRR 30 MAY 2011 QALD-1 17 Closed-domainOpen-domain End-usersApplication developer

18. Contact: danica@dcs.shef.ac.ukdanica@dcs.shef.ac.uk THANK YOU FOR YOUR ATTENTION ! QUESTIONS ? 18 Thanks to Ivan Peikov from Ontotext who helped with the configuration of OWLIM which was necessary for performing the experiments reported in this paper.

19. D EMO United States geography: http://gate.ac.uk/sale/dd/movies/mooney/html/freya.ht ml MusicBrainz: http://gate.ac.uk/sale/dd/movies/mb/html/musicbrainz.h tml DBpedia: http://gate.ac.uk/sale/dd/movies/dbpedia/html/small/fre ya.html