1. MusicOWL The Music Score Ontology
Jim Jones
Institut für Geoinformatik / Universitätsbibliothek (ULB), Universität Münster, Germany
Diego de Siqueira Braga
Institut für Wirtschaftsinformatik, Universität Münster, Germany
Kleber Tertuliano
Musikhochschule, Universität Münster, Germany
Tomi Kauppinen
Aalto University School of Science, Finland
WI ‘17, August 2017, Leipzig

2. Agenda Problem Statement (music score discovery)
Proposed Approach (semantic web technologies)
MusicOWL
Musci2RDF API
Music Score Portal
Future Work
Jones et al. | MusicOWL - The Music Score Ontology

3. Unknown composers?
Managing and analyzing music scores content is a known nightmare for digital libraries (information centers). Many of these libraries have a huge old music score archive, but they treat these music scores with a rather historical than musical documents.
Known composers (Telemann - Georg Philipp Telemann ( ): Brockes-Passion, Detail der Viola da Gamba-Stimme., Elgar)
Easy to find using most of text based searches
Rather historical than musical importance
Unknow composer (Charles Doisy)
- How to find such music scores inside large historical archives if the piece is unknown? -
Jones et al. | MusicOWL - The Music Score Ontology

4. Text-based Search Engines
Cello Concerto (E minor)
Op. 85
1919
Sir. Edward Elgar
Adagio, Moderato
Lento, Allegro Molto
Adagio
Allegro, Moderato, Allegro ma non-troppo …
E Minor (tonality) in Portuguese: Mi menor
Adagio: Beats per minute
Allegro: BPM
Jones et al. | MusicOWL - The Music Score Ontology

5. „Raw Data Now!“ phrases rests clefs time signatures notes dynamics
Conventional meta data on the top - but what about „the rest“?
dynamics
connections
Jones et al. | MusicOWL - The Music Score Ontology

6. Going beyond the conventional metadata …
How many voices / instruments play this piece?
How many movements does it have?
In which tonality(ies) was it written?
How many times a particular melody is played along the piece?
What is its level of complexity?
1 - What can be achieved by going beyond the metadata? Let‘s take a look what one could ask if this information is taken into consideration.
2 - How can we structure a dataset in such a way, that these questions are addressed?
Semantic Web Technologies!
Jones et al. | MusicOWL - The Music Score Ontology

7. Semantic Web Technologies – Linked Open Data
Genesis 3:14
Xxxxxx xxxxxx xxxxxxxx xxx
xxx xxx xxx. Xxxx xxxx x xx
xxx.
- God
“linked open data describes a method of publishing structured data so that
it can be interlinked and become more useful.”
Jones et al. | MusicOWL - The Music Score Ontology

8. Semantic Web Technologies – Linked Data (Benefits)
Attaches meaning to data (allowing thematic searches)
My repository!
Private data and public data can be mixed, enabling companies and individuals to make better decisions and generate innovations.
Links different kinds of data sets making the WORLD a single big repository.
How to formalize these concepts?
Jones et al. | MusicOWL - The Music Score Ontology

9. Semantic Web Technologies – Ontologies
„A set of concepts and categories in a subject area or domain that shows their properties and the relations between them.” - Oxford Dictionary
Human ≡ Man ⊔ Woman
Man ⊓ Woman ⊑ ⊥
Parent ≡ ∀hasChild.⊤
Mother ≡ Woman ⊓ Parent
TBox
John
Mary
Orphan ≡ Human ⊓ ¬∃hasParent.⊤
ChildlessHuman ≡ Human ⊓ ¬Parent
MarriedWoman ⊑ Woman ⊓ hasSpouse.⊤
Poygamist ⊑ Human ⊓ 2≥hasSpouse.⊤
HappyMan ⊑ hasSpouse.(Human ⊓ Orphan)
hasSpouse
hasDaughter
hasFather
hasMother
hasDaughter
Ontology in Computer Science: A formal datamodel to represent knowledge, based on concepts and their relationships.
Tbox axioms : Teminological Knowledge
hasSpouse -> symetric property
hasChild -> asymentric property
hasMother ⊔ hasFather ⊑ hasParent
hasSon ⊔ hasDaughter ⊑ hasChild
hasWife ⊔ hasHusband ⊑ hasSpouse
RBox
Peter
Jones et al. | MusicOWL - The Music Score Ontology

10. OWL - Web Ontology Language
Axioms
TBox: subclass relationships C ⊑ D
RBox: subproperty relationships R ⊑ S, inverse properties R-, transitivity ⊑+
ABox: facts for classes C(a), properties R(a,b), equality a=b, difference a≠b
Class Constructors
Conjunction C ⊓ D, disjunction C ⊔ D, negation ¬C of classes
Property restrictions: universal ∀R.C and existential ∃R.C
Number restrictions: ≥n R and ≤n R
Closed classes (nominals): {a}
Source: OpenHPI (Hasso Plattner Institut)
Jones et al. | MusicOWL - The Music Score Ontology

11. MusicOWL Modelling Music Scores
Issue regarding Music scores + Semantic Web techonologies
Applying OWL to musical data
Jones et al. | MusicOWL - The Music Score Ontology

12. MusicOWL Structure mo:Score mo:movement mo:movement mso:hasScorePart
mso:ScorePart
Jones et al. | MusicOWL - The Music Score Ontology

13. MusicOWL Structure mo:Score mo:movement mo:movement mso:hasScorePart
mso:ScorePart
mso:hasStaff
mso:Staff
Jones et al. | MusicOWL - The Music Score Ontology

14. MusicOWL Structure mo:Score mo:movement mo:movement mso:hasScorePart
mso:ScorePart
mso:hasStaff
mso:hasVoice
mso:Staff
mso:Voice
Jones et al. | MusicOWL - The Music Score Ontology

15. MusicOWL Structure Chord ⊑ NoteSet ⊓ ≥3hasNote.Note
Tetrad ⊑ NoteSet ⊓ (=4)hasNote.Note
mo:Score
chord:Note
mo:movement
mo:movement
chord:Note
chord:Note
mso:hasScorePart
mso:ScorePart
mso:NoteSet
mso:hasDuration
mso:hasStaff
mso:hasVoice
mso:hasNoteSet
mso:Staff
mso:Voice
mso:Duration
Jones et al. | MusicOWL - The Music Score Ontology

16. MusicOWL Structure mso:Measure mo:Score mso:hasNoteSet mo:movement
chord:Note
mo:movement
chord:Note
chord:Note
mso:hasScorePart
mso:ScorePart
mso:NoteSet
mso:hasDuration
mso:hasStaff
mso:hasVoice
mso:hasNoteSet
mso:Staff
mso:Voice
mso:Duration
Jones et al. | MusicOWL - The Music Score Ontology

17. MusicOWL Structure mso:Time Signature mso:hasTimeSignature mso:Measure
mo:Score
mo:movement
chord:Note
mso:hasNoteSet
mo:movement
chord:Note
chord:Note
mso:hasScorePart
mso:ScorePart
mso:NoteSet
mso:hasDuration
mso:hasStaff
mso:hasVoice
mso:hasNoteSet
mso:Staff
mso:Voice
mso:Duration
Jones et al. | MusicOWL - The Music Score Ontology

18. MusicOWL Structure eflatMinor ⊑ Key ⊓ mode.Minor ⊓ tonic.Eb mso:Time
Signature
mso:hasTimeSignature
mso:Measure
mso:hasKey
ton:mode
mo:Score
ton:Mode
ton:Key
mo:movement
chord:Note
chord:Note
mso:hasNoteSet
mo:movement
ton:tonic
chord:Note
chord:Note
mso:hasScorePart
mso:ScorePart
mso:NoteSet
mso:hasDuration
mso:hasStaff
mso:hasVoice
mso:hasNoteSet
mso:Staff
mso:Voice
mso:Duration
Jones et al. | MusicOWL - The Music Score Ontology

19. MusicOWL Structure mso:Time Signature mso:hasTimeSignature mso:Measure
mso:hasKey
ton:mode
mo:Score
ton:Mode
ton:Key
mo:movement
chord:Note
chord:Note
mso:hasNoteSet
mo:movement
ton:tonic
chord:Note
chord:Note
mso:hasScorePart
mso:Clef
mso:ScorePart
mso:NoteSet
mso:hasClef
mso:hasStaff
mso:hasVoice
mso:hasNoteSet
mso:hasDuration
mso:Staff
mso:Voice
mso:Duration
Jones et al. | MusicOWL - The Music Score Ontology

20. MusicOWL Structure mso:Time Signature mso:hasTimeSignature mso:Measure
mso:hasKey
ton:mode
mo:Score
ton:Mode
ton:Key
mo:movement
chord:Note
chord:Note
mso:hasNoteSet
mo:movement
ton:tonic
chord:Note
chord:Note
mso:Dynamic
mso:hasScorePart
mso:Clef
mso:hasDynamic
mso:ScorePart
mso:NoteSet
mso:hasClef
mso:hasStaff
mso:hasVoice
mso:hasNoteSet
mso:hasDuration
mso:Staff
mso:Voice
mso:Duration
Jones et al. | MusicOWL - The Music Score Ontology

21. MusicOWL Structure mso:Time Signature mso:hasTimeSignature mso:Measure
mso:hasKey
ton:mode
mo:Score
ton:Mode
ton:Key
mo:movement
chord:Note
chord:Note
mso:hasNoteSet
mo:movement
ton:tonic
chord:Note
chord:Note
mso:Dynamic
mso:hasScorePart
mso:Clef
mso:hasDynamic
mso:ScorePart
mso:NoteSet
mso:hasClef
mso:hasArticulation
mso:hasStaff
mso:hasVoice
mso:hasNoteSet
mso:hasDuration
mso:Articulation
mso:Staff
mso:Voice
mso:Duration
Jones et al. | MusicOWL - The Music Score Ontology

22. Proof of Concept (Historical Music Scores)
Data structure very complex… how to create data?
Jones et al. | MusicOWL - The Music Score Ontology

23. Dataset Generation PDF, JPEG MusicXML OMR (Sibelius) Music2RDF API
* ULB Münster: 1k music scores from the last 5 centuries
- Many scores written by unknown local composers
- Scores had a rather historical than musical importance.
* Sibelius (by AVID) – choice based on existing license
* MusicXML
- Lacks semantics, e.g. tonalities / circle of fifths
Not easily interoperable (query via XPATH possible)
MusicXML
OMR (Sibelius)
Music2RDF API
Jones et al. | MusicOWL - The Music Score Ontology

24. Dataset Generation - Challenges
OMR often confuses lyrics with notes or articulations
* PDF perfect
* Old German writting (Fraktur?)
OMR inaccurate with old music scores
(expert check needed)
Jones et al. | MusicOWL - The Music Score Ontology

25. WWU Music Score Portal Live demo:
GraphDB Triple Store (Free version)
Search criteria: C C C (whole)
Expected result: PAGE 29 – Notendruck, Movement 23, measure 1, staff 2, voice 4
Jones et al. | MusicOWL - The Music Score Ontology

26. SPARQL Example 1 – Notes Search
mso: <
chord: <
note: <
SELECT ?score ?measure WHERE {
?score mso:hasScorePart ?part .
?part mso:hasMeasure ?measure .
?measure mso:hasNoteSet ?noteset1 .
?noteset1 mso:hasNote ?note1 .
?note1 chord:natural note:A .
?noteset1 mso:nextNoteSet ?noteset2 .
?noteset2 mso:hasNote ?note2.
?note2 chord:natural note:D .}
Jones et al. | MusicOWL - The Music Score Ontology

27. SPARQL Example 2 – Durantion Search
mso: <
chord: <
note: <
SELECT ?score ?measure WHERE {
?score mso:hasScorePart ?part .
?part mso:hasMeasure ?measure .
?measure mso:hasNoteSet ?noteset1 .
?noteset1 mso:hasDuration ?duration1.
?duration1 a mso:Half .
?noteset1 mso:nextNoteSet ?noteset2 .
?noteset2 mso:hasDuration ?duration2 .
?duration2 a mso:Quarter .}
Jones et al. | MusicOWL - The Music Score Ontology

28. SPARQL Example 3 – Phrase Search
mso: <
chord: <
note: <
SELECT ?score ?measure WHERE {
?score mso:hasScorePart ?part .
?part mso:hasMeasure ?measure .
?measure mso:hasNoteSet ?noteset1 .
?noteset1 mso:hasNote ?note1 .
?note1 chord:natural note:B .
?noteset1 mso:hasDuration ?duration1 .
?duration1 a mso:Half .
?noteset1 mso:nextNoteSet ?noteset2 .
?noteset2 mso:hasNote ?note2.
?note2 chord:natural note:E .
?noteset2 mso:hasDuration ?duration2 .
?duration2 a mso:Half.}
Jones et al. | MusicOWL - The Music Score Ontology

29. SPARQL Example 4 – Chords with 4 Notes
mso: <
chord: <
note: <
SELECT DISTINCT ?score ?measure (COUNT(?note) AS ?size) WHERE {
?score mso:hasScorePart ?part .
?part mso:hasMeasure ?measure .
?voice a mso:Voice .
?voice mso:hasNoteSet ?noteset .
?measure mso:hasNoteSet ?noteset .
?noteset mso:hasNote ?note .
?note chord:natural ?natural .}
GROUP BY ?score ?measure ?voice ?noteset HAVING (?size = 4)
Jones et al. | MusicOWL - The Music Score Ontology

30. Wrapping up
Usage of Semantic Web technologies on Music Score discovery.
Semi-automatic approach for Music Score data set creation.
WWU Music Score Portal
Jones et al. | MusicOWL - The Music Score Ontology

31. Future Work Improve data set creation (support other input formats)
Evaluate the possibility of implementing RDF export in existing Music Notation Software.
Your suggestions?
Jones et al. | MusicOWL - The Music Score Ontology

32. Vielen Dank! Portal: http://linkeddata.uni-muenster.de/musicportal/
MusicOWL:
Music2RDF:
Dataset:
Images used in this presentation:
Jones et al. | MusicOWL - The Music Score Ontology