1. Object Denotation Environment
EuroCODE ~ Complete
Object Denotation Environment
A Component-based Semantic Knowledge Platform
Erfahrungen mit einem EU-Proposal aus dem MultiMedia Lab
Guerino Mazzola

2. Autumn 2002
The Topos of Music Geometric Logic of Concepts, Theory, and Performance
In collaboration with
Stefan Göller and Stefan Müller
Contributions by
Carlos Agon, Moreno Andreatta, Gérard Assayag, Jan Beran, Chantal Buteau, Roberto Ferretti, Anja Fleischer, Harald Fripertinger, Jörg Garbers,Werner Hemmert, Michael Leyton, Emilio Lluis Puebla, Mariana Montiel Hernandez, Thomas Noll, Joachim Stange-Elbe, Hans Straub, Oliver Zahorka
1368pp/CD-ROM
ISBN
ToM

3. ... to survive!
Apply!
in order to survive

4. EU 6th FP Project Category 6th Framework Program
-> Priority 2 = IST = Information Society Technologies
1st Call (Dec.2002) Activity code = IST ~
Semantic-based knowledge systems multimedia content mining, distributed computing platforms, mastery of complex information spaces, visualisation of high-dimensional objects, new formal models, covering foundational component-level as well as system-level research
-> Instrument = STREP =
Special Targeted REsearch and innovation Project (new results/knowledge -> develop products)
-> STREP Activity = RTD + innovation-related project management activities demonstration activities
EU 6th FP
Instrument = STRIP =
Special Targeted Research and Innovation Project

5. 1. Knowledge is ordered access to information
3 principles

6. 2. Knowledge is controlled by multimedia objects
3 principles

7. 3 principles 3. The body of knowledge evolves in a cyclic process
Production
3 principles
Reflection
Dissemination
Documentation

8. presto®
presto
Here, we illustrate the theory on the jazz CD „Synthesis“ which I recorded in Its entire structure, in harmony, in rhythmics, and in melodics, was deduced and constructed by use of the composition software presto® (written for Atari computers, but now also working on Atari emulations on Mac OS X), and starting from the 26 classes of three-element motives.
This composition (a grant from the cultural department of the city of Zurich) was not recognized as a computer-generated music by the jazz critics. Only the piano part was played by myself, the entire bass and percussion part was playe by synthesizers, driven by the presto® application via MIDI messages.
During the production of this composition (Synthesis is a four-part, 45-minute piece), I never felt inhibited in my piano playing, in the contrary, it was a great pleasure to collaborate with complex structures of rhythm or melody, objects of a complexity that human percussionists would never be able to play from a score.
1:26

9. RUBATO®
RUBATO L L S
Os X

10. concept modeling GeneralNote Pause Note Onset Duration Onset Loudness
Pitch – – – Ÿ
STRG –

11. concept modeling MakroNote Satellites GeneralNote MakroNote Ornaments
Schenker Analysis
Satellites
GeneralNote –
Onset
Loudness
Duration
Pitch
Note
STRG Ÿ
Pause
concept modeling
MakroNote

12. form semiotics ◊ Den Forms Types E R Mono(E ) Diagrams (Den /E )
F-Name
Den ◊
Forms
Form
Identifier
D-Name
form semiotics
Type
Diagram
Coordinate
Types E
Mono(E )
Diagrams (Den /E ) R

13. frame spaces representation conjunction disjunction collection
FrameSpace for type:
Simple √ = ◊ ® R
simple(√) = R
representation
frame spaces
Limit √ = form-name diagram ® E
limit(√) = lim(f. diagram ® E )
conjunction
Colimit √ = form-name diagram ® E
colimit(√) = colim(f. diagram ® E )
disjunction
Power √ = fn ® E
power(√) = WSpace(fn)
collection

14. Babouska Hjelmslev‘s Babouska Principle Expression Signification
Content
Babouska
Expression
Signification
Content
Expression
Signification
Content
Expression
Signification
Content
Connotation
Motivation
Meta system

15. form semiotics Denotator
The semiotic structure (Hjelmslev stratification) of denotators:
The triadic sign structure (signifier/signification/signified)
reads from left to right
Denotator
form semiotics
Name
Form
Coordinate
Topos
Name
Type/Diagram
Identifier

16. patent Denote Render MakroNote Satellites GeneralNote – Onset Loudness
Duration
Pitch
Note
STRG Ÿ
Pause

17. multimedia science OBJECTS manipulate operate denote CONCEPTS render
parametrize
objectivize
conceive
fold
multimedia science
VECTORS
transform

18. OBJECTS
CONCEPTS
render
objectivize
fold
multimedia science
VECTORS

19. CONCEPTS
fold
multimedia science
VECTORS

20. folding

21. folding x x = (x1,x2,x3,...,xn) n-cube x* = 0.x1x2x3...xn
lexicographic ordering
quantize:
xi = 0,1,2,3,...,d
real axis x*
common ordering

22. OBJECTS
objectivize
multimedia science
VECTORS

23. pinocchio

24. books

25. Movie of Stefan Göller‘s
PrimavistaBrowser
movie
Workflow
500 Pinocchios
Animates

26. satellites Sat Satlist Sat Satlist Geo RTC —2 —*3 —2 —*3 —2 —*3 —2 —*31
satellites
RTC
URL
ETC
Terminator
Time —2
—*3
Trans
Time —2
—*3
Rot
Time —2
—*3
Scale
Time —2
—*3
Color
Time —2
—*4
Sound
Time —2 —*
Tex

27. manipulate
OBJECTS
manipulate objects

28. cockpit - espresso

29. cockpit - espresso

30. cockpit - espresso

31. The EuroCODE Architecture
Mandatory
Tools
Challenging
Prima- Vista- Browser
Info- Codette
Codette- Factory
Cockpit- Factory
XML 2 CODE
Mus-e-Lear-ning
AMO-Codette
Data-Scape
Etc.
Framework: Denotator, Form, Topoi (Ring, Module, etc.) Diagram, ...
LoGeo class library: Logical and geometric operations for predicate construction and manipulation search algorithms
Cockpit 1
Cockpit 2

32. red&t The EuroCODE R&D Strategy = RED&T Generic tools
Deliver
Challenging Components
Test
LoGeo
Extend
XML 2 CODE
AMO Codette
= PredicateBuilder (Processes)
Codette- Factory
Mus-e- Lear-ning
Data- Scape
= StructureBuilder (Architecture)
Etc.
Cockpit- Factory
= InterfaceBuilder (Interaction)
Request

33. workpackages 11 partners:
5 Universities and research centers 1 foundation 1 big company 4 SMEs
15 WP, 122 DEL 3 years,
5.8 M€ ~ 926 MM 0.3 M€ management
management-oriented packages
WORK 1 (project management)
WORK 2 (communication management)
WORK 3 (scientific coordination)
WORK 15 (transfer management)
generic workpackages
WORK 4 (Framework)
WORK 5 (PredicateBuilder)
WORK 6 (PrimavistaBrowser)
WORK 7 (CodetteFactory)
WORK 8 (CockpitFactory)
challenging component targeted packages
WORK 9 (XML2CODE)
WORK 10 (AMOCodette)
WORK 11 (DataScape)
WORK 12 (TerrainDataView)
WORK 13 (KnowledgeGate)
WORK 14 (Mus-e-Learning)
workpackages

34. management General Public EU Commission
Steering Board (with representa- tives from each partner)
Project Manager
Administration
Comunication
Scientific Coordination
Transfer Management
WORK 1
WORK 2
WORK 3
WORK 15

35. Statistik zur 1. IST-Ausschreibung Die erste IST-Ausschreibung im 6
Statistik zur 1. IST-Ausschreibung Die erste IST-Ausschreibung im 6. Forschungsrahmenprogramm endete am 24. April 2003.
Insgesamt sind Projektanträge eingereicht worden, und zwar 77,6% als Papierversion und 22,4% per elektronischer Einreichung.
Ein erster statistischer Überblick über die Anzahl der eingereichten Projektanträge - aufgeschlüsselt nach den einzelnen strategischen Zielen aus dem IST-Arbeitsprogramm und mit Verteilung auf die einzelnen Instrumente (IP, NoE, STREP und andere: CA, SSA):
CMOS und Post-CMOS: 53 gesamt (11 IPs, 8 NoEs, 29 STREPs, 5 andere) Mikro-, Nanosysteme: 82 gesamt (25 IPs, 14 NoEs, 36 STREPs, 7 andere) Breitband für alle: 80 gesamt (12 IPs, 8 NoEs, 45 STREPs, 15 andere) Mobile drahtlose Systeme: 92 gesamt (15 IPs, 15 NoEs, 50 STREPs, 12 andere) Zuverlässigkeit, Sicherheit: 93 gesamt (23 IPs, 8 NoEs, 55 STREPs, 7 andere) Multimodale Schnittstellen: 74 gesamt (23 IPs, 10 NoEs, 36 STREPs, 5 andere) Wissenssysteme: 129 gesamt (18 IPs, 21 NoEs, 80 STREPs, 10 andere) audiov. Syst., Heimplattformen: 78 gesamt (18 IPs, 7 NoEs, 48 STREPs, 5 andere) eBusiness, eGovernment: 248 gesamt (59 IPs, 19 NoEs, 116 STREPs, 54 andere) eSafety: 53 gesamt (11 IPs, 3 NoEs, 35 STREPs, 4 andere) eHealth: 170 gesamt (47 IPs, 20 NoEs, 89 STREPs, 14 andere) eLearning, Kultur: gesamt (54 IPs, 27 NoEs, 115 STREPs, 21 andere) FET vorausschauend: 30 gesamt (25 IPs, 4 NoEs, 1 STREP) Begleitmaßnahmen: 35 gesamt (4 CAs, 31 SSAs)
Summe: gesamt (348 IPs, 169 NoEs, 755 STREPs, 58 CAs, 138 SSAs, 5?) Die budgetmäßige Überzeichnung (Verhältnis von beantragtem Budget zu verfügbarem Budget) variiert in den einzelnen strategischen Zielen und liegt zwischen 5:1 und 20:1.
Gemeinsame Ausschreibung mit Priorität Produkte und Dienste 2010: 92 gesamt (52 IPs, 18 NoEs, 9 CAs, 13 SSAs)
Weiterer Zeitplan für die 1. IST-Ausschreibung: Die Europäische Kommission hat folgenden Zeitplan für die erste IST-Ausschreibung vorgelegt:
12. bis 23. Mai 2003: Begutachtung der Projektanträge
2. bis 13. Juni 2003: Anhörungen (hearings) zu den erfolgreichen (über Schwellwerten liegend) IPs und NoEs
26. Kalenderwoche (=nächste Woche): Projektkoordinator erhält Bewertungsbericht (Evaluation Summary Report) und Begutachtungsbericht geht an das IST Committee. Ab diesem Zeitpunkt können wir Ihnen nähere Auskünfte zu Ihren Projektanträgen geben (Anfragen bitte per an mit Angaben zu Antrags-Nr., Akronym, angesprochener IST Bereich 'strategic objective', Name des Antragstellers)
Juli 2003: Bilaterale Gespräche zwischen Vertretern der Mitgliedstaaten (IST Committee) und Europäischer Kommission
25. Juli 2003: Endgültige Liste der geförderten Projekte (implementation plan)
statistics

36. Schedule I
15 March Consortium Meeting
17 March Request for login and password of coordinator; necessary for all electronic proposal submission acti- vities
18 March Pre-registration (electronic) Need: Acronym, title, abstract, consortium members, some obvious code and key words
21-23 March Travel Mazzola to IRCAM for negotiations
End of March First Draft of proposal; submission to EU site: visibility for all consortium partners
14-17 April Travel to Bonn and Osnabrück

37. Schedule II
14 April Revised Draft; check with national contact point for formal correctness
15 April Final (electronic) submission of proposal
24 April, 17:00 End of submission period
8 May Acknowlegment of receipt
June Evaluation report
From July Negotiation meetings in Bruselles
From October Approval of funding

38. Evaluation Criteria II
Quality of consortium
high quality
committed and well-suited partners
complementarity
SMEs adequately addressed
Quality of management
Project management of high quality
Management of knowledge, intellectual property, and innovative activities
Mobilisation of resources
Resourses foreseen
Resources integrated -> coherence
Financial plan adequate

39. Submission form Parts A & B
Part A
A1: Proposal Nr, acronym, title, duration, call identifier, activity codes, keyword codes, free keywords, abstract (max char.)
A2: Participant portraits (9): Name of organization, address person in charge, phone number, , internet homepage
A3: Financial information: cost model of participants, costs, requested grant to budget, split among RTD, demonstration, management activities

40. Submission form Parts A & B
Part B
Front page: basic coordinates of proposal
Contents page: contents list
Proposal summary page
B.1: Scientific and technological objectives of the project and state-of-the-art (3pp)
B.2: relevance to the objectives of the IST Priority (3pp)
B.3: Potential Impact (3pp)
B.3.1: Contributions to standards
B.4: The consortium and project resources (6pp)
B.5: Project management (3pp)
B.6: Workplan (15pp) —>…
B.7: Other issues (ethical and gender issues, other EU issues)

41. Overview of plan methodology
Part B.6: Workplan
Overview of plan methodology
Gantt chart of timing of different WPs (=work packages)
Pert (=Program Evaluation Review Technique) diagram of components
Detailed work description broken down into WPs:
WP list (with: Nr., title, person months, start, end)
Deliverable list (with: Nr., title, date, nature, dissemination level: public, restricted, confidential)
Description of each WP (objectives, description, deliverables, milestones)

42. ^ * FMT U Osnabrück IRCAM Paris Kom&Alg U Bonn LIM U Milano MML
U Zürich
fabric
Lausanne
finebrain
Basel
ViewTec
Zürich
IGD FhG
CAPcom
INI-GraphicsNet Foundation
Darmstadt ^
Partner
Task
>
Module
Diagram
Functor
Denotator
Form
Satellite
Folding
LoGeo
Retrieval
Cockpit
Data-scapes
DBMS
XML etc.
integration
Knowledge
transfer
Project
Manage-ment

43. Provisional Workpackages
IRCAM Paris (IR) m/m
IR1.1 programming issues 16
IR1.2 cognitive/logical models 16
IR1.3 algebraic models 16
IR1.4 integration of concepts 10
IR1.5 denotator import/export 6
IR2.1 authoring module 10
IR2.2 GUI module 8
IR2.3 sound tools 10
FABRIC|CH Lausanne (FA)
FA1 project management 3
FA2 state-of-the-art 5
FA3 concepts + tech. choice
framework development 8
FA4 tests of framework 4
FA5 application development 8
FA6 dissemination 6
FA7 results assessment &
risk management 1
FMT Osnabrück (FM)
FM1 project management 36
FM2.1 MusEducator
concepts 12
FM2.2 MusEducator
implementation 12
FM2.3 MusEducator
product 12
FM3.1 KnowledgeGate
FM3.2 KnowledgeGate
FM3.3 KnowledgeGate
FM4.1 XML2CODE
design 12
FM4.2 XML2CODE
LIM Milano (LM) m/m
LM1.1 XML2CODE
Design 12 LM1.2 XML2CODE
implementation 12 LM2.1 SPINE (Performance)
concepts 12
LM2.2 SPINE implementation 12
LM3.1 Symbolic & Audio
design 12
LM3.2 Symbolic & Audio 12
implementation 12
MML Zürich (MM)
MM1 PrimavistaBrowser &
InfoCodette 10
MM2 denotator and
topos framework 10
MM3 LoGeo library 40
MM4 CodetteFactory 20
MM5 CockpitFactory 20
MM6 scientific coordination 36
FINEBRAIN Basel (FI)
FI1 requirement analysis 24
FI2 search space concepts 24
FI3 transfer to clients 24
VIEWTEC Zürich (VT) m/m
VT1.1 satellites for GIS
general concepts 10
VT1.2 satellites for GIS
design 18
VT2.1 TerrainDataView
implementation 18
VT2.2 TerrainDataView
product 18
K&A Bonn (KA)
KA1 algorithms for denotator
search 20
KA2 DenoBMS concepts 20
KA3 implementation of KA1 20
KA4 performance tests 12
INI Darmstadt (IN)
IN1 transfer management 18

44. &T Time LoGeo (ProcessBuilder) PRODUCT IMPLEMENTATION DESIGN
Framework (Foundation Classes)
LoGeo (ProcessBuilder)
Prima- Vista- Browser
Info- Codette
Codette- Factory
(Structure-
Builder)
Cockpit- Factory
(Interface-
XML 2 CODE
AMO
Engine
Data-Scape
Know-
ledge-Gate
&T
TerrainData
View
MusEdu-cator
IN1
FA1
FA7
FM1
FA6
PRODUCT
FI3
KA4
IR2.3
FM2.3
MM1
MM2
IR2.2
VT2.2
FM3.3
IR2.1
MM6
VT2.1
IMPLEMENTATION
LM3.2
LM2.2
KA3
FM4.2
IR1.1
FM2.2
MM3
FA4
LM1.2
FM3.2
IR1.4
FA3
FI2
LM2.1
FI1
MM4
MM5
FA5
LM3.1
FM4.1
DESIGN
IR1.2
FM2.1
KA2
VT1.1
LM1.1
FM3.1
VT1.2
IR1.3
KA1
FA2
Time

45. &T Time LoGeo (ProcessBuilder) Framework (Foundation Classes)
XML 2 CODE
AMO
Engine
Data-Scape
Know-
ledge-Gate
Framework (Foundation Classes)
LoGeo (ProcessBuilder)
Prima- Vista- Browser
Info- Codette
Codette- Factory
(Structure-
Builder)
Cockpit- Factory
(Interface-
MusEdu-cator
Time
DESIGN
IMPLEMENTATION
PRODUCT
&T
IN1
FA1
FA7
FM1
FA6
FI3
KA4
IR2.3
FM2.3
MM1
MM2
IR2.2
VT2.2
FM3.3
IR2.1
MM6
VT2.1
LM3.2
LM2.2
KA3
FM4.2
IR1.1
FM2.2
MM3
FA4
LM1.2
FM3.2
IR1.4
FA3
FI2
LM2.1
FI1
FA5
MM4
MM5
LM3.1
FM4.1
IR1.2
FM2.1
KA2
VT1.1
LM1.1
VT1.2
FM3.1
IR1.3
KA1
FA2

46. Consortium Size Scale of activities Duration Participants
at least 3 independent legal entities (9+2)
In at least 3 different Member States or Associate States (D,F,I,CH)
Scale of activities
Up to several Mega-€ (?) from a total of Mega-€ for IST (55 Mega-€ for semantic-based knowledge systems, estimated 4-5 STREPs for 2-3 Mega-€ each)
Costs are calculated from local rates per partner, no global presets!
Essentially 2 cost models: FC = Full costs and AC = Additional costs
AC = option for universities and research institutes (100% of add. costs)
FC = option for SMEs (50% of full costs)
Duration
2-3 years, or more if duly justified (3)
Participants
Universities (4)
Research institutes (1+IGD)
SMEs (3+CAPcom AG)
Knowledge transfer organizations (1)
Potential users and stakeholders (0 +IGD +CAPcom AG)

47. Evaluation Criteria I Relevance to the objective of the program
Scientific
Technical
Socio-economic (the EuroCODE platform is mandatory and essential for the construction of the kowledge society)
Political (strategic fostering of open European systems & SW)
Specified in the work packages (=WPs)
Scientific and technological excellence
Defined and well-focused objective
Clear progress beyond state-of-the-art
Proposed S&T approach adequate to target
Potential impact
Impact on reinforcing competitiveness
Added value of European level (against national)
Exploitation and dissemination ensure optimal use of results

48. Distributed Rubato
Library at IRCAM
Audio Studio
Workstation
MeloTop

50. XML EuroCODE xml document type definition (.dtd) xml document (.xml)
stylesheet (.xsl)
form (.dtx) @form
denotator (.dtx) @substance
satellite (.dtx)
<?xml version="1.0"?>
<!DOCTYPE BOOK [
<!ELEMENT BOOK (P*)>
<!ELEMENT P (#PCDATA)>
]>
<BOOK>
<P>chapter 1 - Intro</P>
<P>chapter 2 - Conclusion</P>
<P>Index</P>
</BOOK>
Personal:.Limit[Leaders, Staff];
Leaders:.Power{ProjectPerson};
Staff:.Power{ProjectPerson};
Resources:.Limit[Budget, Financing, Hardware, Processes];
Budget:.Power{Document};
Financing:.Power{Document};
Hardware:.Power{Document};
Processes:.Power{Document};

51. EuroCODE XML vs. xml documents object configurations character data
content model types
ad hoc rendering via XSL
object configurations
topoi
all categorical types
universal recursive multi- media representation (PCT patent)