1. Metadata improvement for image information retrieval – ISKO 2010 1 Cologne University of Applied Sciences & prometheus Klaus Lepsky – Thomas Müller – Jens Wille Klaus Lepsky – Thomas Müller – Jens Wille Metadata improvement for image information retrieval

2. Metadata improvement for image information retrieval – ISKO 2010 2 Cologne University of Applied Sciences & prometheus Klaus Lepsky – Thomas Müller – Jens Wille Rogier van der Weyden. St. Columba Altarpiece. Adoration of the Magi. Central part. c.1455. Oil on panel. Alte Pinakothek, Munich, Germany Title: St Columba Altarpiece (central panel) Author: ROGIER VAN DER WEYDEN Date: c. 1455 Technique: Oil on oak panel, 138 x 153 cm Type: religious Form: painting Location: Alte Pinakothek, Munich Triptych with the Adoration of the Magi, so-called Columba Altarpiece, Munich, Alte Pinakothek, inv. nr. WAF 1189-1191. Center panel: ca. 140 x 153, Wings each: ca. 140 x 73 cm. Problem I

3. Metadata improvement for image information retrieval – ISKO 2010 3 Cologne University of Applied Sciences & prometheus Klaus Lepsky – Thomas Müller – Jens Wille Adoration of the magi St. Columba Altarpiece Triptych with the Adoration of the Magi St Columba Altarpiece (central panel) St. Columba Altarpiece. Adoration of the Magi. Central part Triptych with the Adoration of the Magi, so-called Columba Altarpiece Columba Altarpiece Adoration of the Magi. Central part Rogier van der Weyden Weyden, Rogier /van Rogier Pasture, Rogier /de le Rogier Roger Rogier Roger Regelet Rogelet Pasture, Rogier /de Weyden, Roger /van der Roger The prometheus database with more than 400.000 pictures from 50 different collections descriptions without any standardization weak subject indexing resulting in low recall ! Problem II

4. Metadata improvement for image information retrieval – ISKO 2010 4 Cologne University of Applied Sciences & prometheus Klaus Lepsky – Thomas Müller – Jens Wille The central Adoration of the Magi, the only depiction of this kind in Rogier's extant work, was a particularly popular subject in Cologne, for since the 12th century the city had kept relics of the Magi, its most treasured possessions, in the cathedral. The Netherlandish artist not only created this work for a patron in Cologne, but also drew inspiration from an outstanding painting done in Cologne at this period, Stefan Lochner's Altarpiece of the Patron Saints of Cologne, sometimes known as the Cathedral Altarpiece. This large retable, painted in the 1440s for the chapel of Cologne town council, unites the patron saints of the city in a magnificent, monumental work from which any trace of historical narrative has been eliminated. Rogier, perhaps at the special wish of his patron, adopted the almost central position of the Virgin Mary in the depiction of the Adoration; this was an unusual situation for her in Netherlandish art. He also echoed other motifs, including the figure of a woman seen in profile, dressed in green and with a plait of hair, who appeared on the left wing of Lochner's altarpiece and again in Rogier's Presentation; details like the sword and fluttering hatband of the youngest king; and, behind the Magi, the figure of a bearded man holding a hat in front of his chest. Traces of the late Romanesque church of St. Gereon in Cologne seem to have entered into the architecture of the temple on the right wing - the polygonal rotunda, the gallery, and the small buttresses on the outside. (http://www.wga.hu/frames-e.html?/html/w/weyden/rogier/11columb/0columb.html) Art historical texts are texts about works of art and therefore may be used as metadata for these works. Idea I Improving information retrieval of digital images by automatically connecting them with text based description

5. Metadata improvement for image information retrieval – ISKO 2010 5 Cologne University of Applied Sciences & prometheus Klaus Lepsky – Thomas Müller – Jens Wille homepage of rdk-web Idea II article Drei Könige (three kings)

6. Metadata improvement for image information retrieval – ISKO 2010 6 Cologne University of Applied Sciences & prometheus Klaus Lepsky – Thomas Müller – Jens Wille identifying corresponding text by finding names of artists in text finding names of works of art in text bringing together pictures and corresponding text identifiying names and bringing together synonymous forms matching text and descriptions of pictures developing a system for an automatic standardization of metadata and a matching algorithm for metadata and text to improve poorly standardized metadata Idea III

7. Metadata improvement for image information retrieval – ISKO 2010 7 Cologne University of Applied Sciences & prometheus Klaus Lepsky – Thomas Müller – Jens Wille wordsearcher: linguistic analysis, grammatical variants are derived to one basic form multiworder: linguistic/lexical analysis, dictionary-based identification of phrases sequencer: algorithmic identification of phrases, based on sequences of the word classes synonymer: identifying and generating of synonyms kings > king #n religious > religious #a, religion #n rogier van der weyden adoration of the magi holy kings > #a #n > holy kings Pasture, Rogier /de le > rogier van der weyden adoration of kings > adoration of the magi lingo - linguistic approach to automatic text indexing

8. Metadata improvement for image information retrieval – ISKO 2010 8 Cologne University of Applied Sciences & prometheus Klaus Lepsky – Thomas Müller – Jens Wille Adoration of Kings The Adoration of Kings The Adoration of the three holy Kings Adoration of the magi Rogier van der Weyden Weyden, Rogier /van Rogier Pasture, Rogier /de le Rogier Roger synonyms of names of pictures synonyms of names of artists importance of synonyms

9. Metadata improvement for image information retrieval – ISKO 2010 9 Cologne University of Applied Sciences & prometheus Klaus Lepsky – Thomas Müller – Jens Wille perseusMatch matching algorithm with levenshtein distance (1) grammatical variants editing distance for characters('königs', 'könige') = 1 (editing distance for characters('kings', 'king') = 1) editing distance for words('königs', 'könige') = 0 (editing distance for words('kings', 'king') = 0) because basic form('königs') = basic form('könige') (basic form('kings') = basic form('king')) (2) synonyms editing distance for characters('könige','weise') = 6 (editing distance for characters('kings','magi') = 5) editing distance for words(könige,weise)= 0 (editing distance for words(kings,magi)= 0) because synonyms('könige') contains 'weise' (synonyms('kings') contains 'magi') using word distance!

10. Metadata improvement for image information retrieval – ISKO 2010 10 Cologne University of Applied Sciences & prometheus Klaus Lepsky – Thomas Müller – Jens Wille "[...] Nic. Lancret stellte in der vor 1732 für den Marquis de Béringhen gemalten Elementenfolge das Wasser durch eine ländliche Ff.szene dar, die auch Allegorie des Liebeswerbens ist (s. Sp. 256; Abb. 52; Georges Wildenstein, L., Paris 1924, S. 70 Kat.nr. 2, s. auch Nr. 3, Abb. 5). [...] Luna­Diana als Herrin über Fischerei und Schiffahrt zeigt ein Stich von Jan Saenredam nach Hendrick Goltzius (um 1559: Abb. 22)." KünstlerIn: Hendrik Goltzius Titel: Allegorie Standort: Basel, Öffentliche Kunstsammlung Gattung: Malerei Datierung: 1611 lingo & perseus match connecting text & pictures

11. Metadata improvement for image information retrieval – ISKO 2010 11 Cologne University of Applied Sciences & prometheus Klaus Lepsky – Thomas Müller – Jens Wille Thank You for listening!