1. towards a zoomable cell abstract cell natural coordinate system Data >48.000 3D Protein Structures from PDB ? A IHGFBCDE >200.000 Images from scientific publications 1

2. Computer Graphics and Visualization TECHNISCHE UNIVERSITÄT DRESDEN Zoomable Cell Stefan Gumhold Michael Schröder Norbert Blenn Anne Tuukkanen Marcel Spehr Matthias Reimann

3. Computer Graphics and Visualization Zoomable Cell, SPP 1335, Kickoff Meeting, 9.12.08, Dagstuhl3 Goals  Data analysis  Natural coordinate system (NCS)  Mapping of images from literature to NCS  3D models of complexes in NCS  Visualization  aggregation of images, volumes and 3D models  Rendering across scale from 10  m to 1Å  Natural adjustment of visualization parameters with dynamic labeling  HCI  support for Virtual Reality environments  speech control and input device development  flexible navigation  community support through web integration  Impact  Interface life scientists „from different scales“  data aggregation and analysis platform  production of illustrative materials

4. Computer Graphics and Visualization Zoomable Cell, SPP 1335, Kickoff Meeting, 9.12.08, Dagstuhl4 Human Cells New Problems Several different instances of the same type each instance is flexible cells are treated badly before imaging very different imaging modalities are used Deformation Framework

5. Computer Graphics and Visualization Zoomable Cell, SPP 1335, Kickoff Meeting, 9.12.08, Dagstuhl5 Various Data Types cell nucleus pore complexes proteins primitives, smooth surfaces implicit surfaces height fields images: 2D, 3D, perspective

6. Computer Graphics and Visualization Zoomable Cell, SPP 1335, Kickoff Meeting, 9.12.08, Dagstuhl6 Data Augmentation  define reference models  for each dataset  scale  imaging modality  features  points  curves  regions  labeling of features  for pairs of datasets  feature mapping  additional alignment information nucleolus envelop pore

7. Computer Graphics and Visualization Zoomable Cell, SPP 1335, Kickoff Meeting, 9.12.08, Dagstuhl7 Integration of Datasets Segmentation Feature Detection Labeling non-rigid Registration

8. Computer Graphics and Visualization Zoomable Cell, SPP 1335, Kickoff Meeting, 9.12.08, Dagstuhl8 Deformation reference model

9. Computer Graphics and Visualization Zoomable Cell, SPP 1335, Kickoff Meeting, 9.12.08, Dagstuhl9 Plan to a Solution start with fully interactive tools add automation step by step with full interactivity for corrections find features that persist over different scales develop learning based segmentation approaches exploit mutual information to register datasets of different dimension and modality

10. Computer Graphics and Visualization Zoomable Cell, SPP 1335, Kickoff Meeting, 9.12.08, Dagstuhl10 Visualization Engine  protein structures  primitive splatting  tubes, surfaces  deferred shading  sorting based transparency  3d surface models  LOD based rendering  depth peeling based transparency  Images & Volumes  volume rendering  compression  transfer functions

11. Computer Graphics and Visualization Example Images

12. Computer Graphics and Visualization Query Based Exploration of Images

13. Available image information Expert labeled text (categorical) Unstructured information of related text (textual) Inherent image features (abstract description of image appearance) More reliable and structured Less reliable and structured

14. Navigation/Exploration Around 100.000 images currently available to us Even with automatic analysis one needs supporting browsing techniques If we have features that measure appropriate image similarities: – Hierarchical Browsing – Fish-Eye View

15. Hierarchical Browsing

16. Fish-Eye View

17. Methods to structure image data set By hand Automatic analysis (off-the-shelf methods) – Unsupervised (Clustering) – Supervised (Multiclass Support Vector machines) Need for appropriate problem oriented feature set

18. Image Feature Definition Vast numbers of image descriptors are available Need for general purpose image descriptors because of wide variety of image origins Standardized Multimedia content description (MPEG-7)

27. Class information from Image Features 1.Definition of semantic classes (assisted and manually, Gene Ontology labels) 2.Relation of abstract image descriptors to semantic classes (training, learning) 3.Evaluation of generalization ability

28. GoImage – Semantic Image Search Comprehensive protein-interaction mapping projects underway What is the cost of completing an interactome map and what is the best strategy for minimizing the cost? How can quality and coverage of interaction data be maximized?

29. GoImage – Semantic Image Search

31. Refinement of a search for membranes through selecting nuclear envelope p.a.