towards a zoomable cell

QuickTime™ and aTIFF (LZW) decompressor

are needed to see this picture.

abstract cellnatural coordinate systemData

>48.000 3D ProteinStructures from PDB

QuickTime™ and a

TIFF (LZW) decompressor

are needed to see this picture.

Qui ckTime™ and a

TIF F (L ZW) decompressor

are needed to see this picture.

?

A IHGFB C D E

>200.000 Images from scientific publications

1

Computer Graphicsand Visualization

TECHNISCHEUNIVERSITÄTDRESDEN

Zoomable CellStefan Gumhold Michael Schröder

Norbert Blenn Anne Tuukkanen

Marcel Spehr Matthias Reimann

Computer Graphicsand Visualization

Zoomable Cell, SPP 1335, Kickoff Meeting, 9.12.08, Dagstuhl 3

Goals Data analysis

Natural coordinate system (NCS) Mapping of images from literature to NCS 3D models of complexes in NCS

Visualization aggregation of images, volumes and 3D models Rendering across scale from 10m to 1Å Natural adjustment of visualization parameters with

dynamic labeling

HCI support for Virtual Reality environments speech control and input device development flexible navigation community support through web integration

Impact Interface life scientists „from different scales“ data aggregation and analysis platform production of illustrative materials

Computer Graphicsand Visualization

Zoomable Cell, SPP 1335, Kickoff Meeting, 9.12.08, Dagstuhl 4

Human CellsNew Problems

Several different instances of the same typeeach instance is flexible

cells are treated badly before imagingvery different imaging modalities are used

Deformation Framework

Computer Graphicsand Visualization

Zoomable Cell, SPP 1335, Kickoff Meeting, 9.12.08, Dagstuhl 5

Various Data Types

cell

nucleus

pore

complexes

proteinsprimitives, smooth surfaces

implicit surfaces

height fields

images: 2D, 3D, perspective

images: 2D, 3D, perspective

Computer Graphicsand Visualization

Zoomable Cell, SPP 1335, Kickoff Meeting, 9.12.08, Dagstuhl 6

Data Augmentation define reference models for each dataset

scale imaging modality features

points curves regions

labeling of features for pairs of datasets

feature mapping additional alignment information

nucleolusenveloppore

Computer Graphicsand Visualization

Zoomable Cell, SPP 1335, Kickoff Meeting, 9.12.08, Dagstuhl 7

Integration of Datasets

Segmentation

FeatureDetection Labeling

non-rigidRegistration

Computer Graphicsand Visualization

Zoomable Cell, SPP 1335, Kickoff Meeting, 9.12.08, Dagstuhl 8

Deformation

reference model

Computer Graphicsand Visualization

Zoomable Cell, SPP 1335, Kickoff Meeting, 9.12.08, Dagstuhl 9

Plan to a Solution

start with fully interactive toolsadd automation step by step with full

interactivity for correctionsfind features that persist over different

scalesdevelop learning based segmentation

approachesexploit mutual information to register

datasets of different dimension and modality

Computer Graphicsand Visualization

Zoomable Cell, SPP 1335, Kickoff Meeting, 9.12.08, Dagstuhl 10

Visualization Engine protein structures

primitive splatting tubes, surfaces deferred shading sorting based transparency

3d surface models LOD based rendering depth peeling based transparency

Images & Volumes volume rendering compression transfer functions

Computer Graphicsand Visualization

Example Images

Computer Graphicsand Visualization

Query Based Exploration of Images

Available image information

• Expert labeled text (categorical)

• Unstructured information of related text (textual)

• Inherent image features (abstract description of image appearance)

More reliable and structured

Less reliable and structured

Navigation/Exploration

• Around 100.000 images currently available to us• Even with automatic analysis one needs supporting browsing techniques• If we have features that measure appropriate image similarities:

– Hierarchical Browsing– Fish-Eye View

Hierarchical Browsing

Fish-Eye View

Methods to structure image data set

• By hand• Automatic analysis (off-the-shelf methods)

– Unsupervised (Clustering)– Supervised (Multiclass Support Vector machines)

• Need for appropriate problem oriented feature set

Image Feature Definition

• Vast numbers of image descriptors are available• Need for general purpose image descriptors because of wide variety of

image origins• Standardized Multimedia content description (MPEG-7)

Class information from Image Features

1. Definition of semantic classes (assisted and manually, Gene Ontology labels)

2. Relation of abstract image descriptors to semantic classes (training, learning)

3. Evaluation of generalization ability

GoImage – Semantic Image Search

Comprehensive protein-interaction mapping projects underway

What is the cost of completing an interactome map and what is the best strategy for minimizing the cost?

How can quality and coverage of interaction data be maximized?

GoImage – Semantic Image Search

GoImage – Semantic Image Search

Refinement of a search for membranes through selecting nuclear envelope p.a.