neue verfahren der bildverarbeitung – auch zur erfassung ... · pdf filesince 1980s...
TRANSCRIPT
© Fraunhofer HHI | 13.07.2017 | 1
Peter Eisert
Neue Verfahren der Bildverarbeitung – auch zur Erfassung von Schäden in Abwasserkanälen?
Peter Eisert
© Fraunhofer HHI | 13.07.2017 | 2
Fraunhofer is Europe‘s largest organization for applied research.
69 institutes and research units in Germany
Research cooperation around the world
24 500 staff members
Budget: more than 2.1 billion €
Research highlights: MP3 audio codec, H.264, H.265 video codec, LTE mobile communications standard
Fraunhofer-Gesellschaft
Peter Eisert
© Fraunhofer HHI | 13.07.2017 | 3
Fraunhofer Institute for Telecommunications,Heinrich Hertz Institute, HHI
1928 Founding of Heinrich Hertz Institute in BerlinSince 1960s Research on fiber optical transmissionSince 1980s Research on digital video codingSince 2003 Member of Fraunhofer-GesellschaftSince 2009 Branch site in Goslar, Germany
Peter Eisert
© Fraunhofer HHI | 13.07.2017 | 4
Facts and Figures
Last updated: 2015
Peter Eisert
© Fraunhofer HHI | 13.07.2017 | 5
Photonic Network and Systems
Photonic Components
Fiber Optical Sensor Systems
Wireless Communications and Networks
Vision and Imaging Technologies
Video Coding and Analytics
Departments
Photo: fotolia.com/SeanPavonePhoto | Edit: Fraunhofer HHI
Peter Eisert
© Fraunhofer HHI | 13.07.2017 | 6
Image Processing Pipeline
geometry
motion / deformation
semantic data
inspection
Peter Eisert
scene acquisition rendering & display
Processing &
Analysis
© Fraunhofer HHI | 13.07.2017 | 7
2D Scene Capturing
Projection of 3D scene into 2D image plane
Loss of 1 dimension
Still images or video
Higher and higher resolution
Peter Eisert
© Fraunhofer HHI | 13.07.2017 | 8
Image Reprojection
Camera calibration determines parameters of projection
Image can be reprojected onto any other surface
But viewing position cannot be changed
Peter Eisert
perspective camera fisheye camera
© Fraunhofer HHI | 13.07.2017 | 9
Image Reprojection
Peter Eisert
perspective camera fisheye camera
© Fraunhofer HHI | 13.07.2017 | 10
Spectral Sensitivity
Regular camera and humans: RGB colors
Multi-spectral imaging can provide material properties
Information invisible to humans
Peter Eisert
RGB image of tumor tumor @ 500nm
NIR imaging
thermal imaging
© Fraunhofer HHI | 13.07.2017 | 11
3D Scene Capturing
3D geometry can be estimated from multiple views (photogrammetry)
Ill-posed problem
Requires determination of corresponding points between views
Epipolar geometry constrains search space
Peter Eisert
xl xr
disparity d=xl-xr
z~
triangulation
© Fraunhofer HHI | 13.07.2017 | 12
Depth Estimation from Stereo
Images captured with high resolution (10-20 MP)
Dense reconstruction from 2 or more views
Arbitrary objects
Reconstruction of entire head or persons (incl. hair, eyes, etc.)
Peter Eisert
© Fraunhofer HHI | 13.07.2017 | 13
High Detail 3D Reconstruction
Peter Eisert
[Blumenthal, Computers & Graphics 2014]
© Fraunhofer HHI | 13.07.2017 | 14
Textured 3D Model from 2 Views
Peter Eisert
© Fraunhofer HHI | 13.07.2017 | 15
3D Models in Security Applications
Peter Eisert
augmented reality visualization security gates
© Fraunhofer HHI | 13.07.2017 | 16
High Quality 3D Reconstruction
Peter Eisert
© Fraunhofer HHI | 13.07.2017 | 17
Aerial Surveillance, Object Detection
Peter Eisert
© Fraunhofer HHI | 13.07.2017 | 18
Analysis of Vegetation and Landscapes
Peter Eisert
© Fraunhofer HHI | 13.07.2017 | 19
Inspection of Laser Welds
Peter Eisert
© Fraunhofer HHI | 13.07.2017 | 20
3D Recording of Footprints
Peter Eisert
© Fraunhofer HHI | 13.07.2017 | 21
Support of Large Range of Object Sizes
Peter Eisert
© Fraunhofer HHI | 13.07.2017 | 22
Tracking and Motion Estimation
Object / camera motion can be determined from point correspondences
3D motion requires additional priors (e.g. static scene, deformation model)
Dense / sparse motion models
Peter Eisert
camera motion object motion
© Fraunhofer HHI | 13.07.2017 | 23
3D Motion Tracking of Robot
3D tracking of motion path
Corresponding points with epipolar constraint
Assumption of near cylindrical pipe shape
Peter Eisert
© Fraunhofer HHI | 13.07.2017 | 24
Enhanced Surface Textures
Projection onto 3D cylinder surface
Correction of lighting artifacts
Compared to simple unwrap
higher resolution
less stitching artifacts
Peter Eisert
Panoramounwrapping with 3D motion
© Fraunhofer HHI | 13.07.2017 | 25
Deformable Surface Tracking
Peter Eisert
© Fraunhofer HHI | 13.07.2017 | 26
Deformable Surface Tracking
Peter Eisert
© Fraunhofer HHI | 13.07.2017 | 27
3D Surface Tracking for Navigated Surgery
Peter Eisert
© Fraunhofer HHI | 13.07.2017 | 28
Image Analysis with Machine Learning
Deep neural networks highly successful for many image analysis tasks
image classification (e.g. detection of defects in pipes)
scene labeling / segmentation
image synthesis
…
Peter Eisert
[ImageNet]
© Fraunhofer HHI | 13.07.2017 | 29
Image Classification
Peter Eisert
(labeled) Training data Classifier
“Fish“
Trai
nin
gTe
stin
g
© Fraunhofer HHI | 13.07.2017 | 30
Convolutional Neural Network
Peter Eisert
© Fraunhofer HHI | 13.07.2017 | 31
Explanation of Black Box
Black box classifier
Explanation of deep neural network decisions
Visualization via relevance feedback
Peter Eisert
© Fraunhofer HHI | 13.07.2017 | 32
Detection of Morphing Attacks
Detection of morphing attacks in ABC Gates
Anomaly detection in passport images
Sematic image forensics
Machine learning
Peter Eisert
Original Morph
© Fraunhofer HHI | 13.07.2017 | 33
Creation of Face Morphs
Morphing pipeline with exchangeable components
Feature detection
Warping
Blending
Automatic alignment of face images based of feature points/feature lines
Automatic adaptive blending that prevents visible cuts
Peter Eisert
Ori
gin
al im
age
sou
rce:
pic
s.st
ir.ac
.uk
Fiel
d w
arp
ing
Piec
ewis
e af
fin
e
Alp
ha
ble
nd
ing
(α=
.5)
Ou
r ad
apti
ve
© Fraunhofer HHI | 13.07.2017 | 34
Detection of Face Morphs
Morphing attack detection using Deep Neural Networks
Creation of training and test datasets>1500 original images
Image pre-processing (filtering, noise) to increase variations in dataset
Comparison of different network architectures
Peter Eisert
© Fraunhofer HHI | 13.07.2017 | 35
VR / AR for Interaction and Assistance
Peter Eisert
© Fraunhofer HHI | 13.07.2017 | 36
Dynamic 3D Scene Capturing
Peter Eisert
© Fraunhofer HHI | 13.07.2017 | 37
TimeLab
Peter Eisert
© Fraunhofer HHI | 13.07.2017 | 38
Omnidirectional Video
High resolution 360° video
No stitching artifacts
2D and 3D
Real-time end to end streaming
Peter Eisert
© Fraunhofer HHI | 13.07.2017 | 39
Sample Productions
Peter Eisert
© Fraunhofer HHI | 13.07.2017 | 40
Summary
Image processing and analysis is a powerful tool for many applications
Significant progress in imaging hardware
High-resolution, ubiquitous and cheap cameras
VR / AR glasses
Increased computing power (mobile devices, graphics boards)
Machine learning enables novel AI technologies
New applications in water resource management ?
Peter Eisert
© Fraunhofer HHI | 13.07.2017 | 41
Fraunhofer-Institut für Nachrichtentechnik,Heinrich-Hertz-Institut, HHI
WE PUT SCIENCEINTO ACTION.
Contact:
Peter [email protected]+49 30 31002 614
Einsteinufer 3710587 Berlin
Peter Eisert