digital image increase - universität klagenfurt · horst bischof redundancy for aerial computer...

Post on 02-Oct-2020

0 Views

Category:

Documents

0 Downloads

Preview:

Click to see full reader

TRANSCRIPT

1 Horst Bischof Redundancy for aerial computer vision

Exploiting redundancy for reliable aerial computer vision

Horst Bischof ���������� ����������������

2 Horst Bischof Redundancy for aerial computer vision

Digital Image Increase

3 Horst Bischof Redundancy for aerial computer vision

Images Worldwide

� �������������� ��� ������������� ��������������� ������������� ��������������� �� !�� ��������������

�"��#�����$!��������%�����

4 Horst Bischof Redundancy for aerial computer vision

Terrestrial Image Acquisition

� �������& ��� �%�'�� (���������!�����)���*� ��� �+� �,����!� ���� �����*�� ��-����� �����

5 Horst Bischof Redundancy for aerial computer vision

Aerial Photogrammetry

� -����������.�� �)#������ � �,� �������+� �&/� �01������� �� ��-�!����2�#����3���)���!��&,+�� (����!�����)(���*� ��� �+��

6 Horst Bischof Redundancy for aerial computer vision

New Sensor Platforms

� -�� ������� �� 4��3����)�� ���� ����������5����5��� �

��� ������5� ��*&�'����� ����� ��+�

6 Towards Fully Automatic Photogrammetric Reconstruction Using Digital Images Taken From UAVs./7�$�������"�%78�1�!���"�#7�8������ �"�97�:�����"���5�47�;�����7�����<�����$&*=&�����7>��

7 Horst Bischof Redundancy for aerial computer vision

Airborne vs. MAVs

#������ ��� ��-�!,�

������3�?����

#/%�*�� �3�/@��@����3�������

A���7���� A����

�&,B�(C�! �3���9��B�(2��!�

�&,B�(�7��! �3���9��B�(��!�

8 Horst Bischof Redundancy for aerial computer vision

Applications

9 Horst Bischof Redundancy for aerial computer vision

Application: Virtual Habitat

[Leberl et al. IEEE Computer 2010]

10Horst Bischof Redundancy for aerial computer vision

Application: Architecture and Cultural Heritage

[Zebedin 2010], [Irschara 2010]

11Horst Bischof Redundancy for aerial computer vision

Application: Construction Site Monitoring

12Horst Bischof Redundancy for aerial computer vision

Application: Mining

13Horst Bischof Redundancy for aerial computer vision

Outline

D 3D Reconstruction �E & �1� 1������!�#� ���E ,�����& �����E =�51�5�����

D Semantic Classification �E *3���������������� ���E =��1����� ���E 41����

D Discussion & Outlook

14Horst Bischof Redundancy for aerial computer vision

3D Reconstruction

15Horst Bischof Redundancy for aerial computer vision

Structure from Motion

6,7�F� ��"�G/1 �!� ��5����������� �1� ������!�1������� �5�5�����H1�����I7�*�7,7"�����>�6#7�*���������� ���7"�G%�1���!�5�����' �������5���5���!���I7�$J-%"�����@>�6F7�&������� ���7"�G*�� ����1��!B�03�������*�� ��-����� �������,I7�&$��=/*9"�����>�6&7�/���'���� ���7"�G:1�5���=�!������,��I7�-%*="����2>�6J7K#7�4���!�� ���7"�G:1�5���=�!�������-��15�����,��I7�0--%"�����>�

5�����H1����� �1��������5�!�����

�,�!�5�����!�������� � ����L��,�� �1� 1���

$F*��� M��*���

16Horst Bischof Redundancy for aerial computer vision

Structure from Motion

-�!�����M��� � ����

�,�&������& �1� 1���

$!�����-��������5������

17Horst Bischof Redundancy for aerial computer vision

Structure from Motion

X1

X4 X

3 X2

X6

X7

X5

-�!������R2, t2, [K2]

minimize g )R"t"[K]"X+�

-�!������R1, t1, [K1]

-�!������R3, t3, [K3]

6F7�&������-%*=�N�2� 1 ����>�

�7�7��������!��������7�����,���� ���

����7�����1����'���

18Horst Bischof Redundancy for aerial computer vision

�*�� �-*��&���51����K��3�

Structure from Motion

Feature Extraction

Coarse Matching

Detailed Matching

Geometric Verification

Images Pose Prior

Geometric Estimation

Epipolar Graph

Image Overlap

Local Descriptors

Camera poses 3D points Matches

Calibration Pose Prior

-�!�1 ������5�,����/��� �� 1���)-�,/+�

19Horst Bischof Redundancy for aerial computer vision

Aerial Photogrammetry

� �����������!������ �����(�����!O�� C�P��������� �@"@���3�2"@����3���� (����!��&,�

20Horst Bischof Redundancy for aerial computer vision

Global Depth Map Optimization

5� �� ��!����1��� �� ��!�

���

��� ��������������

���

)),(()(

)),(( ���� 77�!� �������� ��)F--+�

�������

)( 77��� ���%��� ���)�%+����1����� ��B�

� ������������� ����!�� ������ �5���� �1 �����������

6�7�*����� ���7"�G/�����3����!1�� ��������� �1�1��!1� K������������!�I7�0--%"����C>�

-�� �1�1��0������#�!�� ���/�������

21Horst Bischof Redundancy for aerial computer vision

KKEY view

22Horst Bischof Redundancy for aerial computer vision

DEPTH map

min

5� �� ��!����1��� �� ��!�

���

��� ��������������

���

)),(()(

23Horst Bischof Redundancy for aerial computer vision

DEPTH map

min

5� �� ��!����1��� �� ��!�

���

��� ��������������

���

)),(()(

24Horst Bischof Redundancy for aerial computer vision

25Horst Bischof Redundancy for aerial computer vision

Primal-Dual Optimization D �������1� �������3��� !�� ���������!����������5���

����� K��5�����!��K51�������� �!�6-��!�����"�*��������>�D 4�� 1���������������� !����������������������3�

������!��' �����'��� �1� 1���D -������������ �������������5����!����������������

���5'�����1�������*���

26Horst Bischof Redundancy for aerial computer vision

Distributed Visual SLAM

Set of updateable, geo-referenced virtual cameras

PPose

Low framerate, for map extension and relocalization

Full framerate, “cheap” features, for visual servoing

Very little data needed, provide only data necessary for the

current environment Maintain global map,

use “expensive” features!

27Horst Bischof Redundancy for aerial computer vision

Dense Reconstruction On-the-Fly

28Horst Bischof Redundancy for aerial computer vision

Dense Reconstruction On-the-Fly

29Horst Bischof Redundancy for aerial computer vision

Semantic Segmentation

30Horst Bischof Redundancy for aerial computer vision

Motivation – Semantic Interpretation

Water ?

Building ?

Tree ?Street ?

10 cm/pixel

31Horst Bischof Redundancy for aerial computer vision

Motivation

�… a lot of overlapping images

3D information

32Horst Bischof Redundancy for aerial computer vision

Fusion - Model

D �%K;��5�������!�5���6F�������@>�����

- Robust against outliers - Preserves sharp edges

33Horst Bischof Redundancy for aerial computer vision

Fusion - Model

D 03 ������ ��!1� ���������� ����6Q����?>��

D ��

Some observations

34Horst Bischof Redundancy for aerial computer vision

Fusion – Semantic Interpretation

D =����!�� ����� ��������������!�� ��D -�� �1�1�����!1�� ������ ���*� ��!�5���6*����2>�

� aggregated

refined

35Horst Bischof Redundancy for aerial computer vision

Fusion - Color and Height

Color: Wavelet-Fusion Height: TV-Fusion

36Horst Bischof Redundancy for aerial computer vision

��

Some observations in ortho-view

37Horst Bischof Redundancy for aerial computer vision

Evaluation – Redundant Interpretation

Remember evaluation on single images: Graz 89.5%, Dallas 92.5%

38Horst Bischof Redundancy for aerial computer vision

Large-Scale Results

Building

Green Area

Waterbody

Tree

Streetlayer

Dallas, 4 tiles, each 240 x 240 m, 15 cm

39Horst Bischof Redundancy for aerial computer vision

Building

Green Area

Waterbody

Tree

Streetlayer

SF 10 x 3 tiles 2500 x 700 m 15 cm

40Horst Bischof Redundancy for aerial computer vision

Large-scale Results

Building

Green Area

Waterbody

Tree

Streetlayer

Graz, 7 km2, 155 images, 20 x 10 tiles, 8 cm

41Horst Bischof Redundancy for aerial computer vision

Conclusions & Future Work D /���������5��,������� �1� ������� ������5�!����

� ���� ��������� ����D /������������5��,�����!� ��������

�D =�51�5����������B�8��� ���1 �!� �� ���

�D =��1� ���!�1 � ��������semantic 3D model

D 41 1��B��������#/%��E 4��3����E -�����

42Horst Bischof Redundancy for aerial computer vision

Videos/Code/Papers see

'''7��1@���7���������7��7 1����7� �

43Horst Bischof Redundancy for aerial computer vision

Acknowledgments 41�5������5�5���B��D 4R4�1�5���*��.�� �9�����M�5���%��� �����#� ��5������-�!�1 ���

%����D 4$�K$��*�����!��1�5�5����:#%$��1�5���*��.�� �%#�*�"�*0�/&�&"�

-MF&�=�-�"�9M;$&�$-�D F%5���

���

top related