digital image increase - universität klagenfurt · horst bischof redundancy for aerial computer...
TRANSCRIPT
![Page 1: Digital Image Increase - Universität Klagenfurt · Horst Bischof Redundancy for aerial computer vision 25 Primal-Dual Optimization D 1 3 ˆ ! ˆ ˇ ! 5 K 5 ˆ ! K51 ! 6- !ˇ "* ˙˝˘˛˘>](https://reader036.vdocuments.us/reader036/viewer/2022071212/6025439e99619d322c0d4125/html5/thumbnails/1.jpg)
1 Horst Bischof Redundancy for aerial computer vision
Exploiting redundancy for reliable aerial computer vision
Horst Bischof ���������� ����������������
2 Horst Bischof Redundancy for aerial computer vision
Digital Image Increase
![Page 2: Digital Image Increase - Universität Klagenfurt · Horst Bischof Redundancy for aerial computer vision 25 Primal-Dual Optimization D 1 3 ˆ ! ˆ ˇ ! 5 K 5 ˆ ! K51 ! 6- !ˇ "* ˙˝˘˛˘>](https://reader036.vdocuments.us/reader036/viewer/2022071212/6025439e99619d322c0d4125/html5/thumbnails/2.jpg)
3 Horst Bischof Redundancy for aerial computer vision
Images Worldwide
� �������������� ��� ������������� ��������������� ������������� ��������������� �� !�� ��������������
�"��#�����$!��������%�����
4 Horst Bischof Redundancy for aerial computer vision
Terrestrial Image Acquisition
� �������& ��� �%�'�� (���������!�����)���*� ��� �+� �,����!� ���� �����*�� ��-����� �����
![Page 3: Digital Image Increase - Universität Klagenfurt · Horst Bischof Redundancy for aerial computer vision 25 Primal-Dual Optimization D 1 3 ˆ ! ˆ ˇ ! 5 K 5 ˆ ! K51 ! 6- !ˇ "* ˙˝˘˛˘>](https://reader036.vdocuments.us/reader036/viewer/2022071212/6025439e99619d322c0d4125/html5/thumbnails/3.jpg)
5 Horst Bischof Redundancy for aerial computer vision
Aerial Photogrammetry
� -����������.�� �)#������ � �,� �������+� �&/� �01������� �� ��-�!����2�#����3���)���!��&,+�� (����!�����)(���*� ��� �+��
6 Horst Bischof Redundancy for aerial computer vision
New Sensor Platforms
� -�� ������� �� 4��3����)�� ���� ����������5����5��� �
��� ������5� ��*&�'����� ����� ��+�
6 Towards Fully Automatic Photogrammetric Reconstruction Using Digital Images Taken From UAVs./7�$�������"�%78�1�!���"�#7�8������ �"�97�:�����"���5�47�;�����7�����<�����$&*=&�����7>��
![Page 4: Digital Image Increase - Universität Klagenfurt · Horst Bischof Redundancy for aerial computer vision 25 Primal-Dual Optimization D 1 3 ˆ ! ˆ ˇ ! 5 K 5 ˆ ! K51 ! 6- !ˇ "* ˙˝˘˛˘>](https://reader036.vdocuments.us/reader036/viewer/2022071212/6025439e99619d322c0d4125/html5/thumbnails/4.jpg)
7 Horst Bischof Redundancy for aerial computer vision
Airborne vs. MAVs
#������ ��� ��-�!,�
������3�?����
#/%�*�� �3�/@��@����3�������
A���7���� A����
�&,B�(C�! �3���9��B�(2��!�
�&,B�(�7��! �3���9��B�(��!�
8 Horst Bischof Redundancy for aerial computer vision
Applications
![Page 5: Digital Image Increase - Universität Klagenfurt · Horst Bischof Redundancy for aerial computer vision 25 Primal-Dual Optimization D 1 3 ˆ ! ˆ ˇ ! 5 K 5 ˆ ! K51 ! 6- !ˇ "* ˙˝˘˛˘>](https://reader036.vdocuments.us/reader036/viewer/2022071212/6025439e99619d322c0d4125/html5/thumbnails/5.jpg)
9 Horst Bischof Redundancy for aerial computer vision
Application: Virtual Habitat
[Leberl et al. IEEE Computer 2010]
10Horst Bischof Redundancy for aerial computer vision
Application: Architecture and Cultural Heritage
[Zebedin 2010], [Irschara 2010]
![Page 6: Digital Image Increase - Universität Klagenfurt · Horst Bischof Redundancy for aerial computer vision 25 Primal-Dual Optimization D 1 3 ˆ ! ˆ ˇ ! 5 K 5 ˆ ! K51 ! 6- !ˇ "* ˙˝˘˛˘>](https://reader036.vdocuments.us/reader036/viewer/2022071212/6025439e99619d322c0d4125/html5/thumbnails/6.jpg)
11Horst Bischof Redundancy for aerial computer vision
Application: Construction Site Monitoring
12Horst Bischof Redundancy for aerial computer vision
Application: Mining
![Page 7: Digital Image Increase - Universität Klagenfurt · Horst Bischof Redundancy for aerial computer vision 25 Primal-Dual Optimization D 1 3 ˆ ! ˆ ˇ ! 5 K 5 ˆ ! K51 ! 6- !ˇ "* ˙˝˘˛˘>](https://reader036.vdocuments.us/reader036/viewer/2022071212/6025439e99619d322c0d4125/html5/thumbnails/7.jpg)
13Horst Bischof Redundancy for aerial computer vision
Outline
D 3D Reconstruction �E & �1� 1������!�#� ���E ,�����& �����E =�51�5�����
�
D Semantic Classification �E *3���������������� ���E =��1����� ���E 41����
�
D Discussion & Outlook
14Horst Bischof Redundancy for aerial computer vision
3D Reconstruction
![Page 8: Digital Image Increase - Universität Klagenfurt · Horst Bischof Redundancy for aerial computer vision 25 Primal-Dual Optimization D 1 3 ˆ ! ˆ ˇ ! 5 K 5 ˆ ! K51 ! 6- !ˇ "* ˙˝˘˛˘>](https://reader036.vdocuments.us/reader036/viewer/2022071212/6025439e99619d322c0d4125/html5/thumbnails/8.jpg)
15Horst Bischof Redundancy for aerial computer vision
Structure from Motion
6,7�F� ��"�G/1 �!� ��5����������� �1� ������!�1������� �5�5�����H1�����I7�*�7,7"�����>�6#7�*���������� ���7"�G%�1���!�5�����' �������5���5���!���I7�$J-%"�����@>�6F7�&������� ���7"�G*�� ����1��!B�03�������*�� ��-����� �������,I7�&$��=/*9"�����>�6&7�/���'���� ���7"�G:1�5���=�!������,��I7�-%*="����2>�6J7K#7�4���!�� ���7"�G:1�5���=�!�������-��15�����,��I7�0--%"�����>�
5�����H1����� �1��������5�!�����
�,�!�5�����!�������� � ����L��,�� �1� 1���
$F*��� M��*���
16Horst Bischof Redundancy for aerial computer vision
Structure from Motion
-�!�����M��� � ����
�,�&������& �1� 1���
$!�����-��������5������
![Page 9: Digital Image Increase - Universität Klagenfurt · Horst Bischof Redundancy for aerial computer vision 25 Primal-Dual Optimization D 1 3 ˆ ! ˆ ˇ ! 5 K 5 ˆ ! K51 ! 6- !ˇ "* ˙˝˘˛˘>](https://reader036.vdocuments.us/reader036/viewer/2022071212/6025439e99619d322c0d4125/html5/thumbnails/9.jpg)
17Horst Bischof Redundancy for aerial computer vision
Structure from Motion
X1
X4 X
3 X2
X6
X7
X5
-�!������R2, t2, [K2]
minimize g )R"t"[K]"X+�
-�!������R1, t1, [K1]
-�!������R3, t3, [K3]
6F7�&������-%*=�N�2� 1 ����>�
�7�7��������!��������7�����,���� ���
����7�����1����'���
18Horst Bischof Redundancy for aerial computer vision
�*�� �-*��&���51����K��3�
Structure from Motion
Feature Extraction
Coarse Matching
Detailed Matching
Geometric Verification
Images Pose Prior
Geometric Estimation
Epipolar Graph
Image Overlap
Local Descriptors
Camera poses 3D points Matches
Calibration Pose Prior
-�!�1 ������5�,����/��� �� 1���)-�,/+�
![Page 10: Digital Image Increase - Universität Klagenfurt · Horst Bischof Redundancy for aerial computer vision 25 Primal-Dual Optimization D 1 3 ˆ ! ˆ ˇ ! 5 K 5 ˆ ! K51 ! 6- !ˇ "* ˙˝˘˛˘>](https://reader036.vdocuments.us/reader036/viewer/2022071212/6025439e99619d322c0d4125/html5/thumbnails/10.jpg)
19Horst Bischof Redundancy for aerial computer vision
Aerial Photogrammetry
� �����������!������ �����(�����!O�� C�P��������� �@"@���3�2"@����3���� (����!��&,�
20Horst Bischof Redundancy for aerial computer vision
Global Depth Map Optimization
5� �� ��!����1��� �� ��!�
���
��� ��������������
���
)),(()(
)),(( ���� 77�!� �������� ��)F--+�
�������
)( 77��� ���%��� ���)�%+����1����� ��B�
� ������������� ����!�� ������ �5���� �1 �����������
6�7�*����� ���7"�G/�����3����!1�� ��������� �1�1��!1� K������������!�I7�0--%"����C>�
-�� �1�1��0������#�!�� ���/�������
![Page 11: Digital Image Increase - Universität Klagenfurt · Horst Bischof Redundancy for aerial computer vision 25 Primal-Dual Optimization D 1 3 ˆ ! ˆ ˇ ! 5 K 5 ˆ ! K51 ! 6- !ˇ "* ˙˝˘˛˘>](https://reader036.vdocuments.us/reader036/viewer/2022071212/6025439e99619d322c0d4125/html5/thumbnails/11.jpg)
21Horst Bischof Redundancy for aerial computer vision
KKEY view
22Horst Bischof Redundancy for aerial computer vision
DEPTH map
min
5� �� ��!����1��� �� ��!�
���
��� ��������������
���
)),(()(
![Page 12: Digital Image Increase - Universität Klagenfurt · Horst Bischof Redundancy for aerial computer vision 25 Primal-Dual Optimization D 1 3 ˆ ! ˆ ˇ ! 5 K 5 ˆ ! K51 ! 6- !ˇ "* ˙˝˘˛˘>](https://reader036.vdocuments.us/reader036/viewer/2022071212/6025439e99619d322c0d4125/html5/thumbnails/12.jpg)
23Horst Bischof Redundancy for aerial computer vision
DEPTH map
min
5� �� ��!����1��� �� ��!�
���
��� ��������������
���
)),(()(
24Horst Bischof Redundancy for aerial computer vision
![Page 13: Digital Image Increase - Universität Klagenfurt · Horst Bischof Redundancy for aerial computer vision 25 Primal-Dual Optimization D 1 3 ˆ ! ˆ ˇ ! 5 K 5 ˆ ! K51 ! 6- !ˇ "* ˙˝˘˛˘>](https://reader036.vdocuments.us/reader036/viewer/2022071212/6025439e99619d322c0d4125/html5/thumbnails/13.jpg)
25Horst Bischof Redundancy for aerial computer vision
Primal-Dual Optimization D �������1� �������3��� !�� ���������!����������5���
����� K��5�����!��K51�������� �!�6-��!�����"�*��������>�D 4�� 1���������������� !����������������������3�
������!��' �����'��� �1� 1���D -������������ �������������5����!����������������
���5'�����1�������*���
26Horst Bischof Redundancy for aerial computer vision
Distributed Visual SLAM
Set of updateable, geo-referenced virtual cameras
PPose
Low framerate, for map extension and relocalization
Full framerate, “cheap” features, for visual servoing
Very little data needed, provide only data necessary for the
current environment Maintain global map,
use “expensive” features!
![Page 14: Digital Image Increase - Universität Klagenfurt · Horst Bischof Redundancy for aerial computer vision 25 Primal-Dual Optimization D 1 3 ˆ ! ˆ ˇ ! 5 K 5 ˆ ! K51 ! 6- !ˇ "* ˙˝˘˛˘>](https://reader036.vdocuments.us/reader036/viewer/2022071212/6025439e99619d322c0d4125/html5/thumbnails/14.jpg)
27Horst Bischof Redundancy for aerial computer vision
Dense Reconstruction On-the-Fly
28Horst Bischof Redundancy for aerial computer vision
Dense Reconstruction On-the-Fly
![Page 15: Digital Image Increase - Universität Klagenfurt · Horst Bischof Redundancy for aerial computer vision 25 Primal-Dual Optimization D 1 3 ˆ ! ˆ ˇ ! 5 K 5 ˆ ! K51 ! 6- !ˇ "* ˙˝˘˛˘>](https://reader036.vdocuments.us/reader036/viewer/2022071212/6025439e99619d322c0d4125/html5/thumbnails/15.jpg)
29Horst Bischof Redundancy for aerial computer vision
Semantic Segmentation
30Horst Bischof Redundancy for aerial computer vision
Motivation – Semantic Interpretation
Water ?
Building ?
Tree ?Street ?
10 cm/pixel
![Page 16: Digital Image Increase - Universität Klagenfurt · Horst Bischof Redundancy for aerial computer vision 25 Primal-Dual Optimization D 1 3 ˆ ! ˆ ˇ ! 5 K 5 ˆ ! K51 ! 6- !ˇ "* ˙˝˘˛˘>](https://reader036.vdocuments.us/reader036/viewer/2022071212/6025439e99619d322c0d4125/html5/thumbnails/16.jpg)
31Horst Bischof Redundancy for aerial computer vision
Motivation
�… a lot of overlapping images
3D information
32Horst Bischof Redundancy for aerial computer vision
Fusion - Model
D �%K;��5�������!�5���6F�������@>�����
- Robust against outliers - Preserves sharp edges
![Page 17: Digital Image Increase - Universität Klagenfurt · Horst Bischof Redundancy for aerial computer vision 25 Primal-Dual Optimization D 1 3 ˆ ! ˆ ˇ ! 5 K 5 ˆ ! K51 ! 6- !ˇ "* ˙˝˘˛˘>](https://reader036.vdocuments.us/reader036/viewer/2022071212/6025439e99619d322c0d4125/html5/thumbnails/17.jpg)
33Horst Bischof Redundancy for aerial computer vision
Fusion - Model
D 03 ������ ��!1� ���������� ����6Q����?>��
D ��
Some observations
34Horst Bischof Redundancy for aerial computer vision
Fusion – Semantic Interpretation
D =����!�� ����� ��������������!�� ��D -�� �1�1�����!1�� ������ ���*� ��!�5���6*����2>�
� aggregated
refined
![Page 18: Digital Image Increase - Universität Klagenfurt · Horst Bischof Redundancy for aerial computer vision 25 Primal-Dual Optimization D 1 3 ˆ ! ˆ ˇ ! 5 K 5 ˆ ! K51 ! 6- !ˇ "* ˙˝˘˛˘>](https://reader036.vdocuments.us/reader036/viewer/2022071212/6025439e99619d322c0d4125/html5/thumbnails/18.jpg)
35Horst Bischof Redundancy for aerial computer vision
Fusion - Color and Height
Color: Wavelet-Fusion Height: TV-Fusion
36Horst Bischof Redundancy for aerial computer vision
��
Some observations in ortho-view
![Page 19: Digital Image Increase - Universität Klagenfurt · Horst Bischof Redundancy for aerial computer vision 25 Primal-Dual Optimization D 1 3 ˆ ! ˆ ˇ ! 5 K 5 ˆ ! K51 ! 6- !ˇ "* ˙˝˘˛˘>](https://reader036.vdocuments.us/reader036/viewer/2022071212/6025439e99619d322c0d4125/html5/thumbnails/19.jpg)
37Horst Bischof Redundancy for aerial computer vision
Evaluation – Redundant Interpretation
Remember evaluation on single images: Graz 89.5%, Dallas 92.5%
38Horst Bischof Redundancy for aerial computer vision
Large-Scale Results
Building
Green Area
Waterbody
Tree
Streetlayer
Dallas, 4 tiles, each 240 x 240 m, 15 cm
![Page 20: Digital Image Increase - Universität Klagenfurt · Horst Bischof Redundancy for aerial computer vision 25 Primal-Dual Optimization D 1 3 ˆ ! ˆ ˇ ! 5 K 5 ˆ ! K51 ! 6- !ˇ "* ˙˝˘˛˘>](https://reader036.vdocuments.us/reader036/viewer/2022071212/6025439e99619d322c0d4125/html5/thumbnails/20.jpg)
39Horst Bischof Redundancy for aerial computer vision
Building
Green Area
Waterbody
Tree
Streetlayer
SF 10 x 3 tiles 2500 x 700 m 15 cm
40Horst Bischof Redundancy for aerial computer vision
Large-scale Results
Building
Green Area
Waterbody
Tree
Streetlayer
Graz, 7 km2, 155 images, 20 x 10 tiles, 8 cm
![Page 21: Digital Image Increase - Universität Klagenfurt · Horst Bischof Redundancy for aerial computer vision 25 Primal-Dual Optimization D 1 3 ˆ ! ˆ ˇ ! 5 K 5 ˆ ! K51 ! 6- !ˇ "* ˙˝˘˛˘>](https://reader036.vdocuments.us/reader036/viewer/2022071212/6025439e99619d322c0d4125/html5/thumbnails/21.jpg)
41Horst Bischof Redundancy for aerial computer vision
Conclusions & Future Work D /���������5��,������� �1� ������� ������5�!����
� ���� ��������� ����D /������������5��,�����!� ��������
�D =�51�5����������B�8��� ���1 �!� �� ���
�D =��1� ���!�1 � ��������semantic 3D model
D 41 1��B��������#/%��E 4��3����E -�����
�
42Horst Bischof Redundancy for aerial computer vision
Videos/Code/Papers see
'''7��1@���7���������7��7 1����7� �
![Page 22: Digital Image Increase - Universität Klagenfurt · Horst Bischof Redundancy for aerial computer vision 25 Primal-Dual Optimization D 1 3 ˆ ! ˆ ˇ ! 5 K 5 ˆ ! K51 ! 6- !ˇ "* ˙˝˘˛˘>](https://reader036.vdocuments.us/reader036/viewer/2022071212/6025439e99619d322c0d4125/html5/thumbnails/22.jpg)
43Horst Bischof Redundancy for aerial computer vision
Acknowledgments 41�5������5�5���B��D 4R4�1�5���*��.�� �9�����M�5���%��� �����#� ��5������-�!�1 ���
%����D 4$�K$��*�����!��1�5�5����:#%$��1�5���*��.�� �%#�*�"�*0�/&�&"�
-MF&�=�-�"�9M;$&�$-�D F%5���
���