Prof. Dr. Damian Borth – Artificial Intelligence & Machine Learning [AI:ML] Lab 1

Deep Learning & Earth ObservationArtificial Intelligence & Machine Learning Lab [AI:ML]

Institute of Computer Science (ICS-HSG)

Benjamin Bischke1, Patrick Helber1, Prof. Dr. Damian Borth2

DFKI1 & University of St. Gallen (HSG)2

Prof. Dr. Damian Borth – Artificial Intelligence & Machine Learning [AI:ML] Lab 2

Space 4.0

Prof. Dr. Damian Borth – Artificial Intelligence & Machine Learning [AI:ML] Lab 3

Space 4.0

Deutsche Welle16.May 2018

UN News12.Oct 2018

Prof. Dr. Damian Borth – Artificial Intelligence & Machine Learning [AI:ML] Lab 4

UN Sustainable Development Goals & Earth Observation

Prof. Dr. Damian Borth – Artificial Intelligence & Machine Learning [AI:ML] Lab 5

UN Sustainable Development Goals & Earth Observation

Poverty Indices

CropEstimate

Agriculture

Prof. Dr. Damian Borth – Artificial Intelligence & Machine Learning [AI:ML] Lab 6

UN Sustainable Development Goals & Earth Observation

Water UtilizationPoverty Indices

CropEstimate

Infrastructure & TrafficMonitoring

Pollution Monitoring(Oil Spills, Smok, Pipelines)

Agriculture

Prof. Dr. Damian Borth – Artificial Intelligence & Machine Learning [AI:ML] Lab 7

Natural & ManmadeDisasters

UN Sustainable Development Goals & Earth Observation

Water Utilization

DeforestationClimate ChangeWildlife

MonitoringAlge-Bloom

IceCaps

CarbonStocks

Poverty Indices

CropEstimate

Infrastructure & TrafficMonitoring

Pollution Monitoring(Oil Spills, Smok, Pipelines)

Agriculture

Prof. Dr. Damian Borth – Artificial Intelligence & Machine Learning [AI:ML] Lab 8

Water Utilization

DeforestationClimate ChangeNatural & Manmade

DisastersWildlife

MonitoringAlge-Bloom

IceCaps

CarbonStocks

Poverty Indices

CropEstimate

UN Sustainable Development Goals & Earth Observation

Infrastructure & TrafficMonitoring

Pollution Monitoring(Oil Spills, Smok, Pipelines)

Agriculture

Prof. Dr. Damian Borth – Artificial Intelligence & Machine Learning [AI:ML] Lab 9

MediaEval Benchmark: Multimedia Satellite Task

2017 Challenge 2018 Challenge

Prof. Dr. Damian Borth – Artificial Intelligence & Machine Learning [AI:ML] Lab 10

MediaEval Benchmark: Multimedia Satellite Task

2017 Challenge

Tasks:• Disaster Image Retrieval from Social Media

• Flood Detection in Satellite ImageryParticipation:• 11 teams registered

• More than 60 submissions (both sub-tasks)• Community Building Workshop in Dublin, Ireland

Approaches: Deep Neural Networks (CNN & SegNet)

2018 Challenge

Prof. Dr. Damian Borth – Artificial Intelligence & Machine Learning [AI:ML] Lab 11

MediaEval Benchmark: Multimedia Satellite Task

2017 Challenge

Tasks:• Disaster Image Retrieval from Social Media

• Flood Detection in Satellite ImageryParticipation:• 11 teams registered

• More than 60 submissions (both sub-tasks)• Community Building Workshop in Dublin, Ireland

Approaches: Deep Neural Networks (CNN & SegNet)

2018 Challenge

Prof. Dr. Damian Borth – Artificial Intelligence & Machine Learning [AI:ML] Lab 12

MediaEval Benchmark: Multimedia Satellite Task

2017 Challenge

Tasks:• Disaster Image Retrieval from Social Media

• Flood Detection in Satellite ImageryParticipation:• 11 teams registered

• More than 60 submissions (both sub-tasks)• Community Building Workshop in Dublin, Ireland

Approaches: Deep Neural Networks (CNN & SegNet)

Input: Satellite Images Output: Segmentation masks

2018 Challenge

Prof. Dr. Damian Borth – Artificial Intelligence & Machine Learning [AI:ML] Lab 13

MediaEval Benchmark: Multimedia Satellite Task

2017 Challenge 2018 Challenge

Tasks:• Disaster Image Retrieval from Social Media

• Flood Detection in Satellite ImageryParticipation:• 11 teams registered

• More than 60 submissions (both sub-tasks)• Community Building Workshop in Dublin, Ireland

Approaches: Deep Neural Networks (CNN & SegNet)

Tasks:• Classification for Road Passability (Yes [level of access] vs No)

• Flood Detection in Satellite ImageryParticipation:• 18 Teams registered

• 51 submissions in total (both sub-tasks)• Community Building Workshop in Sophia Antipolis, France

Approaches: Deep Neural Networks (CNN & SegNet & LSTM & )

Input: Satellite Images Output: Segmentation masks

Prof. Dr. Damian Borth – Artificial Intelligence & Machine Learning [AI:ML] Lab 14

MediaEval Benchmark: Multimedia Satellite Task

2018 Challenge

Tasks:• Classification for Road Passability (Yes [level of access] vs No)

• Flood Detection in Satellite ImageryParticipation:• 18 Teams registered

• 51 submissions in total (both sub-tasks)• Community Building Workshop in Sophia Antipolis, France

Approaches: Deep Neural Networks (CNN & SegNet & LSTM & )

2017 Challenge

Tasks:• Disaster Image Retrieval from Social Media

• Flood Detection in Satellite ImageryParticipation:• 11 teams registered

• More than 60 submissions (both sub-tasks)• Community Building Workshop in Dublin, Ireland

Approaches: Deep Neural Networks (CNN & SegNet)

Input: Satellite Images Output: Segmentation masks 14

Prof. Dr. Damian Borth – Artificial Intelligence & Machine Learning [AI:ML] Lab 15

MediaEval Benchmark: Multimedia Satellite Task

2018 Challenge

Tasks:• Classification for Road Passability (Yes [level of access] vs No)

• Flood Detection in Satellite ImageryParticipation:• 18 Teams registered

• 51 submissions in total (both sub-tasks)• Community Building Workshop in Sophia Antipolis, France

Approaches: Deep Neural Networks (CNN & SegNet & LSTM & )

2017 Challenge

Tasks:• Disaster Image Retrieval from Social Media

• Flood Detection in Satellite ImageryParticipation:• 11 teams registered

• More than 60 submissions (both sub-tasks)• Community Building Workshop in Dublin, Ireland

Approaches: Deep Neural Networks (CNN & SegNet)

Input: Satellite Images Output: Segmentation masks 15

Prof. Dr. Damian Borth – Artificial Intelligence & Machine Learning [AI:ML] Lab 16

MediaEval Benchmark: Multimedia Satellite Task

2017 Challenge 2018 Challenge

Tasks:• Classification for Road Passability (Yes [level of access] vs No)

• Flood Detection in Satellite ImageryParticipation:• 18 Teams registered

• 51 submissions in total (both sub-tasks)• Community Building Workshop in Sophia Antipolis, France

Approaches: Deep Neural Networks (CNN & SegNet & LSTM & )

2017 Challenge

Tasks:• Disaster Image Retrieval from Social Media

• Flood Detection in Satellite ImageryParticipation:• 11 teams registered

• More than 60 submissions (both sub-tasks)• Community Building Workshop in Dublin, Ireland

Approaches: Deep Neural Networks (CNN & SegNet)

Input: Satellite Images Output: Segmentation masks

Not Passable /Blocked Road

PassableRoad

Prof. Dr. Damian Borth – Artificial Intelligence & Machine Learning [AI:ML] Lab 17

Remote Sensing & Satellite Image Analysis

Land-Use and Land-Cover Classification(Patch based)

Multimedia Satellite Task 2017/2018

Multi-Task Learningfor Sem. Segmentation

DeepEye for Natural Disasters

Land Use and Land-Cover Segmentation

Multimodal Fusion of Satellite Data

Missing Data during Inference

Estimation of (Micro)-Economic Factors

Lead: Benjamin Bischke & Patrick Helber

Prof. Dr. Damian Borth – Artificial Intelligence & Machine Learning [AI:ML] Lab 18

Water Utilization

DeforestationClimate ChangeNatural & Manmade

DisastersWildlife

MonitoringAlge-Bloom

IceCaps

CarbonStocks

Poverty Indices

CropEstimate

UN Sustainable Development Goals & Earth Observation

Infrastructure & TrafficMonitoring

Pollution Monitoring(Oil Spills, Smok, Pipelines)

Agriculture

Prof. Dr. Damian Borth – Artificial Intelligence & Machine Learning [AI:ML] Lab 19

Water Utilization

DeforestationClimate ChangeNatural & Manmade

Disasters

Infrastructure & TrafficMonitoring

WildlifeMonitoring

Pollution Monitoring(Oil Spills, Smok, Pipelines)

Agriculture

Alge-Bloom

IceCaps

CarbonStocks

Poverty Indices

CropEstimate

UN Sustainable Development Goals & Earth Observation

Prof. Dr. Damian Borth – Artificial Intelligence & Machine Learning [AI:ML] Lab 20

Water Utilization

DeforestationClimate ChangeNatural & Manmade

Disasters

Infrastructure & TrafficMonitoring

WildlifeMonitoring

Pollution Monitoring(Oil Spills, Smok, Pipelines)

Agriculture

Alge-Bloom

IceCaps

CarbonStocks

Poverty Indices

CropEstimate

UN Sustainable Development Goals & Earth Observation

Prof. Dr. Damian Borth – Artificial Intelligence & Machine Learning [AI:ML] Lab 21

Questions?