object extraction from satellite imagery using deep learning
TRANSCRIPT
Object Extraction from Satellite Imagery using Deep Learning
Aly Osama
Presenter
• Aly Osama
• Research Software Development Engineer at Microsoft
• Contact:
• Email: [email protected]
• https://www.linkedin.com/in/alyosama/
• https://github.com/alyosama
Agenda
1. Needed data, its size,
2. How training will go,
3. How evaluation should be carried,
4. What learning tools would you see and why?
5. Literature survey on the subject
6. Code sample/preliminary results
Overview
Key Words:-
1. Satellite Imagery 2. Object Extraction3. Deep Learning4. Python
1. Satellite Imagery
• Objects are often very small (~20 pixels in size ) as example 0.5m/pixel
• Input images are enormous (often hundreds of megapixels).
• Image has more than 3 channels (RGB) called bands.
• Image Format:• Images (GeoTiff , … etc )• Labels ( GeoJSON , WTK )
• On the positive side, • The physical and pixel scale of objects
are usually known in advance• There’s a low variation in observation
angle.
Datasets • SpaceNet
• Source : https://aws.amazon.com/public-datasets/spacenet/
• Tutorial https://medium.com/the-downlinq/building-extraction-with-yolt2-and-spacenet-data-a926f9ffac4f
• 5 Solutions: https://github.com/SpaceNetChallenge/BuildingDetectors
• DSTL Satellite Imagery (Kaggle Competition )• https://www.kaggle.com/c/dstl-satellite-imagery-
feature-detection• Tutorial : https://www.kaggle.com/torrinos/dstl-
satellite-imagery-feature-detection/exploration-and-plotting/run/553107
• DeepSat
• UC Merced land
• Satellite Websites• https://modis.gsfc.nasa.gov/• https://sentinel.esa.int/web/sentinel/home• https://landsat.usgs.gov/
2. Training
• Experiments
1. VGGNet. - (Baseline)• Tune the pretrained model – Transfer
Learning – based on available data
• Data Augmentation like • Random Crops / Scales
• Color Jitter
2. Faster RCNN or YOLO• For Detection and Localization
General Tips
3.Evaluation
• Accuracy :D
• Precession and Recall
• Jaccard index ( Intersection over Union)
4. tools
• Hardware • Deep Learning AMI Amazon Linux Version
• https://aws.amazon.com/marketplace/pp/B01M0AXXQB• Powerful Machine with GPUs
• NVIDIA GTX Titan X
• Software• Caffe + Python
• Pretrained models • Lasagne / Keras Framework
• High level Python• Backend independent “Tensorflow or Theano” • Multi-GPU
• Caffe2 ( 18-4-2017 )
• Utility• QGIS is an open-source tool for managing and editing GeoTIFFs and
geoJSON files
Tools Comparison
5. Literature survey on the subject
1- A Survey on Object Detection in Optical Remote Sensing Images 2016Gong Cheng, Junwei Han*
Cheng et al. 2016
Cheng et al. 2016
• Deep Learning Papers • (Han et al., 2015;Tang et al.,
2015; Wang et al., 2015; Zhou et al., 2015b)
• Datasets• NWPU VHR-10 dataset (Cheng et
al., 2014a)1. • SZTAKI-INRIA building detection
dataset (Benedek et al., 2012)2. • TAS aerial car detection dataset
(Heitz and Koller, 2008)3.• Overhead imagery research
dataset (OIRDS) (Tanner et al., 2009)
• IITM road extraction dataset (Das et al., 2011)5.
FAST AIRCRAFT DETECTION IN SATELLITE IMAGES BASED ON CONVOLUTIONAL NEURAL NETWORKS (Wu et. Al.) 2015
Region Proposals
TODO : Search on R-CNN
FULLY CONVOLUTIONAL NETWORKS FOR BUILDING AND ROAD EXTRACTION: PRELIMINARY RESULTS 2016 Zilong Zhong
• Dataset• Massachusetts’ road dataset and building dataset • each image consists of 3×1500×1500 pixels• contains 1,711 aerial images,
• the FCN’s computation consumption could be much higher than that of the ordinary object recognition models.
Building detection in very high resolution multispectral data with deep learning features 2016• AlexNet -> Features + SVM ( Last layer )
Road network extraction a neural-dynamic framework based on deep learning and a finite state machine wang2015
CNN + FSM ( for sequence )
Do Deep Features Generalize from Everyday Objectsto Remote Sensing and Aerial Scenes Domains? 2016Ot´avio A. B. Penatti
• ConvNet using Caffe and OverFeat
Using convolutional networks and satellite imagery to identify pa.erns inurban environments at a large scaleADRIAN ALBERT*, Massachuse.s Institute of Technology2017
• Dataset :
• UC Merced land use dataset [25] (of 2100 images spanning 21 classes)
• DeepSat land use benchmark dataset ( 4 channels
• (VGGNet and ResNet)
6. Code sample/prelim
inary results
• (Test) DSTL
• https://www.kaggle.com/alyosama/dstl-satellite-imagery-feature-detection/convnet-baseline/
YOLT2
• The actual F1 score of 0.21
• Jaccard index between 0.4 and 0.5
https://medium.com/the-downlinq/building-extraction-with-yolt2-and-spacenet-data-a926f9ffac4fT
CosmiQNet
• Blackbox Fully Convolution Neural Network: CosmiQNet. The inputs are at two resolutions and the output distance transform matches the lower of the input resolutions. The resolution of the 8-band GeoTIFF is roughly one quarter (in each dimension) the resolution of the the 3-band GeoTIFF; the difference in resolution is depicted by the scale of the GeoTIFFs.
https://medium.com/the-downlinq/object-detection-on-spacenet-5e691961d257
Resources
1. DSTL Satellite Imagery Competiton
2. https://medium.com/@avanetten
3. https://www.kernix.com/blog/image-classification-with-a-pre-trained-deep-neural-network_p11
