computer vision experiments in crowdsourced astronomy elasmar.pdf · computer vision experiments in...
TRANSCRIPT
![Page 2: Computer Vision Experiments in Crowdsourced Astronomy Elasmar.pdf · Computer Vision Experiments in Crowdsourced Astronomy Rasmi Elasmar re2300@columbia.edu. M. Blanton. Ivezic et](https://reader030.vdocuments.us/reader030/viewer/2022040614/5f09384f7e708231d425cafd/html5/thumbnails/2.jpg)
M. Blanton
![Page 3: Computer Vision Experiments in Crowdsourced Astronomy Elasmar.pdf · Computer Vision Experiments in Crowdsourced Astronomy Rasmi Elasmar re2300@columbia.edu. M. Blanton. Ivezic et](https://reader030.vdocuments.us/reader030/viewer/2022040614/5f09384f7e708231d425cafd/html5/thumbnails/3.jpg)
Ivezic et al.
![Page 4: Computer Vision Experiments in Crowdsourced Astronomy Elasmar.pdf · Computer Vision Experiments in Crowdsourced Astronomy Rasmi Elasmar re2300@columbia.edu. M. Blanton. Ivezic et](https://reader030.vdocuments.us/reader030/viewer/2022040614/5f09384f7e708231d425cafd/html5/thumbnails/4.jpg)
Ivezic et al.
![Page 5: Computer Vision Experiments in Crowdsourced Astronomy Elasmar.pdf · Computer Vision Experiments in Crowdsourced Astronomy Rasmi Elasmar re2300@columbia.edu. M. Blanton. Ivezic et](https://reader030.vdocuments.us/reader030/viewer/2022040614/5f09384f7e708231d425cafd/html5/thumbnails/5.jpg)
30 terabytes(~1 SDSS)every night
![Page 6: Computer Vision Experiments in Crowdsourced Astronomy Elasmar.pdf · Computer Vision Experiments in Crowdsourced Astronomy Rasmi Elasmar re2300@columbia.edu. M. Blanton. Ivezic et](https://reader030.vdocuments.us/reader030/viewer/2022040614/5f09384f7e708231d425cafd/html5/thumbnails/6.jpg)
~500 petabytes total
![Page 7: Computer Vision Experiments in Crowdsourced Astronomy Elasmar.pdf · Computer Vision Experiments in Crowdsourced Astronomy Rasmi Elasmar re2300@columbia.edu. M. Blanton. Ivezic et](https://reader030.vdocuments.us/reader030/viewer/2022040614/5f09384f7e708231d425cafd/html5/thumbnails/7.jpg)
How do you deal with the universe’s dataset?
![Page 8: Computer Vision Experiments in Crowdsourced Astronomy Elasmar.pdf · Computer Vision Experiments in Crowdsourced Astronomy Rasmi Elasmar re2300@columbia.edu. M. Blanton. Ivezic et](https://reader030.vdocuments.us/reader030/viewer/2022040614/5f09384f7e708231d425cafd/html5/thumbnails/8.jpg)
Requirements● Processing● Transient object detection for
quick (<60 seconds) follow-up● Classification/cataloging● Archiving
![Page 9: Computer Vision Experiments in Crowdsourced Astronomy Elasmar.pdf · Computer Vision Experiments in Crowdsourced Astronomy Rasmi Elasmar re2300@columbia.edu. M. Blanton. Ivezic et](https://reader030.vdocuments.us/reader030/viewer/2022040614/5f09384f7e708231d425cafd/html5/thumbnails/9.jpg)
“Machine learning is really good at partially solving just
about any problem.”- Chris Dixon
![Page 10: Computer Vision Experiments in Crowdsourced Astronomy Elasmar.pdf · Computer Vision Experiments in Crowdsourced Astronomy Rasmi Elasmar re2300@columbia.edu. M. Blanton. Ivezic et](https://reader030.vdocuments.us/reader030/viewer/2022040614/5f09384f7e708231d425cafd/html5/thumbnails/10.jpg)
Crowdsourcing analysis?
![Page 11: Computer Vision Experiments in Crowdsourced Astronomy Elasmar.pdf · Computer Vision Experiments in Crowdsourced Astronomy Rasmi Elasmar re2300@columbia.edu. M. Blanton. Ivezic et](https://reader030.vdocuments.us/reader030/viewer/2022040614/5f09384f7e708231d425cafd/html5/thumbnails/11.jpg)
Galaxy Zoo
![Page 12: Computer Vision Experiments in Crowdsourced Astronomy Elasmar.pdf · Computer Vision Experiments in Crowdsourced Astronomy Rasmi Elasmar re2300@columbia.edu. M. Blanton. Ivezic et](https://reader030.vdocuments.us/reader030/viewer/2022040614/5f09384f7e708231d425cafd/html5/thumbnails/12.jpg)
The first time a human will see these galaxies.
![Page 13: Computer Vision Experiments in Crowdsourced Astronomy Elasmar.pdf · Computer Vision Experiments in Crowdsourced Astronomy Rasmi Elasmar re2300@columbia.edu. M. Blanton. Ivezic et](https://reader030.vdocuments.us/reader030/viewer/2022040614/5f09384f7e708231d425cafd/html5/thumbnails/13.jpg)
Easy-to-answer questions.
![Page 14: Computer Vision Experiments in Crowdsourced Astronomy Elasmar.pdf · Computer Vision Experiments in Crowdsourced Astronomy Rasmi Elasmar re2300@columbia.edu. M. Blanton. Ivezic et](https://reader030.vdocuments.us/reader030/viewer/2022040614/5f09384f7e708231d425cafd/html5/thumbnails/14.jpg)
galaxyzoo.com
![Page 15: Computer Vision Experiments in Crowdsourced Astronomy Elasmar.pdf · Computer Vision Experiments in Crowdsourced Astronomy Rasmi Elasmar re2300@columbia.edu. M. Blanton. Ivezic et](https://reader030.vdocuments.us/reader030/viewer/2022040614/5f09384f7e708231d425cafd/html5/thumbnails/15.jpg)
Willett et al.
![Page 16: Computer Vision Experiments in Crowdsourced Astronomy Elasmar.pdf · Computer Vision Experiments in Crowdsourced Astronomy Rasmi Elasmar re2300@columbia.edu. M. Blanton. Ivezic et](https://reader030.vdocuments.us/reader030/viewer/2022040614/5f09384f7e708231d425cafd/html5/thumbnails/16.jpg)
BenefitsDeal with data that doesn’t typically work well with ML.
Double-check and assess ML predictions.
Use classifications to train better models.
Highlight potentially interesting objects and events for expert follow-up.
Run large-scale experiments.
More detailed analysis than simple classification.
![Page 17: Computer Vision Experiments in Crowdsourced Astronomy Elasmar.pdf · Computer Vision Experiments in Crowdsourced Astronomy Rasmi Elasmar re2300@columbia.edu. M. Blanton. Ivezic et](https://reader030.vdocuments.us/reader030/viewer/2022040614/5f09384f7e708231d425cafd/html5/thumbnails/17.jpg)
Accurate!Amateur consensus agrees with experts 97% of the time.
(Experts agree with each other 98% of the time.)
![Page 18: Computer Vision Experiments in Crowdsourced Astronomy Elasmar.pdf · Computer Vision Experiments in Crowdsourced Astronomy Rasmi Elasmar re2300@columbia.edu. M. Blanton. Ivezic et](https://reader030.vdocuments.us/reader030/viewer/2022040614/5f09384f7e708231d425cafd/html5/thumbnails/18.jpg)
Adaptable
![Page 19: Computer Vision Experiments in Crowdsourced Astronomy Elasmar.pdf · Computer Vision Experiments in Crowdsourced Astronomy Rasmi Elasmar re2300@columbia.edu. M. Blanton. Ivezic et](https://reader030.vdocuments.us/reader030/viewer/2022040614/5f09384f7e708231d425cafd/html5/thumbnails/19.jpg)
zooniverse.org/lab
Make your own!
![Page 20: Computer Vision Experiments in Crowdsourced Astronomy Elasmar.pdf · Computer Vision Experiments in Crowdsourced Astronomy Rasmi Elasmar re2300@columbia.edu. M. Blanton. Ivezic et](https://reader030.vdocuments.us/reader030/viewer/2022040614/5f09384f7e708231d425cafd/html5/thumbnails/20.jpg)
More advanced analysis?
![Page 21: Computer Vision Experiments in Crowdsourced Astronomy Elasmar.pdf · Computer Vision Experiments in Crowdsourced Astronomy Rasmi Elasmar re2300@columbia.edu. M. Blanton. Ivezic et](https://reader030.vdocuments.us/reader030/viewer/2022040614/5f09384f7e708231d425cafd/html5/thumbnails/21.jpg)
Galaxy Zoo Challenge
![Page 22: Computer Vision Experiments in Crowdsourced Astronomy Elasmar.pdf · Computer Vision Experiments in Crowdsourced Astronomy Rasmi Elasmar re2300@columbia.edu. M. Blanton. Ivezic et](https://reader030.vdocuments.us/reader030/viewer/2022040614/5f09384f7e708231d425cafd/html5/thumbnails/22.jpg)
“This competition asks you to analyze the JPG images of galaxies to find
automated metrics that reproduce the probability distributions derived from
human classifications. For each galaxy, determine the probability that it belongs in a particular class. Can you write an algorithm that behaves as well as the
crowd does?”
![Page 23: Computer Vision Experiments in Crowdsourced Astronomy Elasmar.pdf · Computer Vision Experiments in Crowdsourced Astronomy Rasmi Elasmar re2300@columbia.edu. M. Blanton. Ivezic et](https://reader030.vdocuments.us/reader030/viewer/2022040614/5f09384f7e708231d425cafd/html5/thumbnails/23.jpg)
TricksWinning solution used a 42-million layer convolutional neural network to achieve ~75% accuracy.
Most performance gains were from reducing overfitting (dropouts, data modification).
Space has no preferred direction — rotating, translating, zooming, mirroring images adds more valid training data.
S. Dieleman
![Page 24: Computer Vision Experiments in Crowdsourced Astronomy Elasmar.pdf · Computer Vision Experiments in Crowdsourced Astronomy Rasmi Elasmar re2300@columbia.edu. M. Blanton. Ivezic et](https://reader030.vdocuments.us/reader030/viewer/2022040614/5f09384f7e708231d425cafd/html5/thumbnails/24.jpg)
Proven in other fields“Use the ATLAS experiment to identify the Higgs boson”
“Transforming How We Diagnose Heart Disease”
“Predict ocean health, one plankton at a time”
CERN, NHLBI
![Page 25: Computer Vision Experiments in Crowdsourced Astronomy Elasmar.pdf · Computer Vision Experiments in Crowdsourced Astronomy Rasmi Elasmar re2300@columbia.edu. M. Blanton. Ivezic et](https://reader030.vdocuments.us/reader030/viewer/2022040614/5f09384f7e708231d425cafd/html5/thumbnails/25.jpg)
Crowdsourcing data?
![Page 26: Computer Vision Experiments in Crowdsourced Astronomy Elasmar.pdf · Computer Vision Experiments in Crowdsourced Astronomy Rasmi Elasmar re2300@columbia.edu. M. Blanton. Ivezic et](https://reader030.vdocuments.us/reader030/viewer/2022040614/5f09384f7e708231d425cafd/html5/thumbnails/26.jpg)
Why?Free, useful data — people are always watching.
Solves problems of alignment, combination, normalization...
Pushes current limits of machine learning with messy data.
![Page 27: Computer Vision Experiments in Crowdsourced Astronomy Elasmar.pdf · Computer Vision Experiments in Crowdsourced Astronomy Rasmi Elasmar re2300@columbia.edu. M. Blanton. Ivezic et](https://reader030.vdocuments.us/reader030/viewer/2022040614/5f09384f7e708231d425cafd/html5/thumbnails/27.jpg)
How do we know where an image is in the sky?
A search problem.
![Page 28: Computer Vision Experiments in Crowdsourced Astronomy Elasmar.pdf · Computer Vision Experiments in Crowdsourced Astronomy Rasmi Elasmar re2300@columbia.edu. M. Blanton. Ivezic et](https://reader030.vdocuments.us/reader030/viewer/2022040614/5f09384f7e708231d425cafd/html5/thumbnails/28.jpg)
Astrometry.net
![Page 29: Computer Vision Experiments in Crowdsourced Astronomy Elasmar.pdf · Computer Vision Experiments in Crowdsourced Astronomy Rasmi Elasmar re2300@columbia.edu. M. Blanton. Ivezic et](https://reader030.vdocuments.us/reader030/viewer/2022040614/5f09384f7e708231d425cafd/html5/thumbnails/29.jpg)
Reverse-lookupIndexed hash, just like a Google search.
![Page 30: Computer Vision Experiments in Crowdsourced Astronomy Elasmar.pdf · Computer Vision Experiments in Crowdsourced Astronomy Rasmi Elasmar re2300@columbia.edu. M. Blanton. Ivezic et](https://reader030.vdocuments.us/reader030/viewer/2022040614/5f09384f7e708231d425cafd/html5/thumbnails/30.jpg)
Lang & Hogg
![Page 31: Computer Vision Experiments in Crowdsourced Astronomy Elasmar.pdf · Computer Vision Experiments in Crowdsourced Astronomy Rasmi Elasmar re2300@columbia.edu. M. Blanton. Ivezic et](https://reader030.vdocuments.us/reader030/viewer/2022040614/5f09384f7e708231d425cafd/html5/thumbnails/31.jpg)
Lang & Hogg
![Page 32: Computer Vision Experiments in Crowdsourced Astronomy Elasmar.pdf · Computer Vision Experiments in Crowdsourced Astronomy Rasmi Elasmar re2300@columbia.edu. M. Blanton. Ivezic et](https://reader030.vdocuments.us/reader030/viewer/2022040614/5f09384f7e708231d425cafd/html5/thumbnails/32.jpg)
Lang & Hogg
![Page 33: Computer Vision Experiments in Crowdsourced Astronomy Elasmar.pdf · Computer Vision Experiments in Crowdsourced Astronomy Rasmi Elasmar re2300@columbia.edu. M. Blanton. Ivezic et](https://reader030.vdocuments.us/reader030/viewer/2022040614/5f09384f7e708231d425cafd/html5/thumbnails/33.jpg)
Lang & Hogg
![Page 34: Computer Vision Experiments in Crowdsourced Astronomy Elasmar.pdf · Computer Vision Experiments in Crowdsourced Astronomy Rasmi Elasmar re2300@columbia.edu. M. Blanton. Ivezic et](https://reader030.vdocuments.us/reader030/viewer/2022040614/5f09384f7e708231d425cafd/html5/thumbnails/34.jpg)
J. Lodriguss
![Page 35: Computer Vision Experiments in Crowdsourced Astronomy Elasmar.pdf · Computer Vision Experiments in Crowdsourced Astronomy Rasmi Elasmar re2300@columbia.edu. M. Blanton. Ivezic et](https://reader030.vdocuments.us/reader030/viewer/2022040614/5f09384f7e708231d425cafd/html5/thumbnails/35.jpg)
Lang & Hogg
![Page 36: Computer Vision Experiments in Crowdsourced Astronomy Elasmar.pdf · Computer Vision Experiments in Crowdsourced Astronomy Rasmi Elasmar re2300@columbia.edu. M. Blanton. Ivezic et](https://reader030.vdocuments.us/reader030/viewer/2022040614/5f09384f7e708231d425cafd/html5/thumbnails/36.jpg)
Lang & Hogg
![Page 37: Computer Vision Experiments in Crowdsourced Astronomy Elasmar.pdf · Computer Vision Experiments in Crowdsourced Astronomy Rasmi Elasmar re2300@columbia.edu. M. Blanton. Ivezic et](https://reader030.vdocuments.us/reader030/viewer/2022040614/5f09384f7e708231d425cafd/html5/thumbnails/37.jpg)
AstroBin
![Page 38: Computer Vision Experiments in Crowdsourced Astronomy Elasmar.pdf · Computer Vision Experiments in Crowdsourced Astronomy Rasmi Elasmar re2300@columbia.edu. M. Blanton. Ivezic et](https://reader030.vdocuments.us/reader030/viewer/2022040614/5f09384f7e708231d425cafd/html5/thumbnails/38.jpg)
astrobin.com
![Page 39: Computer Vision Experiments in Crowdsourced Astronomy Elasmar.pdf · Computer Vision Experiments in Crowdsourced Astronomy Rasmi Elasmar re2300@columbia.edu. M. Blanton. Ivezic et](https://reader030.vdocuments.us/reader030/viewer/2022040614/5f09384f7e708231d425cafd/html5/thumbnails/39.jpg)
How do we place Jupiterin the sky?
How can we combine images from different sources?
![Page 40: Computer Vision Experiments in Crowdsourced Astronomy Elasmar.pdf · Computer Vision Experiments in Crowdsourced Astronomy Rasmi Elasmar re2300@columbia.edu. M. Blanton. Ivezic et](https://reader030.vdocuments.us/reader030/viewer/2022040614/5f09384f7e708231d425cafd/html5/thumbnails/40.jpg)
Structure-from-Motion
![Page 41: Computer Vision Experiments in Crowdsourced Astronomy Elasmar.pdf · Computer Vision Experiments in Crowdsourced Astronomy Rasmi Elasmar re2300@columbia.edu. M. Blanton. Ivezic et](https://reader030.vdocuments.us/reader030/viewer/2022040614/5f09384f7e708231d425cafd/html5/thumbnails/41.jpg)
Agarwal et al.
![Page 42: Computer Vision Experiments in Crowdsourced Astronomy Elasmar.pdf · Computer Vision Experiments in Crowdsourced Astronomy Rasmi Elasmar re2300@columbia.edu. M. Blanton. Ivezic et](https://reader030.vdocuments.us/reader030/viewer/2022040614/5f09384f7e708231d425cafd/html5/thumbnails/42.jpg)
D. CrandallN. Snavely
![Page 43: Computer Vision Experiments in Crowdsourced Astronomy Elasmar.pdf · Computer Vision Experiments in Crowdsourced Astronomy Rasmi Elasmar re2300@columbia.edu. M. Blanton. Ivezic et](https://reader030.vdocuments.us/reader030/viewer/2022040614/5f09384f7e708231d425cafd/html5/thumbnails/43.jpg)
Agarwal et al.
![Page 44: Computer Vision Experiments in Crowdsourced Astronomy Elasmar.pdf · Computer Vision Experiments in Crowdsourced Astronomy Rasmi Elasmar re2300@columbia.edu. M. Blanton. Ivezic et](https://reader030.vdocuments.us/reader030/viewer/2022040614/5f09384f7e708231d425cafd/html5/thumbnails/44.jpg)
Timelapses!
![Page 45: Computer Vision Experiments in Crowdsourced Astronomy Elasmar.pdf · Computer Vision Experiments in Crowdsourced Astronomy Rasmi Elasmar re2300@columbia.edu. M. Blanton. Ivezic et](https://reader030.vdocuments.us/reader030/viewer/2022040614/5f09384f7e708231d425cafd/html5/thumbnails/45.jpg)
Martin-Brualla et al.
![Page 46: Computer Vision Experiments in Crowdsourced Astronomy Elasmar.pdf · Computer Vision Experiments in Crowdsourced Astronomy Rasmi Elasmar re2300@columbia.edu. M. Blanton. Ivezic et](https://reader030.vdocuments.us/reader030/viewer/2022040614/5f09384f7e708231d425cafd/html5/thumbnails/46.jpg)
Martin-Brualla et al.
![Page 47: Computer Vision Experiments in Crowdsourced Astronomy Elasmar.pdf · Computer Vision Experiments in Crowdsourced Astronomy Rasmi Elasmar re2300@columbia.edu. M. Blanton. Ivezic et](https://reader030.vdocuments.us/reader030/viewer/2022040614/5f09384f7e708231d425cafd/html5/thumbnails/47.jpg)
Martin-Brualla et al.
![Page 48: Computer Vision Experiments in Crowdsourced Astronomy Elasmar.pdf · Computer Vision Experiments in Crowdsourced Astronomy Rasmi Elasmar re2300@columbia.edu. M. Blanton. Ivezic et](https://reader030.vdocuments.us/reader030/viewer/2022040614/5f09384f7e708231d425cafd/html5/thumbnails/48.jpg)
Martin-Brualla et al.
![Page 49: Computer Vision Experiments in Crowdsourced Astronomy Elasmar.pdf · Computer Vision Experiments in Crowdsourced Astronomy Rasmi Elasmar re2300@columbia.edu. M. Blanton. Ivezic et](https://reader030.vdocuments.us/reader030/viewer/2022040614/5f09384f7e708231d425cafd/html5/thumbnails/49.jpg)
R. Elasmar
![Page 50: Computer Vision Experiments in Crowdsourced Astronomy Elasmar.pdf · Computer Vision Experiments in Crowdsourced Astronomy Rasmi Elasmar re2300@columbia.edu. M. Blanton. Ivezic et](https://reader030.vdocuments.us/reader030/viewer/2022040614/5f09384f7e708231d425cafd/html5/thumbnails/50.jpg)
R. Elasmar