Download - Robot eye for the butterfly
![Page 1: Robot eye for the butterfly](https://reader031.vdocuments.us/reader031/viewer/2022022419/58ede2141a28abbe478b470d/html5/thumbnails/1.jpg)
Robot eye for the butterfly
Rutger Vos
![Page 2: Robot eye for the butterfly](https://reader031.vdocuments.us/reader031/viewer/2022022419/58ede2141a28abbe478b470d/html5/thumbnails/2.jpg)
Javanese butterflies
Van Groenendael-Krijger collection • Collected in the 1930s • Photographed in standardized setup
![Page 3: Robot eye for the butterfly](https://reader031.vdocuments.us/reader031/viewer/2022022419/58ede2141a28abbe478b470d/html5/thumbnails/3.jpg)
By extracting salient features from images and using these to train neural networks, automated identification may be possible
Automated identification
![Page 4: Robot eye for the butterfly](https://reader031.vdocuments.us/reader031/viewer/2022022419/58ede2141a28abbe478b470d/html5/thumbnails/4.jpg)
Project structure overview
• Open source, freely available at: github.com/naturalis
• Designed as loosely coupled, swappable modules
• Intended for re-use for multiple cases
![Page 5: Robot eye for the butterfly](https://reader031.vdocuments.us/reader031/viewer/2022022419/58ede2141a28abbe478b470d/html5/thumbnails/5.jpg)
Project structure: reference images
photos [table]id INTEGER NOT NULLmd5sum VARCHAR(32) NOT NULLpath VARCHAR(255)title VARCHAR(100)description VARCHAR(255)
photos_tags [table]photo_id INTEGER NOT NULLtag_id INTEGER NOT NULL
tags [table]id INTEGER NOT NULLname VARCHAR(50) NOT NULL
photos_taxa [table]photo_id INTEGER NOT NULLtaxon_id INTEGER NOT NULL
taxa [table]id INTEGER NOT NULLrank_id INTEGER NOT NULLname VARCHAR(50) NOT NULLdescription VARCHAR(255)
ranks [table]id INTEGER NOT NULLname VARCHAR(50) NOT NULL
![Page 6: Robot eye for the butterfly](https://reader031.vdocuments.us/reader031/viewer/2022022419/58ede2141a28abbe478b470d/html5/thumbnails/6.jpg)
Project structure: image processing
Speeded Up Robust Features
![Page 7: Robot eye for the butterfly](https://reader031.vdocuments.us/reader031/viewer/2022022419/58ede2141a28abbe478b470d/html5/thumbnails/7.jpg)
Project structure: machine learning
![Page 8: Robot eye for the butterfly](https://reader031.vdocuments.us/reader031/viewer/2022022419/58ede2141a28abbe478b470d/html5/thumbnails/8.jpg)
Project structure: optimization
![Page 9: Robot eye for the butterfly](https://reader031.vdocuments.us/reader031/viewer/2022022419/58ede2141a28abbe478b470d/html5/thumbnails/9.jpg)
Project structure: user interface
![Page 10: Robot eye for the butterfly](https://reader031.vdocuments.us/reader031/viewer/2022022419/58ede2141a28abbe478b470d/html5/thumbnails/10.jpg)
Results: SURF features • PCA plots of the “speeded up robust
features” show clustering both at the genus (top) and species (bottom) level
• Some species are so dimorphic that the sexes are treated as separate species (not shown)
• Some individuals are “gynandromorphic”, though there is likely positive collection bias
• Some taxa are much more variable than others
![Page 11: Robot eye for the butterfly](https://reader031.vdocuments.us/reader031/viewer/2022022419/58ede2141a28abbe478b470d/html5/thumbnails/11.jpg)
Results: k-folds cross-validation
• Split the data in k (2, 5, 10) partitions • Train on 1 partition, use k-1 as “out-of-sample” data • Count number of correct/incorrect/unknown identifications
![Page 12: Robot eye for the butterfly](https://reader031.vdocuments.us/reader031/viewer/2022022419/58ede2141a28abbe478b470d/html5/thumbnails/12.jpg)
Next steps
• Application of trained neural networks to the entire VGKS collection (once that is fully digitized)
• Testing other classifiers in addition to ANNs
• Improvement of the end user interface, possibly as a native ‘app’ or on the web
• Extension of the platform to additional cases, such as shells (snails, bivalves)
• Do more with the image feature data: mimicry, character displacement, dimorphism
![Page 13: Robot eye for the butterfly](https://reader031.vdocuments.us/reader031/viewer/2022022419/58ede2141a28abbe478b470d/html5/thumbnails/13.jpg)
Acknowledgements Naturalis sector Collection • Max Caspers • Luc Willemse • Jan Moonen • Digitization volunteers Hogeschool Leiden • Barbara Gravendeel • Patrick Wijntjes • Saskia de Vetter LIACS • Fons Verbeek • Mengke Li • Yuanhao Guo
IBL • Wim van Tongeren WUR • Feia Matthijssen Made possible by • Naturalis internal grant for
application-oriented research • The Van Groenendael-Krijger
Stichting • Kind contributions of photos by
numerous orchid breeders
![Page 14: Robot eye for the butterfly](https://reader031.vdocuments.us/reader031/viewer/2022022419/58ede2141a28abbe478b470d/html5/thumbnails/14.jpg)
Thanks for listening!