![Page 1: From Swarm Robotics to Smart Materialscmma.mims.meiji.ac.jp/eng/events/ICMMA2014/files/Correll.pdfFrom Swarm Robotics to Smart Materials Nikolaus Correll University of Colorado at](https://reader034.vdocuments.us/reader034/viewer/2022052003/6015ef1c5fd13f39b1621572/html5/thumbnails/1.jpg)
From Swarm Robotics to Smart Materials
Nikolaus Correll
University of Colorado at Boulder
![Page 2: From Swarm Robotics to Smart Materialscmma.mims.meiji.ac.jp/eng/events/ICMMA2014/files/Correll.pdfFrom Swarm Robotics to Smart Materials Nikolaus Correll University of Colorado at](https://reader034.vdocuments.us/reader034/viewer/2022052003/6015ef1c5fd13f39b1621572/html5/thumbnails/2.jpg)
The ultimate robotic swarm:a liquid that thinks
![Page 3: From Swarm Robotics to Smart Materialscmma.mims.meiji.ac.jp/eng/events/ICMMA2014/files/Correll.pdfFrom Swarm Robotics to Smart Materials Nikolaus Correll University of Colorado at](https://reader034.vdocuments.us/reader034/viewer/2022052003/6015ef1c5fd13f39b1621572/html5/thumbnails/3.jpg)
Subsets of Liquids that thinkConstruction
Appearance Change
Distributed Computing
Intelligent function
Collective Motion
Viscoelasiticy
![Page 4: From Swarm Robotics to Smart Materialscmma.mims.meiji.ac.jp/eng/events/ICMMA2014/files/Correll.pdfFrom Swarm Robotics to Smart Materials Nikolaus Correll University of Colorado at](https://reader034.vdocuments.us/reader034/viewer/2022052003/6015ef1c5fd13f39b1621572/html5/thumbnails/4.jpg)
Making a liquid that thinks
“Droplets” “Liquid that thinks”
![Page 5: From Swarm Robotics to Smart Materialscmma.mims.meiji.ac.jp/eng/events/ICMMA2014/files/Correll.pdfFrom Swarm Robotics to Smart Materials Nikolaus Correll University of Colorado at](https://reader034.vdocuments.us/reader034/viewer/2022052003/6015ef1c5fd13f39b1621572/html5/thumbnails/5.jpg)
Liquid that thinks
“Droplets” “Liquid that thinks”
Useful novel materials?
![Page 6: From Swarm Robotics to Smart Materialscmma.mims.meiji.ac.jp/eng/events/ICMMA2014/files/Correll.pdfFrom Swarm Robotics to Smart Materials Nikolaus Correll University of Colorado at](https://reader034.vdocuments.us/reader034/viewer/2022052003/6015ef1c5fd13f39b1621572/html5/thumbnails/6.jpg)
Background: Materials that think
• Embedded…– Actuation– Sensing– Computation– Communication
• Periodic, amorphous
Distributed MEMS (Berlin, 1997)
Smart Dust (Pister, 1999)
Amorphous Computing (Abelson, 2000)
Programmable Matter (Goldstein, 2005)
6
![Page 7: From Swarm Robotics to Smart Materialscmma.mims.meiji.ac.jp/eng/events/ICMMA2014/files/Correll.pdfFrom Swarm Robotics to Smart Materials Nikolaus Correll University of Colorado at](https://reader034.vdocuments.us/reader034/viewer/2022052003/6015ef1c5fd13f39b1621572/html5/thumbnails/7.jpg)
Enablers• Cheap sensing,
computation, actuation• Cheap manufacturing• Available polymers
![Page 8: From Swarm Robotics to Smart Materialscmma.mims.meiji.ac.jp/eng/events/ICMMA2014/files/Correll.pdfFrom Swarm Robotics to Smart Materials Nikolaus Correll University of Colorado at](https://reader034.vdocuments.us/reader034/viewer/2022052003/6015ef1c5fd13f39b1621572/html5/thumbnails/8.jpg)
Materials that Sense• Prosthetic / augmented limbs• Improved situational
awareness for robots• More subtle Human / Robot
Interaction• Structural monitoring
![Page 9: From Swarm Robotics to Smart Materialscmma.mims.meiji.ac.jp/eng/events/ICMMA2014/files/Correll.pdfFrom Swarm Robotics to Smart Materials Nikolaus Correll University of Colorado at](https://reader034.vdocuments.us/reader034/viewer/2022052003/6015ef1c5fd13f39b1621572/html5/thumbnails/9.jpg)
Materials that change Appearance
• Smart facades• Camouflage• Smart clothes
![Page 10: From Swarm Robotics to Smart Materialscmma.mims.meiji.ac.jp/eng/events/ICMMA2014/files/Correll.pdfFrom Swarm Robotics to Smart Materials Nikolaus Correll University of Colorado at](https://reader034.vdocuments.us/reader034/viewer/2022052003/6015ef1c5fd13f39b1621572/html5/thumbnails/10.jpg)
Materials that change Shape• Reconfigurable airplane
wings• Reconfigurable
aerodynamic structures• Reconfigurable architecture• Reconfigurable furniture
![Page 11: From Swarm Robotics to Smart Materialscmma.mims.meiji.ac.jp/eng/events/ICMMA2014/files/Correll.pdfFrom Swarm Robotics to Smart Materials Nikolaus Correll University of Colorado at](https://reader034.vdocuments.us/reader034/viewer/2022052003/6015ef1c5fd13f39b1621572/html5/thumbnails/11.jpg)
New forms of artistic expression
K. Sugawara, K. Hata, N. Correll, M. Theodore (2013): 群れ行動の数理モデルとその応用~ 群れお絵かきツールとSwarm Wall ~. 1st International Conference on Human Agent Interaction (HAI), Sapporo, Japan, 2013.
![Page 12: From Swarm Robotics to Smart Materialscmma.mims.meiji.ac.jp/eng/events/ICMMA2014/files/Correll.pdfFrom Swarm Robotics to Smart Materials Nikolaus Correll University of Colorado at](https://reader034.vdocuments.us/reader034/viewer/2022052003/6015ef1c5fd13f39b1621572/html5/thumbnails/12.jpg)
Materials that Sense
![Page 13: From Swarm Robotics to Smart Materialscmma.mims.meiji.ac.jp/eng/events/ICMMA2014/files/Correll.pdfFrom Swarm Robotics to Smart Materials Nikolaus Correll University of Colorado at](https://reader034.vdocuments.us/reader034/viewer/2022052003/6015ef1c5fd13f39b1621572/html5/thumbnails/13.jpg)
Soft Sensing Skin
• Sensor network woven in rubber mesh
• Embedded in EcoFlex silicone rubber
• Surface textured using 60-grit sandpaper during curing
D. Hughes, N. Correll (2014): A Soft, Amorphous Skin that can Sense and Localize Texture . IEEE International Conference on Robotics and Automation (ICRA), Hong Kong, 2014.
![Page 14: From Swarm Robotics to Smart Materialscmma.mims.meiji.ac.jp/eng/events/ICMMA2014/files/Correll.pdfFrom Swarm Robotics to Smart Materials Nikolaus Correll University of Colorado at](https://reader034.vdocuments.us/reader034/viewer/2022052003/6015ef1c5fd13f39b1621572/html5/thumbnails/14.jpg)
Soft Sensing SkinRubbed Texture
Signal Microphone
FFT
Ambient Signal
Signal FFTMeasured Signal Transient Signal
Ambi
ent
Upd
ate
1000 Hz
![Page 15: From Swarm Robotics to Smart Materialscmma.mims.meiji.ac.jp/eng/events/ICMMA2014/files/Correll.pdfFrom Swarm Robotics to Smart Materials Nikolaus Correll University of Colorado at](https://reader034.vdocuments.us/reader034/viewer/2022052003/6015ef1c5fd13f39b1621572/html5/thumbnails/15.jpg)
1. Texture localization
![Page 16: From Swarm Robotics to Smart Materialscmma.mims.meiji.ac.jp/eng/events/ICMMA2014/files/Correll.pdfFrom Swarm Robotics to Smart Materials Nikolaus Correll University of Colorado at](https://reader034.vdocuments.us/reader034/viewer/2022052003/6015ef1c5fd13f39b1621572/html5/thumbnails/16.jpg)
2. Texture Classification
• Logistic regression• Classifier trained on 15
predefined textures• 128 inputs • 15 outputs• 1935 weights stored on
board
𝑦𝑦 = 𝑔𝑔 𝜃𝜃0 + 𝜃𝜃1𝑓𝑓1 + 𝜃𝜃2𝑓𝑓2 + ⋯+ 𝜃𝜃𝑛𝑛𝑓𝑓𝑛𝑛
Brillo: 2.4%Skin: 93.5%…
![Page 17: From Swarm Robotics to Smart Materialscmma.mims.meiji.ac.jp/eng/events/ICMMA2014/files/Correll.pdfFrom Swarm Robotics to Smart Materials Nikolaus Correll University of Colorado at](https://reader034.vdocuments.us/reader034/viewer/2022052003/6015ef1c5fd13f39b1621572/html5/thumbnails/17.jpg)
Classifier Training
• Classifiers trained using 15 textures• 100 samples per texture• 10-fold cross-validation used to determine
accuracy
a b c d e
f g h i j
k l m n o
a) brillob) brushc) cardboardd) coarse wire meshe) cottonf) dense foamg) fine wire meshh) plastici) sandpaperj) silicone foamk) skinl) spongem) terry clothn) textured siliconeo) wood
![Page 18: From Swarm Robotics to Smart Materialscmma.mims.meiji.ac.jp/eng/events/ICMMA2014/files/Correll.pdfFrom Swarm Robotics to Smart Materials Nikolaus Correll University of Colorado at](https://reader034.vdocuments.us/reader034/viewer/2022052003/6015ef1c5fd13f39b1621572/html5/thumbnails/18.jpg)
Classification Results Summary
• Logistic Regression– Overall Classification
Accuracy: 71.2%– Total Number of Weights:
1,935• Possible improvements
– Multi-modal sensing– Joint classifiers– Better classifiers
Confusion Matrix
Non-trivial, material-centric computation
![Page 19: From Swarm Robotics to Smart Materialscmma.mims.meiji.ac.jp/eng/events/ICMMA2014/files/Correll.pdfFrom Swarm Robotics to Smart Materials Nikolaus Correll University of Colorado at](https://reader034.vdocuments.us/reader034/viewer/2022052003/6015ef1c5fd13f39b1621572/html5/thumbnails/19.jpg)
Materials that Change Appearance
![Page 20: From Swarm Robotics to Smart Materialscmma.mims.meiji.ac.jp/eng/events/ICMMA2014/files/Correll.pdfFrom Swarm Robotics to Smart Materials Nikolaus Correll University of Colorado at](https://reader034.vdocuments.us/reader034/viewer/2022052003/6015ef1c5fd13f39b1621572/html5/thumbnails/20.jpg)
Appearance changing materials
![Page 21: From Swarm Robotics to Smart Materialscmma.mims.meiji.ac.jp/eng/events/ICMMA2014/files/Correll.pdfFrom Swarm Robotics to Smart Materials Nikolaus Correll University of Colorado at](https://reader034.vdocuments.us/reader034/viewer/2022052003/6015ef1c5fd13f39b1621572/html5/thumbnails/21.jpg)
Interacting with a Distributed System
N. Farrow, N. Sivagnanadasan, N. Correll (2014): Gesture Based Distributed User Interaction System for a Reconfigurable Self-Organizing Smart Wall. In: Proceedings of the 8th International Conference on Tangible, Embedded and Embodied Interaction (TEI), pp. 245-246 , ACM 2014.
![Page 22: From Swarm Robotics to Smart Materialscmma.mims.meiji.ac.jp/eng/events/ICMMA2014/files/Correll.pdfFrom Swarm Robotics to Smart Materials Nikolaus Correll University of Colorado at](https://reader034.vdocuments.us/reader034/viewer/2022052003/6015ef1c5fd13f39b1621572/html5/thumbnails/22.jpg)
Distributed Gesture Recognition
1. Sample 2. Dilate 3. Interpolate 4. Encode
k-NN
“S”
5. Consensus
Distribute Memory
Distribute Computationwith Rick Han
![Page 23: From Swarm Robotics to Smart Materialscmma.mims.meiji.ac.jp/eng/events/ICMMA2014/files/Correll.pdfFrom Swarm Robotics to Smart Materials Nikolaus Correll University of Colorado at](https://reader034.vdocuments.us/reader034/viewer/2022052003/6015ef1c5fd13f39b1621572/html5/thumbnails/23.jpg)
Accuracy vs. Packet lossHybrid Distributed ComputationDistributed Memory
Tight relationship between material and algorithm design trade-offs
![Page 24: From Swarm Robotics to Smart Materialscmma.mims.meiji.ac.jp/eng/events/ICMMA2014/files/Correll.pdfFrom Swarm Robotics to Smart Materials Nikolaus Correll University of Colorado at](https://reader034.vdocuments.us/reader034/viewer/2022052003/6015ef1c5fd13f39b1621572/html5/thumbnails/24.jpg)
Materials that Shape Change
![Page 25: From Swarm Robotics to Smart Materialscmma.mims.meiji.ac.jp/eng/events/ICMMA2014/files/Correll.pdfFrom Swarm Robotics to Smart Materials Nikolaus Correll University of Colorado at](https://reader034.vdocuments.us/reader034/viewer/2022052003/6015ef1c5fd13f39b1621572/html5/thumbnails/25.jpg)
Variable Stiffness Control• Stiffness control = shape control• Melting of PCL• 2-200MPa change in Youngs’ M.• Local feedback temperature
control• Global distributed shape control
M. A. McEvoy, N. Correll (2014): "Thermoplastic variable stiffness composites with embedded, networked sensing, actuation, and control. In: Journal of Composite Materials, 2014.
25
![Page 26: From Swarm Robotics to Smart Materialscmma.mims.meiji.ac.jp/eng/events/ICMMA2014/files/Correll.pdfFrom Swarm Robotics to Smart Materials Nikolaus Correll University of Colorado at](https://reader034.vdocuments.us/reader034/viewer/2022052003/6015ef1c5fd13f39b1621572/html5/thumbnails/26.jpg)
Principle of Operation
![Page 27: From Swarm Robotics to Smart Materialscmma.mims.meiji.ac.jp/eng/events/ICMMA2014/files/Correll.pdfFrom Swarm Robotics to Smart Materials Nikolaus Correll University of Colorado at](https://reader034.vdocuments.us/reader034/viewer/2022052003/6015ef1c5fd13f39b1621572/html5/thumbnails/27.jpg)
Resulting Shape Change
M. A. McEvoy, N. Correll (2014): Shape Change Through Programmable Stiffness. International Symposium on Experimental Robotics (ISER), Springer Verlag, Marrakech, Morocco, 2014.
![Page 28: From Swarm Robotics to Smart Materialscmma.mims.meiji.ac.jp/eng/events/ICMMA2014/files/Correll.pdfFrom Swarm Robotics to Smart Materials Nikolaus Correll University of Colorado at](https://reader034.vdocuments.us/reader034/viewer/2022052003/6015ef1c5fd13f39b1621572/html5/thumbnails/28.jpg)
Pneumatic Shape Change
Modular soft robotic actuator withembedded sensing and control
Modeling• Geometry• Material properties• Pressure• Curvature• Resulting force
Contraction force Expansion force
Outside Inside
GeometryCurvature
![Page 29: From Swarm Robotics to Smart Materialscmma.mims.meiji.ac.jp/eng/events/ICMMA2014/files/Correll.pdfFrom Swarm Robotics to Smart Materials Nikolaus Correll University of Colorado at](https://reader034.vdocuments.us/reader034/viewer/2022052003/6015ef1c5fd13f39b1621572/html5/thumbnails/29.jpg)
Pneumatic Shape ChangePr
essu
re
TimePressure
Curv
atur
e
Geometry 1
Geometry 2
Feedback control Model prediction
Tight interaction between distributed control and material physics
![Page 30: From Swarm Robotics to Smart Materialscmma.mims.meiji.ac.jp/eng/events/ICMMA2014/files/Correll.pdfFrom Swarm Robotics to Smart Materials Nikolaus Correll University of Colorado at](https://reader034.vdocuments.us/reader034/viewer/2022052003/6015ef1c5fd13f39b1621572/html5/thumbnails/30.jpg)
Materials that Self-Assemble
![Page 31: From Swarm Robotics to Smart Materialscmma.mims.meiji.ac.jp/eng/events/ICMMA2014/files/Correll.pdfFrom Swarm Robotics to Smart Materials Nikolaus Correll University of Colorado at](https://reader034.vdocuments.us/reader034/viewer/2022052003/6015ef1c5fd13f39b1621572/html5/thumbnails/31.jpg)
Vision
1. Self-assembly 2. Cell differentiation 3. Collective behavior
![Page 32: From Swarm Robotics to Smart Materialscmma.mims.meiji.ac.jp/eng/events/ICMMA2014/files/Correll.pdfFrom Swarm Robotics to Smart Materials Nikolaus Correll University of Colorado at](https://reader034.vdocuments.us/reader034/viewer/2022052003/6015ef1c5fd13f39b1621572/html5/thumbnails/32.jpg)
DNA Tile Assembly Model
• Rules to encode neighborhood relationships
• Maximal N rules for structures with N tiles
• Many interesting structures can be made with fewer rules!
Rules
Yuriy Brun. Solving np-complete problems in the tile assembly model. Theoretical Computer Science, 395(1):31–46, 2008.
![Page 33: From Swarm Robotics to Smart Materialscmma.mims.meiji.ac.jp/eng/events/ICMMA2014/files/Correll.pdfFrom Swarm Robotics to Smart Materials Nikolaus Correll University of Colorado at](https://reader034.vdocuments.us/reader034/viewer/2022052003/6015ef1c5fd13f39b1621572/html5/thumbnails/33.jpg)
Example
![Page 34: From Swarm Robotics to Smart Materialscmma.mims.meiji.ac.jp/eng/events/ICMMA2014/files/Correll.pdfFrom Swarm Robotics to Smart Materials Nikolaus Correll University of Colorado at](https://reader034.vdocuments.us/reader034/viewer/2022052003/6015ef1c5fd13f39b1621572/html5/thumbnails/34.jpg)
Robotic Materials
Amorphous Computing
Material Physics
![Page 35: From Swarm Robotics to Smart Materialscmma.mims.meiji.ac.jp/eng/events/ICMMA2014/files/Correll.pdfFrom Swarm Robotics to Smart Materials Nikolaus Correll University of Colorado at](https://reader034.vdocuments.us/reader034/viewer/2022052003/6015ef1c5fd13f39b1621572/html5/thumbnails/35.jpg)
Robotic Materials: New challenges for Education
A soft skin that cansense distance and force Shortest path routing
in a distributed system
An interactive Sushi trayShape-changing materials that
can self-restoreWireless data transmissionpowered by Seebeck effect
Variable Stiffness bySheet Jamming
Computation X Materials
![Page 36: From Swarm Robotics to Smart Materialscmma.mims.meiji.ac.jp/eng/events/ICMMA2014/files/Correll.pdfFrom Swarm Robotics to Smart Materials Nikolaus Correll University of Colorado at](https://reader034.vdocuments.us/reader034/viewer/2022052003/6015ef1c5fd13f39b1621572/html5/thumbnails/36.jpg)
Conclusion
• Robotic Materials pose new opportunities and challenges in distributed algorithms
• Understanding the link between “crowd dynamics” and “material physics”
• Timely problems:– Novel capabilities for robots– Novel materials with revolutionary
functionality for every-day applications
![Page 37: From Swarm Robotics to Smart Materialscmma.mims.meiji.ac.jp/eng/events/ICMMA2014/files/Correll.pdfFrom Swarm Robotics to Smart Materials Nikolaus Correll University of Colorado at](https://reader034.vdocuments.us/reader034/viewer/2022052003/6015ef1c5fd13f39b1621572/html5/thumbnails/37.jpg)
Acknowledgements