mixed-signal neuromorphic vlsi devices for spiking neural ... · ning qiao neuromorphic cognitive...
TRANSCRIPT
![Page 1: Mixed-signal neuromorphic VLSI devices for spiking neural ... · Ning Qiao Neuromorphic Cognitive Systems INI|UZH|ETH|Zürich 15! Two-stage + 2D tree + 2D mesh multi -cast routing](https://reader030.vdocuments.us/reader030/viewer/2022040500/5e1c4c36b9d753125a08af14/html5/thumbnails/1.jpg)
Ning QiaoInstitute of Neuroinformatics
University of Zurich and ETH Zurich
Jun 20, 2018ADAC6 Workshop
1Ning Qiao Neuromorphic CogniHve Systems INI|UZH|ETH|Zürich
Mixed-signal neuromorphic VLSI devices for spiking neural network
![Page 2: Mixed-signal neuromorphic VLSI devices for spiking neural ... · Ning Qiao Neuromorphic Cognitive Systems INI|UZH|ETH|Zürich 15! Two-stage + 2D tree + 2D mesh multi -cast routing](https://reader030.vdocuments.us/reader030/viewer/2022040500/5e1c4c36b9d753125a08af14/html5/thumbnails/2.jpg)
2Ning Qiao Neuromorphic Cognitive Systems INI|UZH|ETH|Zürich
Outlines
! Brain inspired computing
! Neuromorphic engineering! Analog synapse and neuron circuits
! Multi-core Neuromorphic architectures
! Applications
![Page 3: Mixed-signal neuromorphic VLSI devices for spiking neural ... · Ning Qiao Neuromorphic Cognitive Systems INI|UZH|ETH|Zürich 15! Two-stage + 2D tree + 2D mesh multi -cast routing](https://reader030.vdocuments.us/reader030/viewer/2022040500/5e1c4c36b9d753125a08af14/html5/thumbnails/3.jpg)
3Ning Qiao Neuromorphic Cognitive Systems INI|UZH|ETH|Zürich
l In 2017 > 10 zettabytes of data were produced.
l IT infrastructures and consumer electronics absorbed > 10% of the global electricity supply.
l By 2025, over 50 billion of Internet-of-Things (IoT) devices will be interconnected.
l Over 180 zettabytes of data will be generated annually, potentially leading to a consumption of one-fifth of global electricity.
The cost of current computing technologiesis not sustainable
![Page 4: Mixed-signal neuromorphic VLSI devices for spiking neural ... · Ning Qiao Neuromorphic Cognitive Systems INI|UZH|ETH|Zürich 15! Two-stage + 2D tree + 2D mesh multi -cast routing](https://reader030.vdocuments.us/reader030/viewer/2022040500/5e1c4c36b9d753125a08af14/html5/thumbnails/4.jpg)
4Ning Qiao Neuromorphic Cogni2ve Systems INI|UZH|ETH|Zürich
Current trends in computing HWs
![Page 5: Mixed-signal neuromorphic VLSI devices for spiking neural ... · Ning Qiao Neuromorphic Cognitive Systems INI|UZH|ETH|Zürich 15! Two-stage + 2D tree + 2D mesh multi -cast routing](https://reader030.vdocuments.us/reader030/viewer/2022040500/5e1c4c36b9d753125a08af14/html5/thumbnails/5.jpg)
5Ning Qiao Neuromorphic Cognitive Systems INI|UZH|ETH|Zürich
Brain-Inspired compuBng
1mg weight1mm3 volume960’000 neurons10e-15 J/spike<100 uW
l Slow, noisy and variable processing elements
l Massively parallel distributed computation,
local connectivity
l Real-time interaction with the environment
l Complex spatio-temporal pattern recognitionl Foraging, navigation, language, and social
behavior
![Page 6: Mixed-signal neuromorphic VLSI devices for spiking neural ... · Ning Qiao Neuromorphic Cognitive Systems INI|UZH|ETH|Zürich 15! Two-stage + 2D tree + 2D mesh multi -cast routing](https://reader030.vdocuments.us/reader030/viewer/2022040500/5e1c4c36b9d753125a08af14/html5/thumbnails/6.jpg)
6Ning Qiao Neuromorphic Cognitive Systems INI|UZH|ETH|Zürich
Neuromorphic Compu?ng vs. Neuromorphic Engineering
![Page 7: Mixed-signal neuromorphic VLSI devices for spiking neural ... · Ning Qiao Neuromorphic Cognitive Systems INI|UZH|ETH|Zürich 15! Two-stage + 2D tree + 2D mesh multi -cast routing](https://reader030.vdocuments.us/reader030/viewer/2022040500/5e1c4c36b9d753125a08af14/html5/thumbnails/7.jpg)
7Ning Qiao Neuromorphic Cogni2ve Systems INI|UZH|ETH|Zürich
Current trends in neuromorphic processorsNot so radically different aCer-all (not solving the von Neumann boGleneck problem)
![Page 8: Mixed-signal neuromorphic VLSI devices for spiking neural ... · Ning Qiao Neuromorphic Cognitive Systems INI|UZH|ETH|Zürich 15! Two-stage + 2D tree + 2D mesh multi -cast routing](https://reader030.vdocuments.us/reader030/viewer/2022040500/5e1c4c36b9d753125a08af14/html5/thumbnails/8.jpg)
8Ning Qiao Neuromorphic Cogni2ve Systems INI|UZH|ETH|Zürich
Current trends in neuromorphic processorsNot so radically different aCer-all (not solving the von Neumann boGleneck problem)
![Page 9: Mixed-signal neuromorphic VLSI devices for spiking neural ... · Ning Qiao Neuromorphic Cognitive Systems INI|UZH|ETH|Zürich 15! Two-stage + 2D tree + 2D mesh multi -cast routing](https://reader030.vdocuments.us/reader030/viewer/2022040500/5e1c4c36b9d753125a08af14/html5/thumbnails/9.jpg)
9Ning Qiao Neuromorphic Cogni2ve Systems INI|UZH|ETH|Zürich
“Listen to the silicon” (original approach)Mixed-signal analog/digital neuromorphic systems
! Analog/digital computation, digital asynchronous
communication.
! Directly emulate the physics of neural systems.
! Massively parallel collections of non-linear circuits.
! Realistic neural and synaptic dynamics
! Distributed memory
! Co-localized memory and computation
![Page 10: Mixed-signal neuromorphic VLSI devices for spiking neural ... · Ning Qiao Neuromorphic Cognitive Systems INI|UZH|ETH|Zürich 15! Two-stage + 2D tree + 2D mesh multi -cast routing](https://reader030.vdocuments.us/reader030/viewer/2022040500/5e1c4c36b9d753125a08af14/html5/thumbnails/10.jpg)
10Ning Qiao Neuromorphic Cogni3ve Systems INI|UZH|ETH|Zürich
Channel current-voltage rela3onships
![Page 11: Mixed-signal neuromorphic VLSI devices for spiking neural ... · Ning Qiao Neuromorphic Cognitive Systems INI|UZH|ETH|Zürich 15! Two-stage + 2D tree + 2D mesh multi -cast routing](https://reader030.vdocuments.us/reader030/viewer/2022040500/5e1c4c36b9d753125a08af14/html5/thumbnails/11.jpg)
11Ning Qiao Neuromorphic Cogni2ve Systems INI|UZH|ETH|Zürich
Analog circuitsDirect emula2on of synap2c dynamics
[Bartolozzi and Indiveri, 2007]
![Page 12: Mixed-signal neuromorphic VLSI devices for spiking neural ... · Ning Qiao Neuromorphic Cognitive Systems INI|UZH|ETH|Zürich 15! Two-stage + 2D tree + 2D mesh multi -cast routing](https://reader030.vdocuments.us/reader030/viewer/2022040500/5e1c4c36b9d753125a08af14/html5/thumbnails/12.jpg)
12Ning Qiao Neuromorphic Cognitive Systems INI|UZH|ETH|Zürich
Analog circuitsDirect emulaCon of neuron dynamics
![Page 13: Mixed-signal neuromorphic VLSI devices for spiking neural ... · Ning Qiao Neuromorphic Cognitive Systems INI|UZH|ETH|Zürich 15! Two-stage + 2D tree + 2D mesh multi -cast routing](https://reader030.vdocuments.us/reader030/viewer/2022040500/5e1c4c36b9d753125a08af14/html5/thumbnails/13.jpg)
13Ning Qiao Neuromorphic Cogni3ve Systems INI|UZH|ETH|Zürich
Spike-based plas3city VLSI implementa3on
![Page 14: Mixed-signal neuromorphic VLSI devices for spiking neural ... · Ning Qiao Neuromorphic Cognitive Systems INI|UZH|ETH|Zürich 15! Two-stage + 2D tree + 2D mesh multi -cast routing](https://reader030.vdocuments.us/reader030/viewer/2022040500/5e1c4c36b9d753125a08af14/html5/thumbnails/14.jpg)
14Ning Qiao Neuromorphic Cogni3ve Systems INI|UZH|ETH|Zürich
Cor3cal networks: a high degree of clustering
Pyramidal Cell of Layer 3 of Cat Visual Cortex. Dendrites (Green), Axon (Red), Clusters of Boutons (Black) in Layer 3 and 5. Scale bar, 500 µm
[R.J. Douglas and K.A.C. Martin, Neuron, 2007] [Moradi and Indiveri 2014]
![Page 15: Mixed-signal neuromorphic VLSI devices for spiking neural ... · Ning Qiao Neuromorphic Cognitive Systems INI|UZH|ETH|Zürich 15! Two-stage + 2D tree + 2D mesh multi -cast routing](https://reader030.vdocuments.us/reader030/viewer/2022040500/5e1c4c36b9d753125a08af14/html5/thumbnails/15.jpg)
15Ning Qiao Neuromorphic Cognitive Systems INI|UZH|ETH|Zürich
! Two-stage + 2D tree + 2D mesh multi-cast routing schemes using both source-address and destination-address encoding.
! Fully asynchronous hierarchical routers for intra-core (R1), inter-core (R2) and inter-chip (R3) connectivity.
! Embedded asynchronous CAM and SRAM cells distributed across and within cores.
Memory opAmized mulA-core neural architectureHierarchical rouAng with heterogeneous memory structures
![Page 16: Mixed-signal neuromorphic VLSI devices for spiking neural ... · Ning Qiao Neuromorphic Cognitive Systems INI|UZH|ETH|Zürich 15! Two-stage + 2D tree + 2D mesh multi -cast routing](https://reader030.vdocuments.us/reader030/viewer/2022040500/5e1c4c36b9d753125a08af14/html5/thumbnails/16.jpg)
16Ning Qiao Neuromorphic Cogni3ve Systems INI|UZH|ETH|Zürich
Co-localized memory and computationFD-SOI design, ready for beyond CMOS technology
• Multiple parallel I/O pathways• Multiple distributed asynchronous SRAM
LUTs• Distributed multi-bit TCAM cells• Capacitors for state dynamics and learning
• Ideal for integration with (binary) resistive memories
• Ideal for integration with (learning) memristive devices
• Ideal for integration in 3D VLSI technology
![Page 17: Mixed-signal neuromorphic VLSI devices for spiking neural ... · Ning Qiao Neuromorphic Cognitive Systems INI|UZH|ETH|Zürich 15! Two-stage + 2D tree + 2D mesh multi -cast routing](https://reader030.vdocuments.us/reader030/viewer/2022040500/5e1c4c36b9d753125a08af14/html5/thumbnails/17.jpg)
17Ning Qiao Neuromorphic Cogni3ve Systems INI|UZH|ETH|Zürich
Latest NP chip specs
A large-scale, multi-core, neuromorphic processor DynapSEL in 28 nm FDSOI, is reported in ISSCC 2018
![Page 18: Mixed-signal neuromorphic VLSI devices for spiking neural ... · Ning Qiao Neuromorphic Cognitive Systems INI|UZH|ETH|Zürich 15! Two-stage + 2D tree + 2D mesh multi -cast routing](https://reader030.vdocuments.us/reader030/viewer/2022040500/5e1c4c36b9d753125a08af14/html5/thumbnails/18.jpg)
18Ning Qiao Neuromorphic Cogni3ve Systems INI|UZH|ETH|Zürich
Neural dynamicswith appropriate 3me constants
• Radically different from von Neumann architectures.
• Co-localized memory and computation.• No virtual time (time represents itself).• Data/event driven computation.
Paradigm shift
• For interacting with the environment in real-time.• Inherently synchronized with the real-world “natural” events.• To process “natural” sensory signals efficiently (low bandwidth/power).
“Slow” (biologically plausible) time constants
![Page 19: Mixed-signal neuromorphic VLSI devices for spiking neural ... · Ning Qiao Neuromorphic Cognitive Systems INI|UZH|ETH|Zürich 15! Two-stage + 2D tree + 2D mesh multi -cast routing](https://reader030.vdocuments.us/reader030/viewer/2022040500/5e1c4c36b9d753125a08af14/html5/thumbnails/19.jpg)
19Ning Qiao Neuromorphic Cogni3ve Systems INI|UZH|ETH|Zürich
Event-based convolutional network
![Page 20: Mixed-signal neuromorphic VLSI devices for spiking neural ... · Ning Qiao Neuromorphic Cognitive Systems INI|UZH|ETH|Zürich 15! Two-stage + 2D tree + 2D mesh multi -cast routing](https://reader030.vdocuments.us/reader030/viewer/2022040500/5e1c4c36b9d753125a08af14/html5/thumbnails/20.jpg)
20Ning Qiao Neuromorphic Cognitive Systems INI|UZH|ETH|Zürich
Real-Cme autonomous behaving agents
![Page 21: Mixed-signal neuromorphic VLSI devices for spiking neural ... · Ning Qiao Neuromorphic Cognitive Systems INI|UZH|ETH|Zürich 15! Two-stage + 2D tree + 2D mesh multi -cast routing](https://reader030.vdocuments.us/reader030/viewer/2022040500/5e1c4c36b9d753125a08af14/html5/thumbnails/21.jpg)
21Ning Qiao Neuromorphic Cognitive Systems INI|UZH|ETH|Zürich
Connecting neuromorphic processors to neuromorphicsensors and robots
![Page 22: Mixed-signal neuromorphic VLSI devices for spiking neural ... · Ning Qiao Neuromorphic Cognitive Systems INI|UZH|ETH|Zürich 15! Two-stage + 2D tree + 2D mesh multi -cast routing](https://reader030.vdocuments.us/reader030/viewer/2022040500/5e1c4c36b9d753125a08af14/html5/thumbnails/22.jpg)
22Ning Qiao Neuromorphic Cogni2ve Systems INI|UZH|ETH|Zürich
Hardware preliminary (state-of-the-art) results
N0 N
1
SOURCE NEURONS
INPUT SIGNAL
RESERVOIRNEURONS
OUTPUT LAYER
INPUT SIGNAL SHAPE
RESERVOIR RASTER PLOT
OUT NEURONS PREDICTED ACTIVITY
![Page 23: Mixed-signal neuromorphic VLSI devices for spiking neural ... · Ning Qiao Neuromorphic Cognitive Systems INI|UZH|ETH|Zürich 15! Two-stage + 2D tree + 2D mesh multi -cast routing](https://reader030.vdocuments.us/reader030/viewer/2022040500/5e1c4c36b9d753125a08af14/html5/thumbnails/23.jpg)
23Ning Qiao Neuromorphic Cognitive Systems INI|UZH|ETH|Zürich
Distributed ArDficial Intelligence
Autonomoussensory-motor systems
embedded systems &emerging memory technologies
Brain Machine Interfaces & prosthetics
![Page 24: Mixed-signal neuromorphic VLSI devices for spiking neural ... · Ning Qiao Neuromorphic Cognitive Systems INI|UZH|ETH|Zürich 15! Two-stage + 2D tree + 2D mesh multi -cast routing](https://reader030.vdocuments.us/reader030/viewer/2022040500/5e1c4c36b9d753125a08af14/html5/thumbnails/24.jpg)
24Ning Qiao Neuromorphic Cognitive Systems INI|UZH|ETH|Zürich
Team Work: InsCtute of NeuroinformaCcs
• Ning Qiao (INI)• Yulia Sandamirskaya (INI)• Lorenz Mu ̈ller (INI)• Melika Payvand (INI)• Elisa Donati (INI)• Dongchen Liang (INI)• Raphaela Kreise (INI)• Moritz Milde (INI)• Marc Osswald (inSightness)
• Dora Sumislawska (GeorgiaTech, USA)• Fabio Stefanini (Columbia Univ., USA)• Jonathan Binas (Univ. Montreal, CA)• Emre Neftci (UC Irvine, USA)• Saber Moradi (Yale, USA)• Hesham Mostafa (UCSD, USA)• Chiara Bartolozzi (IIT, Italy)• Elisabetta Chicca (Univ. Bielefeld, DE)• Stefano Fusi (Columbia Univ., USA)
![Page 25: Mixed-signal neuromorphic VLSI devices for spiking neural ... · Ning Qiao Neuromorphic Cognitive Systems INI|UZH|ETH|Zürich 15! Two-stage + 2D tree + 2D mesh multi -cast routing](https://reader030.vdocuments.us/reader030/viewer/2022040500/5e1c4c36b9d753125a08af14/html5/thumbnails/25.jpg)
25Ning Qiao Neuromorphic Cognitive Systems INI|UZH|ETH|Zürich
Technology-transfer effort to commercialize
• Dr. Ning Qiao• Prof. Giacomo Indiveri• Dr. Kynan Eng• Dr. Dylan Muir• Dr. Sadique Sheik• Dr. Qian Liu• Felix Bauer• Carsten Nielsen• Ole Richter• Anita Tuomi
![Page 26: Mixed-signal neuromorphic VLSI devices for spiking neural ... · Ning Qiao Neuromorphic Cognitive Systems INI|UZH|ETH|Zürich 15! Two-stage + 2D tree + 2D mesh multi -cast routing](https://reader030.vdocuments.us/reader030/viewer/2022040500/5e1c4c36b9d753125a08af14/html5/thumbnails/26.jpg)
26Ning Qiao Neuromorphic Cognitive Systems INI|UZH|ETH|Zürich
The end
Thank you for your attention
![Page 27: Mixed-signal neuromorphic VLSI devices for spiking neural ... · Ning Qiao Neuromorphic Cognitive Systems INI|UZH|ETH|Zürich 15! Two-stage + 2D tree + 2D mesh multi -cast routing](https://reader030.vdocuments.us/reader030/viewer/2022040500/5e1c4c36b9d753125a08af14/html5/thumbnails/27.jpg)
27Ning Qiao Neuromorphic Cogni3ve Systems INI|UZH|ETH|Zürich
Spiking Neuron Network (SNN)
![Page 28: Mixed-signal neuromorphic VLSI devices for spiking neural ... · Ning Qiao Neuromorphic Cognitive Systems INI|UZH|ETH|Zürich 15! Two-stage + 2D tree + 2D mesh multi -cast routing](https://reader030.vdocuments.us/reader030/viewer/2022040500/5e1c4c36b9d753125a08af14/html5/thumbnails/28.jpg)
28Ning Qiao Neuromorphic Cognitive Systems INI|UZH|ETH|Zürich
Spiking Neuron Network (SNN)
Communication Computation Learning
![Page 29: Mixed-signal neuromorphic VLSI devices for spiking neural ... · Ning Qiao Neuromorphic Cognitive Systems INI|UZH|ETH|Zürich 15! Two-stage + 2D tree + 2D mesh multi -cast routing](https://reader030.vdocuments.us/reader030/viewer/2022040500/5e1c4c36b9d753125a08af14/html5/thumbnails/29.jpg)
29Ning Qiao Neuromorphic Cognitive Systems INI|UZH|ETH|Zürich
• No I/O bottleneck• No memory bottleneck
Intel i7-4960X DYNAP-SEL
Co-localized memory and computationFD-SOI design, ready for beyond CMOS technology
![Page 30: Mixed-signal neuromorphic VLSI devices for spiking neural ... · Ning Qiao Neuromorphic Cognitive Systems INI|UZH|ETH|Zürich 15! Two-stage + 2D tree + 2D mesh multi -cast routing](https://reader030.vdocuments.us/reader030/viewer/2022040500/5e1c4c36b9d753125a08af14/html5/thumbnails/30.jpg)
30Ning Qiao Neuromorphic Cognitive Systems INI|UZH|ETH|Zürich
A closed-loop bi-direcEonal BMI