biologically inspired robotics group,epfl associative memory using coupled non-linear oscillators...
Post on 22-Dec-2015
215 views
TRANSCRIPT
QuickTime™ et undécompresseur TIFF (non compressé)sont requis pour visionner cette image.
Biologically Inspired Robotics Group,EPFLQuickTime™ et undécompresseur TIFF (non compressé)sont requis pour visionner cette image.
Associative memory using coupled
non-linear oscillators
Semester project
Final Presentation
Vlad TRIFA
Project summary
Litterature review Implementation of an associative memory using coupled
oscillators and analysis of performance/drawbacks. Mixture with the BIRG model
Generalization to complex signals Better control on the capacity
Final discussion about relevant issues concerning the performances of both models
Conclusion
Associative Memory
Animal and human memory works by association.Able to retrieve a stored pattern upon presentation of a partial and noisy representation of an input signal.
Many models developed since early 80’s• Concepts taken from statisic mechanics and hebbian learning
rule turned neural networks into dynamic systems.• Useful into understanding dynamics of networks (emergence)
but…• Lack of biologically plausible mechanisms (coupling, binary,…)• Low capacity and performance (Global coupling: N2 parameters)
Oscillators
Oscillating systems are very common in nature and possess very intersting properties.
• Synchronization• Energy efficient mechanism for temporal correlation
Many brain processes rely on interaction of oscillators• CPG• Olfactory and visual cortex • Temporal correlation hypothesis and binding problem
Information can be stored as phase relationships patterns, where coupled oscillators converge.
Analyzed model
Can be found in [Borisyuk, 2001].
Oscillators described by phase, amplitude, and frequency.
Model performance
The capacity of this model is not easy to derive, due to the random phase shifts, and to the dynamics of the nonlinear term:
We do not know what percentage of overlapping is
possible, as memorized patterns can be “overwritten“. Implies that error increases as memory is filled. Robustness due to distributed memorization. But, loss of groups influence strongly the retrieval error.
Discussion
The model is interesting as it is based on oscillating systems, thus can be easily implementable on many oscillating systems (PLL, etc…).
A very nice methodology that is embedded in the system is proposed in order to decide where to store an input signal is proposed.
Random phase shifts ensure some robustness to the system, but too big influence on the performance.
Drawbacks
The all-to-all coupling into groups is not efficient computationally and it uses too many oscillators.
Due to the explicit input signal embedded in the equations, we can only learn sine functions.
The input dimension is annoying. Complexity is increased with no performance increase.
The time is reset after each stimulus. We need to present the input in-phase with the oscillators. We cannot learn sequences.
Improvements
We want to be able to learn complex signals.
Starting from the model in [Righetti et al, 2005], we want to extend the model to form a network.
Discussion
Simpler model, more computationally efficient. We gained a much better control on the amount of
oscillators to dedicate for a frequency component. We are able to memorize complex signals in a robust
and fault tolerant manner, under some constraints.
but… Unfortunately, the capacity depends on the complexity of
the signals to store. We lost the selection of the storage sites based on
phase relationships we had with the previous model.
Future work
We need to find a mechanism (embedded in the dynamics) that can select where each component should be stored depending on the signal.
It would be very intersting to create links between different clusters activated by the same signal, similar to associative connections forming according to the correlation of neural activity between assemblies, enhancing robustness if attenuated components.
Reduce parameters, so we need only to select the amount of oscillators allocated per component.
Conclusion
This work should be considered as an attempt to provide insights on how it is possible to store information encoded as a complex signals in a reliable manner, simply by using oscillating systems with local interactions.
Our approach is interesting as it uses some concepts that are common in biological neuronal networks such as Oscillating components with local interactions No global external process to supervise the learning procedure