N E U R A L N E T W O R K S A N D B U D D H I S T P H I L O S O P H Y
G U S TA V O M U Ñ O Z
A B S T R A C T
• Contemporary results from neural network theory can potentially contribute to a Buddhist understanding of emptiness and the effects of meditation.
I N T R O D U C T I O N
• Neural Networks have the potential to resolve a long-standing paradox of Mahayana Buddhism, the relationship between the conventional and ultimate truth.
• Speculative ideas on connections between neural networks and the effects of meditation.
• How contemplation of a result from contemporary AI research can clarify emptiness, the quintessential idea from classical Buddhism.
B U D D H I S M I N A N U T S H E L L
• Understanding and alleviating sentient beings suffering
• Ignorance of nature of reality as primary cause of suffering
Y O G A C A R A I N A N U T S H E L L
• Yogacara emphasizes misperception of the world as a cause of suffering.
• Rich analysis of psychology of perception.
T W O T R U T H S & T H R E E N AT U R E S
• All buddhist schools: ultimate truth (paramatha-satya) and conventional truth (samvriti-satya).
• Yogacara describes three ways we perceive the world: dependent nature, absolute nature and imaginary nature.
U LT I M AT E T R U T H
• All phenomenons are conditioned, dependent, impermanent and without any essence (non-self).
• This is the very nature of things.
• It is called emptiness (sunyata).
C O N V E N T I O N A L T R U T H
• We construct representations of the world as composed of solid phenomena which are independent, enduring, and characterized by essences.
• Naive perceptions are the conventional truth.
• Taking it as the ultimate truth is the origin of all suffering.
H O W T O R E L AT E B O T H T R U T H S ?
L O T S O F D E B AT E S A L O N G C E N T U R I E S A M O N G D I F F E R E N T
B U D D H I S T S C H O O L S
Y O G A C A R A’ S A P P R O A C H - D E P E N D E N T N AT U R E
• Particular configuration of reality appears to me as a cat, it is not because there is a unconditioned, independent, enduring essence of catness which makes it so.
• From the constant flux of raw sensory data, mind implicitly unifies a set of disparate appearances and labels the unity as a “cat”.
• Viewed as a changing stream of sensory input, the cat is associated with its dependent nature.
Y O G A C A R A’ S A P P R O A C H - A B S O L U T E N AT U R E
• Our idea of cat depends on such streams.
• Configuration of those streams are completely interdependent.
• Configuration of stream at time t1 will depend on the configuration of the stream at least of t0 and other external factors.
• Thus there is no enduring essence of catness within the stream, and this lack of essence is referred as the cat’s and input stream’s absolute nature.
Y O G A C A R A’ S A P P R O A C H - I M A G I N A R Y N AT U R E
• Despite of this, our minds reify the stream and impute an essential catness to it; this mistaken essence is the cat’s imaginary nature.
R E L AT I O N S H I P B E T W E E N T W O T R U T H S
• By reifying the dependent nature (whose true nature is the ultimate nature) we impute the conventional (imaginary) nature.
• In particular the conventional truth, which correspondes to our ordinary experience of the world, is a construction of our own minds.
B U T, H O W T H E I M P U TAT I O N A L N AT U R E A R I S E S F R O M T H E D E P E N D E N T N AT U R E ?
W E N E E D A M E C H A N I C A L A C C O U N T O F I T.
N E U R A L N E T W O R K S T O B R I D G I N G T H E G A P
B R I D G I N G T H E G A P
• Imagine the behavior of a deep neural network as it is trained to recognize cats.
• Like a human, the network will encounter a vast array of non-homogenous raw inputs.
• The region of input space which corresponds to inputs containing cats will be highly structured, but also highly non-localized and discontinuous.
B R I D G I N G T H E G A P
• Unsupervised feature learning + supervised training = learn:
• An appropriate set of high-level features
• And which of these features correspond to images of cats.
• In the end, let us imagine that our network will correctly recognize images containing cats with an error rate similar to that of a human.
F I T T I N G W I T H Y O G A C A R A
• What is involved with the network learning to recognize cats?
• In the first place, one need not make any ontological commitments about an essential catness in order to speak coherently of correctly designating an input as containing a cat.
• One might understand recognition in the same way Buddhism understands conventional designation – a cat is correctly recognized when a common consensus would designate a particular input as a cat.
F I T T I N G W I T H Y O G A C A R A
• Further, one is left with room for a great deal of ambiguity on the boundaries of whatever regions of input might be correctly designated as containing a cat, again supporting the position that there is no essential characteristic the defines such regions.
F I T T I N G W I T H Y O G A C A R A
• If we view the sensory input as being the dependent nature (the various pixels of a receptive field will be in constant flux in dependence on previous configurations of pixels and other causal factors), then the network does not learn the concept of cat by discovering a platonic essence, but rather by learning statistical regularities.
• If the network’s final output neuron uses a binary threshold function, then we can view that output as corresponding to the imaginary nature – the continuous changes in inputs are replaced with a static equivalence relation.
W E M A D E I T !
• Thus it seems that there is some promise in the idea of understanding the two truths and three natures through the lens of contemporary neural network theory.
• In particular it seems possible that the latter can provide a coherent bridge between the three natures.
A M A D H YA M A K A C O N C E R N : E M P T I N E S S O F E M P T I N E S S
• This notion emphasizes that it is important that emptiness itself not be reified and turned into another concept.
• In particular, a proponent of this school (like me) might critique a network interpretation of the two truths as an improper reification of emptiness.
A ( H O P E F U L LY ) FA I R E N O U G H R E S P O N S E
• Neural networks are dynamic, changing phenomena and in no way require reification.
N E U R A L N E T W O R K S A N D T H E E F F E C T S O F M E D I TAT I O N
M E D I TAT I O N : FA M I L I A R I T Y
• Consider attractors of the network viewed as a dynamical system.
• One idea may come from the momentum term sometimes used in back-propagation (back propagation of errors).
• This is a term added to the weight update rule to help the learned weights converge more quickly by preventing them from changing direction too rapidly.
• A notion of momentum may be a way of conceptualizing mental habits and may thus eventually play some kind of role in affording a neural explanation of the effects of meditation.
O N E L A S T C H A L L E N G E : T H E A B I L I T Y O F N E T W O R K S T O R E C O G N I Z E
S P E C I F I C O B J E C T S V I A U N S U P E R V I S E D L E A R N I N G
O B J E C T D E S I G N AT I O N I S A R B I T R A R Y: B U D D H I S M
• The structuring of visual experiences into objects is arbitrary
• In particular there is no inherent reason to divide visual experience into a particular set of features and group those features into the objects that we do.
• The way we divide the world into objects is completely determined by culture or, if you want, environmentally.
R E S U LT S
• Using convolutional networks have shown that all that is required to recover the high-level features is to minimize an energy function to create a sparse coding for inputs.
• A team at Google has had some success in training networks to recognize cats without supervised training.
S O I T M E A N S T H AT…
• One might initially wish to conclude that there is something inherent in the world which compels us to recognize a cat as a cat, contradicting the Buddhist position.
B U T…
• Convolutional networks are specifically modeled on parts of the sentient beings (the very ones who suffer) visual cortex.
• Thus the results should not be interpreted as yielding essential characteristics of the world, but at most as pointing toward something common about the way in which sentient-beings-like visual networks process data.
… A N D …
• The cat features detected by the network were based on the network’s having seen a large volume of cat images.
• Thus while the way we divide the world into objects (as represented by internal feature-maps) may not be culturally determined, it does still seem to be environmentally determined.
• In particular, two people who grow up in completely visually dissimilar environments might still be expected to divide the world into objects differently.
W H E W !
( I S T I L L C A N B E A B U D D H I S T A N D D O A I F O R A L I V I N G )