N E U R A L N E T W O R K S A N D B U D D H I S T P H I L O S O P H Y

G U S TA V O M U Ñ O Z

A B S T R A C T

• Contemporary results from neural network theory can potentially contribute to a Buddhist understanding of emptiness and the effects of meditation.

I N T R O D U C T I O N

• Neural Networks have the potential to resolve a long-standing paradox of Mahayana Buddhism, the relationship between the conventional and ultimate truth.

• Speculative ideas on connections between neural networks and the effects of meditation.

• How contemplation of a result from contemporary AI research can clarify emptiness, the quintessential idea from classical Buddhism.

B U D D H I S M I N A N U T S H E L L

• Understanding and alleviating sentient beings suffering

• Ignorance of nature of reality as primary cause of suffering

Y O G A C A R A I N A N U T S H E L L

• Yogacara emphasizes misperception of the world as a cause of suffering.

• Rich analysis of psychology of perception.

T W O T R U T H S & T H R E E N AT U R E S

• All buddhist schools: ultimate truth (paramatha-satya) and conventional truth (samvriti-satya).

• Yogacara describes three ways we perceive the world: dependent nature, absolute nature and imaginary nature.

U LT I M AT E T R U T H

• All phenomenons are conditioned, dependent, impermanent and without any essence (non-self).

• This is the very nature of things.

• It is called emptiness (sunyata).

C O N V E N T I O N A L T R U T H

• We construct representations of the world as composed of solid phenomena which are independent, enduring, and characterized by essences.

• Naive perceptions are the conventional truth.

• Taking it as the ultimate truth is the origin of all suffering.

H O W T O R E L AT E B O T H T R U T H S ?

L O T S O F D E B AT E S A L O N G C E N T U R I E S A M O N G D I F F E R E N T

B U D D H I S T S C H O O L S

Y O G A C A R A’ S A P P R O A C H - D E P E N D E N T N AT U R E

• Particular configuration of reality appears to me as a cat, it is not because there is a unconditioned, independent, enduring essence of catness which makes it so.

• From the constant flux of raw sensory data, mind implicitly unifies a set of disparate appearances and labels the unity as a “cat”.

• Viewed as a changing stream of sensory input, the cat is associated with its dependent nature.

Y O G A C A R A’ S A P P R O A C H - A B S O L U T E N AT U R E

• Our idea of cat depends on such streams.

• Configuration of those streams are completely interdependent.

• Configuration of stream at time t1 will depend on the configuration of the stream at least of t0 and other external factors.

• Thus there is no enduring essence of catness within the stream, and this lack of essence is referred as the cat’s and input stream’s absolute nature.

Y O G A C A R A’ S A P P R O A C H - I M A G I N A R Y N AT U R E

• Despite of this, our minds reify the stream and impute an essential catness to it; this mistaken essence is the cat’s imaginary nature.

R E L AT I O N S H I P B E T W E E N T W O T R U T H S

• By reifying the dependent nature (whose true nature is the ultimate nature) we impute the conventional (imaginary) nature.

• In particular the conventional truth, which correspondes to our ordinary experience of the world, is a construction of our own minds.

B U T, H O W T H E I M P U TAT I O N A L N AT U R E A R I S E S F R O M T H E D E P E N D E N T N AT U R E ?

W E N E E D A M E C H A N I C A L A C C O U N T O F I T.

N E U R A L N E T W O R K S T O B R I D G I N G T H E G A P

B R I D G I N G T H E G A P

• Imagine the behavior of a deep neural network as it is trained to recognize cats.

• Like a human, the network will encounter a vast array of non-homogenous raw inputs.

• The region of input space which corresponds to inputs containing cats will be highly structured, but also highly non-localized and discontinuous.

B R I D G I N G T H E G A P

• Unsupervised feature learning + supervised training = learn:

• An appropriate set of high-level features

• And which of these features correspond to images of cats.

• In the end, let us imagine that our network will correctly recognize images containing cats with an error rate similar to that of a human.

F I T T I N G W I T H Y O G A C A R A

• What is involved with the network learning to recognize cats?

• In the first place, one need not make any ontological commitments about an essential catness in order to speak coherently of correctly designating an input as containing a cat.

• One might understand recognition in the same way Buddhism understands conventional designation – a cat is correctly recognized when a common consensus would designate a particular input as a cat.

F I T T I N G W I T H Y O G A C A R A

• Further, one is left with room for a great deal of ambiguity on the boundaries of whatever regions of input might be correctly designated as containing a cat, again supporting the position that there is no essential characteristic the defines such regions.

F I T T I N G W I T H Y O G A C A R A

• If we view the sensory input as being the dependent nature (the various pixels of a receptive field will be in constant flux in dependence on previous configurations of pixels and other causal factors), then the network does not learn the concept of cat by discovering a platonic essence, but rather by learning statistical regularities.

• If the network’s final output neuron uses a binary threshold function, then we can view that output as corresponding to the imaginary nature – the continuous changes in inputs are replaced with a static equivalence relation.

W E M A D E I T !

• Thus it seems that there is some promise in the idea of understanding the two truths and three natures through the lens of contemporary neural network theory.

• In particular it seems possible that the latter can provide a coherent bridge between the three natures.

A M A D H YA M A K A C O N C E R N : E M P T I N E S S O F E M P T I N E S S

• This notion emphasizes that it is important that emptiness itself not be reified and turned into another concept.

• In particular, a proponent of this school (like me) might critique a network interpretation of the two truths as an improper reification of emptiness.

A ( H O P E F U L LY ) FA I R E N O U G H R E S P O N S E

• Neural networks are dynamic, changing phenomena and in no way require reification.

N E U R A L N E T W O R K S A N D T H E E F F E C T S O F M E D I TAT I O N

M E D I TAT I O N : FA M I L I A R I T Y

• Consider attractors of the network viewed as a dynamical system.

• One idea may come from the momentum term sometimes used in back-propagation (back propagation of errors).

• This is a term added to the weight update rule to help the learned weights converge more quickly by preventing them from changing direction too rapidly.

• A notion of momentum may be a way of conceptualizing mental habits and may thus eventually play some kind of role in affording a neural explanation of the effects of meditation.

O N E L A S T C H A L L E N G E : T H E A B I L I T Y O F N E T W O R K S T O R E C O G N I Z E

S P E C I F I C O B J E C T S V I A U N S U P E R V I S E D L E A R N I N G

O B J E C T D E S I G N AT I O N I S A R B I T R A R Y: B U D D H I S M

• The structuring of visual experiences into objects is arbitrary

• In particular there is no inherent reason to divide visual experience into a particular set of features and group those features into the objects that we do.

• The way we divide the world into objects is completely determined by culture or, if you want, environmentally.

R E S U LT S

• Using convolutional networks have shown that all that is required to recover the high-level features is to minimize an energy function to create a sparse coding for inputs.

• A team at Google has had some success in training networks to recognize cats without supervised training.

S O I T M E A N S T H AT…

• One might initially wish to conclude that there is something inherent in the world which compels us to recognize a cat as a cat, contradicting the Buddhist position.

B U T…

• Convolutional networks are specifically modeled on parts of the sentient beings (the very ones who suffer) visual cortex.

• Thus the results should not be interpreted as yielding essential characteristics of the world, but at most as pointing toward something common about the way in which sentient-beings-like visual networks process data.

… A N D …

• The cat features detected by the network were based on the network’s having seen a large volume of cat images.

• Thus while the way we divide the world into objects (as represented by internal feature-maps) may not be culturally determined, it does still seem to be environmentally determined.

• In particular, two people who grow up in completely visually dissimilar environments might still be expected to divide the world into objects differently.

W H E W !

( I S T I L L C A N B E A B U D D H I S T A N D D O A I F O R A L I V I N G )