A Primer to Computational Modeling in Psychology and Neuroscience

My goals for today

•  Not to teach you computational modeling

•  Demystifying computational models

•  Central message:

Computational models are not as complicated (nor as fancy) as they sound, and with a little bit of work,

everyone can incorporate it into their research

Outline

•  What is a computational model?

•  Case Study: Reinforcement learning models of decision-making

•  Why computational modeling? •  Picking up computational modeling

What is a computational model?

Building intuition from simple models

y=ax+b

How do we determine a and b? How do we quantify the error of the model? (i.e. how we measure how good the model is)?

Building intuition from simple models

Basic framework

•  Fit the model(s) to dependent measure (i.e behavior or neural activity) to find optimal value for the free parameters

•  Compare different models to see which model best explains dependent measure

Extending to more complicated models

•  Algorithmic description of cognitive processes

•  System of equations rather than one equation

•  Multiple free, non-independent parameters, so difficult to systematically predict model behavior

Case Study:

Reinforcement learning models of decision-making

Classical Conditioning

Learning rule

Observedvalue

Temporal-difference learning algorithm

Expectedvalue

Update

ImmediateReward

DiscountedFutureReward

Intuition: You care about future rewards as well as current reward When do you need to update your values? When you are surprised! How much should you update? Depends on how surprised you were!

Dopamine Response

Schulz,Dayan&Montague,1997

Before training

After training

Dopamine Response

Dopamine Response

Pessiglione,etal.2006 O’Doherty,etal.2004

Wickens et al., 1996

Three-factor learning rule

Cortex = State Information Striatum = Value of States Dopamine = prediction error

(trains values)

Operant Conditioning

Why computational modeling?

Verbally expressed statements are sometimes flawed by internal inconsistencies, logical contradictions, theoretical weaknesses and gaps. A running computational model, on the other hand, can be considered as a sufficiency proof of the internal coherence and completeness of the ideas it is based upon...” (Fum, Del Misser, Stocco, 2007)

TakenfrompersonalpageofConstanRnA.Rothkopf,PhDhTps://fias.uni-frankfurt.de/~rothkopf/

Modeling Techniques

•  Reinforcement Learning

•  Symbolic systems

•  Neural Networks

•  Bayesian Networks •  Agent-based models

•  Many more …

All models are wrong, but some are useful

George Box

Learning how to model

•  Learn how to code

•  Basics of Modeling Computational Modeling in Cognition: Principles and Practice http://www.amazon.com/Computational-Modeling-Cognition-Principles-Practice/dp/1412970768

•  Reinforcement Learning https://webdocs.cs.ualberta.ca/~sutton/book/ebook/the-book.html

•  Neural Network Models https://grey.colorado.edu/CompCogNeuro/index.php/CCNBook/Main

•  Probabilistic Models (i.e. Bayesian Models)

https://probmods.org/

Some additional pieces of information

Some additional pieces of information

Computers playing Atari games

AlphaGo vs. Lee Seedol (4 -1)