Bounded Rationality: Thinking Is Costly

A baseball and bat together cost $11. The bat costs $10 more than the ball. How much does the ball cost?

Write down your answer.

Half of Harvard students said $1, which is the intuitive answer but wrong!

Correct answer is 50 cents: $10.50-$.50 = $10.00

People tend to use “intuitive thinking” or rules of thumb

CONCURRENT SCHEDULES OF REINFORCEMENTStudy of choice centers on

how operant behavior is affected its reinforcement history and by reinforcement history of other operant behaviors.

Research uses concurrent schedules: two or more schedules that operate simultaneously and independently, each for a different response.

Concurrent schedules

of reinforcement

Two schedules are in effect at the same time and the subject is free to switch from one response alternative to the other

Key A Key B

Schedule AVI 60 s

Schedule BFR 10

Choice Behaviorand the

Matching Law

Relative rate of responding on a particular lever equals the relative rate of reinforcement on that lever

The Matching Law is a mathematical statement describingthe relationship between the rate of responding and the rate of reward

developed by Herrstein

Matching Law

RA/(RA + RB) = rA/(rA + rB) orRA /rA = RB /rB

RA = Responses to ARB = Responses to BrA = Reinforcers to ArB = Reinforcers to B

Matching Law

Provides accurate description of behavior of many organisms in many different choice situations.Different speciesAppetitive and aversive stimuli Frequency, magnitude, and delay of

reinforcement

People Aren’t Always RationalStudies find that people make systematic mistakes: People are overconfident.

People give too much weight to a small number of vivid observations.

People are reluctant to change their minds.

Even though people are not always rational, the assumption that they are is usually a good approximation for economic modeling.

Deviations from Matching

Impatience: the desire for instant gratificationRead, Loewenstein & Kalyanaraman (1999)

Choose among 24 movie videosSome are “low brow”: My Cousin VinnySome are “high brow”: Schindler’s List

Picking for tonight: 56% of subjects choose low brow. Picking for next Thursday: 37% choose low brow. Picking for second Thursday: 29% choose low brow.

Tonight I want sugar-coated entertainment… next week I want things that are good for me.

Choice with Commitment

In a standard concurrent schedule of reinforcement, two (or more) response alternatives are available at the same time and the subject is free to switch from one to the other at any time

However, in some (real-life) situations, choosing one alternative makes other alternatives unavailable

In these cases, the choice may involve assessing complex,long-range goals

Can study these types of situations in the lab using aConcurrent-chain schedule of reinforcment

Terminal link

BA

Time

Choice link

Reinforcementschedule A

(VR 10)

Reinforcementschedule B

(FR 10)

Pecking the left key in the choice link puts into effect reinforcement schedule A in the terminal link. Pecking the right key in the choice link puts into effect reinforcement schedule B in the terminal link.

Concurrent-chain schedule

BA

Time

Smallreward

Largereward

Delay

A B

Smallreward

Largereward

DelayDirect-choice procedurePigeon chooses immediate,small reward

Concurrent-chain procedurePigeon chooses the schedulewith the delayed, larger reward

Self-Control

Concurrent chain schedules have been used to study ‘self-control’ in the lab

e.g., choosing a large delayed reward over animmediate small reward

With direct choice procedures, animals often lackself-control. That is, they choose the immediate, butsmaller reward

With concurrent-chain procedures, animals do showself-control. That is, they choose the larger, but delayedreward

Choosing fruit vs. chocolate

TimeChoosing Today Eating Next Week

If you were deciding today,would you choosefruit or chocolatefor next week?

Patient choices for the future:

TimeChoosing Today Eating Next Week

Today, subjectstypically choosefruit for next week.

74%choosefruit

Impatient choices for today:

Time

Choosing and Eating

Simultaneously

If you were deciding today,would you choosefruit or chocolatefor today?

Time Inconsistent Preferences:

Time

Choosing and Eating

Simultaneously

70%choose chocolate

Rachlin and Green (1972)If you delay the reinforcer for both a larger and smaller reinforcer and then you give organism a choice, it may well choose the larger delayed reinforcer

10 second delay

10 second delay

2 seconds foodimmediate

4 seconds foodMust wait 4seconds

4 seconds foodMust wait 4seconds

FR - 15

FR - 15

Self-Control

Why does anyone choose a smaller reward part of the time? Animals and people typically choose a

small immediate reward over a larger delayed reward.

Large rewards are selected when: The choice is made in advance of

reward. Reinforcers are not visible or reward

is already present (pleasurable activity).

What mechanism may account for this short temporal horizon? Two Brain System Theory

of Addiction Increased activity of the

Impulsive System (motivational)

Extented Amygdala Ventral Striatum

Decreased activity of Executive System (rationality)

Ventromedial Prefrontal Cortex (VMPC)

Dorsolaterial Prefrontal Cortex (DLPC)

Self-Control

Temporal Issue Lack of self-control arises from the fact that our behavior

is more heavily influenced by immediate consequences as opposed to delayed consequences.

Immediate Consequence Delayed Consequence

quitting withdrawal Improved health

smoking Nicotine high Deterioration of health

Self-control – preference for larger later reward

Impulsiveness – preference for smaller sooner reward

Which do you prefer

$500 now or $1,000 in two years$500 in four years or $2,000 in six years

People Are Inconsistent Over TimePeople tend to prefer instant gratification,

even when delaying would increase the gratification.

Result: People fail to follow through on plans to do things that are dreary, take effort, or cause discomfort. e.g., people often save less than they plan

To help follow through, people look for ways to commit themselves to their plans. e.g., worker has money taken out of paycheck

before he ever sees it

Choice and Foraging

Laboratory paradigms are often criticized because they do not capture essence of natural contingencies.

More recent research has tried to bring natural contingencies of reinforcement into laboratory.

This move has been prompted by optimal foraging theory.

Choice and Foraging

Optimal foraging theory: feeding behavior is sensitive to relation between amount of energy expended in finding, securing, and consuming food, and amount of energy or nutrition of food.

Patterns of foraging optimize relation between energy gain and energy expenditure.

Optimal foraging theory: feeding behavior is sensitive to relation between amount of energy expended in finding, securing, and consuming food, and amount of energy or nutrition of food. Patterns of foraging optimize relation between energy gain and energy expenditure

Behavioral Economics

Behavioral economics: a new field in which economists apply basic insights from psychology

People aren’t always as rational as traditional economic models assume.

Herbert Simon viewed humans as satisficers,people who make choices that are merely “good enough” rather than optimal.

Other economists have suggested that people are only “near rational” or exhibit “bounded rationality.”

OPERANT BEHAVIOR AND ECONOMICSIt is not surprising that behavior theorists

have applied economic concepts to their own domain.

Matching Law is viewed by some theorists as special case of general economic principles.

OPERANT BEHAVIOR AND ECONOMICS: BASICSDemand (amount purchased at given

price): Elastic (luxuries) Inelastic (necessities)

Commodities: Substitutes (more of one, less of other) Complements (more of one, more of other)

Income (money or responses required to purchase commodities at given price)

OPERANT BEHAVIOR AND ECONOMICS: BASICSDemand (amount purchased at given

price): Elastic (luxuries) Inelastic (necessities)

Commodities: Substitutes (more of one, less of other) Complements (more of one, more of other)

Income (money or responses required to purchase commodities at given price)

Rodents = Bar press

OPERANT BEHAVIOR AND ECONOMICS: BASICSEconomy (relationship between

commodities and income: Closed (fixed income) Open (extra income): most operant studies

Deprivation

People Care About Fairness

People’s choices are sometimes influenced more by their sense of fairness than self-interest.

Example: the ultimatum game

The rules Two players who do not know each

other have a chance to share a prize of $100.

Player A decides what portion of the prize to give to player B.

B must accept the split or both get nothing.

People Care About FairnessPredicted outcome if both players rational A would propose a 99-1 split and B would accept,

because $1 is better than nothing.

Actual outcome from experiments with real people B usually rejects lopsided splits like 99-1

as wildly unfair. Expecting this, A usually proposes giving

$30 or $40 to B. B views this as unfair, but not so much as to

abandon his self-interest, so B accepts.

People Care About Fairness

The results of the ultimatum game apply in other situations.

Example: A firm may pay above-equilibrium wages during profitable years to be fair, or to avoid appearing unfair and risking retaliation from workers.

OPERANT BEHAVIOR AND ECONOMICS Evidence suggests behavior of animals in

conditioning experiments conforms to what economic theory says people will do when confronted with similar choices.

Implies that operant behavior involves a kind of economic decision making.

Animals must decide how to allocate scarce behavioral resources; rules by which they do so are like those people use.

Economic theory and behavior theory may jointly explain human and animal choice.