Other Models of Choice

Molar Maximizing

• Idea that animals maximize their responding– Most reinforcement for least effort– Maximize over a long time horizon

• can explain why subjects switch with concurrent VI-VI schedules but not with concurrent ratio schedules

Molar Maximizing

• molecular and molar theories– molecular theories- animals choose whichever

response alternative is most likely to be reinforced at that time

– Your grade in an individual class– best alternative at the moment

• molar theories- overall strategy– Animals choose responses using long term strategy– Your overall GPA

Melioration

• To meliorate means to make better

• Proposes that subjects shift responding toward one or other alternative until both alternatives are equally attractive in terms of reinforcement per unit of effort or time

• Produces matching on concurrent VI VI schedules

• Predicts exclusive responding on more favorable schedule on concurrent VR VR

Temporal discounting

Temporal Discounting

• Len Green; Mark Dixon; Tom Critchfield

• Theory or model which examines time frame under which people choose longer/later vs. Shorter/smaller

• How long are you willing to wait for a reward

• Assumes that the value of a reward is discounting (decreases) as time between response and reinforcer increase

• http://www.youtube.com/watch?v=G7LN96jEXHc

Temporal Discounting • Accounts for commitment to choice in a two-choice

paradigm. – Commitment to choice is considered a form of self control,

and corresponds with low levels of impulsiveness

• Is explained by a hyperbolic relationship between value and delay of reinforcer

• Eq. 2– V is the present value of future reward– A is it’s amount– D is the delay to reinforcement – k is the value of discounting

(Rachlin & Green, 1972; Rachlin and Logue, 1976; Mazur, 1987, 1988)

Temporal Discounting as a Measure of Impulsivity in Rats: Pitts & McKinney (2005)

• Compared effects of:– Ritalin (1.0, 3.0, 5.6, 10.0, or 17.0 mg/kg)– Morphine (0.3, 1.0, 5.6, 10.0, or 17.0 mg/kg)

• Reinforcers:– SS = 3s access to 0.02cc of sugar water– LL = two different delay schemes, and was four 3s accesses to 0.02cc of sugar

water

• Rats were exposed to a delay discounting task where:– Group 1: exposed to delay scheme 1 for the LL– Group 2: exposed to delay scheme 1 & 2 for the LL: – Delay scheme 1 = 0, 5, 10, 20, 30, 40, or 50s– Delay scheme 2 = 0, 2, 4, 6, 8, 16, or 20s

Pitts & McKinney (2005) cont. • Results:

– Rats chose the LL over the SS only for the 3.0 mg/kg and the 5.6 mg/kg doses of Ritalin.

– Remained impulsive at low or very high doses– Morphine data showed never learned to choose LL over SS

• Ritalin, which is often prescribed in lieu of D-AMPH as an ADHD medication, effects discounting in a manner much like D-AMPH

Animal Data consistent

• Animals tend to be impulsive: choose SS over LL

• Some drugs can alter this pattern; typically these are drugs that increase the tonic level of DA

• Green, et al. (2013) showed that time delay is critical:– Animals choosing NOW vs. later are impulsive– If choose “a little later” vs. “later”- less impulsivve– Seem to respond to “A bird in the hand is better than 2 in the

bush”– This makes sense ecologically

But what about humans?

• Use more “cognitive” task• Ask subjects to choose between SS and LL

– Do you want $10 now or $100 in 10 minutes• $100 in 1 day• $100 in 2 days, etc.

– Ascending and descending series using constant “smaller”– Also vary the probability of the later reward– Look at “switching point”

• At what point is the human no longer willing to wait?

Problems with human studies• Real vs. hypothetical rewards

– Some slight differences in data– But data seem as good for pretend as for real!

• Discounting functions look just like animal functions, with similar fits to themodel

• Group vs. individual analyses

• Stability of temporal discounting patterns– Population differences: gamblers vs. non gamblers– Experience plays a role– K is context-sensitive and experience driven measure of impulsiveness

• Look at an example

Temporal Discounting as a Measure of Impulsivity in Humans

• Shiels et al., 2009– Measured delay discounting in children with ADHD

• Children were measured both on and off medication

– Three groups• More innatentive ADHD• More hyperactive ADHD• Normal control

– All three groups were tested on and off both doses of ADHD medication• Low dose Concerta (39 mg)• High dose Concerta (73 mg)

Shiels et al., 2009 cont.

• Children were exposed to a discounting task– Choose 5 cents now (SS)– Choose 30 cents later (LL)

• Delayed option was manipulated probabilistically – 7s, 14s, or 28s delay– With a constant 35% of being delivered, 65% not being delivered

• Results concluded that:– Children with ADHD significantly chose SS over LL– Each dose of Concerta significantly lowered preference for SS in ADHD

children • Resulted in similar responding for ADHD children and normal controls

Functional Characteristics of Psychological Disorders

• Topographical features of a disorder does not necessarily provide answer as to function of the behavior: e.g., Self-injurious Behavior (SIB)– Escape from demand– Attention– To get treats/toys, etc.– Self stimulation

• But: Behavior analysis has shown that groups of individuals can be differentiated by the function of their behavior:– Violent vs. nonviolent criminals and aggression– ADHD vs. typical kids– Addicts vs. nonaddicts

Does looking at function of behavior help improve diagnosis and treatment

-and how does this tie in the Temporal discounting!

• Examine ADHD:– Can describe topography of behaviors– But vast differences among those diagnoses

• What if examine sensitivity to reinforcement and temporal discounting?– Must define each term– Use models to drive definitions

Explaining ADHD

• Individuals with ADHD are impulsive because:– Look at context of immediate and delayed

consequences– Immediate reinforcers unusually potent:

competing reinforcers heavily discounted due to delay

– Relatively unaffected by delayed punishment– Thus: should see differences in temporal

discounting and reward sensitivity between typical and ADHD individuals

Why is this important?

• Applied data on temporal discounting are informative in 2 ways:– sensitivity to consequences =problem of choice

• Provin context of reinforcement settings, all behavior occurs at expense of another, competing behavior

• Competing behaviors loom large• Response strength = function of strength of alternative behaviors!

– Choice occurs across extended temporal contexts• Delayed consequences have different effect on behavior than immediate

consequences• With passage of time, responses and consequences aggregate, and relation

between them grows more complex• May not be able to parcel out effect of individual response-consequence, but

must take in context

Molar vs. Molecular

• Molar sequences of responses may be just as important as sequences of consequences– Again, context and control over pattern– Variables such as time and effort to change a pattern

of responding are important

• A shift in an established response pattern may have its own value that must be taken into consideration (oh, but this will be behavioral momentum theory in a few weeks!)

Example: Drug Addiction

• Socially important aspects of abuse behavior are molar rather than molecular patterns of responding:– Abuse– Recovery– Relapse

• Amount of responding maintained by drug reinforcers depends on characteristics and availability of alternative reinforcers– Oxycontin abusers do not use illegal drugs or engage in high rates

of illegal behavior– Until they lose access to prescription drugs!

Why an impulsive addict?

• Substance abuse behaviors produce favorable immediate consequences but delay unfavorable consequences

• Non drinking behaviors facilitate favorable delayed consequences but less tangible immediate benefits

• Abuse behaviors compete for time and resources with non-abuse behaviors

Can explain some drinking behavior:

• Research examined 2 specific issues:– Why do habitual heavy drinkers occassionally abstain from drinking

independently or by seeking treatment?– Why do alcohol abusers who have received treatment start drinking again

(relapse)?

• Must track drinking episodes and environmental events that might contribute to them– Over weeks, months, years– Must use self report, but data are quite reliable!

• Vuchinich and Tucker, 1995: Examined temporal patterns of drinking and its relation to a variety of environmental events – Assumed that these events reflected the reinforcing value of engaging drinking

vs. not drinking alternative behavior

Vuchinich and Tucker (1995)

• Indiviudals who ceased drinking had experienced variety of drinking related outcomes that threatened availability of non-drinking related reinforcers– These events accumulated until sufficiently

different to change value of drinking vs. not drinking- then sought treatment

– Hitting “bottom”– Cost of drinking became temporally acute

Vuchinich and Tucker (1995)

• Individuals who started drinking had experienced adverse social and vocational events which preceded onset of drinking

• Again, these events accumulated until sufficiently different to change value of drinking vs. not drinking- then sought treatment– Hitting “bottom” for alternative sources of

reinforcement– Cost of “not drinking” became temporally acute

Why important to understand in these terms?

• Suggest a molar relation between overall patterns of drinking and aggregated consequences for alternative nondrinking behavior– Similar to data from laboratory drug data, suggesting may be same

underlying process

• Understanding this relationship guides development of intervention protocols– Must build behavioral resistance for alternative nondrinking; make

resistance to behavioral disruption

• Suggests focus must be on boosting reinforcement for alternative behaviors, not focusing on “not drinking”

What else might be explained via this model?

• Any choice behavior, really• Eating vs. exercise• Other addictive behaviors• Mental illness and medication compliance• Saving vs. spending• Studying vs. not studying!

• But is there a biological explanation that could support this? Yes!

Dopamine and Reward

A brief review

Reinforcement and Dopamine?

• Olds and Milner:– Brain stimulation = lots of behavior– Animals would work until death to gain access to

this brain stimulation– Thought had discovered pleasure center: • Nucleus accumbens• Mesolimbic pathway

• Dopamine (DA) was neurotransmitter involved in these areas

Reinforcement and DA

• EBS = releasing LOTS of dopamine (DA) – Results in lots of locomotion or exploratory behavior

– Salamone and Schultz’ modern work has shown this release

• DA release modulates “appetitive behavior”– Occurs in modes or modules related to terminal event– E.g., food modes, sex modes– Which mode depends on context of environment

• Search, capture, prepare, consume• DA not affect consummatory behavior

DA is regulated in two ways:

• DA is released in pulses (phasic) and has an overall tone in synapse (tonic)– Result is feedback system

• DA release in response to stimuli in environment– Motivated or energize appropriate behavior– Feedback system follows Rescorla Wagner model

DA regulation

• DA also has a constant level or overall tone in the synapse:– More released when “surprised above what expected– Less when “surprised” below what expected– When get what expected- behavior is “well learned”

and appears to become habit (not sure how this works yet)

• Thus, fluctuations in DA as learn, and then serve as feedback regarding state of environment

Temporal Discounting and DA• Changes in phasic and tonic DA may help explain temporal discounting

• DA tone decays across time– Thus background by which DA phasic spikes are related is changes– Value of a reward changes across time

• With short delay:– DA spike for small reinforcer is obviously different than DA spike for larger/more valuable

reinforcer

• With long delay:– DA tone for larger/more valuable reinforcer decays with time; this makes it less valuable– This may alter the relative value of the immediate reinforcer compared to the delayed

reward – Smaller sooner looks better!

Self-Control

A way to teach LL choices?

Self-Control

• Process by which individual deliberately alters or changes his or her behavior to achieve a specific goal– Sets own goals– Monitors own behavior– Self Administration of own reward

Self Control

• Learning to choose delayed but larger consequences over shorter but smaller consequences– Very difficult for animals to do– May be biological: take what can get, not wait for

“unknown”

• Important human behavior– We can learn to delay reward– Question is: how long are we willing to wait

Causes of Self-Control Problems:

• Problems of Behavioral Excesses: engage in too much of a behavior for too little reward

• Problems of Behavioral Deficits: engage in too little behavior to earn reward

Problems of Behavioral Excesses

• Immediate reinforcers versus delayed punishers for a behavior

• Immediate reinforcers versus cumulatively significant punishers for a behavior

• Immediate reinforcers for problem behavior versus delayed reinforcers for alternative desired behavior

Problems of Behavioral Deficiencies

• Immediate small punishers for a behavior versus reinforcers that are cumulatively significant

• Immediate small punisher for behavior versus immediate but highly improbable major punisher if the behavior does not occur

• Immediate small punisher for a behavior versus a delayed major punisher if the behavior does not occur

A Model for Self-Control

• Two parts1. Clear specification of the problem as a

behavior to be controlled

2. Application of behavioral techniques to manage the problem

A Model for Self-Control

• Emitting a controlling behavior to effect a change in a behavior to be controlled

Steps in Self-Control Program1. Specify problem and set goals

– What do you want to change and how will you know success?

2. Make a commitment to change– Statements and actions that indicate that you want to

change• Write out advantages to change and post in visible space• Make it public• Incorporate frequent reminders of your commitment in your

environment

3. Conduct assessment– When, where, and how does a problem behavior occur?

Design and implement a program

• Manage antecedents - what are the causes?

• Instructions – use self-instructions– Modeling – find someone who is good at behavior and observe– Physical guidance – may use physical barriers to avoid some behaviors

• Manage Immediate surroundings – may need to change– Other people – may need to change– Time of day – may need to change what you do at certain times

• Manage the behavior – for complex behaviors, need to focus on the behavior itself; may need to conduct task analysis and have mastery criteria.

• Manage consequences – eliminate reinforcers of negative behaviors

Prevent relapse and make gains last• Be sure to define behavior clearly

• If have long-term goals, have intermittent short-term goals to check progress– Do not over do it– Try to avoid setback situations

• If can’t recognize setback situations in advance, have coping strategies– Do not overreact to setbacks– Avoid counterproductive self-talk

• Incorporate everyday rewards into the program

• In programs where consequences are only cumulatively significant, – plan postchecks – have specific strategies to follow if the postchecks are unfavorable

• Practice self-control strategies to improve additional behaviors• Involve supportive others

• Sign a behavioral contract with supportive others

Behavioral Contract

• Serves at least four important stimulus control functions:– Ensures all parties agree to goals and procedures

– Ensures that all parties agree on how close they are to reaching the goals

– Provides the client with a realistic estimate of the cost of the program to him or her in time, effort, and money

– Signatures help ensure that all parties with follow procedures

Contract Troubleshooting Guide

• The Contract– Was the target behavior specified clearly?

– If the target behavior was complex, did the contract ask for small approximations to the desired behavior?

– Were specific deadlines identified for the target behavior?

– Did the contract clearly identify situations where the target behavior should occur?

Contract Reinforcement Analysis– Did the contract provide for immediate reinforcement? Are

the reinforcers still important and valuable?

– Could reinforcers be earned often?

– Did the contract call for and reward accomplishment rather than obedience?

– Was the contract phrased in a positive way?

– Do you consider the contract to be fair and in your best interests?

Contract Troubleshooting Guide

• The Mediator (your cosigner)– Did the mediator understand the contract?

– Did the mediator dispense the kind and amount of reinforcement specified in the contract?

– Did the mediator meet with you on the dates specified in the contract?

– Is a new mediator required?

Contract Troubleshooting Guide

• Measurement– Are the data accurate?

– Is your data collection system too complex or too difficult?

– Does your data collection system clearly reflect your progress in achieving the target behavior?

– Do you need to improve your data collection system?

What do the data say about self-control?

• Too often we are unable to control our own behavior unless there are external contingencies that drive our behavior

• If you don’t want to eat the cookies: arrange your contingencies carefully– Don’t by them– Get them out of the house– Let others know your plans

• Strengthen alternative responses and consequences– Focus on what ELSE you could be doing, not the thing you want to stop doing– Learning “coping strategies”

• There isn’t really any “self” control- you don’t stop wanting the thing you are avoiding, you just learn to find something better!!!!!