1 ps 526 behavior analysis and learning caldwell college graduate program in applied behavior...
TRANSCRIPT
1
PS 526 BEHAVIOR ANALYSIS AND LEARNING
CALDWELL COLLEGEGRADUATE PROGRAM IN
APPLIED BEHAVIOR ANALYSIS (ABA)DR. KEN REEVE
2
Chapter 1- Introduction
3
SOME QUESTIONS TO PONDER
• Why are we interested in studying how we come to behave a certain way?
• What do we gain from this knowledge?• How important is it to be able to accurately predict
what someone will do in a given situation?• How does what others do affect what you do? • Any answers to the above questions are concerned
with the systematic study of how we learn to behave, act, respond in a certain way
4
MORE QUESTIONS TO PONDER
• How is the ability to learn adaptive?• The ability to learn is adaptive because it allows us to adjust to
where we live so that we can best optimize our ability to keep on existing
• Can you learn and show no change in behavior?• Can you show a change in behavior and yet not have learned to
do it?• Do you need to make a conscious effort to learn?• Is all learning free will (something you chose to do)?• Can you learn to do something and not know why you do it or
how you learned it?
5
TERMS• LEARNING - is enduring, relatively permanent change
in mechanisms of behavior involving specific stimuli and or responses that results from prior experience with those stimuli and responses
• BEHAVIOR - any activity that passes the dead man's test• (if a dead man can do it, it's NOT behavior• - also called a RESPONSE • REPERTOIRE - set of ALL responses you can do
6
ABCs of BEHAVIOR ANALYSIS
• ANTECEDENT (you are in a particular situation, setting, environment or in the presence of a particular cue or signal)
• this situation gives you some information if you've been there before
• it may have given you some clues as to how to act a certain way to get a particular something to happen
• example: how did you know how to sit in these chairs facing the board, take out a notebook and a pencil and NOT take out other stuff?
7
ABCs of BEHAVIOR ANALYSIS
• BEHAVIOR - this is what you do in a particular setting, situation
• as a result of doing something, a change might occur in the setting
8
ABCs of BEHAVIOR ANALYSIS
CONSEQUENCE - this is also called feedback, outcome- when you produce some behavior, some change
will occur in the situation- it may or may not be noticeable to you, or it may
occur immediately or a while afterwards- the consequence is important because it either
makes the likelihood of your doing that particular behavior again stronger or weaker depending on what the consequence is
9
• Our entire life is FULL of these ABCs, and unfortunately many of them CONFLICT
• What does that mean?• certain types of behavior result in good consequences in one
setting BUT they may result in BAD consequences in another setting
• ex) making jokes in a bar might result in people buying you drinks and flirting with you
• BUT making jokes in a classroom might result in the teacher reprimanding you or other students may treat you poorly
• PROBLEM: in a new situation you may or may not have a good idea as to how to act so you may do very little or behave in a way that worked for you before
• only EXPOSURE to these situations allows you to learn about them (others can also tell you how to act but you need to know whether good consequences previously resulted from listening to these people)
10
BEHAVIOR ANALYSIS
• looks to find the principles as to how both antecedents and consequences affect our behavior
• this is done with both human and animal behavior• - is done with both normal and "abnormal" behavior• - is done by changing environments (antecedents
and consequences) in a systematic and objective way
• How do we do this analysis?• use scientific method, specifically experimental
procedures (MORE LATER)
11
BEHAVIOR ANALYSIS
• - examines the CAUSE of behavior change• - assumes it is due to PRIOR experience• - the causes of behavior can only be inferred from
experimental manipulation• - can only be done with experimental research
because you need to look at both subjects without the cause (control) and subjects with the cause (experimental group)
12
EXPERIMENTAL ANALYSIS OF BEHAVIOR
• seeks to find common or general laws of learning• this MUST be verified experimentally not just
observing• need to be careful that learning, and not any other
reason, is the cause behind behavior change• is a great deal of experimental evidence of
generality of these laws
13
APPLIED BEHAVIOR ANALYSIS
• Same goals as plain old experimental analysis of behavior except that it focuses on SOCIALLY SIGNIFICANT BEHAVIOR
• (as opposed to any interesting behavior the researcher wants to look at)
14
TYPES OF CONDITIONING
• CONDITIONING=LEARNING • RESPONDENT LEARNING • Also called CLASSICAL CONDITIONING;
PAVLOVIAN CONDITIONING; S-R LEARNING; ASSOCIATIVE LEARNING
• when a new antecedent stimulus elicits (brings about) the same response as an old antecedent stimulus because it has been paired with the old stimulus
• RESPONDENT=the elicited behavior
15
TYPES OF CONDITIONING
• OPERANT LEARNING• Also called INSTRUMENTAL LEARNING;
SKINNERIAN LEARNING• Particular behavior becomes more or less likely to
occur in the future as a result of the consequences that have followed the particular behavior in the past
• OPERANT=the behavior that is more or less likely to occur as a result of the consequences it produced
16
BEHAVIOR SELECTED BY CONSEQUENCES
• Similar to evolutionary theory
• Genetic trait is to learned behavior as survival advantage is to day-to-day adaptability
• Within each person’s repertoire, what you do is there because it somehow gives you an advantage
17
REMOTE CAUSATION VS. IMMEDIATE CAUSATION
• Why is it hard in our own lives to see what caused us to act the way we do?
• Unlike in a chemical reaction, in which we can see immediate causal events, what we do in a given situation is often remotely caused (long past learning history that is hard to see or remember)
18
GIANTS IN BEHAVIOR ANALYSIS
• Ivan PAVLOV (1849-1936)• Russian physiologist in late 1800, early 1900s• in 1904, won NOBEL for discovering nerves that
stimulate digestive juices of pancreas• normally, animals will begin to salivate when food is
presented in the mouth• BUT Pavlov noticed that dogs began salivating when
they SAW the food, heard the opening of the food, or even saw the technician who prepares the food
• Any idea why? How is this adaptive?
19
GIANTS IN BEHAVIOR ANALYSIS
• John Watson (1878-1958)
• influential for pushing the study of behavior only (no need to study “psyche”)
• “Psychology as the Behaviorist Views It”
• Little Albert and the White rat
• Advertising
• Child Rearing
20
GIANTS IN BEHAVIOR ANALYSIS
• B.F. Skinner (1904-1990)• father of modern behavior analysis• developed operant learning chamber• research on reinforcement and how
“schedules” operate on what we do• brought BA to the masses• wrote about uses of BA for the betterment
of the world
21
SKINNER MYTHS
1. we have no mind, feelings, thoughts
2. raised his baby in a skinner box, experimented on her, and she went crazy and killed herself
3. was a communist, fascist, totalitarian
4. was a snappy dresser
5. was a skilled novelist
22
ASSUMPTIONS MADE BY BEHAVIOR ANALYSTS
• PRIVATE EVENTS• - yes, what you are thinking, feeling, etc are
occurring but they do NOT cause what you do• - commonly held that thoughts are “commentary”
about what you are doing or will be doing in a given situation and feelings are elicited by respondent conditioning
• your past learning history (and to some degree your genetics) determines what you are likely to do
23
ASSUMPTIONS MADE BY BEHAVIOR ANALYSTS
• FREE WILL• yes, you have “free will” but not in the
sense that most mean it• Behavior analysts imply that it means we
have CHOICES in what we can learn to do, when to do it, and that we can alter our own environments to make us more or less likely to do certain things
24
CHAPTER 2 – EXPERIMENTAL ANALYSIS
OF BEHAVIOR
25
FUNCTIONAL VS. STRUCTURAL APPROACH
• cognitive psychology (and much of education theory) takes a structural approach
• asks: what does the thought process look like• assumes that thinking determines behavior• major flaw: assuming it is cause of behavior, where
does the thought come from? • behavior analysis takes a functional approach• asks: what is the function (purpose) of the behavior;
what does it “buy” the person; how is it adaptive?
26
ELICITED VS. EMITTED BEHAVIOR (RESPONSES)
• respondent behavior is “elicited” by an antecedent event
• operant behavior is “emitted” in the presence of an antecedent event because, in the past, the behavior produced a particular consequence
27
RESPONSE CLASSES
• CLASS=set, group, category• A response class is a group of responses that serve a
similar, common function (produce a similar outcome or occur in the presence of a particular antecedent)
• a respondent response class could be all the behaviors that you do when someone yells DUCK (are elicited behaviors)
• an operant response class could be all the things you do to get your baby to smile at you (are emitted behaviors in the presence of your baby)
28
STIMULUS CLASS
• is a group of stimuli that have a common effect on behavior (can be antecedents or consequences)
• ex.) LOOKOUT, WATCH IT, DUCK: each would likely elicit a set of respondent flinching behaviors (each is an antecedent)
• ex.) COOKIE, APPLAUSE, BIG HUG: each may be used as an effective reinforcer to increase a child’s eye contact (each is a consequence)
• ex.) what group of antecedent stimuli made it likely that you began to do “student behaviors” when you came in today?
29
ESTABLISHING OPERATIONS
• fancy phrase which says that the functional relationship between what we do and the environment we are in DEPENDS on the context
• That is, reinforcers or punishers can be made more or less powerful as can antecedent stimuli
• (most who criticize behavior analysis as being too simplistic fail to mention these “what ifs”)
- deprivation vs. satiation- time- fatigue
- other available response options (anyone clean a lot when a final paper needs to be written?)
30
TACTICS OF BEHAVIORAL RESEARCH
REVIEW:• SCIENCE is a certain way of knowing about
events, such as the causes of our BEHAVIOR, based on OBJECTIVE observations or experience
• ”objective” means everyone can measure or see what you are seeing because you've AGREED on what you are measuring and how to measure it by spelling out its OPERATIONAL DEFINITION
31
TACTICS OF BEHAVIORAL RESEARCH
REVIEW:EXPERIMENTAL- means that rather than just observing things, you
MANIPULATE something to see if that “something” CAUSES something else to happen
ex) if we change ANTECEDENTS or CONSEQUENCES, will either or both of these CAUSE a change to occur in behavior?
- this differs from CORRELATIONAL studies in which we are just looking to see if certain events happen together without necessarily seeing if one cause the other to occur
32
TACTICS OF BEHAVIORAL RESEARCH
What are these events to be measured?- called VARIABLES What is a variable?- is some property of an event that can be measured- this property can change in value or take on different
characteristicsex) how fast you behave or when you behave or how long
you behave or how your behavior changes can all be measured (so these are variables)
ex) aspects of antecedents and consequences can also change
33
TACTICS OF BEHAVIORAL RESEARCH
How can we tell if manipulating one variable causes a change in another?
- manipulate ONLY that one variable and hold all other variable constant (do not let any other variables change in value)
- if a change occurs in the person’s behavior, AND you’ve ruled out all other variables, THEN we can infer that the manipulated variable caused the change
34
TACTICS OF BEHAVIORAL RESEARCH
IV=INDEPDENDENT VARIABLE (“CAUSE”)
- what you manipulate as a researcher, teacher, therapist, or parent to bring about a change in some other event (a person’s behavior)
DV= DEPDENDENT VARIABLE (“EFFECT”)
- what changed (a person’s behavior) as a result of that manipulated variable (the IV)
35
INTERNAL VALIDITY
• VALIDITY = TRUTH = ACCURATE DEPICTION
• internal validity has to do with whether the conclusion drawn about CAUSE and EFFECT relationship between two variables (IV and DV) is correct
• that is, does the conclusion correspond to the actual state of events?
36
CONFOUNDS (are THREATS TO INTERNAL VALIDITY)
• A confound means that what really caused the behavior to change is NOT what you manipulated but was something else
• A confound is when the effects of one or more variables interferes with your ability to determine the cause and effect relationship between IV and DV
37
CONFOUND TYPES
• Each of the following can make it appear as if your IV caused a behavior change (but it really didn’t)
• HISTORY – some other variable was applied to the person without your knowledge
• MATURATION – person’s behavior changed because of physical aging
• INSTRUMENT DECAY – how you are measuring the behavior is accidentally getting more strict or more lax
38
EXTERNAL VALIDITY
• Assuming that the IV did cause a change in the DV, are these results generalizeable to other settings, people, situations, reinforcers, antecedents?
• Need to REPLICATE experiments to gain external validity
39
SINGLE-CASE (SINGLE-SUBJECT) EXPERIMENTAL
DESIGNS • types of experimental designs used in
behavior analysis
• useful in applied clinical settings
• Allows us to study how manipulations affects a single person (as opposed to group studies which only give averages over a group of people)
40
SINGLE-CASE (SINGLE-SUBJECT) EXPERIMENTAL
DESIGNS • REQUIREMENTS: first measure how often behavior in
question is occurring over a period of time (called a BASELINE) for a SINGLE person
• THEN introduce the IV and measure behavior over a period of time (called a TREATMENT, EXPERIMENTAL CONDITION, or INTERVENTION
• This is called an A-B design http://www.unlv.edu/faculty/pjones/singlecase/ss2.gif (but it is NOT a very good design if you stop here) – you need to do more…
41
SINGLE-CASE (SINGLE-SUBJECT) EXPERIMENTAL
DESIGNS • After the behavior changes during
treatment, you can then REVERSE back to baseline over a period of time
• This is called an A-B-A or REVERSAL DESIGN
• Even better is to do more reversals so that you have an ABAB or ABABAB, etc.
42
ABA Reversal Design
43
CHAPTER 3REFLEXIVE BEHAVIOR
AND RESPONDENT
CONDITIONING
44
What’s the difference?• REFLEX = behavior automatically elicited by
some environmental event (antecedent stimulus)- NOT learned; is NOT a behavior you have learned to
do “without thinking”- Is “phylogenetic”
• RESPONDENT = also behavior automatically elicited by some environmental event (antecedent stimulus) BUT…– Is LEARNED through association with some other
stimulus that already elicits a similar response
45
REFLEXES• Believed to be a product of evolution• All species have them but you are more “rigid” as a species
(less adaptable) the more reflexes you have• The antecedent that ELICITS the behavior is called
UNCONDITIONED STIMULUS (translates to “antecedent event that has its eliciting effect on a behavior with no learning needed”)
• The elicited behavior part of the reflex is called the UNCONDIOTNED RESPONSE
• ex) startle, attention, fight or flight, blink, sneeze, shiver, GSR, cry
46
PRIMARY LAWS OF REFLEXES
• Law of Threshold – there is a “trigger point” for each US; below this point and it won’t elicit the UR
• Law of Intensity-Magnitude – the stronger the US the stronger the UR
• Law of Latency – the more intense the US the shorter the latency to the UR
47
OTHER REFLEX CHARACTERISTICS
• HABITUATION – says the more you present the US uninterrupted, the less and less the UR will be
• Believed to occur because of our phylogeny (genetics)
• Nerve impulse of reflex weakens
48
RESPONDENT LEARNING• ONTOGENY – your learning history• In reality, stimuli (events) often predict other stimuli (other
events)• So...you can learn that one event will lead to another and
you can get "ready" for it before it happens • That is, you learn something about one stimulus predicting
another's occurrence through experience and this changes your behavior (def of LEARNING)
• CLASSICAL CONDITIONING is process in which organisms learn about relationships between stimuli and adjust their behavior
49
RESPONDENT LEARNING
50
RESPONDENT LEARNING IN
ACTION IN A BABY• At first baby sees needle and is not upset (sight of
needle is “neutral” or “not meaningful”).• After getting a shot (US), baby gets upset (this UR
occurs naturally without learning—unconditioned response).
• Baby now “associates” the needle with the pain (she learned they “go together”)
• This results in responding to needle as something to be avoided (needle is now a learned or CONDITIONED STIMULUS)
51
RESPONDENT ACQUISITION
• We learn our associations between stimuli bit by bit
• The more pairings, the more we learn they go together
• At some point, you’ve learned this very strongly• The first pairing teaches us the MOST!• Each additional pairing teaches us a little less and
a little less
52
RESPONDENT EXTINCTION
• EXTINCTION – means to “gradually die out” (think of the dinosaurs)
• If we no longer pair the US and CS together, but present them separately, then a person learns that they no longer “go together”
• What is “extinguished”?• The association between the stimuli!
53
A strange phenomenon… SPONTANEOUS RECOVERY
OF RESPONDENT• After EXTINCTION has occurred, if we
wait a while and then present the old CS again, guess what it elicits?
• Yup, the old CR
• Why? Probably because there is some other stimulus present that was originally present when the association was first learned
54
RESPONDENT GENERALIZATION
• To “generalize” means to treat different things the same way
• In this case, if we present stimuli similar to the CS, then we should expect a person to respond similarly to those stimuli
• For example, if a person gets sick eating Cuban food, he may then find himself feeling ill when he tries to eat Mexican food
• This is another example of learning that is very adaptive
55
RESPONDENT DISCRIMINATION
• To “discriminate” means to treat different things differently
• In this case, if we present stimuli different from the CS, then we should expect a person to NOT respond to those stimuli
• For example, if a baby shows excitement at the sound of his father’s voice in the next room, we would not expect this excitement to occur if James Earl Jones is speaking on the television in the nest room
• This is another example of learning that is very adaptive
56
TEMPORAL RELATIONS & RESPONDENT LEARNING
• TEMPORAL = time
• RELATION = how things are arranged with respect to one another
• So…TEMPORAL RELATIONS refers to how CS and US are arranged (presented) with respect to one another as a function of time
57
TEMPORAL RELATIONS & RESPONDENT LEARNING
• SHORT-DELAYED CONDITIONING
• CS comes on just before US and is still on when US comes on (most effective way to learn an association)
CS ___________------_________
US _____________------________
58
TEMPORAL RELATIONS & RESPONDENT LEARNING
• SIMULTANEOUS CONDITIONING
• CS comes on at same time as US (not as effective because CS no longer predicts the US)
CS ______-------________
US ______-------________
59
TEMPORAL RELATIONS & RESPONDENT LEARNING
• TRACE CONDITIONING
• CS goes off before the onset of the US (even LESS effective for learning associations. Why?)
CS ____________------__________
US ___________________------_____
60
TEMPORAL RELATIONS & RESPONDENT LEARNING
• BACKWARD CONDITIONING• US comes on BEFORE the CS (kinda like
yelling DUCK after the ball hits you in the noggin)
• Rarely produces an association CS ____________----___________US ______----_________________
61
Second-Order Respondent Conditioning
• In first order conditioning, a neutral stimulus is paired with an unconditioned stimulus.
• When this occurs the control of the response to the US is transferred to the neutral stimulus, which is now called the CS.
• Second-order conditioning extends this transfer of control to events that have not been directly associated with the unconditioned stimulus. Aha!
• Second-order conditioning involves pairing a neutral stimulus with a CS, rather than pairing a neutral stimulus and US. More…
62
Second-Order Respondent Conditioning
• Why is this important?• Because it means that you can associate a
REMOTE stimulus with some other prior US• This may explain why someone might be afraid of
a dog even though they were never bitten
63
Second Order Conditioning
• CS1 paired with US
• Then CS1 elicits the CR
• IF CS2 then paired with CS1
• Then CS2 ALSO elicits CR– Occurrence in natural environments– Used in advertising: (Warning! Sexist
image to follow)
64
Second Order Conditioning
65
Biological Preparedness
• Humans– Taste aversion– Phobias
66
Applications
• Neuropsychoimmunology
• Systematic Desensitization
• Aversive counterconditioning
67
Systematic Desensitization
• technique based on respondent conditioning• used to treat phobias• Two major components: deep muscle relaxation
and fear hierarchy.• The first step is to teach the client deep muscle
relaxation. The theory was that relaxation was incompatible with fear responses and that the conditioned stimuli that elicited fear would eventually evoke relaxation which is called reciprocal inhibition.
68
The Fear Hierarchy
• After a client learns to relax, a fear hierarchy is constructed.
• Once the situation of maximum fear is identified, a series of verbal stimuli are arranged that describe settings that progressively evoke less and less fear.
• There are usually 10-30 items on the anxiety hierarchy, but some therapists have used over 100 items.
• Typically each item is written on a separate index card and the cards are sorted so that those causing the least fear are at the top and those causing the most fear are on the bottom.
69
The Fear Hierarchy• Once the fear hierarchy has been constructed, systematic
desensitization can begin.• At this point, the therapist reads the least anxiety provoking
description and asks the client to imagine himself or herself in that circumstance.
• The client is also told to lift a finger if the slightest amount of anxiety is felt.
• If the person feels anxious, the same item is repeated until at least three repetitions occur without anxiety.
• Following three or more successful repetitions, the next item on the hierarchy is presented. This continues until the last, and most fearful, item is completed.
70
Conditioning and Compound Stimuli
• To investigate the effects of context in respondent behavior, researchers have used situations involving compound stimuli.
• Compound Stimuli - two (or more) conditioned stimuli (tone and light) are presented together and acquire the capacity to evoke a single conditioned response.
71
Overshadowing
• Overshadowing - when a compound stimulus is used as the CS in a conditioning experiment.
• For example, a light and tone (CS) may be turned on at the same time and paired with an unconditioned stimulus such as food.
• Which PART of the compound will elicit salivation?
72
Overshadowing
• Pavlov found that the most salient property of the compound stimulus came to regulate exclusively the conditioned response.
• For example, if a loud tone and a faint light were used as the compound CS, then the tone would evoke salivation while the light would not. The tone is said to overshadow conditioning to the light. This happens even though the weak light will function as a CS if it is originally presented by itself and paired with a US.
73
Blocking• Blocking- a CS associated with a US blocks a subsequent
association between the US and a new CS. • In blocking, a CS is paired with a US until the conditioned
response reaches maximum strength. Following this conditioning, a second stimulus is presented at the same time as the original CS, and both are paired with the unconditioned stimulus.
• On test trials, the original CS evokes the CR but the second stimulus does not.
• For example, a tone (CS) may be associated with food (US) until the tone reliably evokes salivation. Next, the tone and light are presented together and both are associated with food. On test trials, the tone will elicit salivation but the light will not.
74
• Conditioned Suppression- a previously neutral stimulus (e.g. tone, light) is paired with an aversive US such as an electric shock. After several pairings, the originally neutral stimulus becomes a conditioned aversive stimulus (CS). The CS is said to evoke a conditioned emotional response (CER) that is commonly called anxiety or fear.
Conditioned Suppression (Learned Emotional “Shutdown”)
75
Blocking• Using a conditioned-suppression procedure, Kamin (1969)
discovered the phenomenon of blocking. • Two groups of rats were used: a blocking group and a
control group.• In the blocking group, rats were presented with a tone
(CSave) that was associated with electric shocks for 16 trials.• Following this, the rats received eight trials during which the
compound stimulus tone and light were followed by shock.• The control group did not receive the 16 light-shock
conditioning trials but did have the eight trials during which the compound stimulus tone and light were paired with shock.
76
Kamin (1969)• Both groups were tested for conditioned suppression of
lever pressing in the presence of the light. That is, the light was presented alone and suppression of bar pressing for food indicated the occurrence of the conditioned emotional response (CER).
• Kamin found that the light suppressed bar pressing in the control group but did not affect lever pressing in the blocking group. In other words, prior conditioning with the tone blocked or prevented conditioning to the light.
• Functionally, the light was a CSave in the control group but not in the blocking group.
77
Sensory Preconditioning
• Sensory preconditioning is another example of stimulus control by compound events.
• Sensory Preconditioning- two stimuli such as light and tone are repeatedly presented together without the occurrence of a known US (preconditioning).
• Later, one of these stimuli is paired with an unconditioned stimulus and the other stimulus is tested for conditioning.
• Even though the second stimulus was never directly associated with the US, it comes to evoke a conditioned response.
78
CS-US Correlations
• Contiguity - temporal relationship
• Contingency - correlation
79
Rescorla-Wagner Model of Conditioning
• CS acquires a limited amount of associative strength on each trial.– Associative strength – magnitude of elicited CR
80
The Rescorla-Wagner Model• The basic idea of the Rescorla-Wagner model is
that a conditioned stimulus acquires a limited amount of associative strength on any trial (pairing with US).
• Associative strength describes the relation between the CS and the magnitude of the conditioned response. In general, associative strength increases over conditioning trials and reaches some maximum, asymptotic level.
81
Rescorla-Wagner Model of Conditioning
• CS gains gains certain amount of associative strength on any one trial.
• Strength gained diminishes with each pairing as associative strength is increased.
82
Acquisition
Trials
0 2 4 6 8 10
Ass
oci
ativ
e S
tren
gth
0
2
4
6
8
10VMAX
Si=0.25VMAX=10.00VSUM= 0.00
83
Drug Tolerance and Classical Conditioning
• A psychoactive drug such as heroin is a US when it is in the bloodstream; its effects are a UR (the “high”).
• BUT, body also has an UR in which it FIGHTS the drug and decreases its potency (more…)
84
Drug Tolerance and Classical Conditioning
• OD cases have resulted from drug use outside the user’s typical drug administration environment. A high dose that was tolerated in the CS setting is lethal in a setting which is still a neutral stimulus.– Repeated pairings of a drug and a CS causes the conditioned response to
gain strength and oppose the unconditioned drug effect– Reduction of drug effects (UR) is not due to habituation, but rather to the
counteractive effects (CR) on the setting and injection process (CS)
• What if the normal situation is present (CS) without the drug US?– Elicited respondents are often called “cravings”, a process known as
conditioned withdrawal– CS elicits reactions normally countered by US, but when US is not
delivered when CR reactions occur, subjects experience withdrawal
85
Conditioned Immunosuppression• Another example of environmental influences altering what is
generally thought to be internal and autonomously controlled processes– CS is paired with a US drug that suppresses immune system function
(eg. Production of antibodies)– After several pairings the CS is presented alone, immune system
response is measured
• Can the immune system be conditioned to increase immune reaction?– Buske-Kirschbaum, Kirschbaum, Stierle, Jabaij, and Hellhammer
study (1994)• Flavor CS paired with adrenaline injection US• After several pairings, presentation of CS alone raised cell production normally
caused by adrenaline!
86
87
Chapter 4
REINFORCEMENT
AND EXTINCTION
OF OPERANT BEHAVIOR
88
Learning Objectives
• Respondent vs. Operant Learning
• Reinforcement vs. Punishment
• “Side Effects” of Operant Learning Processes
• Extinction
• Partial Reinforcement
• Forgetting vs. Extinction
89
TYPES OF BEHAVIOR ANALYSIS
• RESPONDENT CONDITIONING - organism does NOT control presentation of stimuli but only responds to them
• OPERANT CONDITIONING – more “active” and goal-directed
90
OPERANT CONDITIONING
• INVOLVES LEARNING WHAT TO DO BASED ON CONSEQUENCES OF PRIOR “DOING”
• LEARNED BY HAVING BEHAVIORS FOLLOWED BY PARTICULAR CONSEQUENCES
• EXAMPLES? (see next slide!)
91
OPERANT CONDITIONING EXAMPLES
• After Ringo makes a sound resembling a word, his mom gives him a big hug. Ringo makes the sound again (wouldn’t you?)
• After Bobby sticks his finger in the dog’s nose, the dog barks loudly and snaps at him. Bobby does NOT do that again!
• After Jonathan fusses, his therapist removes all curriculum materials. Jonathan fusses again the next time work is put in front of him.
• Susan screams in the supermarket and her mother gives her some candy to quiet her. She screams again the next time they enter the supermarket.
92
What is a Contingency?
• A contingency is an IF-THEN relationship
• Usually describes how reinforcement and punishment are arranged
• If behavior X occurs in a particular setting, THEN consequence Y will follow
• The 3-TERM contingency is the ABC relationship of antecedent-behavior consequence
93
OPERANT CONDITIONING TYPES
• REINFORCEMENT- a consequence that increases the chance that the behavior will be made again
• PUNISHMENT- a consequence that decreases the chance that the behavior will be made again
94
OPERANT CONDITIONING TYPES
• REINFORCEMENT- a consequence that increases the chance that the behavior will be made again
- can be POSITIVE contingency (present something such as hugs, praise, kisses, tokens, access to TV or video games, stars, stickers)
- or NEGATIVE contingency (take something away such as pain, hunger, fear, annoyances, wet diaper)
95
96
OPERANT CONDITIONING TYPES
• PUNISHMENT- a consequence that decreases the chance that the behavior will be made again
- can be POSITIVE contingency (present something such as yell, loud noise, frown, pain, restraint)
- or NEGATIVE contingency (take something away such as access to TV or video games, privileges, access to dessert) (also called RESPONSE COST or TIME OUT)
97
Summary Table
BEHAVIOR INCREASES
BEHAVIOR DECREASES
CONSEQUENCE INVOLVES
ADDING SOMETHING
POSITIVE
REINFORCEMENT
POSITIVE PUNISHMENT
CONSEQUENCE INVOLVES REMOVING
SOMETHING
NEGATIVE REINFORCEMENT
NEGATIVE PUNISHMENT
(TIME OUT; REPSONSE
COST)
98
ANOTHER OPERANT CONDITIONING
TYPE: EXTINCTION
• Like punishment, likely to decrease behavior but is done by ignoring child’s behavior (assuming this behavior had previously been reinforced)
• NEVER use this if behavior child is doing is dangerous
99
“SIDE EFFECTS” OF OPERANT CONDITIONING
• REINFORCEMENT HAPPY
• PUNISHMENT NOT HAPPY
• EXTINCTION NOT HAPPY
100
Behavioral Effects of Extinction• Extinction produces several behavioral “side
effects” in addition to a decline in a rate of response• Extinction Burst (“tantrum”)
When extinction is started, operant behavior tends to increase in frequency (called extinction burst)
• Response TopographyIn addition to extinction bursts, operants
become increasingly variable in form (“topography”) as extinction proceeds
101
Other Behavioral Effects of Extinction
• Emotional ResponsesA variety of emotional responses occur
under conditions of extinction. One important kind of emotional behavior that occurs during extinction is aggression.
• Why are some behaviors “resistant to extinction”? (see next…)
102
Partial Reinforcement Extinction Effect
• Resistance to extinction substantially increases when intermittent schedule of reinforcement has been used to maintain behavior.
• Extinction occurs more rapidly when behavior had previous CONTINUOUS REINFORCEMENT than on intermittent reinforcement
• 2004 Honda vs. 1971 Chevy
103
Discriminative Stimuli• Antecedent event (setting) that precedes an operant
behavior• What is present in your surroundings when a behavior
is reinforced (or punished)• When learning occurs, the discriminative stimuli alter
the likelihood that the operant behavior will occur• Discriminative stimuli are said to “set the occasion” for
behavior• discriminative stimulus symbol is SD.• Fancy way of saying that we learn a type of rule: “In a
particular situation, performing a certain behavior usually produces a particular consequence.”
104
Discriminative Stimuli• Discriminative stimuli change the probability that
an operant will be emitted.• When an SD is followed by an operant response
that produces reinforcement, the operant is more likely to occur the next time the stimulus is present.
• When an operant does not produce reinforcement, the stimulus that precedes the response is called an S∆ (“S-delta”). In the presence of an S-delta, the probability of emitting an operant declines.
105
Reinforcement, Intrinsic Motivation, and Creativity
• Many educators and social psychologists argue that rewards/reinforcement reduce individual self-determination, motivation, and creativity– Rewards/reinforcement are interpreted as
“controlling” which leads to the reduction in creative performance
106
Rewards and Intrinsic Motivation
• Opponents of rewards often cite experimental data showing rewards as having a negative effect(s)– Results are not consistent, some studies find negative
effects, some find positive effects– Tangible rewards given for meeting a certain criterion
level of performance or exceeding the performance of others actually maintained or enhanced intrinsic interest
– Rewards tied to level or quality of performance increase intrinsic motivation or leave intrinsic interest unaffected
107
Rewards and Creativity
• Commonly accepted as fact, the phrase “rewards lessen creativity” is a broad generalization– Divergent thinking is the most widely studied form of
creativity• Involves varied novel responses to a problem or question that
has multiple possible solutions
• Opponents of rewards report that offering an individual a reward results in reduced divergent thinking
• Robert Eisenberger suggests failures to find increased creativity may occur when the reward is not actually contingent on creative performance
108
Premack’s Principle
• “more probable behaviors will reinforce less probable behaviors”
• “less probable behaviors will punish more probable behaviors”
– Premack suggests it is possible to describe reinforcing events as actions of the organism rather than as discrete stimuli
109
Procedures in Operant Conditioning
• The Method of Successive Approximation, aka Shaping
• Many novel forms of behavior may be shaped by the method of successive approximation. Shaping is key part of acquisition and necessarily involves differential reinforcement.
• Variability is a good thing, it is the “stuff” that novel operants are selected from
• As behavior continues to be reinforced variability tends to decrease.
110
Applying Extinction• Extinction is an important role in behavior
modification• When put to bed by his parents a 20-month-old boy
would “throw” a temper tantrum requiring the parents to stay up with him.
• The parental attention given to the boy served as a reinforcement to the tantrums.
• Extinction was implemented by the parents leaving the room after the child was put to bed.
• The first extinction session, the tantrum lasted 45 min. and on the third session only 10 min. After ten days the boy had no tantrums.
111
Applying Extinction
• The child’s aunt reinforced his crying by staying in the room and his tantrums reoccurred.
• This intermittent reinforcement increases resistance to extinction of the tantrums
• The second extinction procedure took longer due to the intermittent reinforcement.
• The first procedure was more effective since the child was continuously reinforced.
112
Remembering and Recalling
• In much of psychology, people and other organisms are said to store information about events in memory– The use of the noun memory is an example of
reification or treating an action as if it were a thing
– In behavior analysis, the verb remembering (or forgetting) is used to refer to the effect of some event on behavior after the passage of time
113
Remembering and Recalling• For humans, we may say that a person recalls his/her trip to
Costa Rica, when a picture of the trip occasions a verbal description of the vacation– From a behavioral perspective, recalling the trip is behavior
(mostly verbal) emitted now with respect to events that occurred in the past
• Remembering and recalling are treated as behavior processes rather than some mysterious thing (I.e. memory) within us– Behavior analysts assume that the event recalled is one that was
described when it first occurred– Recalling refers to reoccurrence of behavior (mostly verbal) that
has already occurred at least once
114
Chapter 5 - Schedules of Reinforcement
116
REINFORCEMENT INTHE “REAL” WORLD
• So far we've talked about rather simplistic cases in which reinforcers are delivered after each response
• in actuality, how reinforcers are presented is far more complex
• rarely do we see a 1:1 relationship between R and Sr
117
SCHEDULES OF REINFORCEMENT
• these are rules about HOW and WHEN reinforcer follows a R
• usually we study these schedules by first conditioning an animal with CONTINUOUS reinforcement and then seeing how different schedules maintain or change levels of responding
• Why are these schedules important? Who needs to know about them?– teachers, parents, managers, industry, etc.
118
Types of Schedules of Reinforcement:
• Most simple schedule:• CONTINUOUS REINFORCEMENT (CRF) - is
simplest schedule• involves a 1:1 ratio between R and Sr• this ratio stays constant
• INTERMITTENT (PARTIAL) SCHEDULES:• these involve cases where reinforcement is delivered
only part of the time or intermittently
119
RATIO SCHEDULES
• here the reinforcer depends on a certain number of responses to occur with no regard to WHEN the responses are made
• Fixed Ratio - ratio of R/Sr remains constant– is denoted by the number of responses to get one reinforcer
(FR x)
• ex) Paper boy gets paid after delivering 30 papers• ex) factory worker makes 10 televisions to get paid• ex) salesman gets commission after every 5 cars he sells• ASK: what is CRF really? answer: FR 1
120
RATIO SCHEDULES
• What type of responding does FR produce?• CRF (FR 1) produces steady responding• "leaner" FR schedules produce pattern called
BREAK and RUN or STEPPING• that is, there is a RATIO RUN up to the point of
reinforcement, followed by a POST-REINFORCEMENT PAUSE
• the "leaner" the schedule, the greater the run and the longer the pause
• the harder you work to get the reward, the longer you "rest"
121
FIXED RATIO SCHEDULES
122
RATIO SCHEDULES
• What happens if you abruptly and severely lean out a FR schedule?
• you get RATIO STRAIN• person will stop responding before getting the
reinforcer• A lean FR schedule functions as EXTINCTION• person says "Oh, I guess I don't get the reward
anymore so I will stop responding"• THEREFORE, if you plan on leaning out a
schedule you need to do it GRADUALLY to avoid ratio strain
123
RATIO SCHEDULES• In real world, the ratio to get rewards usually varies to some
degree so it is important to study:• VARIABLE RATIO - ratio of R/Sr varies about some
average• is denoted by the AVERAGE number of responses to get one
reinforcer (VR x)• ex) slot machine• ex) number of turns to get dough right, • ex) number of good deeds to get someone to notice you, etc.• how many responses needed to get the reward is now
UNPREDICTABLE• sometimes it can be few responses, sometimes many• for this reason, you do NOT get break and run but instead
get STEADY responding
124
VARIABLE RATIO SCHEDULES
125
INTERVAL SCHEDULES
• here, delivery of Sr depends not only on the response being made but also WHEN the response is made
• reinforcement RESETS the interval clock• 1. FIXED INTERVAL - a certain constant or fixed time period
must elapse since the last reinforcer was presented before a response can result in another reinforcer
• IMPORTANT: the person still needs to make the response after the time has elapsed
• denoted by the time period (FI y)• ex) checking the mailbox before 2:00pm• ex) going in for a paycheck before Thursday• ex) taking out a cake before the 45 minutes are up• ex) checking to see if the water has boiled• ex) weekly scheduled quizzes• ex) weekly meetings• ex) dog checking for master to be home
126
INTERVAL SCHEDULES
• What type of responding does FI produce?
• produces SCALLOPING
• person learns when in time Sr is available so person to respond at higher and higher rates when that time is near
127
FIXED INTERVAL SCHEDULES
128
INTERVAL SCHEDULES
• VARIABLE INTERVAL - a varying time period must elapse since the last reinforcer was presented before a response can result in another reinforcer
• IMPORTANT: again, the person still needs to make the response after the time has elapsed to get reinforcer
• ex) child in car asking "are we there yet?"• ex) pop quizzes• ex) wondering when to make the "first move" during a
date• ex) checking to see if chicken is done
129
VARIABLE INTERVAL SCHEDULES
• What type of responding does VI produce?
• steady responding without pauses because you never know exactly when the reinforcer will be available
130
INTERVAL SCHEDULES
• So far we've implied that you can make the response at any time once the interval has elapsed BUT
• in real world there is often only a short window of time in which the reinforcer is available
• - this is called a LIMITED HOLD INTERVAL SCHEDULE
• ex) checks from payroll are available around the 20th of each month (VI schedule) but once they come they are only available for two weeks or they are forfeited (LIMITED HOLD)
131
INTERVAL vs. RATIO SCHEDULES
• How does the RATE of RESPONDING affect the occurrence of Sr?
• RATIO SCHEDULES - responding at a higher rate ALWAYS results in more Sr
• INTERVAL SCHEDULES - reinforcement depends more on WHEN the R is made as opposed to how many R are made
• So logically...which schedule yields HIGHER RESPONSE RATES?
• the RATIO SCHEDULE• Implications of this?• management wants a ratio schedule (get more work out of
workers) whereas labor does not
132
VARIABLE RATIO vs. VARIABLE INTERVAL
SCHEDULES• Why do VR schedules yield higher response rates
than VI?• may have to do with IRT (inter-response time)• VI - there is a benefit in waiting (conserve energy)
because the longer you wait the more likely it is that reinforcement will be available when you actually respond
• VR - there is no benefit to waiting because the longer you wait the longer you are putting off getting the reinforcer
133
REVIEW
134
Chapter 8:Stimulus Control
135
Stimulus Control• controlling stimuli = antecedent events that
precede operant behavior and affect its occurrence• A controlling stimulus (S) is said to alter the
probability of an operant, in the sense that the response is more (or less) likely to occur when the stimulus is present.
136
Stimulus Control• Discriminative stimulus (SD) - a controlling
stimulus that sets the occasion for reinforcement of an operant response.
• S-delta (SΔ)or extinction stimulus - a stimulus that sets the occasion for extinction of an operant response.
• SP – a stimulus that sets the occasion for punishment of an operant response
137
Emitted versus evoked•Operants can and do occur in the absence of any eliciting stimulus (they are said to be “freely emitted”). •However, when an SD comes to control occurrences of an operant (alters the response’s probability of occurring,) then we say that the “SD evokes the operant.” •The term evoke indicates that operant response is under stimulus control of an antecedent stimulus.•We can also say that “the SD sets the occasion for reinforcement of the particular response.”
138
Differential Reinforcement and Discrimination
• When a person responds in one situation but not in another, we say that the person shows a discrimination between the situations.
• simplest way to teach differential responding (aka discrimination) is to reinforce an operant in one situation and withhold reinforcement in the other.
• Stimulus control refers to a change in behavior that occurs when either an SD or SΔ is presented.
• When SD is presented, probability of response increases; when an SΔ is presented, the probability of response decreases.
139
Generalization vs. Discrimination
• DISCRIMINATION = particular response in one situation but not in another
• We “discriminate among/between settings, people, stimuli”
• GENERALIZATION = particular response occurs across different situations
• We “generalize across settings, people, stimuli”
140
Stimulus Generalization
• Particular response is evoked by new stimuli that presumably share common properties with the original discriminative stimulus used in training.
• For example, child says “monkey” when shown novel pictures of monkeys not used in original teaching situation
• Generalization and discrimination refer to differences in the precision of stimulus control (e.g. under what situation does one behavior “end” and another one “begins”?)
• Discrimination and generalization are inversely related (as one goes up the other goes down)
141
Generalization Gradient
• A generalization gradient shows the relationship between the probability of response and stimulus value.
142
Generalization Gradient
• Another example…
143
Generalization Gradient
• There are many stimulus characteristics that might control behavior (size, color, presence or absence of certain features, location, etc.)
• Generally, the more you vary these values, the LESS GENERALIZATION you will see
144
Studying Stimulus Control
• To study generalization/discrimination, the researcher may arrange the presentation of SD or SΔ so that one follows the other. This is called successive discrimination.
• In alternative procedure, simultaneous discrimination, the SD and the SΔ are presented at the same time and the organism responds to one or the other.
145
Problems with “Regular” DiscriminationTraining Procedures
• When the SD and the SΔ are alternately presented, the organism initially makes many errors.
• That is, person continues to respond in the presence of the SΔ because generalization often occurs unless you teach otherwise.
• As discrimination training progresses, a differential response occurs to the SD and SΔ.
146
Problems with “Regular” DiscriminationTraining Procedures
• BASIC RESEARCH IN “REGULAR” DISCRIMINATION TRAINING: pigeon taught to peck (R) a green key (SD) for food (Sr).
• Once behavior is well established, color on key is changed to blue (SΔ) and pecking not reinforced
• blue and green colors are alternately presented• During early training sessions, onset of extinction
will generate emotional behavior that interferes with ongoing operant behavior. (Bird is Mad!)
147
Problems with “Regular” DiscriminationTraining Procedures
• Extinction is an aversive procedure!
• Pigeons flap their wings aggressively and will work for an opportunity to attack another bird during the presentation of SΔ !
• Birds will even peck a different key if pecking turns off the extinction stimulus (SΔ), implying that the stimulus is aversive. (this is an avoidance behavior!)
148
Problems with “Regular” DiscriminationTraining Procedures
• Because “emotional behavior” is elicited by the frustrating trial and error sessions, discriminative responding takes a long time to develop!
• So…is there a better option?
149
Yes: Errorless Discrimination• Here, the teacher does not allow the child to make mistakes by
responding to the extinction stimulus SΔ.
• Errorless discrimination involves gradually introducing the SΔ initially at a very weak “intensity” so that responding to it is very low in probability. Thus, person is actually practicing NOT responding to it.
• Over repeated trials, intensity of the SΔ gradually increased.
• Eventually SΔ is presented in full intensity and the person will not respond to it.
• So, a discrimination between the (SD) and the SΔ was acquired without the errors of responding to the SΔ.
150
STIMULUS FADING
• When some stimulus ALREADY occasions a response, but we want to have a DIFFERENT stimulus occasion the response, then we might use a STIMULUS FADING PROCEDURE
• PROMPT = an “artificial” SD that we use to make a
behavior likely to occur when the “natural” SD is ineffective
• We then gradually reduce (FADE OUT) the prompt so that
the control of the behavior transfers to the “natural” SD
• Sometimes we FADE IN the “natural” SD
151
More on FADING
• Sometimes we can “morph” an “artificial” SD into the
“natural” SD
• This can be done with morphing software.
• Often done where a picture is morphed into a word over successive trials so that the word will control a verbal label such as CAT.
152
COMPLEX STIMULUS CONTROL
• Refers to situations in which the behavioral function of an antecedent stimulus changes depending on the presence of another stimulus.
• That is, in some situations, a stimulus might be an SD and in other situations the SΔ
• This is called CONDITIONAL DISCRIMINATION LEARNING because the correct response is “conditional on” (dependent on) another antecedent
• A CONDITIONAL DISCRIMINATION is an “IF-THEN” rule
153
EXAMPLE OF CONDITIONAL DISCRIMINATION
Please select the PLUS SIGN by pressing it:
+ &
Note that here the stimulus “+” is an SD for pressing the + sign AND the stimulus “&” is an SΔ for pressing the & sign.
The correct response is conditional on (dependent on) the instructions give.
What instructions would REVERSE the behavioral functions of the stimuli?
154
EXAMPLE OF CONDITIONAL DISCRIMINATION
Please select the animal that says MEOW by pointing to it:
CAT DOG
Note that here the stimulus “CAT” is an SD for pressing CAT AND the stimulus “DOG” is an SΔ for pressing DOG.
The correct response is conditional on (dependent on) the instructions give.
What instructions would REVERSE the behavioral functions of the stimuli?
155
CONDITIONAL DISCRIMINATION =
MATCHING TO SAMPLEOften in conditional discrimination training, the
conditional stimulus is referred to as a SAMPLE and the choices we respond to are called COMPARISON STIMULI
This is also called MATCHING TO SAMPLE (MTS) because you are trying to make the correct response in the presence of the sample
Did you know? A multiple-choice exam question is a type of matching to sample or conditional discrimination trial!
156
EXAMPLE OF CONDITIONAL DISCRIMINATION
For a child, the stimulus of MOMMY being present is usually an SD for talking to her
But WHAT IF mommy and the child are in church or temple?
Note that the correct response to mommy (being quiet) is conditional on (dependent on) the setting in this case.
What instructions would REVERSE the behavioral functions of the stimuli?
157
Delayed Matching to Sample (DMTS) and Remembering
• This variation of the matching to sample task has been used to investigate behavior said to reflect cognition and memory– Here, the sample stimulus is turned off before the
comparisons are presented
– This time period is known as the retention interval
– Theoretically, organism is covertly doing something that helps to retain the information about the sample
158
• Traditional cognitive psychology talks about the internal rules defining what things “go together” and why
• Behavior analysts refer to concepts as “sets of stimuli that occasion a common response”
• Those “sets of stimuli” can be incredibly complex such as exemplars of “immoral conduct”
• The “common response” can also be complex
Concept Formation
159
• Behavior analysts identify the characteristics of the stimuli in the “concept” that controls the behavior
• They also examine how certain kinds of training (discrimination training, programming for generalization) affect the likelihood of stimulus classes (see work of Reeve, haha!)
Concept = stimulus class
160
• PERCEPTUAL CLASS = stimuli in the set share some physical characteristics
• RELATIONAL CLASS = stimuli in the set share some abstract relationship, such as examples of “bigger than”.
• EQUIVALENCE CLASS = stimuli do NOT share any physical characteristics. Stimuli are instead taught to belong together just because society says so.– Example: 1 = one = spoken word “WUN”
stimulus class types
161
Non-Human Studies on Natural Concept Formation
• Herrnstein - Pigeons– Pigeons can learn to form concepts of a person as
discriminated from non-persons, i.e., statues, mannequins, etc., a specific person from others, etc. The abstract stimulus class of “person” readily learned.
– Pigeons can learn to form natural concepts of persons, trees, fish, etc., with a high degree of accuracy. Artificial concepts such as buildings also learned.
Pigeons appear to respond to two-dimensions objects (photographs) as representations of three dimensional objects
162
Stimulus Control andLife
• Reading
• Studying
• Sleeping (Insomnia)
• Writing
163
Stimulus Control and Multiple Schedules
• Behavior analysts often use multiple schedules of reinforcement to study stimulus control in the laboratory.
• On a multiple schedule, two or more simple schedules are presented one after the other and each schedule is accompanied by a distinctive stimulus.
164
Behavioral Contrast
• Occurs within subjects
• Operant responding is the dependent variable
• Measured on multiple schedules of reinforcement– Two or more simple schedules that alternate– Subject does not get to “choose” on which
schedule to respond
165
Behavioral Contrast (cont.)
• Negative Behavioral Contrast– A decrease in response rate in one component of
a multiple schedule of reinforcement because the conditions of reinforcement in another component have gotten better
• Positive Behavioral Contrast– An increase in response rate in one component of
a multiple schedule because the conditions of reinforcement in another component have gotten worse
166
The Moral Behind Contrast Effects
• Contrast effects are instrumental in demonstrating that reinforcement in other situations can alter present behavior– Even when the other reinforcement is currently
available
167
End of notes for MIDTERM exam