ufdcimages.uflib.ufl.edu · web viewdown), wherein the sibling imitates the child’s behaviors...
TRANSCRIPT
Running head: BEHAVIORAL CODING 1
Behavioral Coding of Siblings of Children with Autism
Melanie C. Hankins
University of Florida
BEHAVIORAL CODING 2
Introduction
The purpose of this pilot study was to determine if the siblings of children with Autism
Spectrum Disorder (ASD) could learn and appropriately utilize interventions that might improve
the social behavior of the children with ASD. The ultimate goal of teaching these interventions
to the siblings is to cause improvement of social interactions in the still-developing child with
Autism Spectrum Disorder. Ten minute video-recordings of the interactions between the sibling
and the child with ASD were made before and after the sibling was taught specific interventions
that were coded according to the frequency in which the specific intervention occurred. This
pilot study, “In-Home Training for Siblings of Children with Autism”, was based on the results
of a larger, sentinel study, “In-Home Training for Fathers of Children with Autism: A Follow up
Study and Evaluation of Four Individual Training Components”, which incorporated similar
father-child in-home training. The principle investigator of this study was Dr. Jennifer Elder
(See Appendix A). If the full study is approved by the NIH, it should receive $2 million in
funding and would last a total of five years (Elder, 2010).
Autism Spectrum Disorder (ASD) is a developmental disorder that contains three general
domains of abnormal behaviors as explained in the Diagnostic and Statistical Manual of Mental
Disorders, 5th Edition (DSM V) criteria: social interaction, communication, and
restricted/repetitive behaviors or interests. ASD includes: Asperger’s Syndrome, Autism,
Pervasive Developmental Disorder Not Otherwise Specified (PDDNOS), and Childhood
Disintegrative Disorder (American Psychiatric Association, 2013). Autism affects 1 in 50
children, and its prevalence appears to have been increasing, although the cause of this increase
remains unclear. School-aged boys are more commonly affected than girls, with boys being four
times more likely to have ASD (Blumburg et al., 2013). Helping children with ASD requires a
BEHAVIORAL CODING 3
high level of interdisciplinary care, particularly among specialties associated with pediatrics. It
is also important to find ways to improve the quality of life of affected families, as autism affects
entire families, not just the autistic children. Providing family members with education,
encouragement, and behavioral skills to allow them to adequately manage care is especially
important. According to Autism Speaks (2010), early detection of any abnormalities in
development is critical to improve such children’s quality of life. These families often must rely
on the skills of a nurse who is educated and informed about Autism Spectrum Disorder.
Description
This study utilized specific interventions taught to the siblings of the autistic children. Some
of these interventions consisted of behaviors that sometimes occurred spontaneously, even
without specific training. During each session, the children were first recorded during a two
minute “settling-in” period, and those two minutes were not coded for the interventions. Prior to
the training of each specific intervention to the siblings, a baseline video was recorded of the
sibling and autistic child’s interaction during play, during which the video was coded by
counting the frequency of each intervention. The siblings were then taught one of the
interventions. After the sibling had received the training for each intervention, a ten minute
video was recorded that was coded for the interventions. The siblings carried out the
interventions they were taught during the recordings, and the interventions were coded, even if
these interventions observed had not yet been taught to the sibling of the autistic child. The
interventions were introduced one at a time and are described below:
Imitation/Animation (IA): A movement cycle that begins within 5 seconds of an
Autistic Disorder (ASD) child initiation (e.g. child utters a vocalization, jumps up and
BEHAVIORAL CODING 4
down), wherein the sibling imitates the child’s behaviors in an animated manner (e.g.
with exaggerated, lively movement).
Following the Child’s Lead (FL): A movement cycle that begins when the sibling
joins the ASD child and plays the game or plays with the toy the ASD child has
selected (e.g. child picks up a book and starts to look at it, sibling gets next to the
child, looks too and comments on the pictures; ASD child stacks one block on
another and the sibling joins in and stacks a third block on top) within 5 seconds of
the ASD child’s behavior and ends with the completion of the sibling’s action.
Commenting on the Child (C): A movement cycle that begins with an ASD child’s
action or vocalization and ends with a related and immediate sibling comment (e.g.
“So, you found a red car” rather than “Is that car red?”).
Methodology
These interventions were coded by students with conferred inter-rater agreement. According
to the Center for Educator Compensation Reform (2011), inter-rater agreement is the degree to
which two or more raters give the same rating to the same specific observable situation using the
same rating scale. Before coding the videos, the raters were trained by a research assistant with
the aid of the principal investigator. After coding 27 videos, the conferred inter-rater agreement
was calculated, and the definitions of the training components were further refined. After the
definitions were refined, the coders calibrated using eight videos that were not previously coded,
the two minute settling-in videos. There was then independent coding of eight randomly-
selected established videos, two from each family. Independent inter-rater agreement was then
calculated and analyzed. The inter-rater agreement was calculated by the number of agreements,
BEHAVIORAL CODING 5
divided by the number of agreements plus the number of disagreements (Fleiss, Levin, Paik,
2003).
The hypothesis of this study was that after teaching the interventions to the sibling of the
autistic child, there would be an improvement in social interaction in the still-developing
children with autism spectrum disorder. The sample contained four ethnically diverse autistic
spectrum disorder child-sibling pairs.
After the conferred inter-rater agreement and independent inter-rater agreement were
calculated, the results were analyzed. The results were different than expected. Our conferred
inter-rater agreement had a high level of reliability, whereas our independent inter-rater
agreement had a very lower level of reliability. Additional work is needed to improve inter-rater
agreement prior to conducting the larger study and further suggestions to increase the quality of
the research gathered are included below.
Findings
The calculated conferred inter-rater agreement was 98.3 for Family A, 93.1 for Family B,
97 for Family C, and 96 for Family D overall. Imitation/Animation was 95 for Family A, 96.4
for Family B, 95.4 for Family C, and 95.6 for Family D. Following the Child’s Lead had an
agreement of 100 for Family A, 90.1 for Family B, 95.7 for Family C, and 94.5 for Family D.
Commenting on the Child had an agreement of 100 for Family A, 92.8 for Family B, 100 for
Family C, and 97.9 for Family D (See Figure 1).
BEHAVIORAL CODING 6
Figure 1. Conferred Inter-rater Agreement
The calculated independent inter-rater agreement was significantly lower, Family A was
74.6, Family B was 72.2, Family C was 56.8, and Family D was 39.7 overall.
Imitation/Animation had an agreement of 67.4 for Family A, 84 for Family B, 91.5 for Family C,
and 79.5 for Family D. Following the Child’s Lead had an agreement of 70.8 for Family A, 70.2
for Family B, 78.8 for Family C, and 27.2 for Family D. Commenting on the Child had an
agreement of 85.5 for Family A, 62.5 for Family B, 0 for Family C, and 12.5 for Family D (See
Figure 2).
Figure 2. Independent Inter-rater Agreement
BEHAVIORAL CODING 7
Figure 3. Inter-rater Agreement
Commenting on the Child had the lowest inter-rater agreement (See Figure 3). This
could be partially due to the fact that this intervention had the lowest frequency during the
videotaping sessions. This intervention appeared difficult for still-developing children to word
comments to meet this definition; however, according to the Child Development Institute (2013),
children aged 8-10 should be able to form complex sentences at the adult level, so it is unclear as
to why the siblings had difficulty with this.
The findings indicated that the siblings of the ASD children were able to learn and use
these interventions. There was an increase in the frequency of the interventions after they were
taught the interventions, and the frequency counts remained higher with each subsequent
intervention that was taught. Commenting on the Child had the lowest frequency counts out of
the three interventions, but showed an increase in frequency after the intervention was taught.
Suggestions
There are multiple suggestions that, if implemented in the larger study, could improve the
reliability and accuracy of the data gathering process, and also improve the quality of the
research. It would be useful to replicate the study with more subjects. This pilot study was very
BEHAVIORAL CODING 8
limited in the sample size, as there were only four sibling/autistic child pairs. There could also
be stricter inclusion criteria. One of the subjects was non-verbal, which could have altered the
results. Improving the quality of the video recording might be helpful in improving coding
accuracy. One of the challenges of coding the videos related to the quality of the videos. At
times it was difficult to understand what the subjects were saying, so it was unclear if the siblings
were commenting on the autistic child, or imitating the autistic child, potentially altering the
results. Another suggestion is to use the two minute “settling-in” videos as reinforcing/teaching
time. Often, the sibling may not have remembered the prior intervention that was taught, and
this could be utilized to further instruct the subjects. Additional reminder sessions and
encouraging daily practice with families could be very beneficial. As stated previously, the
subjects sometimes appeared to have difficulty utilizing the previously taught interventions. The
current definitions used in the coding leave considerable room for subjective interpretation by
the rater, even after the further improvements noted above. These definitions could be further
improved, which could positively impact the inter-rater agreement. More formal training of
coding with the principal investigator, incorporating several practice videos with feedback on
coding accuracy would be very helpful. More formal coding training could improve the
reliability of the study. Incorporating a short training video that clearly defined the interventions
with examples of Imitation/Animation, Following the Child’s Lead, and Commenting on the
Child that could be played before each coding session would remind the coders of the correct
techniques, which would benefit the study. Also, recalibrating if there is a long interval between
coding sessions, perhaps utilizing the “settling-in” two minute videos, would remind the coders
of the interventions and correct techniques to use while coding.
BEHAVIORAL CODING 9
Because Commenting had the lowest inter-rater agreement, it may be beneficial to drop this
intervention from the larger study or code for another element, such as coding for a general
response or praise. It may be beneficial to follow-up with families after the interventions and
video-recording are finished to analyze if the siblings are still utilizing the interventions. This
could be analyzed at three and six months. Social interaction could also be analyzed at these
times to discover if there was any improvement in social interaction of the autistic child. It
would also be beneficial to study the way in which the siblings were impacted by these
interventions by analyzing their quality of life and exploring whether or not they felt empowered
by utilizing these interventions. The refined definitions are included below:
I/A: Movement cycle that begins within 5 seconds of an ASD child’s initiation, (e.g.,
child utters a vocalization, jumps up and down, changes body position) where the
sibling imitates/matches the child’s behavior and/or vocalization in an animated
manner (e.g., with exaggerated affect, lively movement). Each movement and
vocalization count as an individual event.
FL: Movement cycle that begins when the sibling joins the ASD child and plays the
game or plays with the toy the ASD child has selected (e.g., child picks up a book and
starts to look at it, sibling gets next to the child, looks too and reads book; ASD child
stacks one block on another and sibling joins in and stacks a third block on top)
within 5 seconds of the ASD child’s behavior and ends with the completion of the
sibling’s action, and is not to be confused with imitating ASD child’s direct action.
Sibling goes along with ASD child’s initiation of an event and contributes to play.
Commenting: Movement cycle that begins with an ASD child’s action or vocalization
and ends with a related and immediate sibling comment (e.g., “So, you found a red
BEHAVIORAL CODING 10
car” rather than “Is that a red car?”). Sibling making a verbal remark directly related
to the ASD child’s behavior and/or actions. “W” questions (what, when, why, how,
where) are not included in commenting; additionally, asking “yes or no” questions are
also not included in commenting.
Conclusion
The raters viewed one of the in-home filming sessions with the preceptor, and were able to
see the research process from start to finish. One of the observers coded 27 videos of sibling-
autistic disorder child interaction. With the assistance of the other coder, they calculated
conferred inter-rater agreement and independent inter-rater agreement. The coders then
determined how to solidify data gathering and coding techniques in order to ensure standardized
methods for future replication and to prevent bias. They then refined the definitions of the
training components to be more applicable to school-aged children. Finally, they offered
suggestions for the future NIH-funded study.
In order to carry out the functions of the study, it was important that to have a good
background on ASD and knowledge about characteristics that are prevalent in this group. The
coders had to learn how to code data by observing behavior carefully and counting the
occurrences of specifically defined behaviors, and calculating conferred inter-rater agreement
and independent inter-rater agreement. Microsoft Excel was used to formulate graphs of the
collected data. Additionally, the coders attempted to remain unbiased during the data-gathering
process.
During the research process, there were some difficulties with the study itself. There were a
few times where it was difficult for the coders to meet with their preceptor due to their very busy
nursing school schedules. There were also a few lapses between coding sessions, as schedules
BEHAVIORAL CODING 11
did not always warrant having a set time to code every week. Also, the inter-rater accuracy in
the study was lower than expected; however, this provided an opportunity to develop invaluable
insight as to how to improve the study.
There was a great deal to learn during this study, from the acquisition of information about
Autism Spectrum Disorder, to how to conduct research. It was invaluable to see research being
conducted and learn how to reduce bias. Improving the data-gathering process during the coding
of the videos to increase accuracy of the information was important, as was the ability to work
effectively with a partner. The importance of family involvement in these types studies is clear,
not just the importance of subject participation. The families encouraged the subject to
participate and set the schedule for the video recording sessions.
In order to improve the quality of the research, several techniques were used to keep the
researchers focused on the study. The raters communicated frequently, attempting to understand
the needs of all persons involved, and displayed patience throughout the entire process. The
coders met frequently (at least every week) to collaborate on coding sessions, and there were
other meetings to communicate with the preceptor to discuss progress in the research and to
discuss other logistical issues. These meetings were very helpful because they ensured the
coders were moving at an appropriate pace, and the project remained on target. If there had been
any other adversity during this project, it would have been important to communicate these
issues and come up with a plan to address them.
Recommendations
The honors program requires a great deal of time, patience, and dedication, so it is important
to ensure enough time to finish the project. It is also important to start working on the honors
project as early as possible. Starting over the summer is helpful, as it made time management
BEHAVIORAL CODING 12
considerably easier during the fall and spring semesters. Also, it is very important to pick a topic
for the project that is interesting. It is much easier to do the project when you are passionate
about it. It is very important to meet with your preceptor regularly. The preceptor is a great
resource and will provide significant assistance and support during the project. Finally, it is
important for the student to be as objective as possible, it is easy to be biased in your research,
however, and this can significantly alter your results.
Summary
This study by Dr. Jennifer Elder, “In-Home Training for Siblings of Children with Autism”,
was a pilot study based on a sentinel study, “In-Home Training for Fathers of Children with
Autism: A Follow up Study and Evaluation of Four Individual Training Components” (Elder,
2010). The current study incorporated several training components that were taught to the
siblings of the autistic children. These siblings performed the interventions while being
videotaped during interactions with the autistic child. These videos were then coded for the
different interventions taught, Imitation/Animation, Following the Child’s Lead, and
Commenting on the Child. After the coding of the videos, the data collected was analyzed using
conferred inter-rater agreement, and later twenty randomly-selected videos were also coded
using independent inter-rater agreement. The results indicate that the children were able to learn
and utilize specifically taught interventions with their siblings with Autistic Spectrum Disorder.
After this data was collected, suggestions were made for further improvement of the study.
BEHAVIORAL CODING 13
References
American Psychiatric Association. (2013). Diagnostic and Statistical Manual of Mental
Disorders (5th ed.) Washington DC: Author.
Autism Speaks. (2010). How is autism treated? Retrieved from:
http://www.autismspeaks.org/family-services/tool-kits/100-day-kit/treatments-therapies.
Blumburg, S.J., Bramlett, M. D. Kogan, M. D., Schieve, L.A., Jones, J. R., Lu, M.C. (2013).
Changes in prevalence of parent-reported autism spectrum disorder in school-aged US
children. Centers for Disease Control and Prevention, Retrieved from:
http://www.cdc.gov/nchs/data/nhsr/nhsr065.pdf
Center for Educator Compensation Reform. (2011) Research synthesis: D. measurement.
Retrieved from:
http://www.cecr.ed.gov/researchSyntheses/34008_CECR_RS_Inter_Rater_measurement
_508.pdf
Child Development Institute. (2013). Language development in children. Orange, CA. Retrieved
from: http://childdevelopmentinfo.com/child- development/language_development /
Elder, J.H., Donaldson, S.O., Kairalla, J., Valcante, G., Bendixen, R., Ferdis, R., Self, E.,
Walker, J., Palau, C., & Serrano, M. (2010). In-home training for fathers of children
with autism: A follow up study and evaluation of four individual training components.
Journal of Child and Family Studies, 20(3), 263-271.
Elder, J.H., Valcante, G., Yarandi, H., White, D., & Elder, T.H. (2005). Evaluating in-home
training for fathers of children with autism using single-subject experimentation and
group analysis methods. Nursing Research, 54(1), 22-32.
Fleiss, J.L., Levin, B., Paik, M.C. (2003). The measurement of inter-rater agreement. Statistical
BEHAVIORAL CODING 14
Methods for Rates and Proportions (18, 598-625). Retrieved from:
http://hpm.fk.ugm.ac.id/hpmlama/images/Biostatistik/Tutorial_4_AS/2-chapter18.pdf
National Institute of Mental Health. (2007). Autism spectrum disorders (pervasive
developmental disorders). Retrieved from:
http://www.nimh.nih.gov/health/topics/autism-spectrum-disorders-pervasive-
developmental-disorders/index.shtml
BEHAVIORAL CODING 15
Appendix
Appendix A
In-Home Training for Fathers of Children with Autism: A Follow up Study and Evaluation of
Four Individual Training Components
Literature regarding fathers of children with autism remains sparse, and because mothers
are the more common intervening parent, few training methods have focused on fathers. Thus,
we sought to evaluate effects of in-home training directed at fathers and their ability to train
mothers in the same manner in which they were trained. Fathers were taught four skills
commonly associated with in-home training interventions for parents of children with autism:
following the child's lead, imitation with animation, commenting on the child, and expectant
waiting. Father skills were evaluated twice a week for 12 weeks during videotaped in-home
father-child play sessions. Analyses included visual inspection of graphed data and statistical
analyses of father skill acquisition, mother skill acquisition, and child behaviors with both
parents. A multivariate repeated measures analysis of 18 dyads revealed significant increases in
frequencies of fathers' imitation with animation, expectant waiting, and commenting on the child.
Child initiating rates increased significantly as did frequencies of child non-speech vocalizations.
Analysis of mothers revealed significant increases in frequencies of imitation with animation,
expectant waiting, and following the child's lead. Child behaviors had similar results for father
and mother sessions. Findings are consistent with those from the sentinel study indicating that
fathers can effectively implement skills that promote father-child social interactions and that
children respond positively to this approach (Elder, 2010).