1. identify research to support safety assessments. 2. what are other states doing? ◦ studies that...
TRANSCRIPT
1. Identify Research to Support Safety Assessments.
2. What are other states doing? ◦ Studies that show state instruments are valid
and reliable? ◦ Have states looked at the “process” or time
involved with safety assessments.◦ Has safety process effected “face to face” time
with families. ◦ Do states have outcomes on their research?
Tasks for the Group
Research shows workers:1. May ignore the probability of the abuse.2. Could be overconfident in our skills.3. Could have difficulty weighing all the factors.4. Have been shown to make different decisions
under the same conditions.5. Are reluctant to revise their assessments
once made.6. Display skepticism over new information.
(Evidence for Practice , An Executive Summary. U.C. Berkley, 2005)
Why Risk and Safety Assessments?
Consensus Based Models (Included are only those models used in more than one location).
1. Child at Risk Field (CARF) Action for Child Protection2. Washington Risk Assessment Matrix (WRAM).3. California Family Assessment Factor Analysis (Fresno
Model)4. Child Emergency Response Assessment Protocol (Illinois
Model) 5. *Family Assessment Response (After a safety
Assessment)
Actuarial (SDM)◦ Children’s Research Center – Structured Decision
Making.
In all Models Safety Assessments are treated as part of Risk Assessment
(except PA and NC)
Two Major Types of Safety Assessments
“Safety” represents a point in time which is valid & reliable only for that moment.
Designed to improve clinical judgment & decisions.
Attempt to “standardize” a process for decision making.
Observe family behaviors/functioning though lists of expected behaviors.
Similarities of Consensus Basedand Actuarial Models
Examine the ability of the caretaker to protect the child (parents, foster parents, etc…, )
Recognize the role and importance of the extended family to assist in safety.
Examine current situations that affect safety.
Agree that Safety and Risk Assessments are not needed with each contact.
Both see risk and safety assessment as part of an ongoing problem solving process.
More Similarities
Safety/Risk Assessment in a Problem Solving Model
Referral Screen
Investigation
Assessment
Intervention
Planning
Implementation
Evaluation
Follow-up
Other services
Close Case
Reassess Safety/Risk
Ongoing reassessment of problems/issues/t
heories.
Safety/Risk Assessment Differences in the Models
Consensus Based Actuarial
Safety is based on information gathered during the assessment.
Construct validity
More emphasis on the caretaker than the extended family (Action for Child Protection 2007). http://www.actionchildprotection.org/
Safety is based on information gathered during the assessment but variables proven to be valid and reliable are given a score.
Construct and empirical validity (strong statistical validity behind the questions).
Statistically focuses questions to reduce worker error. (Shlonskya A. & Wagner 2005)
Safety/Risk Assessment Differences
Consensus Based Actuarial Models
Child is “safe” or “unsafe” (less emphasis on levels of risk)
Child Safety is “Dynamic”
Less subject to bias, decisions preferences and ethical concerns.
Levels Safety and risk can be predicted.
It’s a reliable and valid checklist
Take a comprehensive approach. Items are based on maltreatment theories. Items often shared across different
instruments or forms. Sometimes numerical scores are given. Tend to use a single tool for all types of
maltreatment reports. Can structure information for clinical
assessments of risk. Help document information for decisions.
Consensus-Based Instruments
Some argue that more information equates to better decisions.
Criticized for:◦ poorly defined measures (nebulous, ambiguous,
subjective).◦ Inconsistency in types of variables◦ Use same variables to predict all types of abuse,
neglect, sexual abuse.◦ Less weight given to recurrence of maltreatment.◦ Reliance on variables for which there is no
research. Evidence for Practice , An Executive Summary. U.C. Berkley, 2005)
Consensus-based instruments
Less bias (Fontes, L.A. (2008). Use statistics to weigh factors to predict future. Often statistical analysis is done in locality where
the instrument is used. Use fewer factors than Consensus Based. Factors are scored and summed into an overall risk
score. Families are rated low, med, high and receive
different service responses. More reliable and valid questions. Raelene Freitag, Director Children's Research Center (2011) Email
correspondence.
Actuarial Instruments
Criticized for:◦ Not using or curtailing the clinical judgment of the
worker.
◦ Basis for judgment on a factor that is statistically associated with recurrence of maltreatment, and may not appear to be causally related to the outcome. This may cause caseworkers to discount the value because they can’t understand the theory, math or reason behind the score.
Evidence for Practice , An Executive Summary. U.C. Berkley, 2005)
Actuarial Instruments
The research literature clearly favors Actuarial Models over Consensus Based.
“Settled Science”
Rittner, B., Children and Youth Service Review, Vol. 24 No. 3, March 2002 pages 189 – 207. (Compared Mich’s actuarial model to Washington’s and Fresno California’s CB models).
Evidence for Practice, Risk and Safety Assessment in Child Welfare: Instrument Comparisons, No. 2 July 2005. Argues for more research.
Johnson, W. (2011). The validity and utility of the California Family Risk Assessment under practice conditions in the field: A prospective study. Child abuse & neglect, 35, 1, 18-28.
D’Andrade, A., Austin, M.J., & Benton, A. (2008). Risk and Safety Assessment in Child Welfare: Instrument Comparisons. Journal of Evidence-Based Social Work 5 (102) 31-56.
Actuarial v. Consensus Based Models
Alaska – Did have full system but may have switched to Action
Arkansas – Says the are implementing
California – Some counties
Colorado - Used in el Paso County only.
Connecticut
DC
Florida – Some communities (not statewide) Family Assessment Response?
Georgia
Indiana – One of the first states, currently updating their model.
States Doing SDM Green = County Systems
Kansas - Two private agencies (Managed Care system?)
Louisiana – Using the model
Massachusetts – Most of system (Union is fighting)
Michigan – One of the first states.
Minnesota – Family Assessment Response State - One of the first states
Missouri – Uses parts of SDM Model
Nebraska (KVC)
New Hampshire
New Jersey
New Mexico - Began several years ago.
States Doing SDM Green = County Systems
New York – Family Assessment Response
Ohio – Incorporated into Competency Based Training.
Rhode Island
Washington – SDM risk assessment only
Wisconsin (some counties)
Utah – Just started
Vermont
Virginia - Family Assessment Response
Raelene Freitag, Director Children's Research Center (2011) Email correspondence.
States Doing SDM (Actuarial)Green = County Systems
SDM in other places New Brunswick is just
starting
Manitoba has part of system
Ontario has risk assessment and may be using more
Saskatchewan is starting
British Columbia has risk and may have picked up others
Bermuda
Queensland
South Australia
New south wales
Northern Territories (in roll out)
Raelene Freitag, Director Children's Research Center (2011) Email correspondence.
Alabama – Currently doing a pilot of the model. Alaska (has not been confirmed) Hawaii Maryland (at the University of Maryland with a
demonstration project) Texas South Dakota West Virginia Wisconsin
http://www.actionchildprotection.org/
States Doing Action For Child Protection (Consensus Based Model)
Arizona
Delaware (Focused Safety Assessment) (PIP)
Georgia
Idaho (Safety Assessment tool) (PIP)
Illinois - Child Emergency Response Assessment Protocol (CERPA), the model of PA’s risk assessment. Improved outcomes with use but no reliability and validity. (Evidence for Practice , An Executive Summary. U.C. Berkley, 2005) and A Comparison of Approaches to Risk Assessment in Child Protection and Brief Summary of Issues Identified From Research on Assessment in Related Fields Child Welfare League of America, 2005)
States Doing Other Consensus Based Models
Iowa
Kentucky (Family In Need Assessment) Family Assessment Response. Very detailed and specific with a great deal legislated.
Maine
Missouri (Part of Action) http://www.dss.mo.gov/cd/info/cwmanual/section2/ch9/sec2ch9.pdf
North Carolina - Family Assessment Response. ** Very detailed Safety Assessment Directions.
South Carolina - Family Assessment Response.
States Doing Other Consensus Based Models
WRAM Wash. St.
FRESNOCal. Model
CARF Action
CERAP Illinois
SDM Actuarial
Scales 7 Domains
5 Domains
5 Domains
3 Scales (1 safety)
ITEMS 37 23 14 14 20
PredictiveValidity
Poor Poor Little Some Highest
Convergent Validity
Poor No Research
Poor No Research
No Research
Inter-rater Reliability
Some but poor
No Research
None No Research
Yes - ok
Outcomes No Research
No Research
None Yes Yes – good.
Racial/Ethnic
Mixed – good
No Research
No Research
No Research
Mixed – good.
Risk/Safety Comparisons (From - Evidence for Practice , An Executive Summary. U.C. Berkley, 2005)
SDM and Family Group Decision Making – Minnesota, SDM News, Issue 24. Feb. 2011.
Use with other Models
Massachusetts looked at SDM. About half the workers thought the processes was helpful, half did not. Raelene F. & Healy T. (2008) Massachusetts Department of Children and Families Assessment Field Test Results
CARF (Action Model): Favored by less experienced workers (Doueck et al. 1993).
Illinois – Found workers inflated scores to get cases accepted for service. (Lyle & Graham 2000).
Assessment forms often missing. (Lyle and Graham 2000).
Studies of the Process
Quality of implementation more important than actuarial or consensus-based.
Consensus that high quality training is needed.
All models need supervisory and management support
Need supervisors and workers engaged in planning and implementation.
(Evidence for Practice , An Executive Summary. U.C. Berkley, 2005)
Studies of the Process
But in California a county process model must:
1. Determine a response to a referral.2. Make an initial safety decision.3. Assesses safety/risk in placements.4. Make a referral disposition.5. Be used in ongoing case planning decisions.6. Be used in reunification7. Be used in case closure. (Evidence for Practice , An Executive Summary. U.C. Berkley, 2005)
Are Multiple Models being used in a State? Yes! Wisconsin & California
Competition between Structured Decision Making (Actuarial) and
Action for Child Protection (Consensus Based)
Action for Child Protection Children’s Research Center
Action is home to the National Resource Center for Child Protective Services.
Consensus Base qualitative research
CRC (SDM) is a non-profit affiliated by the National Council on Crime Delinquency.
Actuarial models quantitative research.
Action for Child Protection (no author listed) (2007) The Safety Decision. http://www.actionchildprotection.org/PDF/May_2007_The_Safety_Decision.pdf
http://www.actionchildprotection.org/
Child Welfare League of America (no author) (2005) A Comparison of Approaches to Risk Assessment in Child Protection and Brief Summary of Issues Identified From Research on Assessment in Related Fields, A report prepared for Pennsylvania.
D’Andrade, A., Austin, M.J., & Benton, A. (2008). Risk and Safety Assessment in Child Welfare: Instrument Comparisons. Journal of Evidence-Based Social Work 5 (102) 31-56.
Doueck, H.J. English, D. DePanfalis, D. & Moote, G. (1993). Decision-making in child protective services: A comparison of selected risk assessment systems. Child Welfare, 72(5), 441-52.
Evidence for Practice , An Executive Summary. U.C. Berkley, 2005.
Fontes, L.A. (2008). Assessment Instruments and Structured Decision Making in Child Abuse and Culture. New York: Guilford Press.
Freitag, R. & Healy T. (2008) Massachusetts Department of Children and Families Assessment Field Test Results
Freitag, Raelene (2011) Director Children's Research Center, email correspondence.
References
Johnson, W. (2011). The validity and utility of the California Family Risk Assessment under practice conditions in the field: A prospective study. Child abuse & neglect, 35, 1, 18-28.
Lyle C. G. & Graham E. (2000). Looks can be deceiving: Using a risk assessment instrument to evaluate the out-comes of child protection services. Children and Youth Services Review, 22(11/12), 935-49.
New York OCFS Family Service Review http://www.ocfs.state.ny.us/main/cfsr/far.shtm
North Carolina DSS On-line manual
Rittner, B., Children and Youth Service Review, Vol. 24 No. 3, March 2002 pages 189 – 207.
SDM News, (no author listed) Issue 24. Feb. 2011.
Shlonskya A. & Wagner D. (2005) The next step: Integrating actuarial risk assessment and clinical judgment into an evidence-based practice framework in CPS case management, Children and Youth Services Review, 27 (2005) 409– 427.
References
Baird, C., Wagner, D., Healy, T., & Johnson, K.(1999). Risk assessment in child protective services: Consensus and actuarial model reliability. Child Welfare, 78(6), 723-748.
Johnson, K., Caskey, R., & Wagner, D. (2003). Addendum to the evaluation of Michigan’s foster care Structured Decision Making case management system. Madison, WI: Children’s Research Center. http://www.nccd-crc.org/crc/pubs/mi_2002_fc_eval_addendum.pdf
Johnson, K. (2005). A retrospective support assessment study of foster and relative care providers. Madison, WI: Children’s Research Center. http://www.nccd-crc.org/crc/pubs/fcrp_support_assmnt_sept05.pdf
Johnson, W. (2004). Effectiveness of California’s child welfare Structured Decision Making® model: A prospective study of the validity of the California family risk assessment. Sacramento, CA: California Department of Social Services. http://www.nccd-crc.org/crc/pubs/ca_sdm_model_feb04.pdf
Lee, S., Aos, S., & Marna Miller, M. (2008). Evidence-based programs to prevent children from entering and remaining in the child welfare system: Benefits and costs for Washington. Olympia: Washington State Institute for Public Policy, Document No. 08-07-3901.
Loman, L. A., & Siegel, G. L. (2004). An evaluation of the Minnesota SDM Family Risk Assessment. Institute of Applied Research. www.iarstl.org/papers/FinalFRAReport.pdf
Rossi, P., Schuerman, J., & Budde, S. (1996). Understanding child maltreatment decisions and those who make them. Chicago: Chapin Hall Center for Children, University of Chicago.
Rycus, J. S., & Hughes, R. C. (2003). Issues in risk assessment in child protective services: Policy white paper. Columbus, OH: North American Resource Center for Child Welfare. http://www.nccd-crc.org/crc/pubs/ra_issues_whitepaper_2003.pdf
Other Readings
Rycus, J. S., & Hughes, R. C. (2007). Issues in risk assessment in child protective services. Journal of Public Child Welfare, 1(1), 85-116.
Stewart, A. & Thompson, C. (2004). Comparative Evaluation of Child Protection Assessment Tools. Queensland: Griffith University.
Wagner, D., Johnson, K., & Caskey, R. (2003). Evaluation of Michigan’s foster care Structured Decision Making case management system. Madison, WI: Children’s Research Center.
http://www.nccd-crc.org/crc/pubs/mi_2002_fc_eval.pdf
Wagner, D. & Johnson, K. (1999). Using actuarial risk assessment to target service interventions in pilot California counties. 13th National Roundtable on CPS Risk Assessment. http://www.nccd-crc.org/crc/pubs/13th_roundtable_ca_risk.pdf
Wagner, D. & Meyer, B.L. (1998). Using actuarial risk assessment to identify unsubstantiated cases for preventative intervention in New Mexico. 12th National Roundtable on CPS Risk Assessment. http://www.nccd-crc.org/crc/pubs/nm_1998_roundtable_risk.pdf
Wagner, D., Hull, S., & Luttrell, J. (1995). The Michigan Department of Social Services risk based Structured Decision Making system: An evaluation of its impact on child protection service cases. Ninth National Round Table on CPS Risk Assessment.
http://www.nccd-crc.org/crc/pubs/mi_1995_cps_eval.pdf
Wood, J.M. (1997, April). Risk predictors for re-abuse or re-neglect in a predominantly Hispanic population. Child Abuse & Neglect. 21 (4) 379-389.
Other Readings