Upload File

conducting meaningful cost analysis · best you can do the back of envelope approach your own back...

  • Home
  • Documents
  • Conducting Meaningful Cost Analysis · best you can do The Back of Envelope Approach Your own back of envelope calculations are unlikely to involve geometry. ØMost likely, you will
33
Conducting Meaningful Cost Analysis Michael T. Braun, Ph.D. Satomi Wakita, Ph.D. Children and Family Research Center, School of Social Work at the University of Illinois at Urbana-Champaign 2017 CQI Conference

Upload: others

Post on 21-Aug-2020

0 views

Category:

Documents


0 download

Report

TRANSCRIPT

Page 1: Conducting Meaningful Cost Analysis · best you can do The Back of Envelope Approach Your own back of envelope calculations are unlikely to involve geometry. ØMost likely, you will

Conducting Meaningful Cost AnalysisMichael T. Braun, Ph.D. Satomi Wakita, Ph.D.

Children and Family Research Center, School of Social Work at the University of Illinois at Urbana-Champaign

2017 CQI Conference

Page 2: Conducting Meaningful Cost Analysis · best you can do The Back of Envelope Approach Your own back of envelope calculations are unlikely to involve geometry. ØMost likely, you will

Purpose

Ø reviewwhatstepsareneededtoperformacostanalysis,inparticularfocusingonthe“ingredients”ofdata.

Ø talkaboutthebarriersforcalculatingcostsandconductingtheanalysis.

Page 3: Conducting Meaningful Cost Analysis · best you can do The Back of Envelope Approach Your own back of envelope calculations are unlikely to involve geometry. ØMost likely, you will

1.CQIandcostanalysis2.Ingredientsofcost3.Ourproject4.Barriersforcalculatingcosts

5.Whatwelearned6.Questionsanddiscussion

Outline“Costanalysisisathoroughdescriptionofthetypeandamountofallresourcesusedtoproduceprogramservices.”

Source: Yates, B. T., Delany, P. J., & Dillard, D. L. (2001). Using cost " procedure " process "outcome analysis. In Bruce A. Thyer (Ed.), the Handbook of Social Work Research Methods(pp. 207-238). Thousand Oaks, CA: Sage.

Page 4: Conducting Meaningful Cost Analysis · best you can do The Back of Envelope Approach Your own back of envelope calculations are unlikely to involve geometry. ØMost likely, you will

1.Identifyand

understandtheproblem

2.Researchthesolution

3.Developthetheoryofchange

4.Adaptordevelopthe

solution

5.Implementthesolution

6.Monitorandassess

thesolution

Source: CQI Training Academy. (2014). CQI cycle of learning and improvement tip guide.

Figure 1. CQI Cycle of Learning and Improvement

Page 5: Conducting Meaningful Cost Analysis · best you can do The Back of Envelope Approach Your own back of envelope calculations are unlikely to involve geometry. ØMost likely, you will

Strategiesforcalculatingcosts

Page 6: Conducting Meaningful Cost Analysis · best you can do The Back of Envelope Approach Your own back of envelope calculations are unlikely to involve geometry. ØMost likely, you will

Personnel (salaries and benefits)

Materials and supplies

Depreciation costs of durable equipment

Facilities / office space

Travel Utilities

1. Itemize resources used to implement the program.

Page 7: Conducting Meaningful Cost Analysis · best you can do The Back of Envelope Approach Your own back of envelope calculations are unlikely to involve geometry. ØMost likely, you will

2. Collect data on resource costs.

Direct costq Employee salariesq Employee benefitsq Initial service costsq Costs associated with

servicesq Costs of out-of-home

placement

Page 8: Conducting Meaningful Cost Analysis · best you can do The Back of Envelope Approach Your own back of envelope calculations are unlikely to involve geometry. ØMost likely, you will

Direct cost(continued) q Time the worker spent

on the - Assessment- Interview- Communication

(e.g., email, phone call, etc.)

Page 9: Conducting Meaningful Cost Analysis · best you can do The Back of Envelope Approach Your own back of envelope calculations are unlikely to involve geometry. ØMost likely, you will

Indirect costq Administrativeq Management

§ Training§ Accounting§ HR

q Infrastructure costs§ IT

q Facilities

Page 10: Conducting Meaningful Cost Analysis · best you can do The Back of Envelope Approach Your own back of envelope calculations are unlikely to involve geometry. ØMost likely, you will

3. Collect data on resource allocation.

Category ActivityServicestoenrolledfamilies Parentgroupmeeting

• Proportion of salaries/fringe benefits for the caseworker

• Value of any volunteer labor• Educational materials• Client support materials• Cars• Computers/laptops• Office supplies• Office space

Page 11: Conducting Meaningful Cost Analysis · best you can do The Back of Envelope Approach Your own back of envelope calculations are unlikely to involve geometry. ØMost likely, you will

4. Estimate the value of direct and indirect costs.

5. Create an estimate of total costs.

Page 12: Conducting Meaningful Cost Analysis · best you can do The Back of Envelope Approach Your own back of envelope calculations are unlikely to involve geometry. ØMost likely, you will

6.Estimatecostsofindividualprogramcomponentsoractivities.

7.Estimatecostsperparticipant.

8.Conductanalysesandexaminecostvariation.

Source:Calculating the Costs of Child Welfare Services Workgroup. (2013). Cost analysis in program evaluation: A guide for child welfare researchers and service providers. Washington, DC: Children’s Bureau, Administration for Children and Families, U.S. Department of Health and Human Services.

Page 13: Conducting Meaningful Cost Analysis · best you can do The Back of Envelope Approach Your own back of envelope calculations are unlikely to involve geometry. ØMost likely, you will

üFamiliarizeoneselfwiththeinterventionsthatwillbeevaluated:Whatarethegoals?

üDecidewhetheracostanalysisisworthdoing.üRecognizetheaudience.Whataretheirneeds? Whatprogramorservicewillbeanalyzed?

üWhatdaterangewillbecovered?üWhatcomparisonswillwemake?

Checklist for establishing a cost analysis

Page 14: Conducting Meaningful Cost Analysis · best you can do The Back of Envelope Approach Your own back of envelope calculations are unlikely to involve geometry. ØMost likely, you will

Ø Costevaluationwasoneoftheoutcomesintheevaluationproject.

Ø Totestthetheorythatanewprogramwouldproducehighercostsduringtheinitial assessmentandlowerthecostsduringthefollow-up period.

Cost analysis for our evaluation project

Page 15: Conducting Meaningful Cost Analysis · best you can do The Back of Envelope Approach Your own back of envelope calculations are unlikely to involve geometry. ØMost likely, you will

Initialcaseandfollow-upserviceperiodsforthecostanalysis

Initial Report Date Assessment Close Date

365 Days after Initial Report Date

Initial Case Period Follow-up Period

Page 16: Conducting Meaningful Cost Analysis · best you can do The Back of Envelope Approach Your own back of envelope calculations are unlikely to involve geometry. ØMost likely, you will

Analysisdesign(unit=family)Treatment Group

Comparison Group

Total cost

Total cost

Selectrandomly

Page 17: Conducting Meaningful Cost Analysis · best you can do The Back of Envelope Approach Your own back of envelope calculations are unlikely to involve geometry. ØMost likely, you will

ØFirst,weasked:Whatdowecareabout?ØWecareaboutpiecesthatchangewiththeNewProgram

ØWedon’tcareaboutanythingthatisthesamebetweenBusinessasUsualandtheNewProgram

ØForexample,nonewfacilitiesareneeded,nonewsupplies

Cost Types

Page 18: Conducting Meaningful Cost Analysis · best you can do The Back of Envelope Approach Your own back of envelope calculations are unlikely to involve geometry. ØMost likely, you will

ØSoaskyourself,WhatdoIcareabout?ØDoIcareaboutcoststostarttheprogramFROMSCRATCHsomewhereelse?

ØOrarethesecostsmoreaboutsmallchangestoBusinessasUsual?

ØForexample,let’ssaytheprogramisaboutmotivationalinterviewing

Cost Types

Page 19: Conducting Meaningful Cost Analysis · best you can do The Back of Envelope Approach Your own back of envelope calculations are unlikely to involve geometry. ØMost likely, you will

ØSecond,weasked:OfthethingsweDOcareabout,whatcanweaccess?ØStatelevelevaluationsmeanlotsofdifferentsystemsandlotsofdifferentgatekeepers

ØForexample,wecouldn’tgetinformationaboutexactemployeesalary

Cost Types

Page 20: Conducting Meaningful Cost Analysis · best you can do The Back of Envelope Approach Your own back of envelope calculations are unlikely to involve geometry. ØMost likely, you will

ØSoaskyourself,WhatcanIgetaccessto?ØIdeally… Butofcourse,wedon’tliveinanidealworld!

ØForexample,howmuchburdencanweputonworkers?

Cost Types

Page 21: Conducting Meaningful Cost Analysis · best you can do The Back of Envelope Approach Your own back of envelope calculations are unlikely to involve geometry. ØMost likely, you will

ØThird,weasked:Howaccuratecanourcostcategorymeasuresbe?ØOfthethingswedocareaboutandcangetaccessto,howaccurateistheinformationwehave?

ØForexample,workertimespentwithfamilies

Cost Types

Page 22: Conducting Meaningful Cost Analysis · best you can do The Back of Envelope Approach Your own back of envelope calculations are unlikely to involve geometry. ØMost likely, you will

ØSoaskyourself,HowaccuratecanIgetwithoutgoingcrazy?ØDoyouwanttotaketheAccountantApproach,theBackofEnvelopeApproach,orsomethinginbetween?

Cost Types

Page 23: Conducting Meaningful Cost Analysis · best you can do The Back of Envelope Approach Your own back of envelope calculations are unlikely to involve geometry. ØMost likely, you will

ØAccountforeverypossiblecostineverypossibleway!

ØYoumightneedthisapproach!

The Accountant Approach

Page 24: Conducting Meaningful Cost Analysis · best you can do The Back of Envelope Approach Your own back of envelope calculations are unlikely to involve geometry. ØMost likely, you will

ØQuickanddirtyØThismightbethebestyoucando

The Back of Envelope Approach

Your own back of envelope calculations are unlikely to involve geometry.

Page 25: Conducting Meaningful Cost Analysis · best you can do The Back of Envelope Approach Your own back of envelope calculations are unlikely to involve geometry. ØMost likely, you will

ØMostlikely,youwillchooseacombinationofthetwo

ØSomecategoriesallowexactcalculationsØOthersrequireroughapproximationsØThat’sokay,aslongasyounoteyourcertaintyaboutallcategories

Or somewhere in between

Page 26: Conducting Meaningful Cost Analysis · best you can do The Back of Envelope Approach Your own back of envelope calculations are unlikely to involve geometry. ØMost likely, you will

ØBacktoourexperience…ØWorkertimewithfamilieswasalistofworkercontactswithfamiliesØButwecouldnotknowA)howlongthecontacttook,B)ifthecontactwasfacetofaceorviasomeothermethod,andC)ifworkersreliablyusedthesamecontacttypeforsimilarcontacts

Cost Types

Page 27: Conducting Meaningful Cost Analysis · best you can do The Back of Envelope Approach Your own back of envelope calculations are unlikely to involve geometry. ØMost likely, you will

ØSowhattodo?ØFindthemostcommoncontacttypesØUsean“expertpanel”andaskthemfortheirbestestimates!

ØAveragetheirestimatesforanaveragetimeforeachcontacttype

Cost Types

Page 28: Conducting Meaningful Cost Analysis · best you can do The Back of Envelope Approach Your own back of envelope calculations are unlikely to involve geometry. ØMost likely, you will

ØThisresultedinacostanalysiscontainingthisdata:

1. Workercontactswithfamilies*averagetimepercontact*averageworkersalary=Costofworkertime(forinitialandfollow-upperiods)

2. Servicecostsperfamily(forinitialandfollow-upperiods)

The Result

Page 29: Conducting Meaningful Cost Analysis · best you can do The Back of Envelope Approach Your own back of envelope calculations are unlikely to involve geometry. ØMost likely, you will

ØAndwhatdidwefind?1. Workersdidspendmoretimewithfamiliesin

thenewprogram2. Thismeantthenewprogramcostmoreinthe

beginning3. Butalsonoevidencethatnewprogram

reducedcostsinthefollow-upperiod

The Result

Page 30: Conducting Meaningful Cost Analysis · best you can do The Back of Envelope Approach Your own back of envelope calculations are unlikely to involve geometry. ØMost likely, you will

ØThiswasthehardestquestiontoanswerØAfewthoughts…1. Peopleunderstandcostmoreintuitivelythan

othermeasures2. ButweKNOWthatthisisonlyarough

estimateoftruecost3. Sowehavetobecautiousinanyconclusions

weoffer

Is this useful at all?

Page 31: Conducting Meaningful Cost Analysis · best you can do The Back of Envelope Approach Your own back of envelope calculations are unlikely to involve geometry. ØMost likely, you will

What we learned

ØGarbageIn/GarbageOut(GIGO)ØWatchoutforwaysinwhichvarianceiscapped

ØAndwatchoutfortimesvarianceishuge!ØConsiderthereasonsyoumay(ormaynot)wanttoconvertallunitsintodollarsandcentsØOryoumightsay,MakeyourcostanalysisunitsDollarsand(Common)Sense

Page 32: Conducting Meaningful Cost Analysis · best you can do The Back of Envelope Approach Your own back of envelope calculations are unlikely to involve geometry. ØMost likely, you will

ThankYou!Whatquestionsdoyouhave?

Otherquestionstothinkabout/discuss:1. Whatistheroleofcostanalysisinyour

organization?2. Whataresomebarriersinconductingcost

analyses?3. Anythoughtsonmakingyourcostanalysis

moremeaningful?

Michael [email protected]

Satomi [email protected]

Page 33: Conducting Meaningful Cost Analysis · best you can do The Back of Envelope Approach Your own back of envelope calculations are unlikely to involve geometry. ØMost likely, you will

ReferencesØ CalculatingtheCostsofChildWelfareServices

Workgroup.(2013).Costanalysisinprogramevaluation:Aguideforchildwelfareresearchersandserviceproviders.Washington,DC:Children’sBureau,AdministrationforChildrenandFamilies,U.S.DepartmentofHealthandHumanServices.

Ø Levin,H.M.&McEwanP.J.(2001).Cost-effectivenessanalysis:Methodsandapplications(2nd ed.).ThousandOaks,CA:Sage.


Using Strategies and AND/OR Decomposition for Back of the Envelope Reasoning

An Unlikely Hero

GOMS Analysis on the back of the envelope

Thermal Performance of Building Envelope Details for … Thermal... · performance of building envelope details ... BUILDING ENVELOPE DESIGN DETAIL ... envelope details for mid- and

Please find following items in the envelope...please use the enclosed return envelope to send it back. Participants who fail to return their timing chip will be charged 500 yen. If

A GUIDE TO BACK EXERCISESorlafoley.com/wp-content/uploads/2014/09/Back2Yourself... · 2020-05-04 · o Surgery is risky and unlikely to heal most back pain or prevent recurrence

Guesstimation: Solving the world’s problems on the back of a … · 2017. 9. 28. · Guesstimation: Back of the envelope solutions to the world’s problems Princeton University

ENVELOPE / BREV for artists€¦ · fold it back up to make the envelope Fold the flaps together and seal your envelope carefully, using one piece of tape as illustrated so it can

Most Unlikely Friendships

Dynamic Scoring: A Back-of-the-Envelope Guide The Harvard

Unlikely Secret Agent

Unlikely Friendships Calendar

Swiss International Stationaries Branding Manual · INDEX Sales Kit A4 Envelope A4 Letter Head Business Cards DL Envelope Back of Evelope Compliments Tent Card Welcome Tent …

Back of the Envelope Calculation: Ferrari IPOpwsclub.com/wp-content/uploads/2015/10/Ferrari-Presentation.pdf · Back of the Envelope Calculation: Ferrari IPO Jiri Tresl, PhD CFA CAIA

Successive Interference Cancellation: A Back of the Envelope Perspective

Apr 17, 2015 00:00 BST COMMENT: Back on behind vinyl's ...newsroom.northumbria.ac.uk/.../back-on...s-unlikely-comeback-11947… · commercial comeback. In an era in which even digital

The Back of an Envelope (at Ignite NewsFoo)

Guesstimation: Solving the world’s problems on the back of a cocktail napkin … · 2018-11-19 · Back of the envelope solutions to the world’s problems Princeton University

Back of the Envelope Business Planning

Tech Report NWU-EECS-07-11:Back of the Envelope Reasoning for Robust Quantitative Problem Solving

An Unlikely Cause of Back Pain: Eikenella corrodens ... · clinical microbiology & infectious diseases : official publication of the European Society of Clinical Microbiology. 1992;

Richard Ellenson, Unlikely AT Inventor, Looks Ahead and Back · Richard Ellenson, Unlikely AT Inventor, Looks Ahead – and Back An interview with Richard Ellenson, parent of a child

Back of the envelope: estimation & orders of magnitudepleclair.ua.edu/ph111/L2_estimation/back-of-the-envelope.pdfGuesstimation: Solving the World's Problems on the Back of a Cocktail

Medically Unlikely Edits (MUEs) - Optum360 · Medically Unlikely Edits (MUEs)—Professional Medically Unlikely Edits (MUEs) The Centers for Medicare & Medicaid Services (CMS) began

Science on Supercomputers: Pushing the (back of) the envelope Jeffrey P. Gardner Pittsburgh Supercomputing Center Carnegie Mellon University University

Upcycled Envelope Clutch - WeAllSew · Envelope Clutch step two Apply fusible interfacing to the back of the ivory satin following the manufacturer’s instructions to stabilize it

Peter Wittich Back of the Envelope Physics

Unlikely dissapeared

Unlikely Friendships Brochure

BACK OF THE ENVELOPE (BOTE) MODEL FOR ONE ......BACK OF THE ENVELOPE (BOTE) MODEL FOR ONE COUNTRY TWO FACTORS AND FOUR COMMODITIES (124) WITH MATHS APPENDIX SHOWING HOS ORIGIN AND

Craft Brewery Insights - beveragebusinessbuilders.com · Craft Brewery Insights: ... Balance Sheet DR / CR Natural balance Key • • • • • • • • → Back of the Envelope

Unlikely Help

Back of the Envelope Reasoning Praveen Paritosh Ken Forbus QR04 Practice Talk Thursday, July 8, 2004

Petrocapita March 2011 - Oil Prices, Positive Feedback Loops and the Back of an Envelope

Energy in 2017: two steps forward, one step back · Those factors were unlikely to persist. Indeed, last year’s Stats ... stocks fell back to more normal levels last year. ... barrels