si 75 robert wood johnson foundation 060113

CASE: SI-75 DATE: 03/29/05 (REVD 09/12/07)

Lecturer Laura Arrillaga-Andreessen and Victoria Chang prepared this case as the basis for class discussion rather than to illustrate either effective or ineffective handling of an administrative situation. The Stanford Graduate School of Business gratefully acknowledges the assistance of Giving 2.0 (giving2.com) in the development of this case.

Copyright 2013 by the Board of Trustees of the Leland Stanford Junior University. Publically available free cases are distributed through ecch.com. No part of this publication may be reproduced, stored in a retrieval system, used in a spreadsheet, or transmitted in any form or by any means electronic, mechanical, photocopying, recording, or otherwise without permission of the Stanford Graduate School of Business. Every effort has been made to respect copyright and to contact copyright holders as appropriate. If you are a copyright holder and have concerns, please contact the Case Writing Office at [email protected] or write to the Case Writing Office, Stanford Graduate School of Business, Knight Management Center, 655 Knight Way, Stanford University, Stanford, CA 94305-5015.

ROBERT WOOD JOHNSON FOUNDATION

The Robert Wood Johnson Foundation (RWJF) was launched in 1972 as a national philanthropic organization devoted exclusively to improving the health and health care of all Americans. It was started by Robert Wood Johnson, who had built Johnson & Johnson into the worlds largest health and medical care products company. The foundation prioritized its grantmaking into four areas: (1) to ensure that all Americans have access to quality health care at reasonable cost, (2) to improve quality care and support for people with chronic health conditions, (3) to promote healthy communities and lifestyles, and (4) to reduce the personal, social, and economic harm caused by substance abusetobacco, alcohol, and illicit drugs.1 RWJF funded a broad range of grantees including nonprofit organizations, universities, research organizations, professional associations, state and local government agencies, health care providers, and community groups. To accomplish its goals, the foundation supported training, education, research (excluding biomedical research), and projects that demonstrated the effective delivery of health care services. Its focus was not on paying for individual care, but rather on strengthening health care systems and promoting the conditions that would advance better health. Areas of interest were reflected in four grantmaking portfolios: (1) targetedaddressing specific improvements in targeted health and health care challenges over a defined time period, (2) human capitalattracting, developing, and retaining high-quality leadership and workforce to improve health and health care, (3) vulnerable populationspromoting community-based projects that improve health and health care outcomes for societys most vulnerable people; and (4)

1 Robert Wood Johnson Foundation, Our Mission, http://www.rwjf.org/about/mission.jhtml (July 26, 2006).

Robert Wood Johnson Foundation SI-75

p. 2

pioneerpromoting fundamental breakthroughs in health and health care through innovative projects.2 Because of RWJFs health care roots, it utilized the construct of clinical trials as a model for strategic development. According to the foundations Web site, Research is central to the foundations mission. In fact, [more than] one of every four grant dollars provided by the Robert Wood Johnson Foundation funds research and evaluation initiatives. Findings from these initiatives highlight lessons learned from our investments and inform our future work in health and health care. At the same time, our research efforts provide a useful knowledge base that is designed to assist the efforts of our colleagues in the health and health care policy fields.3 RWJFs work to build a knowledge base about health and health care included research, policy analysis, and program evaluation. A total of 10 research, policy analysis, and evaluation officers worked at the foundation, and each officer worked on more than one strategic interest area. By 2005, RWJF had firmly established the importance of building the knowledge base of the health and health care fields within its strategy and was committed to communicating its findings to both internal and external constituencies. Linda Bilheimer, a senior program officer for health care coverage and RWJFs research and evaluation officer, noted the additional importance of translating research findings for policymakers and decision-makers: Effective research translation and synthesis that can facilitate informed decision-making by policymakers and health care leaders can actually be more difficult than conducting original research. Unfortunately, this discipline does not have the same stature within academic institutions as original research. Addressing this issue is an important challenge for us.4

RESEARCH, POLICY ANALYSIS, AND EVALUATION

James Knickman, senior vice president for research and evaluation at RWJF believed in the motto, Better information provides the groundwork for better decisions.5 The foundation made grants to research analysts after staff had identified specific health problem as priorities during its fall grantmaking strategy session and concluded that improved research could potentially help society resolve or ameliorate the problem. From 1994 to 1998, the foundation awarded 1,154 grants involving research, policy analysis, and evaluation totaling $366 million and accounting for more than 30 percent of the foundations awards.6

2 Robert Wood Johnson Foundation, Portfolio Overview, http://www.rwjf.org/portfolios/overview.jhtml (July 26, 2006). 3 Robert Wood Johnson Foundation, Research, http://www.rwjf.org/research/index.jsp?gsa=1 (July 26, 2006). 4 All quotations from representatives of the Robert Wood Johnson Foundation are from interviews by the authors unless otherwise cited. 5 James R. Knickman, Research as a Foundation Strategy, as it appears in the RWJF annual anthology of articles entitled To Improve Health and Health Care 2000, http://www.rwjf.org/files/publications/books/2000/chapter_07.html (July 26, 2006). . 6 Ibid. According to Knickman: Many foundation grants involved both research and nonresearch activities. The 1,154 grants involving research, policy analysis, or evaluation included all those where research, evaluation, or policy analysis played some role. The $366 million figure only counts the subset of dollars devoted to research, evaluation, or policy analysis for each of the 1,154 grants.


p. 3

The foundations research strategy did not entail encroaching on the strategic space of large organizations such as the National Institutes of Health (NIH) as a source of funding for medical research. NIH had significant funding capacity and, in 2002 alone, allocated $19 billion in extramural research grants.7 In addition, the RWJF chose not to become a think tank for federal policy due to its desire to avoid partisan politics. Rather, according to Knickman, the foundations research strategy focused on improving the nations ability to understand key health and health care issues so that decisions could be made concerning the way Americans maintain health and obtain health care.8 Thus, the foundation focused on mapping out and assessing health care policy issues for the public and key decision-makers. The foundations rationale for focusing on research was to test the effectiveness of foundation-funded programs, learn about health and health care, obtain baseline information about health or health care issues, understand public policy choices, understand health-related behavior, and build the health care field through training programs.

RESEARCH, POLICY AND ANALYSIS, AND EVALUATION PROJECTS

The types of research, policy analysis, and evaluation projects that the foundation funded were divided into the following categories: (1) research projects, (2) policy analysis and planning, (3) program evaluation, (4) research-driven demonstrations, and (5) training.

Research Projects

Each fall, foundation staff members embarked on a two-year grantmaking plan that included research projects as part of the process. Such projects were often initiated by staff members, who in turn solicited proposals and sought funding for implementation. In some cases, the foundation developed a national advisory committee to make funding selections. Other research projects stemmed from ad-hoc proposals that entered the foundation through unsolicited mail. The success rate of unsolicited projects being funded was approximately 10 percent. In other cases, the foundation brought together researchers to form networks to investigate a particular topic such as the causes and origins of tobacco dependence.

Policy Analysis and Planning

According to Knickman, most of the foundations funds for policy analysis and planning were allocated to large national programs that provide resources to states and state-based nonprofit organizations to examine options for improving health, health care, and health policy.9 An example of a large national program was State Initiatives in Health Care Reform, which funded policy analysis at the state level. Large national programs often made grants to intermediary organizations, which then conducted policy analysis and research to map out key issues.

7 Outside typical boundaries, thus outside of NIHs core focus areas. 8 Knickman, op. cit. 9 Ibid.


p. 4

Program Evaluation

The foundation viewed program evaluations as a way to determine whether, and how, a program could be replicated or modified. Evaluations focus on what can be learned and communicated from a program; they do not monitor individual grant management and administrative performance, said Knickman. The administrative staff at the foundation and the national program offices that are set up to facilitate the operations of each multi-site initiative perform these functions.10 Program staff members worked with an outside evaluator to determine the best approach and methodology for a national program evaluation. Some evaluations were formal and focused on outcomes assessment, while others were less formal and relied on self-assessment. Through formative evaluations, the foundation encouraged grantees to obtain information about ongoing progress to measure intermediate outcomes and adjust strategy accordingly. Since the foundations inception, it had conducted over 100 external evaluations of its programs.

Research-Driven Demonstrations

In some cases, the foundation funded research-driven demonstrations or demonstration programs simultaneously coupled with evaluations of the demonstration. According to Bilheimer, the intent of research-driven demonstrations was to try a new idea, structure it in a way that you can evaluate it, and then present the findings to the field. As the foundation expressed it: Information to improve decisions in the health field sometimes requires carefully designed field trials of new approachesan innovative new service delivery system, for example, or a new way of financing health care, or an intervention to change health-related behavior.11 Such situations required research-driven demonstrations, which resembled clinical trials. Grantees selected to manage demonstrations were typically research experts in their specified fields.

Research Support through Training Programs

The most flexible research funding was provided in the form of training programs for health researchers. Knickman explained, Generally, the foundation funds training programs to correct perceived deficiencies in the United States health care workforce. In the case of research training programs, however, the training is designed to enable researchers to define and carry out research projects that will advance their careers and create new generations of senior faculty who will address research questions important to the foundation.12

MEASURING RESEARCH, POLICY ANALYSIS, AND EVALUATION INVESTMENTS

The foundation wanted to determine whether its investment in research, policy analysis, and evaluation actually led to improvement in health and health care, and if so, whether the payoffs were larger than with other types of grantmaking. Knickman acknowledged the difficulty of measuring such efforts: It is difficult to argue with the fact that information does matter.

10 Ibid. 11 Ibid. 12 Ibid.


p. 5

However, it is much more difficult to know whether marginal investments in information really affect the decisions society makes about health care.13 Outcomes related to the foundations research efforts had included the establishment of entire new fields of analysis; for instance, public policy research related to tobacco control. In addition, research investments had led to the development of important new data sets and a better understanding of health and health care (e.g. market forces in health care, state public policy initiatives to combat substance abuse, and alternative approaches to caring for people at the end of life). Knickman cited the foundations willingness to nurture young researchers as another benefit to society in the long run. However, he noted, Perhaps we will never know whether our research investments make tangible contributions to solving the social problems the foundation cares about. We may have to trust the more commonsense observations that societies, businesses, and institutions that invest in research and information tend to do better than those that lag in these investments.14

AN EXAMPLE: COVERING KIDS AND FAMILIES PROGRAM OVERVIEW

The foundations program Covering Kids and Families (CKF) was designed to increase the number of eligible children and adults who enrolled in federal and state health care coverage programs. Among the tens of millions of uninsured Americans, there were significant numbers of children and adults who were eligible for publicly funded health care coverage programs but were not enrolled. In 2003, 45 million Americans were uninsured, including 8.4 million children. More than 5 million uninsured children were in families with income below 200 percent of the federal poverty level. Most of those low-income children were eligible for Medicaid and the State Childrens Health Insurance Program (SCHIP). RWJF launched Covering Kids and Families (CKF) in 2001 as an extension of the Covering Kids Initiative (CKI), which had been started in 1997 to help states and local communities increase the number of children who benefited from health coverage programs. CKI had begun as a $13 million initiative planned for up to 15 states and had grown into a $47 million initiative with projects in 50 states, the District of Columbia, and more than 170 local pilot communities. CKI employed a state and local coalition model to promote childrens coverage, using three primary strategies: (1) conduct and coordinate outreach programs, (2) simplify enrollment and renewal processes, and (3) coordinate existing health care coverage programs. To support the work of CKI, the foundation undertook a multi-year, $26 million public education campaign to inform income-eligible parents that their uninsured children could potentially qualify for low-cost or free health care coverage and to prompt them to inquire about enrollment. The 2001 Covering Kids and Families program was a $68 million initiative that operated through statewide projects in 45 states and the District of Columbia, liaison grants in five states, and more than 140 local community projects. Grantees under this program were eligible for up to four years of funding. Using the same three CKI strategies, CKF expanded the programs mission to include the enrollment of eligible parents in public coverage programs. In addition, it placed greater emphasis than the earlier program on reenrollment issues. 13 Ibid. 14 Ibid.


p. 6

Evaluation of CKF and Internal Programmatic Changes

By 2005, the foundation had not yet released CKF findings to the public except for the occasional general article as they were still in the relatively early stages of evaluation. However, Bilheimer emphasized that regular feedback was essential within the CKF program:

We did not want to wait until the end of four years to get an evaluation report. Instead, what we really wanted was ongoing formative feedback about the program in addition to an outcomes evaluation. We wanted to understand the implementation process, the barriers that our grantees face, how they address them, and how the economic and policy environment in which they are operating is changing. We also wanted to know about their technical assistance needs. Further, we wanted to hear from low-income families about their perceptions of Medicaid and SCHIP, including enrollment barriers and problems accessing care once enrolled. Finally, we sought early insights on the effectiveness of the coalition model that we are using.

Toward the close of CKI, the foundation conducted a program assessment, which included a methodological tool called a reverse site visit. The foundation staff then spent almost a year working with evaluation experts to design a full evaluation of the new CKF initiative. Phase I of that evaluation was funded at $3.6 million for a two- to three-year period. Funding for a second phase of the evaluation was to be considered by the foundations board in the summer of 2005. Researchers from three major national research organizationsMathematica Policy Research, the Urban Institute, and Health Management Associateswere conducting the evaluation. Techniques included a range of qualitative methods, such as focus groups, site visits, reverse site visits, and the use of highlight memos, in addition to quantitative analysis. The evaluators also designed a Web-based reporting system, to facilitate ongoing reporting by grantees on their activities.

Reverse Site Visits The original Covering Kids Initiative did not have an evaluation component built into the program. According to Bilheimer, When it was time to consider the renewal of the program, there were no evaluation findings to guide us and no data for us to analyze. Thus, Bilheimer and her team conducted a fast turnaround assessment of the program, using program documents and a new technique called a reverse site visit, which the foundation came to use in the CKF program on an annual basis. Bilheimer described this assessment strategy as unusual, as it ran counter to the conventional evaluation approach of sending evaluators to sites. A reverse site visit was exactly thatbringing people in versus sending people out. Grantees from around the country, a panel of experts, and an expert moderator were brought in to the foundation to probe and pull thoughts out of peoples heads, as well as enable grantees to share their lessons with fellow field practitioners. Foundation staff presence was kept to a minimum during reverse site visits. I was able to observe a reverse site visit in progress, and it was extraordinary, said Bilheimer. The issues that came out were so much richer because the grantees were interacting with each other and with the panel, rather than being interviewed individually by panel members or being seen alone out in their sites by a group of experts.


p. 7

One key issue that emerged from the first reverse site visit was related to access. Grantees indicated that one of the key reasons why families did not enroll in Medicaid or SCHIP was due to access problems Medicaid and SCHIP enrollees had experienced in the past. Bilheimer described one organization that had resolved the problem in a grassroots manner. Each week, the grantee would call primary care physicians within the radius of a particular neighborhood and ask whether the physicians were taking new Medicaid patients. After that, she would go from door to door to encourage families to enroll their children in Medicaid and SCHIP, handing each parent the name and number of an available physician. After the reverse site visit, the foundation included an access component in the new CKF program to support the capacity of grantees to analyze and solve access issues related to enrollment.

Highlight Memos and Data Analysis As a result of the foundations desire for feedback and lessons learned from the first Covering Kids Initiative, Bilheimer and her team commissioned a series of qualitative highlight memos from the evaluators for the new CKF program. Highlight memos gave the foundation real-time feedback from focus groups, site visits, survey results, and information from the Web-based reporting system. By early 2005, the foundation had received 11 such memos and four reports. The memos and reports were shared with the National Program Office, as well as with grantees. Simultaneously, the foundation also conducted a larger quantitative evaluation that fed into the qualitative evaluation and vice versa. The qualitative evaluations and the formative feedback from the site visits and surveys drove many of the questions for the quantitative data analysis. Administrative data on Medicaid enrollment from the Centers for Medicare and Medicaid Services (CMS), which the agency agreed to provide, were critical for this analysis. Using enrollment data, the evaluators developed an epidemiologic15 model to identify increases and decreases in Medicaid/SCHIP enrollment. In turn, the foundation agreed to share its findings and conduct seminars for CMS staff on related implications for Medicaid and SCHIP. Bilheimer said, I think its a win-win for both parties. We get access to the data, and they get feedback on what we have learned from our initiatives, which they value.

External Communication

Communication formed a critical component of the CKF strategy. Since the inception of the CKF Back-to-School Campaign in 2000, messages about the availability of low-cost and free health care coverage reached millions of families across the nation annually. The communications team worked with CKF coalitions, organizations, and corporate partners as well as with local outreach efforts across the country to reach these families. Extensive media coverage, generated by campaign activities, resulted in information about low-cost or free health care coverage for uninsured children, reaching a potential cumulative audience of more than 489 million readers, viewers, and listeners. More than 1,880 television stories, 60,630 radio stories and interviews, and more than 2,050 print stories appeared in publications across the United States.

15 Epidemiology is the branch of medicine that deals with the study of the causes, distribution, and control of disease in populations.


p. 8

The efforts of more than 27 national and regional companies fortified the effort, resulting in CKF messages reaching a potential cumulative audience of more than 689 million people. Notable efforts included those of Capital One Financial Corporation and CVS/pharmacy. Capital One Financial Corporation communicated the availability of low-cost and free health care coverage programs by providing messages about health care coverage in English and Spanish in billing statements to 200,000 existing Spanish-speaking cardholders. CVS/pharmacy alerted families about low-cost and free health care coverage when parents filled or refilled prescriptions for their children (ages 0-18), at the check-out counter, through public services announcements on its public address systems, on outdoor electronic billboards, and on its Web site. According to the foundation, data released by the Urban Institutes National Survey of Americas Families suggested that the Covering Kids & Families Campaign and similar efforts were making a difference. For example, just prior to the launch of the first Covering Kids Back-to-School Campaign in 2000, nearly 5 million additional children had been enrolled in Medicaid and SCHIP. The number of uninsured children had declined by 1.8 million, and awareness of SCHIP among eligible families increased from 47 percent in 1999 to 71 percent in 2002. In terms of communicating evaluation findings related to CKF efforts, Bilheimer added:

Well disseminate the evaluation findings in a variety of waystraditional approaches such as peer-reviewed journal articles, presentations at national meetings, and at our own meeting for our research and evaluation grantees, etc. In addition, well disseminate findings through meetings and discussions with health policy staff on Capitol Hill and in the Administration, and possibly at the state level. We plan to develop a series of policy briefs that are short and accessible, and present findings in ways that are meaningful to state and federal policymakers. Articles, policy briefs, and other findings will all be available on our Web site.

THE FUTURE

As of 2005, research, policy analysis, and evaluation had become a central component of the Robert Wood Johnson Foundations strategy. As the foundation continued its work, it aspired to effectively communicate relevant findings to internal stakeholders, including foundation staff and board members, in order to help shape the organizations strategic direction. Knickman and his team also recognized the importance of disseminating research and evaluation results to external constituents, while measuring the impact of their work in informing policymakers and decision-makers in the health care field.

ASSIGNMENT QUESTIONS

1. Discuss the difference between research-driven demonstrations and program evaluations. What factors might influence which type of evaluation the foundation decides to pursue?

2. What indicators might the foundation utilize to assess the success of its evaluation efforts?

How could foundation staff members measure the impact of the Covering Kids and Families (CFK) program on policymakers and other health care leaders?


p. 9

3. Brainstorm ideas for managing information within the foundation to ensure that staff

members at all levels have the knowledge to make informed decisions.


p. 10

Exhibit 1 RWJF Interest Areas

Addiction Prevention & Treatment

Increasing the number of treatment settings employing approaches that have been proven to work.

Building Human Capital

Attracting, developing and retaining high-quality leadership and workforce to improve health and health care.

Childhood Obesity Halting the increase in prevalence of overweight among children.

Disparities Reducing racial and ethnic disparities in the care of targeted diseases.

Health Insurance Coverage

Supporting efforts to achieve stable and affordable health care coverage for all Americans.

Nursing Reducing the shortage in nursing staff and improving the quality of nursing-related care by transforming the way care is delivered at the bedside.

Pioneer Promoting fundamental breakthroughs in health and health care through innovative projects.

Public Health Improving the strategic use of information and accountability measures by leaders to enhance performance and raise the visibility and impact of public health.

Quality Health Care Accelerating performance improvements on nationally adopted measures of outpatient chronic care through local market demonstrations.

Tobacco Use Recruiting new funding partners to sustain the state and national tobacco policy change infrastructure and maintain policy gains and momentum through targeted grantmaking.

Vulnerable Populations

Promoting community-based projects that improve health and health care outcomes for society's most vulnerable people.

Source: Robert Wood Johnson Foundation, Interest Area Overview, http://www.rwjf.org/portfolios/interestareas.jhtml (July 26, 2006).


p. 11

Exhibit 2 Rationale for Research, Policy Analysis, and Evaluation

Test the Effectiveness of Foundation-Funded Programs The research and evaluation group at RWJF tested the effectiveness of foundation-funded programs. The foundation hired outside reviewers to determine such effectiveness in the early years of the foundation. Since the foundations inception, it has conducted nearly 100 external evaluations of its programs. Its important to note that such evaluations focused on what could be learned and communicated from a program; they did not monitor individual grant management and administrative performance. Administrative staff at the foundation and the national program offices handled management and administration of the programs. Learn About Health and Health Care In more recent years, according to Knickman: many foundation programs focused less on testing new models of care and more on actively promoting social change. This shift has led to a more varied approach to [research, policy analysis, and] evaluationwhere the first central question is how each major foundation initiative could be considered a learning opportunity.16 One example included experts visiting the foundations Clinical Scholars Program. After interviews and analysis, the experts provided an assessment of the programs contributions to developing physician leaders of health services research and made suggestions for strengthening the program.17 Another series of grants were related to the Health Tracking program, which looked at how managed care and other market-oriented changes affected health and health care. Obtain Baseline Information In some instances, the foundation funded surveys to obtain basic information about health or health care issues. For example, the foundation conducted a series of surveys over several years that measured the number of Americans who experienced problems in gaining access to health care. Other surveys included one which analyzed the way physicians practiced medicine, as well as one which sought to understand how young physicians decided which specialty to enter. Such surveys shaped overall grantmaking program strategy. For example, a physician survey convinced the foundation to support training programs for generalist physicians, and the access-to-care surveys led the foundation to try to expand primary care by funding primary care group practices within hospitals.18 The RWJFs research benefited external organizations, individuals, and other parties. According to Knickman, the foundation was committed to making its data sets, related to its research, publicly available as quickly as possible. All foundation-funded surveys that were judged to be

16 James R. Knickman, Research as a Foundation Strategy, as it appears in the RWJF annual anthology of articles entitled To Improve Health and Health Care 2000, http://www.rwjf.org/files/publications/books/2000/chapter_07.html (July 26, 2006) 17 Ibid. 18 Ibid.


p. 12

useful to other external researchers were put into the public use data files at the University of Michigans Institute for Survey Research. Understand Public Policy Choices According to Knickman, over time, the foundations definition of research expanded to include policy analysis, which involved background analysis to support the development of new public policies at the federal and state levels.19 The foundation defined policy analysis as research that analyzed and documented the multitude of options regarding a particular social problem. The foundation funded many such programs. One was called State Initiatives in Health Care Reform, which aimed to improve access to health care for vulnerable populations in 16 states. The organization conducted research and analysis, then provided information to executive and legislative leaders in various states. Understand Health-Related Behavior The foundation was committed to reducing the negative effects caused by smoking, alcohol, and illegal drugs. As a result, the foundation was interested in determining which public policies and behavioral and environmental interventions could reduce peoples use of such substances. The foundations interests led it to fund behavioral and policy research related to substance abuse. One example included a study on tobacco policy research. Build the Health Care Field: Training Programs Most foundation-funded research, policy analysis, and evaluation focused on specific health problems, but another aspect of the area included training programs to educate future researchers. Training programs represented 17 percent of foundation grantmaking and were utilized to build the health care field generally. Targets for training programs included academic scholars in medical schools and postdoctoral fellows in various social science fields.

19 Ibid.

si 75 robert wood johnson foundation 060113

Documents