the role of evaluation in legislative decision making

The Role of Evaluation in Legislative Decision MakingAuthor(s): Allan GreenSource: Public Administration Review, Vol. 44, No. 3 (May - Jun., 1984), pp. 265-267Published by: Wiley on behalf of the American Society for Public AdministrationStable URL: http://www.jstor.org/stable/975492 .

Accessed: 16/06/2014 02:12

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at .http://www.jstor.org/page/info/about/policies/terms.jsp

.JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range ofcontent in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new formsof scholarship. For more information about JSTOR, please contact [email protected].

.

Wiley and American Society for Public Administration are collaborating with JSTOR to digitize, preserve andextend access to Public Administration Review.

http://www.jstor.org

This content downloaded from 91.229.229.210 on Mon, 16 Jun 2014 02:12:12 AMAll use subject to JSTOR Terms and Conditions

http://www.jstor.org/action/showPublisher?publisherCode=black

http://www.jstor.org/action/showPublisher?publisherCode=aspa

http://www.jstor.org/stable/975492?origin=JSTOR-pdf

http://www.jstor.org/page/info/about/policies/terms.jsp


PUBLIC MANAGERS' FORUM 265

The Role of Evaluation in Legislative Decision Making

Allan Green, Oregon Legislative Research Office-

Legislative performance auditing and program evaluation in Oregon is presently limited to sunset reviews of occupational licensing agencies and a few other state agencies, such as the liquor control board and the education coordinating commission. Expanded program evaluation was proposed during the 1983 session of the legislature and is the subject of an interim study. This paper discusses issues relating to the sunset review process and legislators' expressed interest in program evaluation.

The Legislative Research Office in Oregon differs from legislative audit or program evaluation offices in other states. The primary function of legislative research is to provide policy research on issues facing the legislature. Sunset reviews are also a responsibility, but one which requires less than half of staff time. Whereas legislative evaluation offices in other states usually report to a single committee, this office is under the direction of the legislature generally and responds to requests from all members. An issue for legislative evaluation offices in many states, that of being distinctly different from other legislative staff, is not an issue in Oregon. Rather, legislative research staff must be cog- nizant of their two distinct roles: conducting general policy research and preparing sunset evaluations.

Legislative research staff are generalists, not account- ants or auditors. Like legislative evaluation offices in many states, legislative research comprises nonpartisan, professional staff. Because they are nonpartisan, researchers must separate themselves from any member's particular policy objective. This may be difficult when a researcher has worked closely with a legislator and has arrived at similar conclusions. But independence and objectivity are vital to the credibility of a nonpartisan office. Most of the issues discussed here involve the relationship between legislators and nonpartisan staff. These issues include staff recommendations and advocacy of findings, determination of a successful evaluation, and the trend toward more fiscally-oriented evaluations.

Staff Recommendations

The issue of whether legislative staff should offer policy recommendations, while probably of greater concern to staff than to legislators, is fundamental. Several problems may be encountered when staff make recommendations. The first relates specifically to situations, such as sunset reviews, where the legal burden is on the

MAY/JUNE 1984

agency to justify its existence. Staff recommendations rather than the agency's justification may become the focus of legislators' attention. A staff recommendation concerning one particular aspect of an agency's program may result in legislators losing sight of the broader issue-whether the agency is needed. Such focus on staff recommendations is not, in itself, undesirable since staff should be prepared to defend their recommendations. But this focus on staff may have the effect of dif- fusing the legal requirement on the agency.

A successful evaluation is one that provides the information legislators need to reach a conclusion, even if that conclusion differs from the staff's.

A second problem with recommendations is that legislators and staff may have very different perspec- tives. Legislators are elected to make policy decisions. These decisions are made on a daily basis and on a wide range of issues that are often interrelated. The researcher, on the other hand, may have for some time worked almost exclusively on one issue. Staff insight is important, but the staff's perspective may be somewhat narrower than the legislators'.

Staff recommendations can sometimes be trivial. If researchers are compelled to make recommendations on each criterion of evaluation or each component of a program, the result may be that minor issues are raised simply to fill a requirement of the review. And the focus of an evaluation is made negative by the need to find fault or provide legislators with an issue for a press release.

Another problem relates to the political ramifications of staff recommendations. There is a potential for the media or other observers to judge legislators according to their support or opposition to the staff recommendations. Researchers are not accountable in the same manner as legislators since they do not have to answer to the electorate. Researchers may evaluate an agency from a strict standard, whereas legislators may conclude that

Allan Green has been director of Oregon's Legislative Research Office for seven years. He previously served as a research analyst for the Colorado Legislative Council and taught political science at the Uni- versity of Puget Sound in Tacoma, Washington. He is a member of the Executive Committee of the Western Governmental Research Association.



266 PUBLIC ADMINISTRATION REVIEW

even though it may not be fulfilling its mission, the agency is popular or not harming anyone and should not be subject to major revision.

Finally, there is the problem of staff morale when recommendations are ignored or rejected by legislators. It is difficult for researchers to have thoroughly studied an issue, arrived at a logical conclusion, and made recommendations that are good public policy, only to have the recommendations rejected for political or other reasons.

Some of these problems may be avoided by presenting legislators with a series of policy options, rather than specific recommendations. With policy options the legislator can select the option that will be most politically palatable, and staff are still involved in the policy-making process. One drawback to this approach is that legislators may not select the option most clearly justified by the research. But for nonpartisan staff who conduct general policy research in addition to program evaluation, policy options may be a viable alternative to staff recommendations.

Staff Advocacy

A second issue concerns the proper level of staff involvement after completion of the agency evaluation. Does staff have a role in actively advocating either legislative enactment or agency implementation of its findings?

With evaluations prepared for a legislative oversight committee, staff have the advantage of a working relationship with the committee and easy access to members. Such researchers should know how to relate to each member and should know which members will be interested in their findings. But these advantages can disappear in the face of agency or interest group lobbying.

A recent report by the National Association of Social Workers describes how a Colorado affiliate was able to effectively combat staff recommendations concerning the licensing of that occupation. The report instructs associations in other states on contributing to and working in legislative campaigns, hiring lobbyists, and obtaining the mutual cooperation of rival professional groups.* Such tactics can place staff at a distinct dis- advantage. Frequently, organizations which might support the staff findings are not interested enough in the issue to become involved. And as a result, hearing rooms are crowded with advocates of one side of the issue. The staff side may be easily dismissed unless legislators are highly receptive to the staff or vitally interested in the issue.

Staff follow-up of findings on agency management is another concern. If, for example, staff find that the agency is not abiding by the proper administrative procedures, they should document violations in their

*Stephen R. Block et al., "Surviving a Sunset Review: The Colorado Experience" (Silver Spring, Md.: National Association of Social Workers, 1982).

report. To see that the agency reforms its practices requires continued staff involvement with the agency. This may also require more time and resources than are available. In Oregon, the legislature has not asked for follow-up on staff findings, and legislative research monitors previously evaluated agencies only on a limited basis.

Measure of Success

Should the determinant of a successful legislative evaluation be acceptance by the legislature, usefulness to the agency under review, or credibility and adherence to professional standards? Although an evaluation should meet the standard of usefulness to the agency and the regulated occupation, its primary purpose is to provide information beneficial to legislators in their consideration of an agency. This does not mean that the measure of success depends on legislators' agreement with staff findings and recommendations. A successful evaluation is one that provides the information legislators need to reach a conclusion, even if that conclusion differs from the staff's.

Staff reports should meet professional standards. These require that issues be thoroughly researched, that information sources be properly documented, and that information be presented in a succinct manner. Legis- lators should not have to spend an undue amount of time studying reports. And reports should be carefully organized so that legislators can easily refer to them.

If information presented by the staff is well documented, is clearly defensible, and facilitates the discus- sion of issues by all sides, then staff should consider the evaluation to be a success. If on the other hand legislators are not reading staff reports or following staff presentations, then one should question whether the issue evaluated was really of interest to the legislators or whether the reports or presentation were adequate. Staff often are not in control of the subject area to be investigated, but reports and presentations can and should be modified to meet legislative needs and standards.

Justifying the Time and Cost of Evaluations

A nagging question among legislative evaluators may be whether staff can really justify the costs and time required to conduct evaluations. This is a question that, fortunately, staff need not be concerned with since it is the legislators' responsibility to determine how staff resources are allocated. If the legislature concludes, for whatever reason, that it wants program evaluations by its staff, then it is the staff's responsibility to conduct them.

Changing Issues and Programs

Based on the interest of legislators in Oregon and what appears to be a national trend, there has been a shift in what legislators expect of program evaluation.

MAY/JUNE 1984



PUBLIC MANAGERS' FORUM 267

As state governments have been confronted with revenue shortfalls and the reduction or elimination of federal funding for programs, legislators' focus has shifted from the goals or effectiveness of programs to methods for reducing their cost. This shift will sig- nificantly affect the discipline of program evaluation.

In many states, legislative audit or evaluation com- mittees have directed program evaluation. These com- mittees have been concerned with the public policy aspects of programs under review. With focus shifting to programs' fiscal aspects, oversight responsibility will correspondingly shift to a different group of legislators. In most states, this will be the small group of members on the budget (or Ways and Means) committee. Budget committee members often have no other substantive committee assignments and incur the resentment of other members of the legislature who believe that agencies and programs should be reviewed from a broad perspective. The other members may be interested in the

findings of a policy evaluation, but have little involvement in evaluations relating to the intricacies of budgeting.

The implications of this shift for staff are enormous. When program evaluation is policy oriented, it is usually a function of the legislative auditor, a separate evaluation agency, or, as in Oregon, a general research office. With a shift to fiscal emphasis, program evaluators may become advisory to and, ultimately, part of the budget staff. As staff turnover occurs, persons with a budgetary background will be sought.

This shift in evaluation emphasis may be desirable public policy. But different legislators will be involved, reviews will have a new budgetary focus, and evaluation staff will change from program to fiscal. Legislative evaluations with less policy orientation can only be another budgetary tool and not a substantive program analysis.

CALESt CA PUBLISHERV New! THE GREAT EDUCATION DEBATE: Washington and the Schools by Benjamin D. Stickney and Laurence R. Marcus. This examination of federal involvement in education discusses budget reduction, program consolidation, and deregulation; the influence of the courts on education; the appropriate role of the federal government in education; and related topics. June '84, about $22.75

New! THE SERVICE TRAP: From Altruism to Dirty Work by Paula Dressel. This book describes and highlights many of the frustrations faced by service pro- viders and offers insights into everyday problems. The implementation of social welfare policies and future prospects for service workers are among the specific topics discussed. '84, $16.75

CONTROVERSIAL NUTRITION POLICY ISSUES edited by Giorgio R. Solimano and Sally A. Lederman. The diverse articles in this volume analyze current in- ternational issues in public health nutrition. Among the topics discussed are malnutrition, U.S. food aid, sac- charin use, RDAs for pyridoxine, the vitamin C con- troversy, breast feeding, and child nutrition. '83, $34.75

New! PLANNING FOR JUSTICE: The Problems of Justice With Specific Approaches to the Issues by Robert S. Clark. The author provides an analytic view of the problems of justice and outlines specific concep- tual approaches that can be utilized in problem resolu- tion. Justice planning, challenges and remedies, communication, and causation and correlation are among the topics discussed. April '84, about $14.75

New! HUMAN SERVICE PLANNING AND EVALUATION FOR HARD TIMES by Alan Booth and Douglas Higgins. Step-by-step procedures for cutback planning and management, assessing program effectiveness, monitoring client outcomes, allocating resources, evaluating agencies, and establishing priorities are set forth in this book. April '84, about $23.50

THE GAMBLING SCENE: Why People Gamble by To- mas M. Martinez. The author examines the gambling scene in terms of its identity-defining capacity, studies gambling as a career, explores the relationship between the mass media and gambling, addresses the interac- tion of gambling and politics, and discusses gambling as a social problem. '83, $24.50

Order direct for fastest results * Write or call (217) 789-8980 * Books sent on approval Postage paid on MasterCard, Visa & prepaid orders * Catalog sent on request

| 2600 South First Street * Springfield . Illinois . 62717

MAY/JUNE 1984



the role of evaluation in legislative decision making

Documents