A Collaborative Approach toward Not-For-Profit and Local Government Program Evaluation

Sean Curtis Amir

The sociologist, Talcott Parsons in his work The Social System stressed that

“organizations are organizations are organizations.” At the time of his writings, Parsons

1

maintained “all major contributions to the field were conceived to apply broadly across all types

of organizations” (Henry, 2013). To this extent, Parsons notions of the organization would mean

that all organizations, whether political, transactional, or nonprofit, are alike, and previous

research should be applied regardless of the organizational missions. This essay seeks to expand

on Parsons’ notions of the organization, and discuss whether program evaluation can be applied

universally to all organizations. If this is the case, this essay posits that program evaluation, then,

can be benefitted by applying nonprofit evaluation methods to public agencies, and vice versa.

Public and nonprofit organizations engage in program evaluation procedures similarly, either

through internal evaluators, or outside analysts. An internal organization member benefits from

high access to information, and has the ability to conduct a long-term evaluation. While an

external analyst may provide an objective evaluation, with more time devoted to their research as

opposed to other organizational missions. What this essay proposes, is that local governments

and nonprofits can work collaboratively together toward transformative public agency

evaluations, while sustaining the social and communitarian missions of nonprofits through

resource dependence via collaboration and co-optation.

First this essay will discuss the benefits of collaboration and co-optation, specifically

within the constraints of interpenetration between governments and nonprofits. Organizations are

becoming increasingly interdependent, thus the idea of collaborative frameworks are essential.

The complexities of modern problems require a diverse, objective, and multidisciplinary skill-set

for effective evaluations, therefore introducing nonprofits into the process benefits problem-

solving program evaluations. Nonprofits as evaluators, while “outsiders” are not economically

(although resource dependent) motivated, the missions of nonprofits and public agencies are both

2

inherently democratic, and thus evaluations are centered on community improvement. Second,

this essay will discuss resource dependence theory in nonprofits. Finally, this essay will attempt

to identify strategies, and determine whether nonprofits organizations can assist public agencies

in program evaluation. This collaborative approach is a departure from traditional evaluation

tactics that rely either on inside evaluators or paid outside analysts. It is essentially a mixed

methods approach, benefitting the local government with an objective evaluator, and the

nonprofit with critical resources to function. Thus, this essay relies on resource dependence

theory as the link between effective program evaluation, and improving the community.

The term organization has manifested itself in a variety of social structures. We find

nonprofits, charitable groups, government agencies, and other formal networks universally

recognized as organizations. For Bromley and Meyer, organizations are increasingly blended and

blurred across disciplines and motives. They argue this has occurred for two reasons, first,

“individuals have a great many rights and capacities as empowered social actors, and, second,

that social activities can and should be managed through the application of science-like

principles covering the natural and social environment” (Bromley and Meyer, (2014). The

authors go on to explain the roles of public and nonprofits as organizations as universal cultural

principles, principles that at their core, promote efficiency, effectiveness, accountability,

decision-making strategy, and evaluation:

A charity responded to some social ill, but a modern nonprofit organization should do so

in a way that is accountable, systematic, and effective; a firm maximized profits,

but a proper one should now also display elements of corporate

responsibility; a government could provide public services through a

3

centralized bureaucracy, but a public organization should do so while involving many

stakeholders (Bromley and Meyer, 2014).

Thus, standard evaluation practices can theoretically be applied to all organizations,

whether business, government, or charitable. Moreover, the authors cite Knutson who maintains

“some NPOs are perceived as being increasingly institutionalized into ‘hybrids’ of private and

public organizations due to resource-based relationships with the private and public sectors,

including financing, competing, or contracting relationships” (Bromley and Meyer, 2014). The

oft cited and popular public management strategy NPM (New Public Management) has certainly

contributed to the recent collaborative approach to public services. Osborne’s The New Public

Governance discusses the plural state, where “multiple interdependent actors contribute to the

delivery of public services, and a pluralist state, where multiple processes inform the policy-

making system.” It is this idea of external environmental pressures that may contribute to the

ideology that inter-organizational relationships must be implemented. As Osborne contends,

NPM is a direct response to an increasingly “complex plural, and fragmented nature of public

policy implementation and service delivery in the twenty-first century” (Osborne, 2010).

Therefore it may prove to be beneficial for governments and nonprofits to take on a pluralistic

and partnership-based approach to both policy, and agency evaluations. Certainly the rise of

NPM and the emphasis toward measuring effectiveness have changed standard management

practices. However Suarez notes that while “hiring consultants, using quantitative metrics for

program evaluation, performing, annual audits, and integrating other management practices are

no guarantees of success in the business sector, let alone the nonprofit sector… but might confer

legitimacy and aid nonprofits in procuring government grants and contracts” (Suarez, 2011).

4

COLLABORATION, CO-OPTATION, AND NETWORKS

Robert Agranoff gives a definition of cooperation, to mean “the act of working jointly

with others, usually to resolve a problem or find a corner of activity.” Agranoff’s definition

works principally for the emphasis of inter-organizational relationships, or networks, a term that

continues to be popularized in the public sector lexicon (Suarez, 2011). Networks, and other

collaborative management strategies are facilitated and operated by multiple organizations that

would be previously be unable to, or lacked the resources to solve the problems alone (Shafritz

and Hyde, 2011). At the same time, nonprofits have also taken on business-oriented, and

scientific management strategies focused on efficiency, effectiveness, and accountability that

reaffirm Parson’s notions that all contributions to the field may be applied to all organizations

regardless of their mission.

Networks, according to Agranoff are important vehicles for “resource pooling, mutual

exploration, and knowledge creation,” these strategies quantify the new market-based approaches

nonprofits are increasingly engaged in. Bromley and Meyer cite Weisbrod, who in 1997, posited

that nonprofits will see a great demand for their services, and thus must seek new revenue

streams. Weisbrod maintains nonprofits must seek insulation from pressures to deviate from their

role of social advocate, and help to take an economic path, creating a more balanced contribution

to private enterprise and government. In fact, research of the relationships between government

and nonprofits stress the benefits of collaborative grants and contracts, strategies that contributed

to the growth and survival of their causes. Moreover, Suarez maintains that grants and contracts

“can provide access to policy makers and the political process, enabling nonprofits to advocate

5

for causes and legislative change” (Suarez, 2011). While this growing emphasis on NPM have

made their way to the nonprofit sector, their increased professionalization, and market-based

approaches must continue in order for government agencies to trust them as an outside analyst.

Suarez quotes an executive director of a nonprofit health organization who stated:

I think that for the period of time that I came in, a lot of us came in just with whatever

credentials we happened to have at the time. So I think that we all learned on-the-job the

nature of the business as well as developing some skill sets. The newer executive team

I’m bringing on comes with more formal preparation for the field. They’re coming

in with Master’s of Health Administration, they’re people who’ve had health plan

experience before. While for those of us that have been here for a long

period of time, we just grew in and evolved with the organization (Suarez, 2011).

The more professionalized the nonprofit, the more likely they are to receive government

funding, and the more likely governments will be inclined to form a partnership. The resource

dependence that characterizes nonprofit organizations are critical for the existence. No

organization is self-sufficient under the resource dependence theory, therefore other actors are

needed to survive (Berner and Bronson, 2005). Literature concerning nonprofit resource

dependence has grown considerably during the NPM era. Growth strategies include “activities to

innovate, expand, or diversify the resource base of a non profit” (Akingbola, 2013). While

research has been limited in its scope concerning nonprofit evaluation, Akingbola maintains the

range of activities are extensive. The author notes, “new product/service innovation; merger;

joint venture partnership; new subsidiary; and for-profit business venture to support a nonprofit

6

enterprise. A growth strategy positions a nonprofit to optimize the resource base by leveraging

emerging community needs, funding and revenue opportunities. It requires a higher level of

integration and coordination” (AkingBOLA, 2013) Central to these strategies, though, is the idea

of partnerships, coordination, and collaboration, most notably from its principle resource base,

which in this case are local governments. Management of this growth strategy allows nonprofits

to put forth their maximum resources to program evaluation, ensuring the service remains

specialized, and maintaing the organization mission. This allows nonprofits to diversify their

resource base, that is, “extending the existing service to a different demography or market”

(Akingloba, 2013). However, resource dependency and collaborations with public agencies are

not without their costs to the nonprofit.

The nonprofits most valued characteristic outside of its organizational mission, is its

autonomy. Collaboration results in a loss of this autonomy specifically in increasingly formal

networks. Carman notes “A key challenge for an individual organization in choosing among

different collaboration forms, therefore is to keep the dynamic balance between managing

resource dependence and sustaining organizational autonomy” (Carman, 2011). Moreover co-

optation and dependence on government funding may have impactful consequences in

community representation on a nonprofit organizations governing board. Guo notes, “the

adoption of a co-optation strategy in response to government funding dependence leads to an

increase in the number of corporate, professional, and social elites” (Guo, 2007). This dynamic

can potentially hurt an organizations representation of community members, and perhaps impact

program evaluation involving community members.

In study examining the presence of nonprofit collaborations with government, it was

found that, overall, 77% of the nonprofits interviewed claimed they collaborated with other

7

nonprofits, while 53% collaborated with public sector agencies, and 32% collaborate with private

businesses (Suarez, 2011). The same study, a nonprofit director serving foster children was

quoted discussing their partnership with a public agency. “Our biggest partnership is with the

Department of Human Services (DHS) itself. We are allowed to post flyers in the office, attend

meetings from time to time to publicize our things, because we’re serving those same kids. It’s a

very positive relationship with the San Francisco DHS” (Suarez, 2013). Again, it becomes clear

that both public agencies and nonprofits have been infused with neoliberal thoughts

globalization, outsourcing, and intergovernmental cooperation. Its also evident that these

strategies have made their way to the nonprofit sector as well. Suarez maintains, an “expanding

role of markets, new demands for accountability, and a much greater focus on management” in

nonprofits have demonstrated a level of professionalism that can contribute to increased

collaboration with public agencies. Thus, as indicated by Agranoff, partnerships and networks

have the ability to jointly resolve complex social issues that singular public agencies could not

do (Shafritz and Hyde, 2011).

Carman notes there seems to be two competing influences from reliance on government

funding, though. “First, the pressure toward formalized collaboration is likely to increase as an

organization receives government funding; second, the deterrent from funding requirements

actually increases with the diversity of government funding streams.” What this essentially

indicates is that less diversified resource bases in nonprofits indicates are stronger likelihood that

these organizations will enter formal collaboration activities (Carman, 2011).

COLLABORATIVE PROGRAM EVALUATION

8

Public program evaluation is essentially a three-step process, Henry maintains this

process can be reduced to choosing the evaluators, defining the problem, and designing the

evaluation (Henry, 2013). This process is generally meant to discover findings about program

effectiveness, however in most cases, local government offices are insufficiently equipped with

personnel capable of providing such information. Berner and Bronson note that a “lack of

resources is a common barrier to quality evaluations (or any, in fact)” (Berner and Bronson,

2005). Further, valuations can prove to be taxing on the morale of organization members.

Thoughts of budget cuts, downsizing, and self-doubt create tense and anxious program staff

members upon the realization their program will be evaluated. However, Henry notes these

worries are often overstated. Henry says “not even 1 percent of some 600 municipal programs led

to termination of the evaluated activities, but 78 percent of these evaluations led to the

adjustments of program activities. This reaffirms the idea that evaluations are meant as a means

for improving productivity of programs, not simply a budget reducing method. Henry maintains

all states use some method of managing for results, and the proportion for which “major agencies

conduct productivity analysis more than doubled over three decades” (Henry, 2013). Berner and

Bronson contend evaluations are typically performed in five steps:

• Agree on and articulate the program goals and objectives;

• Agree on and declare the program theory or theory of change;

• Specify and agree on the criteria that will be used to measure success and the standards that

must be met;

• Gather data according to the criteria to see if the standards have been met; and

9

• Interpret the data and present results in a meaningful and useful way (Berner and Bronson,

2005)

In standard evaluation procedures, the local government’s manager’s office, the budget

office, or finance department are considered the requesting agency. The requesting agency will

typically contact an evaluator at the discretion of a governing board, or for their own

understanding of the program or project. If the evaluation is conducted internally, the evaluator

from one of the offices will contact the program representative and information concerning the

program in question. However, the smaller the local unit, the less likely internal evaluations will

take place. Simply, local government offices do not have the time, the personnel, or expertise for

in-depth program evaluation (Berner and Bronson, 2005).

As Henry notes, inside and outside program evaluators each have their benefits. The

inside evaluator may attain detailed knowledge of the organization, its culture, and they have the

benefit of conducting long-term evaluations. Outside evaluators are better able to devote their

time and resources to research, Henry notes they “may be more fictive mediators because of the

objectivity” (Henry, 2005).

Often referred to as “participatory evaluation,” or “collaborative evaluation,” this essay

seeks to introduce the agency or the representative from the program being evaluated into the

evaluation process. As noted by Berner and Bronson, collaborative evaluation seeks to include

the representatives of all stakeholders, including “program staff, affected citizens, politicians, and

interest groups” (Berner and Bronson, 2005). These stakeholders, not just the requesting agency

and the evaluator, are involved in the five-step evaluation procedure. Critics of the collaborative

10

approach contend human interaction, absent of an objective outside evaluator threatens to

undermine the evaluation process. “It opens evaluation to more

subjectivity and threat of bias. Sharing control inherently means losing some control” (Berner

and Bronson). Its benefits are well cited, O’Sullivan and Rodriguez-Campos maintain:

Collaborative Evaluation systematically invites and engages stakeholders in program

evaluation planning and implementation. Unlike "distanced" evaluation

approaches, which reject stakeholder participation as evaluation team members,

Collaborative Evaluation assumes that active, on-going engagement between

evaluators and program staff, result in stronger evaluation designs, enhanced data

collection and analysis, and results that stakeholder understand and use

(O’Sullivan, 2012).

Collaborative evaluation is an approach that offers, among others, many advantages in

terms of access to information, quality of information gathered, opportunities for

creative problem-solving, and receptivity to findings. In the last decade,

collaborative evaluation has grown in popularity along with similar participatory,

empowerment, and utilization- focused evaluation approaches (Rodriguez-Campos,

2012).

The evaluation process itself is not different form the traditional procedures involving

information gathering and analysis, the difference lies in the relationships fostered between the

evaluator and the stakeholders. While organizational transformation through citizen deliberation,

11

and less hierarchical power relationships is certainly beneficial, Berner and Bronson are quick to

temper expectations. The underlying message of collaborative evaluations is simply, “improving

the quality of the evaluation and increasing the likelihood that the results are useful for all

involved” (Berner and Bronson, 2005). A key feature of collaborative evaluation is that is

evolutionary in its applications. That is, over time, program representatives and stakeholders

learn to collect and report data. For example, the book co-evaluation seeks to empower

stakeholders in evaluating their own programs over-time, and contributing that data to facilitate

learning. In the end, there are benefits and drawbacks, both politically and for nonprofits. A

collaborative evaluation requires nonprofits to relinquish some its autonomy, while public

agencies must consider time and resources needed to engage evaluators and program staff. It is

uncertain whether collaborative evaluation yields different results than traditional approaches,

and while much research is still needed, specifically case studies on agencies currently engaged

nonprofit collaborative evaluation, the primary mission for social improvement may be stronger

through the increased communication.

References

12

1. Akingbola, K. (2013). A model of strategic nonprofit human resource management.

Voluntas: International Journal of Voluntary and Nonprofit Organizations, 24(1), 214-240.

2. Barber, B. (1994). Talcott parsons on the social system: An essay in clarification and

elaboration. Sociological Theory, 12(1), 101-105.

3. Berner, M., & Bronson, M. (2005). A case study of program evaluation in local government:

Building consensus through collaboration. Public Performance & Management Review,

28(3), 309-325.

4. Bies, A. L., & Blackwood, A. S. B. (2008). Accountability, ethics, evaluation, and

governance in nonprofit management education: Trends and treatment. Journal of Public

Affairs Education, 13(3/4), 519-547.

5. Bromley, P., & Meyer, J.W. (2014) “They Are All Organizations”The Cultural Roots of

Blurring Between the Nonprofit, Business, and Government Sectors

6. Brooks, A. C. (2002). Can nonprofit management help answer public management's "big

questions"? Public Administration Review, 62(3), 259-266.

7. CARMAN, J. G. (2011). Understanding evaluation in nonprofit organizations. Public

Performance & Management Review, 34(3), 350-377.

13

8. Frumkin, P., & Kim, M. T. (2001). Strategic positioning and the financing of nonprofit

organizations: Is efficiency rewarded in the contributions marketplace? Public

Administration Review, 61(3), 266-275.

9. Gerth, H. H., & Hughey, M. W. (1997). On talcott parsons' "the social system". International

Journal of Politics, Culture, and Society, 10(4), 673-684.

10. Geer, B. W., Maher, J. K., & Cole, M. T. (2008). Managing nonprofit organizations: The

importance of transformational leadership and commitment to operating standards for

nonprofit accountability. Public Performance & Management Review, 32(1), 51-75.

11. Guo, C. (2007). When government becomes the principal philanthropist: The effects of

public funding on patterns of nonprofit governance. Public Administration Review, 67(3),

458-473.

12. Hatten, M. L. (1982). Strategic management in not-for-profit organizations. Strategic

Management Journal, 3(2), 89-104.

13. Henry, N. (2013). Public administration and public affairs. Boston: Pearson.

14. Keating, E. K., & Frumkin, P. (2003). Reengineering nonprofit financial accountability:

Toward a more reliable foundation for regulation. Public Administration Review, 63(1), 3-

15.

14

15. LaFleche, V. (1991). Collaborative effort as a means of enhancing productivity. Public

Productivity & Management Review, 15(2, New Approaches to Productivity: Proceedings of

the Fourth National Public Sector Productivity Conference), 261-263.

16. Lohmann, R. A. (2007). Charity, philanthropy, public service, or enterprise: What are the big

questions of nonprofit management today? Public Administration Review, 67(3), 437-444.

17. Maentz, A. M. (1987). The volunteer: Agent for action. Public Health Reports (1974-), 102(,

Supplement: Women's Health. Proceedings of the National Conference on Women's Health),

78-80.

18. McDaniel, R. R.,Jr. (1986). Administrative Science Quarterly, 31(1), 143-144.

19. Mirabella, R. M. (2013). Toward a more perfect nonprofit: The performance mindset and the

"gift". Administrative Theory & Praxis, 35(1), 81-105.

20. Newman, W. H., & Wallender, H. W.,III. (1978). Managing not-for-profit enterprises. The

Academy of Management Review, 3(1), 24-31.

21. Osborne, S. (2010). The new public governance? : emerging perspectives on the theory and

practice of public governance. London New York: Routledge.

15

22. O’Sullivan, R.G. (2012). Collaborative evaluation within a framework of stakeholder-

oriented evaluation approaches.

23. Peterson, T. O., & Fleet, D. D. V. (2008). A tale of two situations: An empirical study of

behavior by not-for-profit managerial leaders. Public Performance & Management Review,

31(4), 503-516.

24. Rivera, M. A. (1986). Public Productivity Review, 10(1, Personnel Strategies and

Productivity Improvement), 110-111.

25. Rodríguez-Campos, L. (2012). Advances in collaborative evaluation.

26. Shafritz, J. & Hyde, A. (2011). Classics of public administration. Boston: Wadsworth Pub

Co.

27. Sink, D. (1987). The American Political Science Review, 81(1), 271-272.

28. Suarez, D.F. (2011) Collaboration and Professionalization: The Contours of Public Sector

Funding for Nonprofit Organizations

29. Wymer, W., & Mottner, S. (2009). Marketing inclusion in the curricula of U.S. nonprofit

management programs. Journal of Public Affairs Education, 15(2), 185-202.

16

30. Young, D. R. (1999). Nonprofit management studies in the united states: Current

developments and future prospects. Journal of Public Affairs Education, 5(1), 13-23.

17