Why Organizations Thrive: Lesson 4

Why Organizations Thrive – Lesson 4
August 2010
By Jonathan Poisner, www.poisner.com

I’m in the process of writing a long article entitled: Why Organizations Thrive. The article details fifteen lessons I learned while growing the Oregon League of Conservation Voters (OLCV), buttressed by my observations of dozens of other groups both in Oregon and across the country.

Collectively, I believe these lessons are a very useful set of principles that any Executive Director can use to improve their organization’s capacity to fulfill its mission.

This article is Lesson 4.

Lesson 4 is to evaluate, evaluate, evaluate.

What do I mean by that?

Let’s start with the assumption that you’re either a growing or mature organization, with multiple programs. It’s time to engage in strategic planning for the future, and your board needs to wrestle with whether to invest further in one or several of these programs, or to try something new.

How do they make that decision?

The board and staff can, of course, consult a Magic 8 ball.

Or, they can sit around and shoot the breeze, trading over-generalized statements about what appears to work or not work, usually based on a small set of anecdotes and their own biases.

I’ve never witnessed a group use a Magic 8 ball, but I’ve absolutely seen groups put in the position of shooting the breeze when choosing strategies.

Why? Because nobody’s taken the time and effort to evaluate.

Good evaluation is at the heart of any organization that thrives. You need to figure out what’s working well and not well. This isn’t just so that you’re ready to evaluate when it’s time for strategic planning, but also so you can make yearly budgeting decisions and mid-course improvements to long-term or repeating programs.

Yet, many groups do very little formal evaluation of their programs.

Even when they do, often the programs evaluated don’t track back to their accounting and/or budgeting, so it’s impossible to compare them on a per dollar basis to the organization’s other programs.

So why do so few organizations invest dollars in evaluation?

Resource shortages are part of it. Evaluation can be done on reasonable budgets, but it does take time and that’s time and money away from doing your program. Should you really take 5% of your program budget and devote it to evaluation when that 5% can do X — X, of course, varies wildly based on the type of nonprofit.

Evaluation may also seem daunting to some because they lack the training. Good evaluation isn’t rocket science, but doing it well does require some serious thought. When I was an Executive Director, I can only remember once in my dozen years when I was offered a session on evaluation at a conference or training.

Another fear many groups have – particularly advocacy groups – is that evaluation will force them to reduce their goals to quantitative terms, since they assume that all evaluation must be quantitative.

Lastly, and perhaps not least, evaluation can be scary for staff. If the results don’t match up with expectations, what will the board think? What about funders?

In my experience, each of these fears should be set aside.

Spending that 5% is absolutely worth it. By learning more about what works you ensure that next year’s programs will be more effective. I also found that when results weren’t what we wanted, both the board and funders very much appreciated staff’s candor. Rather than undercutting our fundraising, I believe it helped it.

A good evaluation process doesn’t require you to only evaluate quantitatively – to the contrary, qualitative evaluations are an essential part of any overall evaluation strategy. The American Evaluation Association has an entire track at their annual conference devoted to evaluating policy and advocacy organizations.

As for the training, the glib answer is: “I can help. ” (Look on my website for the evaluation training and services I offer, or give me a call and I can talk to you about how to access lists of potential evaluators).

The less glib answer is that evaluation is actually rather straightforward.

Here’s the two minute version.

Create a list of your programs.

Identify for each the major activities you will undertake.

The exception is fundraising and where a foundation grant specifically required some evaluation.

For each activity, list the outcomes of the activities, both quantitatively and qualitatively.

Figure out how you’ll collect data. What will you measure numerically and when and how will the data be collected? What other methods (Surveys? Interviews? Evaluation forms?) can be used to collect qualitative data?

Develop a plan to collect it. When will you collect the data? Who will collect it?

How will the results be reported?

How will the “report” be communicated beyond staff/board, if at all?

There is, of course, potentially more to it than that. There’s a growing subset of evaluation work that focuses on running controlled experiments where you try out different things and measure which has a statistically significant impact versus a control group. For that, you may need somebody with a little bit of statistics in their back pocket.

There’s also the question of when and if to bring in an outsider. Aside from offering expertise or time to assist with the design of your evaluation system, reasons an outsider may be necessary include: (1) confidentiality in the data collection process, (2) greater credibility in reporting the results to funders, (3) the funders may require an outside evaluation, or (4) lack of staff time.

But for the vast majority of organizations, without outside assistance you can get most of the benefit of evaluation from just rolling up your sleeves and systematically setting up systems to incorporate evaluation into your annual work plans.

<<Back to Why Organizations Thrive

Be Sociable, Share!

Content © Copyright 2010-2013 • Jonathan Poisner Strategic Consulting LLC. All rights reserved.