The formula for success isn't as simple as dollars in - dollars out = bottom line - particularly for nonprofits working in the area of advocacy or providing direct services.
If you're the boss of an organisation whose only concern is fundraising, then measuring your success is easy. Pick up a pen, grab an old envelope, and write out the following simple equation: {spend x, raise y (over z years)}. Substitute your own figures for x, y and z, and bingo! You now know all you need to know about whether to reward or fire your fundraising manager.
But for organisations that conduct advocacy and/or provide direct services, the challenge of measuring success is very different and almost always much more complex. Perhaps that's why some nonprofits make only the most cursory attempt to do so, whilst others dodge the issue all together.
Scoring election scorecards
I've been thinking about this a lot recently, ever since I received an ‘election scorecard' through my post from a well-known environmental nonprofit. You've probably received one yourself - several nonprofits produced them in the run up to the general election.
As I held the scorecard in my hand, and read through the list of issues the nonprofit wanted me to consider when deciding who to vote for, I couldn't help but wonder: as the various departments allocated precious dollars to this exercise, how many took the time to work out exactly how they would measure its success? And come to think of it, exactly what would constitute a successful scorecard campaign anyway? New supporters? A flood of donations? Or would a slight rise in the profile of the highlighted issues be deemed enough?
It's a question I put directly to the manager who was directly responsible for the scorecard that arrived in my post that day. I asked him to imagine I was his boss, and that I had called him into my office to justify the expenditure on scorecards vs. other campaign devices.
He acknowledged it was a good, if tough, question. His response was to point out that the campaign wasn't about translating the scorecard into votes; it was a service for members and would hopefully raise the profile of the issues highlighted.
"Yes", I said, "that's all very well. But I'm your boss and I need some proof that this was money that couldn't have been spent more productively. So how are you going to measure the success of these two goals?" His answer was, in a nutshell, through anecdotal feedback and media mentions.
Measuring success might be tough - but it is necessary.
I fully appreciate, of course, that environmental nonprofits will always face a tough job measuring the success of profile-raising campaigns. After all, we can't know if a climate change campaign was successful until such time as climate change has been demonstrably arrested - or the last human on earth dies of its affects.
Yet as difficult as measuring success certainly is, I firmly believe that it's part of every good fundraiser's job to come up with ways to do just that. Especially when you are an environmental nonprofit, and your job is to save the planet. There is simply too little money around to waste a single cent on a campaign that doesn't have a real, tangible and measurable impact.
Success for service providers
So it is hard for advocacy nonprofits. But what of service providers? On the face of it, measuring the success of what these organisations do must surely be far easier. For example, let's say you run a helpline, and you've just launched a campaign to raise its profile. If the number of calls goes up in the wake of the campaign, all you have to do is divide the increase by the cost of the campaign, and you've got a figure to measure your success against, yes?
But hang on, what about the quality of the calls? What if the information was not relevant As soon as you go down this qualitative, rather than quantitative, avenue, you realise pretty soon that measuring the success of a service provider isn't as simple as you might have thought.
A case study: Epilepsy Action
So how do you measure outcomes in terms of quality of life? To answer that question, I want to talk to you about Epilepsy Action, a fantastic nonprofit that is very close to my heart. Partly because they were my first client in Australia, partly because a very dear and close friend had a terrifying operation to treat his debilitating epilepsy (it worked by the way). And partly because they have a very thorough approach to quality control: they bend over backwards to measure outcomes.
Carol Welsh from Epilepsy Action told me that after each person is helped, e.g. at a camp or a memory workshop, participants complete surveys and mark how relevant and useful parts of the service were. She explained that: "our staff look at the surveys and then feed into the next activity. Also we use our phone room to call about a month later and ask [participants] questions specifically to see how the service is being delivered."
Carol went on, explaining that at the moment they are "Trying to reach out to people in rural communities to promote services by video."
Sounds sensible, but it gets better. As I write, Epilepsy Action is doing various marketing initiatives to build up a database of relevant people in welfare and health services and network through networking. But get this: "... we measure staff time on each task and evaluate whether it produces the outcome we are expecting." Fantastic stuff - they actually measure staff time against marketing activities that are not just fundraising. Now, how many organisations can say they are doing that?
Measuring outcomes - not just output
I am not an expert on epilepsy, and I imagine there are conflicting approaches to value from services but at least Epilepsy Action is measuring its outcomes (not just outputs) clearly. They have a sound platform upon which to plan and improve their future campaigns and service provisions. What's more, when potential donors ask them exactly what they have achieved for the quality of lives of their beneficiaries, they can give them a direct and demonstrable answer.
Which prompts me to ask once again - how many other advocacy and service providing organisations are able to do that?
Reading this article, you are probably a fundraiser. But ask yourself - are you raising money that is having a good value impact? I am not talking about dollars in, dollars out. I am talking about impact.
One final piece of advice: a great structure to use and evaluate yourself against is Givewell's Impact Analysis. This is not Givewell in Australia (fantastic data gatherers!) - it is Givewell.org in the USA. Check out what constitutes ‘impact' on their website here.
(c) Sean Triner September 2010
(This is my most recent Agitator column published in F&P Magazine E-bulletin.)
