The joy of polls

I’ve always been a fan of opinion polls. And after Nate Silver’s triumphant predictions of the US election outcome you’d think everyone would be. But the aid/development sphere has still a way to go to catch up.

In a recent blog post I explained a little about some of the challenges in the post-2015 global consultation. A while ago  week I was a “discussant” at the recent Tech Salon organized on using technology for qualitative M&E and I talked a little more about this project. In preparing my thoughts and listening to the discussion one specific idea kept coming into my mind – that is the importance of opinions and perceptions as part of monitoring and evaluation. I’ve written previously about the need to “listen to beneficiaries”  mostly from the point of view of it being the right way to do participatory development that also has a chance of being sustainable – but it also happens to be good, if not traditional M&E.

In a traditional approach to project monitoring we often look very closely at the supply side of things  – how much of our budget have we spent, how much did we deliver, how many people did we train, was it done on time and according to plan. Or if we are able to measure impact we might look at things the number of kids completing school, or the death rates from preventable diseases. An advantage of this type of measure is that they are usually quantifiable and measurable, and there is a clear logic or change theory around how these measures relate to your project activities.

But what about looking at demand. Do people want what we are giving them, or do they want something else? Are they satisfied with what we are giving them and how it is provided?

Asking people about their opinions, perceptions and feelings is also a useful and complementary way of project monitoring that might tell you a different story that a supply/delivery focused measurement based on “hard” data. Imagine a school project where the school has been built, the teachers hired, the curriculum designed and the cash transfers for the poorest have been distributed, but school attendance and achievement are not progressing at the same pace. You’ve spent your budget, delivered all your project outputs, but something still isn’t right. If you want to find out what’s happening, then you will need to ask people why don’ t they send their kids to school  – is the curriculum wrong, are the school hours inconvenient, do they value education, are there household labour needs for the children, or is there just no better jobs to get if you get an education. Maybe people wish that the aid project was on something else entirely.

Participatory research is of course one way to dig into what beneficiaries want. But this is also costly, time-consuming and usually is only able to cover a small sample of people. It’s good for digging deep to understand an issue  – but sometimes you also want to get a broad and, if you are lucky, representative view of what people are thinking and feeling. This is where opinion polls and surveys come in. Surveys are of course widely used for political purposes – but they are also used a lot by companies seeking to understand the reach and appeal of their brand and their products (which do you like better “Coke” or “Pepsi”, Which three adjectives on this list best describe our product?).

So why not use polls to ask about how you are doing with your project. Or even further, how can we find out about the image and reputation of your organization? Do people think you are effective? Do they think you are easy or hard to work with? And your “products” – do they think your programmes are effective, are they timely? are they responsive to what people need and what they want?

Many large NGOs do use opinion research to measure the appeal of their organization and their work for fundraising purposes – and use this to carefully tailor their appeals and marketing, although they are not always too keen to publicize this.  But use of polls in developing countries is much less common – partly due to logistical challenges – but also possibly because the incentives to do so are less, as there is a greater incentive to increase funding than to satisfy beneficiaries, who don’t pay.

But some organizations are now starting to also take this on. UNDP country offices run surveys of their national partners (which are curiously not to be found online). The UN recently carried out a partnership survey for the “Quadriennial Comprehensive Policy Review of operational activities for development”  – essentially a review of how well programme country governments think the UN is supporting them with its development work. This gave some interesting feedback on how we are doing which provoked some internal discussion – but also (thankfully only) a few who questioned whether government’s opinions of  how we are doing  are as valid as our internal monitoring data.

But the incentives to use polling for monitoring are still limited, unless donors are prepared to ask for them and even to finance them. They can be very informative to  help understand whether a programme is reaching and is appreciated by beneficiaries – but ultimately this will only be given importance if this feedback has consequences – and one of the best  ways would be for donors to ask for this – or for then to finance this type of research to be carried out independently themselves as an additional way of evaluating programmes and organizations they fund.

And now back to the post-2015 agenda for a brief advertisement.  While the process to develop the post-2015 development agenda is commendable for its use of  online discussions, consultations with technical experts and technical analytic papers – there is also an important role for opinion polling here too. Polling can reach a wide audience  that don’t have the means or the time to participate in face to face or online discussions but who can express their views via an online and/or SMS based poll on what their priorities for the post-2015 might be.

But luckily someone has already thought of that! The UN, the World Wide Web foundation and ODI have teamed together to launch the “My World 2015” public poll. Go there and vote for the changes that you would most like to see included in the post-2015 agenda.!