The long and winding road to evidence based development
There has been quite a bit of discussion online about the Financial Times article “How Aid got Smarter” featuring an interview with UNICEF’s Executive Director Tony Lake.
The article makes some important points about the need to improve the use of evidence to making decisions in aid, in particular in discovering what works and what doesn’t, admitting it and acting on it.
What is perhaps a pity about the article is that it can be read to imply that until recently aid decisions were largely faith-based, but now suddenly, at last, the role of science and evidence is being taken seriously in development. As usual the reality is a bit more complex than that.
Discussions around the need to make development work more evidence based have been around as long as I’ve been working in development (and probably a lot longer than that). And the progressions towards improved use of knowledge and evidence in development often seems like a case of two steps forward, one step back.
Over my past 20 or so years working in aid some notable improvements in the attention to evidence include an increased investment in and focus on evaluation resulting in more professionalized evaluation departments with greater resources and thus more and better evaluations; greater investment in supporting statistical data collection including in agreeing on harmonized standards and common sets of indicators to track over time; greater attention in various forms to supporting better internal communication and knowledge management to help staff have better access to, and make better use of available development knowledge. There are probably many others.
But many challenges remain. A few of the most thorny (and recurring) challenges in using knowledge in development work seem to be:
- How far we are able to “know” what works and what doesn’t. We don’t have the resources and skills to measure everything scientifically – and some of the knowledge we need is highly contextual and based on experience as well as on science (See my previous blog “The truth is out there” about the limits of what we can know).
- But even when we have a large body of relevant, available knowledge it is not always used in decision-making. It’s important to understand the reasons for these and try to tackle them along with work to increase the supply of knowledge (see my previous blog “Creating a demand for knowledge”).
- In our desire to understand something, and to “break it down” so we can tackle it in manageable pieces or sell it to donors, or the public, we often forget that many of the things we are dealing with are “complex adaptive systems” where the whole works differently from the sum of the parts and where a good practice in one context might not work in another. Of course, this doesn’t mean we shouldn’t use evidence – but we need to understand it in context, and apply it flexibly rather than expecting to find universal answers. (See my previous blog “Who’s afraid of complexity in aid”)
But while evidence based aid isn’t a new idea, and even though we are still not there yet, there is still good reason to be optimistic that aid is becoming, and will continue to become more evidence informed. Here are a few reasons why:
1. The results agenda – donors and beneficiaries alike are putting increasing pressure on aid agencies to managing for and report on results – in particular to be sure that ever scarcer aid money is being well invested (see this blog by Owen Barder on some of the benefits and challenges of the results agenda for improving aid).
2. Aid transparency – as more and more aid agencies sign up for IATI then it becomes easier to see who is dong what and where which is an aid to accountability and to improved coordination – but also to research as there is a whole lot of new data to crunch to understand more about how aid works (or doesn’t) especially when linked to the results agenda.
3. Open data and research – more and more development data is being made freely available for public use which provides a whole range of raw material for researchers. Increasingly (although still slowly) publicly funded research (and even data sets and analyses) is also being opened up for public access – which means there is a lot more chance that it will be used.
4. Real time data analysis – Often one of the big challenges in using evidence is that by the time you know enough about a problem its already too late (think global food/economic crisis). New “big data” techniques to more quickly understand what is happening – at least enough to act, if not enough to scientifically “know”. (See this previous blog on the possibilities of “real time”).
5. Beneficiary feedback – this is one area where there is great (as yet mostly untapped) promise, and a number of interesting initiatives. Too often external solutions are imposed on beneficiaries, using science as a basis, but without enough attention to getting real time feedback from the people who the programme is designed to help on whether they want the programme, if it is likely to work, and whether they are satisfied with it, or whether they have their own ideas about how to improve it. More listening can make projects more likely to work, and more participation can also help them be more sustainable in the long term giving beneficiaries a say and a stake in the project’s success (see my previous blog “listening to the people we work for” for more).
6. Lastly, there are a lot of smart, committed individuals talking about and working on how to improve aid. Sure, there always have been, but it seems (to me at least) that the volume and depth of this discussion has increased over the past few years, including from the detractors who in their own way, through their own critiques are advancing the discussion and thinking about how to do aid work well. And with more and more aid agency heads such as Tony Lake are speaking up in favour of smart aid – we can hope for more discussion about what smart aid really means- and for aid workers to feel more empowered to advocate for it inside their own organizations.
The quest for smarter aid is not new, and it will not be achieved overnight. Evidence based development work is more an ongoing journey rather than a destination. But the lights of Oz are looking a little bit brighter.
(Image: Dave Cutler)