Delivering development through case studies

(picture: from @andyR_AGI twitter feed)

I just came from a two-day meeting in Berlin launching the “Global Delivery Initiative” which is being spearheaded by the World Bank and the German technical cooperation agency GIZ.

You are probably wondering what the Global Delivery is – I did (one wag asked if it was something about safe childbirth). While it sounds like a way of doing programming it is actually about building an alliance and common knowledge base between development organizations around what works in development. It is related to, but different from the recently launched “Doing Development Differently” initiative (about which I shall blog separately).

The key insight driving this initiative is that while there has been a lot of research and evaluation on the “what” of delivery and a lot is known about what approaches “should” work, especially around technical issues and programme design, much of what goes wrong in development is related to the “how” i.e. how programmes are actually delivered on the ground in the messy reality. Relatively less is understood about what makes some similarly designed projects successful while others fail.

Many of these implementation challenges are messy human problems and don’t lend themselves easily to experimental design or traditional research methods. In fact much is based on “tacit” knowledge that lies in people’s experiences (here is one of my first blog posts “the truth is out there” which explains this in more detail).

The key approach being taken through the global delivery initiative is that in order to capture this tacit knowledge and make it shareable and reusable is the development of case studies on the “how” of delivery. The aim is to develop case studies that are of high quality, focus on the how rather than the what,  and according to common standards and format to make them shareable both within, but more importantly across development organizations. The initiative is proposing to create a global online repository of  delivery focused case studies using a somewhat standardized template and methodology. The aim would be to collect and share examples of how delivery challenges have been overcome on the ground to build up an evidence base of what works – but not as “best practice” but as a resource of example approaches which could be adapted to local context, and longer term as the number of examples grows as a resource that could be analyzed and mined to spot common themes and solutions to delivery challenges.

The aim of the meeting was to present the approach together with case study examples from participants to help refine the approach as well as to get more contributing partners aboard and to talk through more on how to make the initiative be successful – including what needs to be done collectively by partners and what needs to be done inside individual organizations to strengthen their ability to create and effectively use case studies. However at the meeting it was clear that “case studies” meant different things to different people, have different uses and employ different approaches – and there is a balance to be struck between coming up with a shared approach that allows cross organizational learning versus specific needs of individual organizations. There was a marketplace of example case studies from which the diversity of approaches was clear  – some were simply an approach to research to understand a problem while others were much more focused on documenting a programme, and others on how a problem had been addressed.

A few of the key issues raised in the meetings were:

• Who identifies the “problems” that should be documented? The general feeling was that this needed to be done in a participatory way with beneficiaries and country team leaders rather than being top down. Importantly the case studies should focus on problems and not projects
• What is the difference between case studies and more formal evaluation techniques? There was some confusion in the discussion but the general sense was that these are complementary activities not alternatives, and that case studies shouldn’t seek to be as rigorous and comprehensive as evaluations
• How to make use of existing knowledge sharing techniques that result in self-reflection such as appreciative inquiry or after action review in the development of case studies as the currently proposed methodology didn’t fully make use of these – and self-reflection is an important part of a learning case study.
• How to incorporate learning from failure – there was general agreement we should, but also that this was extremely challenging in publicly funded development work. Case studies focusing on failure might not be feasible, but including lessons from the less successful aspects of a programme or a comparative analysis across different locations to identify the determinants of success or otherwise was seen as valuable. A more challenging approach, but also more fruitful would be to design more experimental, iterative approaches to problem solving in development from the outset, which may have a higher risk of failure, but also greater potential benefits, and greater learning.
• Who can/should write case studies? it was felt that not everyone has the right skill set to develop good case studies. Also there is a balance between using insiders who know the context and outsiders who can be more impartial and may have better documentation skills. Case studies are not objective in the same way as evaluations are however and they do need to draw on the reflections of those involved in the case. We also heard from Jennifer Widner that Princeton is developing a MOOC for writing case studies which will be interesting to check out.
• Case studies are very labour intensive – and there was some discussion about weighing the value against the amount of time they take to develop (one estimate was that a single case study takes at least 350 person hours of work)
• There are a lot of organizational challenges in the use of case studies – partly because use and dissemination is often not fully thought through at the beginning of the process – but more importantly because organizational practices and incentives are not aligned to support the use of tacit learning in our work (a much larger issue that I’ve discussed previously on this blog), and because our tendency to present everything as a success and as an advocacy opportunity to funders hampers our ability to self-critically reflect and learn.
• My final observation was that unsurprisingly it was pointed out that case studies are only one of a range of approaches to fostering learning from experience and sharing delivery related knowledge. It was felt that other approaches such as communities of practice, peer learning and support, use of innovation approaches such as human-centred and participatory design also needed to be part of any initiative to improve delivery.

Going forward there was strong support for learning more about delivery challenges and sharing and applying that knowledge, although the nature of the partnership and who would do what was a little less clear (understandable since there was a diverse array of partners and the meeting only lasted a day and a half)

The World Bank and GIZ have started producing case studies and are planning more in 2015 along with an online repository – they are now trying to get others to sign up and do the same.

There was an agreement that case studies should be complemented by other work – and participants agreed to do more to share with each other both what they learn about delivery related knowledge but also how they learn about delivery in terms of tools, techniques and approaches they are using and how well they work.

Participants also agreed to advocate within their organizations for the idea of learning from experience on the how of development and more generally doing development differently.

All told, this looks like a promising initiative to improve sharing of tacit knowledge between development organizations – and one which I need to follow-up on to see if and how UNICEF could be involved. But for it to be successful it will need more partners, and notably absent were local and southern partners who would be key to any learning on delivery. In addition while case studies can form a good basis for this learning, the initiative will need to go beyond collecting and sharing case studies to focus on the idea of fostering learning from experience in development – both in identifying and sharing that learning – but also in overcoming some of our institutional challenges in actually applying it – and ensuring that there is continual learning in how we do our work.

Finally I mentioned at the beginning of the blog that this was related to “Doing Development Differently”, an initiative which seeks to rethink how we do development. While the global delivery initiative can contribute to that – I think the groups need to keep asking themselves whether they really are contributing to doing development differently by collecting new, more grounded information on what works and in a way that helps inspire and inform action, but doesn’t direct it – and avoid creating another knowledge repository that goes unused, or is used as a set of development recipes from donor to beneficiary (see this old blog of mine on why “best practice” databases are not the way to go), or a way of telling stories that make our (current traditional) work look good without real learning.

So let’s give it a try!