KM on a dollar a day

Musing on knowledge management, aid and development with limited resources

Archive for December 2014

Delivering development through case studies

with 2 comments


(picture: from @andyR_AGI twitter feed)

I just came from a two-day meeting in Berlin launching the “Global Delivery Initiative” which is being spearheaded by the World Bank and the German technical cooperation agency GIZ.

You are probably wondering what the Global Delivery is – I did (one wag asked if it was something about safe childbirth). While it sounds like a way of doing programming it is actually about building an alliance and common knowledge base between development organizations around what works in development. It is related to, but different from the recently launched “Doing Development Differently” initiative (about which I shall blog separately).

The key insight driving this initiative is that while there has been a lot of research and evaluation on the “what” of delivery and a lot is known about what approaches “should” work, especially around technical issues and programme design, much of what goes wrong in development is related to the “how” i.e. how programmes are actually delivered on the ground in the messy reality. Relatively less is understood about what makes some similarly designed projects successful while others fail.

Many of these implementation challenges are messy human problems and don’t lend themselves easily to experimental design or traditional research methods. In fact much is based on “tacit” knowledge that lies in people’s experiences (here is one of my first blog posts “the truth is out there” which explains this in more detail).

The key approach being taken through the global delivery initiative is that in order to capture this tacit knowledge and make it shareable and reusable is the development of case studies on the “how” of delivery. The aim is to develop case studies that are of high quality, focus on the how rather than the what,  and according to common standards and format to make them shareable both within, but more importantly across development organizations. The initiative is proposing to create a global online repository of  delivery focused case studies using a somewhat standardized template and methodology. The aim would be to collect and share examples of how delivery challenges have been overcome on the ground to build up an evidence base of what works – but not as “best practice” but as a resource of example approaches which could be adapted to local context, and longer term as the number of examples grows as a resource that could be analyzed and mined to spot common themes and solutions to delivery challenges.

The aim of the meeting was to present the approach together with case study examples from participants to help refine the approach as well as to get more contributing partners aboard and to talk through more on how to make the initiative be successful – including what needs to be done collectively by partners and what needs to be done inside individual organizations to strengthen their ability to create and effectively use case studies. However at the meeting it was clear that “case studies” meant different things to different people, have different uses and employ different approaches – and there is a balance to be struck between coming up with a shared approach that allows cross organizational learning versus specific needs of individual organizations. There was a marketplace of example case studies from which the diversity of approaches was clear  – some were simply an approach to research to understand a problem while others were much more focused on documenting a programme, and others on how a problem had been addressed.

A few of the key issues raised in the meetings were:

  • Who identifies the “problems” that should be documented? The general feeling was that this needed to be done in a participatory way with beneficiaries and country team leaders rather than being top down. Importantly the case studies should focus on problems and not projects
  • What is the difference between case studies and more formal evaluation techniques? There was some confusion in the discussion but the general sense was that these are complementary activities not alternatives, and that case studies shouldn’t seek to be as rigorous and comprehensive as evaluations
  • How to make use of existing knowledge sharing techniques that result in self-reflection such as appreciative inquiry or after action review in the development of case studies as the currently proposed methodology didn’t fully make use of these – and self-reflection is an important part of a learning case study.
  • How to incorporate learning from failure – there was general agreement we should, but also that this was extremely challenging in publicly funded development work. Case studies focusing on failure might not be feasible, but including lessons from the less successful aspects of a programme or a comparative analysis across different locations to identify the determinants of success or otherwise was seen as valuable. A more challenging approach, but also more fruitful would be to design more experimental, iterative approaches to problem solving in development from the outset, which may have a higher risk of failure, but also greater potential benefits, and greater learning.
  • Who can/should write case studies? it was felt that not everyone has the right skill set to develop good case studies. Also there is a balance between using insiders who know the context and outsiders who can be more impartial and may have better documentation skills. Case studies are not objective in the same way as evaluations are however and they do need to draw on the reflections of those involved in the case. We also heard from Jennifer Widner that Princeton is developing a MOOC for writing case studies which will be interesting to check out.
  • Case studies are very labour intensive – and there was some discussion about weighing the value against the amount of time they take to develop (one estimate was that a single case study takes at least 350 person hours of work)
  • There are a lot of organizational challenges in the use of case studies – partly because use and dissemination is often not fully thought through at the beginning of the process – but more importantly because organizational practices and incentives are not aligned to support the use of tacit learning in our work (a much larger issue that I’ve discussed previously on this blog), and because our tendency to present everything as a success and as an advocacy opportunity to funders hampers our ability to self-critically reflect and learn.
  • My final observation was that unsurprisingly it was pointed out that case studies are only one of a range of approaches to fostering learning from experience and sharing delivery related knowledge. It was felt that other approaches such as communities of practice, peer learning and support, use of innovation approaches such as human-centred and participatory design also needed to be part of any initiative to improve delivery.

Going forward there was strong support for learning more about delivery challenges and sharing and applying that knowledge, although the nature of the partnership and who would do what was a little less clear (understandable since there was a diverse array of partners and the meeting only lasted a day and a half)

The World Bank and GIZ have started producing case studies and are planning more in 2015 along with an online repository – they are now trying to get others to sign up and do the same.

There was an agreement that case studies should be complemented by other work – and participants agreed to do more to share with each other both what they learn about delivery related knowledge but also how they learn about delivery in terms of tools, techniques and approaches they are using and how well they work.

Participants also agreed to advocate within their organizations for the idea of learning from experience on the how of development and more generally doing development differently.

All told, this looks like a promising initiative to improve sharing of tacit knowledge between development organizations – and one which I need to follow-up on to see if and how UNICEF could be involved. But for it to be successful it will need more partners, and notably absent were local and southern partners who would be key to any learning on delivery. In addition while case studies can form a good basis for this learning, the initiative will need to go beyond collecting and sharing case studies to focus on the idea of fostering learning from experience in development – both in identifying and sharing that learning – but also in overcoming some of our institutional challenges in actually applying it – and ensuring that there is continual learning in how we do our work.

Finally I mentioned at the beginning of the blog that this was related to “Doing Development Differently”, an initiative which seeks to rethink how we do development. While the global delivery initiative can contribute to that – I think the groups need to keep asking themselves whether they really are contributing to doing development differently by collecting new, more grounded information on what works and in a way that helps inspire and inform action, but doesn’t direct it – and avoid creating another knowledge repository that goes unused, or is used as a set of development recipes from donor to beneficiary (see this old blog of mine on why “best practice” databases are not the way to go), or a way of telling stories that make our (current traditional) work look good without real learning.

So let’s give it a try!

Written by Ian Thorpe

December 16, 2014 at 8:45 am

Love the One you’re with (on using official software)

with 7 comments


I think we have gotten quite used to assuming that our organizations will not provide the latest technology tools we need to do our jobs, and so if we want to get things done we will have to download or use free or even paid consumer software, even though this is officially frowned upon.

For quite a while externally available tools looked much better than what our organizations were able or willing to provide. Ease of use, attractive design,capability to work with external partners, real-time collaboration were all things that consumer tool have that our office tools did not.

As a result I saw people using a plethora of unofficial tools to do their work: Google Docs, Evernote, PBWiki, Yousendit, Dropbox, Ning, WordPress, you name it – even using Facebook for official internal networking (see here on why I don’t think this is a good idea), and YouTube to share internal videos. The upside was that people were being entrepreneurial finding ways to collaborate to get things done – but this also comes with a big downside – especially from a knowledge management perspective.

When a lot of work is going on in outside applications there are a number of risks. If it is a free service, then most likely your data is being used to advertise to you and others and you may have little control over the privacy and security of your information – nor any real guarantee that it will even always be there.

Another challenge is that all this organizational knowledge is being kept in separate silos as each department or team use different tools, or even when they use the same tools have individual accounts that are not connected to one another and not widely known about across the organization. When staff or consultants leave, tools are changed or abandoned, passwords are forgotten and content is lost. And none of these are integrated with our official tools so we need to learn different passwords and manually copy or transfer data between systems.

But I think we have now hit a turning point where the benefits of using official tools outweighs the limitations they might impose. For the first time I remember, we have many official tools that can  do almost the same as, and sometimes even do more than the consumer tools we have grown to love. In UNICEF we are using Office365 which includes tools such as Yammer for social networking, Lync for video conferencing, One Drive for file sharing and SharePoint for document management and collaborative authoring (and IBM and others offer similar tools). These are all quite powerful, they are more or less integrated together and with our other systems, and they are officially sanctioned and supported. What’s more they now also allow users to participate from any location and most devices, and they also allow collaboration with external partners – all stumbling blocks to using official systems in the past.

But while the software has greatly improved our behaviour is yet to catch up. There are still many who continue to use unofficial tools. The reasons for this are several including simply not being aware the new tools are available and what they can do to not knowing how to use them, to feeling more comfortable with the external tools they already know and preferring them as they “work better”, and perhaps not trusting what IT is pushing to them.

In the past I’ve also been one of those people who used a lot of outside tools and complained about what we being provided as official tools. While I still think we need to do more to make our official tools better, for me there was a turning point when I realized the benefits of using official systems for reliability, privacy, knowledge retention and ability to collaborate were more important that the inconvenience in learning new tools and coping with the mostly minor issues I have with them.

In my own mind I apply a kind of 80% rule. If the official tool meets 80% of my feature and usability needs then it’s better to use if for work than going it alone with my preferred tool. Only in those instances where the official tool is lacking key functionality, or where no real official alternative exists will I find my own external solution. For example we don’t have a corporate tool for staff surveys and while SharePoint can be used to do this we don’t have the templates and setup yet to provide this as an option for staff who want to do their own surveys so I would use Surveymonkey, Limesurvey or some other tool. By contrast now we have One Drive there is no real justification to use private Dropbox accounts to share files internally, even if the interface might be slightly nicer.

It’s of course totally fine to use whatever tool you like for personal use – and also to use external tools to participate in work groups set up by others outside your organization (e.g. I participate in external online forums such as KM4Dev or on UNDP’s Teamworks as I couldn’t expect them to join me on Yammer instead!) But if you are setting something up for your organization, even if you are inviting external participants then you are better off using your official tools.

So how do we overcome the reluctance of staff to use perfectly adequate existing tools. A few thoughts:

1. Communicate, communicate, communicate about the tools that are available and what they can do

2. Train and support people to use them

3. Show real life examples of how they are being used to support work and get better results i.e. have the users speak to how they can be used and the benefits rather than only promoting this from an IT perspective.

4. To users – apply my 80% rule – do you really need to use your preferred tool? Why not at least give the official systems a try – and let the network benefits do the rest.

5. To IT – listen to user feedback and try to make the tools as responsive to user needs as possible – in particular look at what external tools are doing well that official tools don’t and try to work on those.

6. Again to IT – look carefully at what unofficial tools staff are using and why – not to control and forbid – but to better understand what are common technology needs of staff or new ways they are finding to use technology to improve their work. These are the things the official systems need to catch up with. Use this as a way of identifying future tools and services that would be really valuable (did I mention that we need a good corporate survey tool?).

Post script: After posting this I got in a heated twitter discussion with @Wayan_vota who strongly disagreed and who has some good points (although I still disagree with him). I realized that this post might sound like I’m cheerleading for Microsoft and corporate IT rules and regulations. I’m certainly doing neither. My main point is that as much as I might personally love other tools, and everyone in the organization has their own favourites, I realized that if we want to get people to collaborate across the organization, and we want to be able to find anything – even after people leave then we need to get over fighting “the man”. Instead we need to figure out how to get the best official tools we can that everyone use, and then make the best use of what we have got, including how they are managed – and keep the pressure up to make these as useful as possible.


Written by Ian Thorpe

December 3, 2014 at 9:00 am

%d bloggers like this: