Archive for the ‘Learning’ Category
(picture: from @andyR_AGI twitter feed)
I just came from a two-day meeting in Berlin launching the “Global Delivery Initiative” which is being spearheaded by the World Bank and the German technical cooperation agency GIZ.
You are probably wondering what the Global Delivery is – I did (one wag asked if it was something about safe childbirth). While it sounds like a way of doing programming it is actually about building an alliance and common knowledge base between development organizations around what works in development. It is related to, but different from the recently launched “Doing Development Differently” initiative (about which I shall blog separately).
The key insight driving this initiative is that while there has been a lot of research and evaluation on the “what” of delivery and a lot is known about what approaches “should” work, especially around technical issues and programme design, much of what goes wrong in development is related to the “how” i.e. how programmes are actually delivered on the ground in the messy reality. Relatively less is understood about what makes some similarly designed projects successful while others fail.
Many of these implementation challenges are messy human problems and don’t lend themselves easily to experimental design or traditional research methods. In fact much is based on “tacit” knowledge that lies in people’s experiences (here is one of my first blog posts “the truth is out there” which explains this in more detail).
The key approach being taken through the global delivery initiative is that in order to capture this tacit knowledge and make it shareable and reusable is the development of case studies on the “how” of delivery. The aim is to develop case studies that are of high quality, focus on the how rather than the what, and according to common standards and format to make them shareable both within, but more importantly across development organizations. The initiative is proposing to create a global online repository of delivery focused case studies using a somewhat standardized template and methodology. The aim would be to collect and share examples of how delivery challenges have been overcome on the ground to build up an evidence base of what works – but not as “best practice” but as a resource of example approaches which could be adapted to local context, and longer term as the number of examples grows as a resource that could be analyzed and mined to spot common themes and solutions to delivery challenges.
The aim of the meeting was to present the approach together with case study examples from participants to help refine the approach as well as to get more contributing partners aboard and to talk through more on how to make the initiative be successful – including what needs to be done collectively by partners and what needs to be done inside individual organizations to strengthen their ability to create and effectively use case studies. However at the meeting it was clear that “case studies” meant different things to different people, have different uses and employ different approaches – and there is a balance to be struck between coming up with a shared approach that allows cross organizational learning versus specific needs of individual organizations. There was a marketplace of example case studies from which the diversity of approaches was clear – some were simply an approach to research to understand a problem while others were much more focused on documenting a programme, and others on how a problem had been addressed.
A few of the key issues raised in the meetings were:
- Who identifies the “problems” that should be documented? The general feeling was that this needed to be done in a participatory way with beneficiaries and country team leaders rather than being top down. Importantly the case studies should focus on problems and not projects
- What is the difference between case studies and more formal evaluation techniques? There was some confusion in the discussion but the general sense was that these are complementary activities not alternatives, and that case studies shouldn’t seek to be as rigorous and comprehensive as evaluations
- How to make use of existing knowledge sharing techniques that result in self-reflection such as appreciative inquiry or after action review in the development of case studies as the currently proposed methodology didn’t fully make use of these – and self-reflection is an important part of a learning case study.
- How to incorporate learning from failure – there was general agreement we should, but also that this was extremely challenging in publicly funded development work. Case studies focusing on failure might not be feasible, but including lessons from the less successful aspects of a programme or a comparative analysis across different locations to identify the determinants of success or otherwise was seen as valuable. A more challenging approach, but also more fruitful would be to design more experimental, iterative approaches to problem solving in development from the outset, which may have a higher risk of failure, but also greater potential benefits, and greater learning.
- Who can/should write case studies? it was felt that not everyone has the right skill set to develop good case studies. Also there is a balance between using insiders who know the context and outsiders who can be more impartial and may have better documentation skills. Case studies are not objective in the same way as evaluations are however and they do need to draw on the reflections of those involved in the case. We also heard from Jennifer Widner that Princeton is developing a MOOC for writing case studies which will be interesting to check out.
- Case studies are very labour intensive – and there was some discussion about weighing the value against the amount of time they take to develop (one estimate was that a single case study takes at least 350 person hours of work)
- There are a lot of organizational challenges in the use of case studies – partly because use and dissemination is often not fully thought through at the beginning of the process – but more importantly because organizational practices and incentives are not aligned to support the use of tacit learning in our work (a much larger issue that I’ve discussed previously on this blog), and because our tendency to present everything as a success and as an advocacy opportunity to funders hampers our ability to self-critically reflect and learn.
- My final observation was that unsurprisingly it was pointed out that case studies are only one of a range of approaches to fostering learning from experience and sharing delivery related knowledge. It was felt that other approaches such as communities of practice, peer learning and support, use of innovation approaches such as human-centred and participatory design also needed to be part of any initiative to improve delivery.
Going forward there was strong support for learning more about delivery challenges and sharing and applying that knowledge, although the nature of the partnership and who would do what was a little less clear (understandable since there was a diverse array of partners and the meeting only lasted a day and a half)
The World Bank and GIZ have started producing case studies and are planning more in 2015 along with an online repository – they are now trying to get others to sign up and do the same.
There was an agreement that case studies should be complemented by other work – and participants agreed to do more to share with each other both what they learn about delivery related knowledge but also how they learn about delivery in terms of tools, techniques and approaches they are using and how well they work.
Participants also agreed to advocate within their organizations for the idea of learning from experience on the how of development and more generally doing development differently.
All told, this looks like a promising initiative to improve sharing of tacit knowledge between development organizations – and one which I need to follow-up on to see if and how UNICEF could be involved. But for it to be successful it will need more partners, and notably absent were local and southern partners who would be key to any learning on delivery. In addition while case studies can form a good basis for this learning, the initiative will need to go beyond collecting and sharing case studies to focus on the idea of fostering learning from experience in development – both in identifying and sharing that learning – but also in overcoming some of our institutional challenges in actually applying it – and ensuring that there is continual learning in how we do our work.
Finally I mentioned at the beginning of the blog that this was related to “Doing Development Differently”, an initiative which seeks to rethink how we do development. While the global delivery initiative can contribute to that – I think the groups need to keep asking themselves whether they really are contributing to doing development differently by collecting new, more grounded information on what works and in a way that helps inspire and inform action, but doesn’t direct it – and avoid creating another knowledge repository that goes unused, or is used as a set of development recipes from donor to beneficiary (see this old blog of mine on why “best practice” databases are not the way to go), or a way of telling stories that make our (current traditional) work look good without real learning.
So let’s give it a try!
One thing organizations are not good about is figuring out how to retain or reuse knowledge of staff who leave the organization, whether through retirement or leaving for a job in another organization. Many organizations do very little at all to address this, but even for those that do, there are a few fundamental flaws in how organizations look at this problem:
1. They consider that whatever knowledge you generate or use while you are in a job belongs to the organization and so should be kept there
2. They consider knowledge to reside either in documents and products produced by staff – or in a slightly more enlightened state they consider that it resides in individuals and can be downloaded from the when they leave.
3. They create a firewall between current and former staff, including retirees making it hard to maintain personal networks and relationships that are often where the real source of knowledge exchange lies.
Probably the most effective strategy to retain knowledge is to be the kind of organization that retains and develops its best staff – and that could be a series of blog pots in its own right – but staff will leave whether through retirement, to seek new opportunities, or as a result of restructuring – so we need to think of ways to address this.
At KM World a couple of weeks ago (yes I’m still blogging about that) there were a few interesting presentations about how to tap into the knowledge and expertise of former staff – in particular a very interesting one by Lesley Schneier a retiree of the World Bank about a system they had set up to tap into the expertise of Bank retirees to support current projects on a pro-bono basis, building on the enthusiasm of many retirees to keep in touch and be helpful and the interest of current staff to get free expert advised from seasoned professionals. There were lots of good ideas on how to make us of the desire of retirees to stay involved to support current work but also a couple of important lessons on the challenges too.
One challenge at the Bank is the interaction between staff and retirees takes place on an externally facing platform which is separate from the platforms that staff use in their day-to-day work (because I presume IT security?) which means that it is much more of an effort for current staff to go there, and many don’t even know that it is an option.
I don’t have any magic solutions to the problem of “knowledge walking out the door” but it seems to me that there are a few practices we could try to change which could make an important difference:
Most IT systems use your current official email as a means of authenticating you into social platforms. This is great as it is a good shortcut to check whether you are a current employee and so should be permitted to be part of the internal intranet and all the secrets and confidences in contains – but when you leave you lose your official email and access to your internal resources, but also to whatever internal social networks you have.
However in reality the nature of a social network is that is social, that is it is based on personal and professional relationships that have developed over time. Once you have left an organization your social connections (and willingness to collaborate with them) do not disappear overnight with a change of job or of email address. Neither do your past contributions suddenly become useless – even if over time they might decline in relevance.
So a couple of practices to consider might be to allow some kind of continued access to communities or similar sites after you retire or leave the organization. Obviously there would be some things which are too internal or confidential to share, especially in commercial operations; but in my area of work, international development, any technical discussions on how to do our work better should not be confidential and so there is no reason to lock people out after they leave the organization – after all they may still be willing to provide free advice to support current projects.
Maybe an even better way to handle this would be to create communities that are by default open in the first place and have your staff use those for non-internal non-confidential collaboration – that way you can more easily engage with and learn from external experts even if they have never worked for your organization – and also your internal expertise can be of benefit to others without needing formal cooperation thus increasing the influence of your knowledge work.
One particular bad practice I have seen that should be avoided at all costs is to delete a persons contributions to online discussions after they leave an organization, or after they choose to leave a network, or to delete their user profile. It’s useful to mark on someone’s profile if they are currently working for the organization and if they are currently active in the network as this is useful information when reading the contributions – but don’t delete either the contribution or the profile of the person who created it.
Another challenge with retaining knowledge is the handover process. In some places we don’t have any consistent handover procedures at all and if there is a large gap in hiring (as is often the case in the UN) then a newcomer is often forced to try to recreate from scratch the knowledge base of their predecessor. A few thoughts on this:
1. It’s good to have some standard handover process in place with some standard elements of what should be included and a timing to do this that is before the last minute. Even better if these are in some widely accessible database. Better still if there are some “teeth” to make sure that this is complied with. This could be as simple as a standard template – but there are also some more creative techniques out there involving interviewing, video etc. that make it more alive and less procedural.
2. It is however unreasonable to expect that someone can download all relevant knowledge into a form or interview since a lot of the important tacit, just in time knowledge can’t be captured in a handover note – in fact you don’t always know what is most important until you need it. Having human contact between someone leaving a post and a new person coming can help a lot with this. Ideally having an overlap when two people are in post together so that the newcomer can see in practice how the job is done can be extremely valuable, and it would be good if we could change our hiring practices to allow for this – especially for critical office positions that require a lot of “feeling” for the job that can’t be easily captured in written form. If an overlap isn’t possible then a face to face handover meeting, or better still a meeting with some follow-up calls can greatly help.
3. Lastly since this is “World Out Loud Week” – one of the best ways to mitigate against knowledge loss is not to try to capture everything when the person leaves, and when memories of important events and learnings have often been retrospectively interpreted to fit your own world view – wouldn’t it be better instead to capture knowledge all the time as you work? That way it is much easier to see the individual steps on how a project was developed, how and when key decisions were made and what were the key drivers. Encouraging all staff to work out loud (inside the organization if they are not willing or able to do it publicly) and making all that available to the rest of the organization to learn from is one of the best ways of preserving living knowledge that comes from how we actually do our work (rather than how we talk about it afterwards).
Yesterday I had an interesting conversation with colleagues at UNDP who are organizing an event about “scaling up”.
The idea of scaling up successful pilots or innovations has long been one of the holy grails of aid work, and it seems we’re still quite not sure how to do it, or at least how to do it consistently, or how to “pick winners” i.e. ideas that can be scaled successfully.
The conversation reminded me of a presentation I attended and an internal blog I wrote some years ago about how to spread good ideas when I was back with UNICEF. It occurs to me that quite often in looking at scaling up a successful pilot or prototype we tend to focus on i) identifying those ideas which have been successfully piloted and ii) for which we can use the evidence of success to mobilize resources from donors and domestically.
However even when new ideas have been shown to work in a successful pilot or prototype (or have even been “proven” through extensive research and clinical style testing), i’s not a guarantee that they will scale. A big challenge is the issue of “adoption” i.e. how can you persuade others to apply them other than with scientific evidence and cash – because those are not enough.
Below is a slight reworking of my old blog post that looks at some of the challenges of spreading (or diffusing) good ideas:
Colleagues working on communication for Development (C4D) at the meeting also felt that the conclusions of the paper were highly relevant to their work, and it seems there could be a promising common interest in different parts of the organization to pursue these ideas further. One obvious challenge both within UNICEF, and in disseminating innovations externally is that often quite a few of the ideal conditions for successful diffusion are not present, and we may have a varying ability to foster them. We therefore also need to do some thinking about what we can do that is useful even when we know we are not able to create the kind of environment that we would ideally like in order for new approaches to be adopted.
From the paper, one important aspect of whether a new innovation or approach is adopted is the nature of the idea itself. Below are highlighted some aspects of an innovation which can have an important impact on whether it is adopted (The text below is adapted from Nancy’s presentation with some added commentary from my side).
Is the new approach more effective than the current strategy? Is it more cost-effective? If it is not perceived to have an advantage, it could be dead in the water, though even having an advantage does not guarantee adoption. (Note: this benefit needs to be clear to the adopter and the benefit for the adopter might well be different from the perceived benefit to management or to the promoter of the approach)
Is it compatible with user’s values, norms, ways of working, and perceived needs?
Simple is good! If it is complex, can it be broken down into smaller bites? The perception of complexity can also be partly overcome with demonstrations and hands-on experience.
Is there space given to try it out on a limited basis?
Are the effects of the innovation readily observable? (and preferably measurable)
Is there room to adapt it to local realities or to refine it? This seems particularly important for dissemination of “good practices” spread through horizontal networks (and is very relevant to our work on lessons learned – and would seem to imply that these should be seen more as an idea bank than templates or how-to guides)
Fuzzy boundaries (related concept)
i.e. an innovation should have a fixed core of common elements used in all cases, but with other elements that can be adapted around the edges to meet different circumstances and needs. These should preferably emerge from repeated trails in different contexts. What’s is important here is that there is a set of core principles to the innovation that make it successful, but a n accepted range of modifications that make it suitable for different applications.
If there is a high degree of uncertainty in adopting the new approach, then it can be perceived as personally risky and thus it is less likely to be adopted (especially in a risk-averse environment).
If a new approach is relevant to user’s work and improves their performance, it is more likely to be adopted, especially if it is feasible and easy.
To spread an idea you need a support system for new implementations – help desk, training, customization, implementation advice (even if you can see the benefit, it’s hard to do something new if you don’t know how – in this context I’d also add that being able to be in contact and share experiences with others who are have tried or are also trying out the new approach can be invaluable).
An implication of this seems to be that incremental changes are easier to promote than radical ones – since they fit more easily into existing norms and values and the benefit can appear more tangible, even if the potential improvement is less. At the same time just because and idea is easier to spread, it doesn’t mean that it is better. There are some interesting trade-offs here between ideas that can be easily implemented and spread and those which might have a more profound impact.
So, what do you think of these?
Do they hold true from your own experience?
What do you think are the lessons we can learn for our own work in trying to share new approaches between offices, for adopting good ideas from the external world, or in getting our partners to take on proven approaches that for they themselves are new?
Post script: back when I wrote this I had planned to write something about the types of people and organizations that effectively spread new ideas. I never got to this but I’ve now put it on my list for a future blog post.
A few months ago I was appointed “Learning Manager” for my office, responsible for leading an office learning plan and helping foster the creation of a culture of learning in the office as well as helping facilitate staff access to learning opportunities. This is not a full-time position, rather a set of additional responsibilities added on to my existing job.
The “Learning Manager” role is something that UNDP created for every office some years ago as a way of strengthening organizational learning in individual offices, and this plus the considerable wealth of online courses available to staff via the intranet on the “Learning Manager System” displays a strong commitment to foster learning by UNDP.
At the same time learning managers are often quite junior staff (I’m in the relatively rare position of being a learning manager and an actual manager too). Typically this role is lumped in with the Human Resources assistant position (or HR associate as they are called in UNDP) which is also typically a local staff recruitment. Last week I participated in an orientation/skills development course for new learning managers which was a great opportunity to speak to other learning managers and find out how they do their job.
What I heard was both daunting but also encouraging. Many new learning managers were struggling to get traction on learning in their offices due to resource constraints, mixed levels of support from managers, lack of focus on learning due to workload and the challenge of getting things done without any formal authority and on top of doing their “regular” job (and I might add unrealistic expectations from the organization on what a learning manager can physically manage to do).
But I also encountered a highly motivated and resourceful group who were finding different ways to achieve results in challenging circumstances. The shared challenge that all were trying to address is how to maximize office learning with limited time and money and no formal authority. I’m sharing here some of the ways, both strategic and tactical that learning managers are getting the job done.
1. One of the key challenges is making the case for learning with the head of office and management team. Different approaches used for this include making the case for learning as an investment in office productivity (appeal to logic), reminding that it is part of the “rules” and measured in the office scorecard and comparing how the office is doing to similar offices e.g. in the same region (appeal to authority), or emphasizing the effect it will have on staff morale and creating goodwill in the office as well as helping the staff at a personal level to deal with changes in the organization (appeal to emotion).
2. Another challenge is balancing the roles of facilitator and enforcer. Learning managers are expected to ensure that all staff do their mandatory online trainings (ethics, gender, security etc.) and that the office has a learning plan and that individuals have learning goals in their performance appraisals – yet don’t have the authority to make people do this, especially those who are reluctant. Ultimately though the most fulfilling role for the learning manager, and probably the one that achieves the best learning results is to foster a learning culture by responding to people’s needs, interests and aspirations and acting as a facilitator and coach to help people learn rather than trying to force them to do the compulsory things they may not be enthusiastic to do.
3. At a tactical level when budgets are tight it is often not cost-effective to send individuals on external training courses out of the country, and local opportunities may be limited. However you can benefit from extensive expertise and experience that is already in the office. Example approaches to this include i) organizing a “skillshare” session where staff members share a skill they have (possibly from a previous job) with the rest of the office, either as a training course or as a coach ii) have staff who do go on external trainings or who go on work travel to debrief the office on their learning as a routine event or requirement iii) taking advantage of visitors from HQ or regional offices and asking them to carry out a training or briefing as part of their visit iv) inviting speakers from local partners.
4. Pooling resources – e.g. sharing learning opportunities with other UN agencies or with government and NGOs. This could be by organizing joint trainings or by having a reciprocal arrangement to allow people from other organizations to join trainings organized by the office in exchange for being able to send people to their trainings, and routinely sharing information on learning events with one another.
5. Make use of online resources – this can involve using online courses developed by the organization or licensed through an external provider for example in UNDP staff have access to a wealth of UNDP and externally developed courses through their Learning Manager System. Other options include use of MOOCs (massively open online courses), webinars or other online and remote learning opportunities.
6. Mentoring and coaching – setting up individual peer-to-peer learning exchanges within the office or between offices in the same region. This can be valuable as it provides ongoing support rather than just an episodic training. One more sophisticated way to do this is to use the self-assessment peer-assist methodology (from Collison and Parcell’s Learning to Fly) where offices (or individuals) self-assess their learning needs against a set of criteria and then are paired up according to needs and strengths.
7. Organizing regular learning events or learning days (e.g. once per month) where staff devote time to learning to ensure learning is regular and recognized. Other similar approaches are sending out weekly TED talks, articles., presentations or other short pieces of interest to stimulate learning without consuming much time.
8. Some offices seek to regularly send staff on “detailed assignments” or give them “stretch assignments”. these are short-term opportunities to take on a more challenging assignment to fill in for a temporary vacancy or a colleague on extended or maternity leave either within the same office or in another office. These may also be used in place of hiring external consultants for specific needs e.g. preparations for a major UN event. These provide on the job learning that can be particularly helpful for national staff who have deal with the catch-22 of needing international experience needed to move to an international posting.
9. Find ways to reward learning by publicly acknowledging those who have completed learning activities (such as having them receive an award from the head of office at a staff meeting) or those who have contribute to sharing their knowledge and skills. There was also some discussion on the pros and cons of “name and shame” for those who don’t complete mandatory trainings, although I’m not personally in favour of this.
10. Network of learning managers – perhaps the most powerful way of sharing good ideas, learning opportunities or even just to get moral support is through networking between learning managers in different offices. Having access to experience and advice from other offices is an excellent way to improve learning whether by sharing templates and examples, or helping share resources or by providing feedback on potential courses or trainers. Perhaps the most valuable support though is in sharing advice on how to get management support and how to motivate learners.