KM, M&E, the art and science of delivery

I was meeting with a KM team from another UN agency a couple of days ago when the conversation turned to two interesting and related questions

1. What is the relationship between knowledge management and monitoring and evaluation?

2. To what extent should the focus of knowledge management be about improving the use of academic or scientific knowledge in development work?

Many organizations, especially in the UN are linking monitoring and evaluation with knowledge management both in terms of content and in terms of organizational structure. This is both an opportunity and a challenge. On the one hand information from monitoring and evaluation processes is a critical input to knowledge management processes, and similarly knowledge management tools and techniques can help support better monitoring and evaluation. At the same time there are subtle differences between these approaches (see my past blog post comparing KM and evaluation for more details). A major difference is that monitoring and evaluation has a greater focus on accountability – is the project on track, were the outputs delivered, was the money spent well, did the project have the desired impact. Knowledge management focuses more on learning and reflection and on how to share what is learned with other projects. These are in fact very complementary, and they require some overlapping skills such as the ability to collect and analyze data and summarize and interpret it – but again knowledge management practitioners put a greater emphasis on “soft”  skills such as understanding human psychology and group dynamics, networking skills and skills in both interpersonal and mass communication. Another challenge is that if the KM team is seen by staff across the organization as part of the “results” team or performance monitoring team then this can mean that people are less likely to trust them with their stories of failures and setbacks in case that be “used against them” but a large part of learning requires candid reflection on positive and negative experiences in a safe environment.

In practical terms having M&E and KM sit and work together can be very effective to get a more complete picture of organizational knowledge and how to leverage it, but this only works if the KM people are allowed to function like KM people and their different role and skill set is respected rather than being seen as part of an overall monitoring and accountability system. Similarly KM people need to work closely with human resources, communications and technology teams across any organization so the structure and working methods need to allow for that (I hope to write a future blog post about the plusses and minuses of locating the KM function in different parts of an organizational structure including communication, staff development, IT, executive office, programmes etc. –unsurprisingly there is no “best” approach as each has it’s advantages and disadvantages).

Taking the second question – an increasing refrain heard in aid agency strategic plans, or government plans for that matter is that they need to be more “evidence-based”. On one level this is a no brainer – if you have knowledge about a problem, how it is caused and what strategies are effective in tackling it, then why wouldn’t you use it?

A more interesting question might be to look at why available evidence isn’t it being used, or what if any are the limitations of what you can determine from research that can be applied in addressing problems in the real world.

Research and scientific method is very powerful, and woefully underused in identifying what types of technical interventions work well in development and under what conditions. It makes sense to use experimental research techniques to determine which interventions are most effective and also to tweak their design to improve their efficiency both at a general level and at a local level to adapt them to context.

But in every project there a certain amount of “knowledge” that is needed to implement them successfully that isn’t easily measured through a scientific approach and which can’t be implemented in a standardized fashion across different contexts. (see an early blog of mine “The truth is out there, or maybe not” for more discussion on the limits of applicable scientific knowledge). The most important of these are politics, culture and personalities (i.e. the actual people involved in implementing the project or who are critical to its success). Dealing effectively with these issues requires a combination of local knowledge, experience, qualitative research and insights, peer assistance and advice and flexible adaptation. This is where knowledge management techniques such as communities of practice, peer-assists, after-action reports and even lessons learned databases and expert rosters come into their own. Similarly innovation tools such as human centred design and rapid prototyping can also be put to use to address the aspects of project design and implementation that don’t readily lend themselves to rigorous research, or for which standardized approaches can easily be designed.

Again, soft KM techniques, personal judgment and expertise and hard science should not be seen as competing approaches but as complementary ones. The challenge is figuring out how to combine them effectively, especially which approach to use and when, and what to do if they generate seemingly different conclusions. For me, this is an area we could do to think more about. Yes, to doing more and better research on development approaches, and yes to doing more to put the conclusions in the hands of decision makers and persuading to use them in their decisions – but at the same time we also need to think more about how to tackle the soft side of the “science of delivery” such as how do we adapt approaches to make them successful in a local context taking account of politics and power as well as culture and social norms – and how do we manage the people side of the project effectively, and how do we continually adapt our programmes to deal with the changing situation on the ground, as well as ensuring that we are constantly learning from each new experience and incorporating that learning into our future programmes. This is the “art-meets-science” of delivery where we still have a lot to learn.

Your ideas – better before they were famous (deal with it)

“Every great cause begins as a movement, becomes a business, and eventually degenerates into a racket.” – Eric Hoffer

If you are a die-hard fan of a band you are quite likely to say – yes, they are great, but their best work came before they were famous, before they “sold out”.

If you are an early adopter or promoter of a new idea –knowledge management, social media, cash transfers, mobile phones for development, innovation etc. etc. then you probably feel the same once everyone has jumped on the bandwagon.

The thing is, most new things of potential merit whether ideas, technologies, bands, fashions, political ideologies, you name it go through a similar cycle.

Generation of something new – Promotion by a small dedicated following who really “get it” and who like it because of its uniqueness, while everyone else is either critical, sceptical  or totally unaware of its existence, growing  buzz shared by a wider “early adopter group” (what Gladwell would call the “mavens”) – adoption and promotion of the idea to a more mainstream audience that uses its originality as a marketing tool but also smooths off the edges – co-option of the idea by the mainstream often including dumbing it down, and diluting it in order to make it more acceptable, and ensuring that a profit can be made from it – low quality clones are created – finally it either becomes so mainstream it is no longer noticed, or it jumps the shark and disappears from view.

So what happens when something moves from being an emerging idea to an everyday occurrence? One aspect is that the first movers and early adopters all complain about how dumbed down and commercialized their original idea has been, and how it isn’t any good any more, or how the late adopters don’t really understand it and are just following like sheep. Like a club that isn’t cool any more now that everyone goes there.

But here’s the thing. Unlike with a favourite hidden restaurant or eclectic band, if you do have a really great idea for technology, or development, or promotion of human rights or for participation – then your goal should be for word to be spread as widely as possible – not just to the cool kids. And if you want your idea to spread, then you also need to be prepared for it to be adapted and owned by others, for it to be “dumbed down” so it can be more ready accepted, and for it to be commercialized so that it can be financially sustainable. Your idea is no longer your idea – it is no longer pure – and it’s probably “less good” than it was – but the difference is it has now become accepted and widely used.

This recent article in Salon that has been doing the rounds about the culture of TED talks and the oversimplification and marketing of “creativity” is a fine example. While its true that the notion of creativity has been oversimplified, packaged and sold – it seems strange to me to be dismissive of the fact that the recognition and focus on the importance of creativity has never been stronger, even if the content has been watered down to make it accessible to a wider audience.

Another related critique is that ideas that are initially revolutionary become appropriated by the existing hierarchy and thus become tools of the status quo rather than tool for change (or as Billy Bragg famously put it “The revolution is only a t-shirt away”). This is no doubt true, but at the same time in taking on parts of a revolutionary idea, the status quo and balance of power also subtly changes – the change may be more evolutionary than revolutionary – but it is real nevertheless. And often simple ideas and technologies have quite revolutionary impacts, but not necessarily the ones that were expected, nor do these occur in a short or predictable timeframe.

So what can a change activist do when you see your great idea taken up and messed up by others?

Maybe you should let it go, and accept that in order for an idea to be successful it will need to be taken up and adapted by others for both better and for worse (and worse for you might be better for someone else), recognizing the adoption of others is in fact one of the best measures of the quality of the idea. Maybe you can help adapt the idea, and yes, even dumb it down or commercialize it yourself to help it spread (as well as possibly to make a living out of it). That way you can also help do your best to ensure that the most critical (to you) parts of your idea are preserved.

You can keep pushing forward to further refine and develop your idea in order to improve and evolve it so it keeps being ahead of the curve – so it remains revolutionary or leading edge while everyone else is moving to where you were 5 years ago. Just remember though that in 5 years time you want them still to be following behind you.

Or if you need to you can do something else. Create or promote another different idea and help develop it and make it more practical and popular.

In conclusion: Don’t be upset if everyone starts using and adapting your idea, and if they figure out how to make a buck out of it. But that doesn’t mean you have to give up striving for your ideas. It’s good to keep looking for the revolutionary idea that might change the world, but to change the world you might have to be prepared to give up ownership of your idea as well.

Can I help you? (maybe, maybe not)

I’m currently participating in a “UN Transformational Leadership course” which I’ve mentioned in past blogs. One of interesting self-discoveries I’ve made from this course is that sometimes we are the ones that create the barriers to change or to pursuing our big goals.

Through some introspection I realized that one of the biggest challenges I face is that I find it hard to say no when people ask me to help them. And the more I help people, the more people ask me to help, and the less focused I am on pursuing my own goals.

This desire to help others isn’t entirely altruistic. Like most people I want to feel valued, and like many I want recognition for what I do and for my expertise. I’m probably also a bit of a procrastinator. This has translated into me looking to help and advise others which has then translated into more and more external asks. Having a public blog and social media accounts and working in the field knowledge management also means that I get a lot of requests for help.

So what would be a good, healthy way of managing this, without stopping being helpful as at all? Here is a little framework I’m prototyping for myself. This time I’m asking for YOUR help to give me feedback and share your tips for dealing with “too many questions”.

To start I realized that my time is broadly divided into three major blocks 1. Work 2. Family time (as a husband and father of three) and 3. sleep. Those are all pretty important, and anything that doesn’t fit within those categories takes valuable time away from them, so I need a few good measures as to how to think about requests for my time. If I were a consultant, or worked for employer that allowed outside activities then an important category might be “do I get paid?” but I can’t get remunerated for what I do beyond my salary, and that doesn’t depend so much on how helpful I am.

So here are the questions I plan to use to screen requests for help:

1. Is it directly related to my job? Is it explicitly part of my workplan, my job description, or is it at least related to the broad purpose of my job. If not, does it at least contribute to the priorities of and mission of the UN?

2. Do I know you? How do I know you? – are we friends, colleagues or past collaborators? Is our current or future professional relationship likely to be mutually beneficial (even if not equal), have you helped me in the past?

3. Is your question interesting? Are you asking me something I’m curious about myself, or something that I’m passionate about either professionally or personally?

4. Is it something that I can easily answer? Is it related to my expertise? Easier questions are more likely to get answered than complex involved ones, but at the same time, very broad generic questions (tell me everything about KM) are less likely to be answered. It’s also not good to ask a question for something you could have easily researched yourself.

5. Will my input be useful and used?, and will I get any feedback on it? I can’t tell you the number of times I’ve been asked for inputs, never to hear what happened to them, or even what happened to the project they were requested for.

6. I hate to finish with this but — Is there something in it for me? Not money since I can’t take it. But will it help me develop my skills and knowledge? Will it help me in my current work? Will it help me make new connections? Will it help me develop my career or find my next job? Or at least do I get more than a thank you e-mail?

I’m going to try to run future queries through this filter and only take on those that score well on the above criteria. I’ll let you know how it goes. If I don’t respond to your e-mail don’t be offended, I want to help, I really do – but I also have to get stuff done.

 

P.S. Here is a sample of the types of questions I regularly get:

Can you help me get a job? Can you review my CV? Can you comment on my KM strategy?, Can you comment on my publication?, Can you tell me who in the UN (or elsewhere) is working on X? Can you tell me what software tool I should use for Y? Can you answer my (10 page) questionnaire? Can you tell me how to do Z knowledge management related task? Could you give me feedback/input on my project? Can you tell me where to find all the research on A? Could you help publicize my publication/project? Can you speak at our conference (and pay for your own travel)? [addendum since I wrote this bog: Can you meet me to check out/help me market my new must have KM related software]

In a future blog I’ll give my generic answers to some of these questions, so you can read those before asking me

Should you always try to pick the best investment in aid?

A recent twitter exchange on how to pick what aid projects to fund prompted me to think about whether or not it really is a good idea to invest all/most aid money in those interventions with the biggest proven return to investment.

Some organizations, such as the Copenhagen Consensus and GiveWell, have invested a lot of effort on reviewing research, expert opinion and feedback to help rate and identify the “best” investments” in development and the “best organizations” to which you might donate your money. Their laudable aim is to encourage a more rational approach to charitable giving and aid allocation in order to get the best results for the limited resources available.

While there are many data uncertainties in this type of calculation (not all types of interventions have been studied, and comparable information is not available on all organizations) this type of effort can be very valuable in terms of provoking discussion around effective approaches and in terms of helping guide individuals and organizations to invest in effective strategies.

So it might seem at first glance that any donor, whether governmental, corporate or individual would be well advised to review the available ratings and to invest most of not all of their money into the top rated activities and organizations. It probably comes as no surprise that I don’t agree! (if I did this would be a much shorter blog post).

While it’s true that the top rated interventions/organizations most likely deserve greater attention and funding than they currently get – there are also some good reasons why this shouldn’t be the only determinant of where the money goes. Here are a few reasons why:

1. Diminishing returns on investment – even though childhood vaccination and micronutrient supplementation might be great investments, there is only so much that can be invested in these before the large parts of the population have already been reached, and the added costs and complications of reaching the unreached mean they are no longer the “best” investments to make, at least in terms of efficiency.

2. Equity/human rights – making allocation decisions purely on the basis of cost efficiency is likely to lead to targeting interventions at the easy to reach, rather than at the most needy. Cost effective maybe – but certainly not fair. In the early days of the MDGs many countries tried to reach the goals by focusing on those parts of the population where it was easier to meet the targets overlooking those who were most marginalized and discriminated against because they were harder to help (the tyranny of averages) – and this is one of the reasons for the current discussions about inequality and equity in the context of the post-2015 goals, and why many organizations  are now looking at how to focus more on reducing inequality in their approaches.

3. Some of the most important problems faced in the developing world are not necessarily the most cost-effective to address nor do we always know how to measure them– at least yet. Part of this might be that for some complex problems such as conflict prevention or gender equality we don’t have a simple set of agreed interventions that can be easily replicated. That doesn’t mean we shouldn’t try to work on them. Quite often different “problems” are interrelated and may have shared underlying causes. While individually it might make sense to invest in an individual intervention, the long-term sustainablity of that intervention and “net development gain” for the country in question may well depend on attention to these underlying complex issues. For example proven health interventions are unlikely to be implemented successfully in situations of chronic instability and lack of security, while they are unlikely to be sustainable until efforts are made to address issues such as  health system capacity and governance, domestic financing and resource mobilization with a sound public financial system , or attention to public attitudes and behaviour (e.g. you can’t have an effective vaccination programme without public communication and attention to anti-vaccine rumours).

4. Doing what we know and what we care about – Expertise and financial resources for development are not fully fungible. People are more ready to contribute to some countries, interventions or organizations than others and make these decisions based on factor other than cost-effectiveness. People, and organizations are more likely to give to issues they care about (or that resonate politically) and naturally make choices based on their perceived importance of the issue itself not only the proposed interventions,. While micronutrients may be cheaper they may well attach a greater importance to addressing violence, HIV/AIDS or some other topic, because it touches them personally, they feel it is more critical or it easier to engage others about. another important element is expertise – if your country or organization (or you as an individual) know more about agricultural development than health then it makes sense that you will provide assistance in your area of comparative advantage regardless of whether it is the most cost-effective or sought after area of work since you will be able to do more good doing what you know well rather than doing something else you don’t know poorly.

5. Diversifying your portfolio – investors know that even if you have a few solid stocks it’s always good to diversify your portfolio to reduce risks if your pick suddenly does badly (e.g. new research finding your cost-efficiency measures were not so good after all, or that they had unintentional negative consequences). It also makes sense to spend some part of your resources on new innovative “start-up” approaches that are unproven, and which may even fail, in order to find new and better ways to support development. This requires a willingness to take risks on unproven but promising new approaches, and preferably on multiple diverse ones including some long shots – otherwise you might miss out on discovering whole new and potentially more cost-effective approaches.

6. What the beneficiary wants – last but not least: just as donors care about some issues more than others, so do beneficiaries. Developing country governments may disagree on priorities and need to operate within their own domestic political constraints. But also individual beneficiaries might also disagree with you about what is their most pressing need, even when you explain the evidence! Ultimately the aim of development assistance is to help individuals, communities and governments to be able to manage their own development and so they should have the last say on what is done to them, for them.

I’m all in favour of measuring cost effectiveness, continually looking at how to improve it, and using it more  in aid allocation decisions, but like any aid intervention – it’s no panacea. Often choices about what to fund are governed by subjective decisions about what is important, and always issues are interrelated and cannot be decided upon in isolation – so while I encourage the technicians to do their work, let’s not be fooled that they will or should be the ones making the decisions about where aid money will go.

Going social: 10 tips for bloggers

I spotted this blog post by Blythe Fraser (@b1ythe) on UNDP’s internal “UN Teamworks” social networking platform – I thought it had some great tips for bloggers which are also relevant outside of UNDP. She kindly agreed to let me include it as a guest post on “KM on a dollar a day”. Blythe is  the Online Communication Specialist in UNDP’s Bratislava Regional Centre. She curates UNDP’s “Voices from Eurasia” blog. 

Photo: Udo Herzog, CC

Just for the moment, let’s put aside why you should be blogging and look at the art of blogging itself.

After editing 555 of your posts on Voices of Eurasia, I learned a lot about your work, and am impressed by all of you. I would also like to share what I learned:

1. Just do it. Don’t overthink it. Overthinking leads to you writing a very long academic paper that details all of your work, your thoughts, upcoming projects, and predictions, lessons learned, other people’s thoughts, and all the creatures on Noah’s Ark – with footnotes included! All of this is great, but just not all at once. (You can do a series of posts like Alex on behavioural science.) You shouldn’t feel too much pressure, other than to just write it and send it to us 🙂

2. Shorter is better! You started sending in longer and longer posts, so I established an 800 word limit. The global UNDP blog advises 300 to 350 words. People don’t read content online the same way they do in print. The bottom line: less is more on the web.

3. Get to the gist – and fast. UNDP is full of people who are used to writing academic papers, and sometimes this is the default style of writing – an overview of the topic first, then the details, then finally the results and conclusions. On the web, you only have a few seconds to grab someone’s attention before they leave. To hook someone to start reading, or even to open your post, your title and first line must be engaging, provocative and clearly communicate the topic of your post. It also needs to have keywords for search engine optimization. (I found a great formula for writing headlines.)

4. It’s from your own perspective. It should be written in the first person. Use “I” and not “we.” This is the expected style of a blog post, and you are also blogging as yourself, an expert in your field, and not on behalf of the whole of UNDP. It also makes it easier to include your opinions, which makes the post more interesting. People have different perspectives, so your blog posts will reflect the diversity of your personality and style. (But people do expect clear and concise writing on the web, so don’t go too crazy. :))

5. The tone is familiar. Writing in the first person helps, but also try to be informal, like you’re at your kitchen table talking to your friend, or your grandma. 🙂

6. Ask at least one question somewhere in your post. Your title can be in the form of a question, you can have several questions throughout, and the end of the post is a perfect place for a question. Whatever you do, don’t conclude your post! Many people make a beautiful argument and tie it up neatly with a ribbon. Done and done! Leave people room to engage with your post. Questions are a great way to encourage commenting. And be prepared to respond and talk to people who leave comments.

7. Understand that your blog post is part of a bigger conversation. Linking to other bloggers, articles and research shows that you are part of the conversation. Including links also allows people to dig deeper into the subject. This can include links to your own content (such as project pages, other posts or updates). Nilgun and Alex’s post is a good example of how to use links. Your post can be a response to someone else’s post (like Alisher’s response to Michael Clemens), and remember to be social and comment on other people’s posts.

8. Sharing is as important as writing your posts. Communications people invest a lot of time and energy in marketing your blog posts on social media. Sometimes this means others pick up your posts and publish them on their own sites. But we need your firepower too. That’s why we’re always trying to get you on social media and to share with your existing networks and partners. This is all part of opening up, so we can all listen and learn from people and other organizations.

9. Photos, videos, and multimedia make your post better. There is an art and science to using images on the web, but the bottom line is that articles with images get more views. It’s also difficult to market your posts on social media without them. Luckily our country offices have a lot of great photos, but we also need to keep issues of copyright in mind.

10. You’ve heard this one before, but it bears repeating: no acronyms and avoid the jargon.

You might also enjoy 11 Smart Tips for Brilliant Writing.