KM on a dollar a day

Musing on knowledge management, aid and development with limited resources

Measuring value in communities of practice

with 12 comments

Last week we were lucky enough to have Etienne Wenger lead us in a workshop looking at measuring the value of and making the  business case for communities of practice.

The background to this is that we introduced the concept of communities of practice into UNICEF less that two years ago. We’ve seen tremendous enthusiasm and take up from across the organization which has spread mainly as a grass roots initiative. But, we still lack the level of senior level sponsorship and support that we would like or possibly need  for this to become a major strategic approach to how we collaborate and share knowledge across the organization.

One of the particular challenges facing many community initiatives in different kinds of organizations is measuring and demonstrating that the communities add value and measuring the value that they generate. The aid-development field is almost obsessed with “results-based management” and similar measurement approaches right now and so we need to be able to justify what we are doing in these terms – but also for ourselves to know how we are doing, and what we can improve.

Etienne Wenger (together with Beverly Trayner and Maarten de Laat)  recently developed a value creation framework exactly to help assess and promote value creation in communities. It’s not published yet so I won’t share it here in its entirety- but I wanted to share some highlights of the framework and how we looked at in our recent workshop.

The paper defines several “cycles” of where value is created by communities. Briefly these are:

1.  Immediate value – such as where someone gets a question answered, or finds information they need though a community.

2. Potential value – where the exchange of knowledge can potentially contribute to someone’s work but we don’t know yet what the outcome of this will be. An example is where someone finds an approach they learned about of interest and wants to try it out.

3. Applied value – where someone has applied knowledge they have gained through the community and have changed their work as a result.

4. Realized value – where someone has applied knowledge from their community to change how they do their work and this has had a measurable positive effect on the work’s success (according to whatever indicators of success are used to measure the activity concerned).

5. Reframing value – where the interactions of the community have caused the participants (and the organization) to re-examine its definition of success or its way of doing business and have changed the practice as a result.

Of course ideally you want to be able to go as far as you can through these cycles – but the community needs to be up and running for longer and you need to put more effort into measurement to determine the changes and the contribution of the community to them as you move through the cycles. We tried applying the cycles to our own communities and found that we were mostly still at the first few but with a few examples of the later cycles emerging. The participants were also able to see ways in which some of the work they were doing could be enhanced to help make it more impactful.

The other big element of the framework and our discussions was how do we collect the information/data we need to be able to measure this value through the different cycles. Here we looked briefly at quantitative measures, what kind exist, what they tell us and how to collect them – but the most interesting part was the use of “value-creation stories” to illustrate how value is created in practice. Etienne developed some draft templates for different types of community stories which we test drove during the workshop. These stories illustrate different facets of the communities and cut across the value creation cycles. They types of stories we looked at were:

  • Community stories – about how the community has created value, either at individual or community wide level
  • Leadership stories – personal narrative of someone leading a community
  • Sponsorship stories – personal narrative of someone sponsoring a community or an initiative
  • Stories of the initiative – stories about the community of practice initiative itself.

Of course those who prefer hard data might ask, how can stories be useful if they subjective. The answer to this is to collect multiple stories out of which you can get a more complete and convincing picture of the results of a community. Etienne referred to this as “systematic anecdotal evidence”.

We started to collect some of our own stories during the workshop which led to some very interesting conversations about what is working and what isn’t and showed how interviewing and storytelling ws a powerful way of getting at this information which might not otherwise be surfaced.

Our next exercise was to create “stories of the future” – i.e. stories of the potential we wanted to see from communities – done as a kind of visioning exercise for us to see where we want to go with individual communities as well as the communities initiative itself. This was very helpful in identifying the gaps between where we currently are and where we want to be. It highlighted in particular the challenges we have with high level sponsorship and the difference this could potentially make to our work.

There were many other thought provoking  discussions around our communities work during the workshop including some interesting discussions around ho communities work within hierarchical organizations which I’ll come back to in future blog posts.

Our next steps will be to develop a measurement system for our community work, and to use the information we collect from this build a case for investing in our communities that builds on both the results achieved to date, and their future potential. It’s going to be a lot of work but I’m excited about the possibilities, and the other workshop participants were too.

Update: Bev Trayner has posted the full paper on her website. Here’s her blog post. Here’s a link to the full paper:
Promoting and assessing value creation in communities and networks: a conceptual framework

Written by Ian Thorpe

May 5, 2011 at 8:50 am

12 Responses

Subscribe to comments with RSS.

  1. I appreciate the list of five cycles where value is created. Plus, anytime we can get a better handle on measuring the value of initiatives, the better off we are when answering that seemingly inevitable question from management. Still, there is something very dissatisfying about the notion that there are those who would question the value of participation in a community of practice without having some agreed upon measurement associated with it. Are we so obsessed with ROI that we can’t even agree on that without hard data in front of us? It seems like common sense would indicate that there is value in people joined by a common interest communicating with each other about a subject and learning from their shared experiences and insights. But, if common sense really was common, more people would have it.

    Jeff Ross

    May 7, 2011 at 9:01 am

  2. Thanks for sharing Ian! I think distinguishing the different types of value is valuable, though I don’t see the difference between applied and realized value?

    I guess you know our paper for IKM? we also interviewed Wenger. What I liked was: measure what you can measure, make sure you have enough story so that you understand the figure and make sure you discuss the findings to interpret the consquences.

    Joitske Hulsebosch

    May 11, 2011 at 6:13 am

    • Hi Joitske
      I probably didn’t explain well the distinction between applied and realized value.
      Applied value is when you apply something you learn from the community in your work.
      Realized value is when you have applied it and there is a measurable improvement in your work as a result.
      So in the first case you have decided to use something because it seems promising but you don’t necessarily know yet if it actually works better in practice. Does that make more sense?

      Ian Thorpe

      May 11, 2011 at 6:19 am

      • Hi Ian, I understand the difference but wonder if it is so practical to distinguish between the two. In any case when I use some idea from CoP (eg. I used wordle in a certain way in a training- idea picked up in a community) it is definitely with the idea that it will improve my work – proven or not.

        Joitske Hulsebosch

        May 11, 2011 at 8:04 am

      • @joitske I think the distinction is to do with time, and the difference between potential and reality. It takes time for the actual benefit of applying something you lean in a community to manifest itself visibly and sometimes it might not actually be of benefit at all (except that you might learn from it and be able to feed that back to the community).

        I also think it’s stronger to say – I adapted something from the community and saved $x or x amount of time than to say I’m trying out something I got from the community because I believe it has the potential to save time and money. The first is much more concrete, so much more convincing.

        Ian Thorpe

        May 11, 2011 at 10:11 am

  3. Great input, thank you. I have experienced that frustrating feeling of spending a lot of time and energy “justifying” the use of something. In my case, it was the use of democratic dialogue for change processes. And I can see the same coming related to CoPs.
    It seems that we have to work on this in order to get support from senior level and from sponsors/donors. I wonder if someone has applied the Theory of Change to this. I guess CoPs and networking have a strong influence on the “assumptions” in planning processes (according to ToC).
    Saludos, Anaí

    Anai Linares

    May 11, 2011 at 2:55 pm

    • HI Anai and Ian,

      That’s something that came up strongly in our interviews: people within development thought it was very important and normal to measure and justify. People from business felt you have to trust people’s intuition and observations. If managers and leaders feel the CoP is doing well, it’s fine. Why waste time and efforts measuring it? One person even said that evaluations are always mentioned when people want to end the CoP. Hence hard to win at that time when sponsorship has started doubting… Gaining informal support may be very important.

      Joitske Hulsebosch

      May 11, 2011 at 3:08 pm

  4. […] best use it we often end up hiring external c onsultants. Use of tools such as expertise rosters, communities of practice and peer assists can make it much easier to find and make use of our in-house experts (see […]

  5. […] created value in practice (this seems very akin to the idea in Wenger’s and De Laat’s value creation framework) as these benefits can be both large, and through use of a storytelling approach quite […]

  6. […] the systematic collection of stories to help measure the value of communities of practice – see this blog. This was based on an approach developed by Etienne Wenger and Beverly Trayner. Unfortunately I […]

  7. […] background-position: 50% 0px; background-color:#222222; background-repeat : no-repeat; } (via @Kaplan_Univ) – Today, 3:46 […]

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: