Measuring value in communities of practice
Last week we were lucky enough to have Etienne Wenger lead us in a workshop looking at measuring the value of and making the business case for communities of practice.
The background to this is that we introduced the concept of communities of practice into UNICEF less that two years ago. We’ve seen tremendous enthusiasm and take up from across the organization which has spread mainly as a grass roots initiative. But, we still lack the level of senior level sponsorship and support that we would like or possibly need for this to become a major strategic approach to how we collaborate and share knowledge across the organization.
One of the particular challenges facing many community initiatives in different kinds of organizations is measuring and demonstrating that the communities add value and measuring the value that they generate. The aid-development field is almost obsessed with “results-based management” and similar measurement approaches right now and so we need to be able to justify what we are doing in these terms – but also for ourselves to know how we are doing, and what we can improve.
Etienne Wenger (together with Beverly Trayner and Maarten de Laat) recently developed a value creation framework exactly to help assess and promote value creation in communities. It’s not published yet so I won’t share it here in its entirety- but I wanted to share some highlights of the framework and how we looked at in our recent workshop.
The paper defines several “cycles” of where value is created by communities. Briefly these are:
1. Immediate value – such as where someone gets a question answered, or finds information they need though a community.
2. Potential value – where the exchange of knowledge can potentially contribute to someone’s work but we don’t know yet what the outcome of this will be. An example is where someone finds an approach they learned about of interest and wants to try it out.
3. Applied value – where someone has applied knowledge they have gained through the community and have changed their work as a result.
4. Realized value – where someone has applied knowledge from their community to change how they do their work and this has had a measurable positive effect on the work’s success (according to whatever indicators of success are used to measure the activity concerned).
5. Reframing value – where the interactions of the community have caused the participants (and the organization) to re-examine its definition of success or its way of doing business and have changed the practice as a result.
Of course ideally you want to be able to go as far as you can through these cycles – but the community needs to be up and running for longer and you need to put more effort into measurement to determine the changes and the contribution of the community to them as you move through the cycles. We tried applying the cycles to our own communities and found that we were mostly still at the first few but with a few examples of the later cycles emerging. The participants were also able to see ways in which some of the work they were doing could be enhanced to help make it more impactful.
The other big element of the framework and our discussions was how do we collect the information/data we need to be able to measure this value through the different cycles. Here we looked briefly at quantitative measures, what kind exist, what they tell us and how to collect them – but the most interesting part was the use of “value-creation stories” to illustrate how value is created in practice. Etienne developed some draft templates for different types of community stories which we test drove during the workshop. These stories illustrate different facets of the communities and cut across the value creation cycles. They types of stories we looked at were:
- Community stories – about how the community has created value, either at individual or community wide level
- Leadership stories – personal narrative of someone leading a community
- Sponsorship stories – personal narrative of someone sponsoring a community or an initiative
- Stories of the initiative – stories about the community of practice initiative itself.
Of course those who prefer hard data might ask, how can stories be useful if they subjective. The answer to this is to collect multiple stories out of which you can get a more complete and convincing picture of the results of a community. Etienne referred to this as “systematic anecdotal evidence”.
We started to collect some of our own stories during the workshop which led to some very interesting conversations about what is working and what isn’t and showed how interviewing and storytelling ws a powerful way of getting at this information which might not otherwise be surfaced.
Our next exercise was to create “stories of the future” – i.e. stories of the potential we wanted to see from communities – done as a kind of visioning exercise for us to see where we want to go with individual communities as well as the communities initiative itself. This was very helpful in identifying the gaps between where we currently are and where we want to be. It highlighted in particular the challenges we have with high level sponsorship and the difference this could potentially make to our work.
There were many other thought provoking discussions around our communities work during the workshop including some interesting discussions around ho communities work within hierarchical organizations which I’ll come back to in future blog posts.
Our next steps will be to develop a measurement system for our community work, and to use the information we collect from this build a case for investing in our communities that builds on both the results achieved to date, and their future potential. It’s going to be a lot of work but I’m excited about the possibilities, and the other workshop participants were too.
Update: Bev Trayner has posted the full paper on her website. Here’s her blog post. Here’s a link to the full paper:
Promoting and assessing value creation in communities and networks: a conceptual framework