KM on a dollar a day

Musing on knowledge management, aid and development with limited resources

World Bank TL;DR

with 10 comments

I know I’m a bit late commenting on the discussion of a recent World Bank paper that found that a third of all World Bank reports are never downloaded (I just switched jobs), but I’m fascinated by some of the challenges of use of evidence that it brings to light.

At least some of the non-blogged conversation that I’ve heard about this  seems to be i) what a scandal, think of all that waste producing research that no-one reads and ii) aren’t we glad that we are so much better than that.


But not so fast…

 

First of all it’s commendable that the Bank did this analysis and was then willing to make the results public. Many organizations have not asked themselves the same questions and done research to answer them, and I imagine of those that have, few would be so public about the findings.
Secondly I’m not that surprised by the results. I suspect this is common for a lot of potentially important (and not so important) research and policy papers, whether by done by development organizations, or by individual researchers.
Yet the implications of this are far worse … imagine if only 70% of papers are ever downloaded – how many are then actually read at all, and of these how many are fully read and understood, and for those, what proportion have caused the reader to reflect and change their point of view or more importantly have influenced their actions.
So what could be about this?
I wanted to briefly touch on three areas where we can all do better in terms of getting our reports read and use.:


1. Packaging
Presenting research in long PDFs in black and white with dense text is unreadable fonts without a decent summary is always a great way to hide potentially useful information.
By contrast – spending a bit of time to develop a good summary that highlights the key findings and their relevance in non-technical language is an invaluable investment since people usually look at the summary first before deciding whether to read further. Putting the summary on it’s own web page rather as well as in the PDF itself also increases the chance it will be read. It may be even better to produce a separate document with all the key take-aways for advocacy and decision-making with a link back to the technical sources for those that need them.
It may seem more trivial but another important factor in whether or not something gets read is the design and layout. Something with attractive layout and colour as well as helpful diagrammes and appealing images is a lot more likely to be read. Making the document in easier to browse formats  such as  HTML or e-reader versions can also help. Using internal navigation aids such as meaningful section headings and a table of contents makes it much easier for people to scan a document to find what interests them (you didn’t imagine that someone would really read it all from beginning to end did you?). Finally if you don’t have a good catchy title that gives an idea of what the reader can gain from the document then it might not be opened at all.


2. Dissemination
Uploading a PDF on a website is not enough to get people to notice it let along download it. Getting research into the hands of people who can use it requires targeted dissemination.  It’s useful to think first about who are intended audiences of a paper (too often there aren’t any) or at least who ought to be interested in the results.
One obvious way to do this is to draw up a list of people to whom the list should be disseminated and then send it to them. But this requires some work on building up a good mailing list. It also requires a well written message that will attract the attention of the reader among the many messages they receive.
Although targeted dissemination is probably most effective, it’s also worth spending some time making it easier for people to find your stuff by chance too.   This means having papers easily findable on your website by having a clear place to find your research including having content tagged by relevant topic terms and ensuring your pages are search engine optimized with internal and external search.
Another important strategy is to get other people to do your marketing for you. Asking someone else to recommend a paper and forward it to their networks can be very effective since people are often more influenced by personal recommendations than by corporate dissemination. And at the very least you will have encouraged at least one more person to look at your report, even if they didn’t pass it on.
Similarly social media can be another effective way of spreading your work through social media such as through blogging about the main findings and their relevance or encouraging others to do so, or encouraging discussion via twitter or in online discussion forums (one wag suggested that the bank producing and blogging about a paper showing no-one downloads many of their reports was in fact an elaborate ruse to boost readership).
Another key tactic is to link a paper to events, whether its relevance to current hot topics of debate or linking it to a specific event (an academic conference, a UN day etc.). This should be done not only when a piece of work is produced but continually as new relevant opportunities arise.


3. Relevance – probably most important is whether the research actually speaks to an actionable need. Does the research design and results help lead to decision-making or does it answer a relevant policy question? While many things are interesting to know, it’s useful to reflect on how a piece of research or policy work is intended be used by whom and what we might expect them to do with it before the work is started, let alone before the paper is written. Ideally any work is driven by a specific knowledge need, and one that has actionable consequences.
A final comment is that monitoring the impact of the research programme, while extremely challenging is also extremely important. Large sums of money are spent on commissioning and publishing research, and in times of scarce aid budgets there is an increasing need to demonstrate the results of this. This monitoring should not only look at how many times a paper is downloaded or linked, but also try to track how it is contributing to policy debate and improved programming (I couldn’t finish a blog about research without saying more research needed!).


In summary – while research needs to be rigorous that is not enough to guarantee it will be useful. It also needs to be relevant, well presented and well disseminated – and we need to measure not just the production but also the use.

Written by Ian Thorpe

May 22, 2014 at 12:11 pm

10 Responses

Subscribe to comments with RSS.

  1. Thanks a lot!
    Sent from my iPhone

    Tardi, Rachele

    May 22, 2014 at 12:49 pm

  2. […] advocacy community about the value and utility of these types of policy reports (see here, here and here). Many representatives of research and advocacy organisations focused on international development […]

  3. Great ideas, indeed… I liked your thoughts, but I missed a picture in the post! (Well, it´s so nice written that I don´t need it, but it surely help a bit… ;-) )

  4. Good points. In fact, the research only tracked downloads from a single central server, when often the reports are hosted in multiple places even within the Bank. For example, documents posted on country office websites, and linked to from those sites, would not be counted at all, nor would hard copies. In an assessment we did of one of our studies, the report resided on six different servers within the Bank (plus many outside the Bank), and nearly all of the downloads in the first year were from the country office server, not the central server covered in the research paper. (In fact, we did make a special site, along the lines recommended in the post above, and the site also includes the assessment of impact I mentioned above. http://www.worldbank.org/vn/vdr2010 )

    The choice of the central server (think of it as the big file cabinet in the back room where everything gets stored) rather than, say, the country office server (the display case in the front of the shop) made sense for the research purpose at hand which was to examine the correlates of downloads and to focus on those that are “widely read”. Tracking downloads from every server in a systematic way would be nearly impossible. Unfortunately, rather than reporting on the interesting findings about reports that are widely read, the “one third never downloaded” figure got picked up, with reactions along the lines you describe in your post. The researchers made clear that they were only looking at one server, however, so “never” cannot be claimed in any universal sense.

    That being said, we can certainly do a lot better at disseminating, and the post above provides some good advice on how to do that.

  5. Very interesting! Recently, I think about ‘packaging’ a lot, and prioritizing the information we want to deliver.

    Eunwoo Kim

    May 25, 2014 at 10:07 pm

  6. […] I know I'm a bit late commenting on the discussion of a recent World Bank paper that found that a third of all World Bank reports are never downloaded (I just switched jobs), but I'm fascinated by …  […]

  7. […] I know I'm a bit late commenting on the discussion of a recent World Bank paper that found that a third of all World Bank reports are never downloaded (I just switched jobs), but I'm fascinated by …  […]

  8. Hi Ian, good observations. However, I think that especially organizations similar to the World Bank (I’m talking UN organizations in general) are quite aware that the picture probably looks similarly bleak, if not worse, on their side, but very few actually have the courage to look at their stats, or actually share them! So you’re right, we have to give the WB some credit for being open and honest about it, and triggering a long overdue debate. Was planning to write my own blog post reflection on this, hope I’ll still get to it ;)

    Johannes

    May 29, 2014 at 10:47 am

  9. And here are my two cents to this debate. Thanks for triggering me to finalize my post :) http://jschunter.blogspot.com/2014/05/rethinking-knowledge-products-after-pdf.html

    Johannes

    May 29, 2014 at 5:07 pm

  10. Most of us are overwhelmed with information so we develop filters to scan out most information that comes across our desk. Email gets skimmed not read. People don’t download PDFs because it means a big time commitment and they may not see the value in doing so. Once opened I agree it’s important to list the main points very quickly

    Just because you make a report downloadable doesn’t guarantee that they will open it. You have to engage your intended audience first to demonstrate the value to them. Then it will be downloaded. here are some possible steps

    1. Know your intended audience. What are they looking for, their needs, challenges that are impacting them?
    2. Make the title more interesting and engaging and descriptive of the overall theme that aligns with their needs
    3. Use internal social collaboration tools to raise awareness and sell the idea of how important this article is – drip out small bits of information to entice people t want more
    4. Send it to a few key influencers and get their commitment to review it and send it directly to others who should be reading it

    There are other ideas. Hope this helps

    Wayne Tarken

    June 6, 2014 at 9:59 am


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 13,774 other followers

%d bloggers like this: