I know I’m a bit late commenting on the discussion of a recent World Bank paper that found that a third of all World Bank reports are never downloaded (I just switched jobs), but I’m fascinated by some of the challenges of use of evidence that it brings to light.
At least some of the non-blogged conversation that I’ve heard about this seems to be i) what a scandal, think of all that waste producing research that no-one reads and ii) aren’t we glad that we are so much better than that.
But not so fast…
First of all it’s commendable that the Bank did this analysis and was then willing to make the results public. Many organizations have not asked themselves the same questions and done research to answer them, and I imagine of those that have, few would be so public about the findings.
Secondly I’m not that surprised by the results. I suspect this is common for a lot of potentially important (and not so important) research and policy papers, whether by done by development organizations, or by individual researchers.
Yet the implications of this are far worse … imagine if only 70% of papers are ever downloaded – how many are then actually read at all, and of these how many are fully read and understood, and for those, what proportion have caused the reader to reflect and change their point of view or more importantly have influenced their actions.
So what could be about this?
I wanted to briefly touch on three areas where we can all do better in terms of getting our reports read and use.:
Presenting research in long PDFs in black and white with dense text is unreadable fonts without a decent summary is always a great way to hide potentially useful information.
By contrast – spending a bit of time to develop a good summary that highlights the key findings and their relevance in non-technical language is an invaluable investment since people usually look at the summary first before deciding whether to read further. Putting the summary on its own web page as well as in the PDF itself also increases the chance it will be read. It may be even better to produce a separate document with all the key take-aways for advocacy and decision-making with a link back to the technical sources for those that need them.
It may seem more trivial but another important factor in whether or not something gets read is the design and layout. Something with attractive layout and colour as well as helpful diagrammes and appealing images is a lot more likely to be read. Making the document in easier to browse formats such as HTML or e-reader versions can also help. Using internal navigation aids such as meaningful section headings and a table of contents makes it much easier for people to scan a document to find what interests them (you didn’t imagine that someone would really read it all from beginning to end did you?). Finally if you don’t have a good catchy title that gives an idea of what the reader can gain from the document then it might not be opened at all.
Uploading a PDF on a website is not enough to get people to notice it let along download it. Getting research into the hands of people who can use it requires targeted dissemination. It’s useful to think first about who are intended audiences of a paper (too often there aren’t any) or at least who ought to be interested in the results.
One obvious way to do this is to draw up a list of people to whom the list should be disseminated and then send it to them. But this requires some work on building up a good mailing list. It also requires a well written message that will attract the attention of the reader among the many messages they receive.
Although targeted dissemination is probably most effective, it’s also worth spending some time making it easier for people to find your stuff by chance too. This means having papers easily findable on your website by having a clear place to find your research including having content tagged by relevant topic terms and ensuring your pages are search engine optimized with internal and external search.
Another important strategy is to get other people to do your marketing for you. Asking someone else to recommend a paper and forward it to their networks can be very effective since people are often more influenced by personal recommendations than by corporate dissemination. And at the very least you will have encouraged at least one more person to look at your report, even if they didn’t pass it on.
Similarly social media can be another effective way of spreading your work through social media such as through blogging about the main findings and their relevance or encouraging others to do so, or encouraging discussion via twitter or in online discussion forums (one wag suggested that the bank producing and blogging about a paper showing no-one downloads many of their reports was in fact an elaborate ruse to boost readership).
Another key tactic is to link a paper to events, whether its relevance to current hot topics of debate or linking it to a specific event (an academic conference, a UN day etc.). This should be done not only when a piece of work is produced but continually as new relevant opportunities arise.
3. Relevance – probably most important is whether the research actually speaks to an actionable need. Does the research design and results help lead to decision-making or does it answer a relevant policy question? While many things are interesting to know, it’s useful to reflect on how a piece of research or policy work is intended be used, by whom, and what we might expect them to do with it. Ideally this is done before the work is started, let alone before the paper is written. Ideally any work is driven by a specific knowledge need, and one that has actionable consequences.
A final comment is that monitoring the impact of the research programme, while extremely challenging is also extremely important. Large sums of money are spent on commissioning and publishing research, and in times of scarce aid budgets there is an increasing need to demonstrate the results of this. This monitoring should not only look at how many times a paper is downloaded or linked, but also try to track how it is contributing to policy debate and improved programming (I couldn’t finish a blog about research without saying more research needed!).
In summary – while research needs to be rigorous, that is not enough to guarantee it will be useful. It also needs to be relevant, well presented and well disseminated – and we need to measure not just the production but also the use.