Peer-to-peer review

I’m in my tenth year as an academic, which means that my view of how things worked in the world of research pre-2002 is based purely on output — that is, the papers and books I’ve read that were published before my time. As a result, I really don’t know if the phenomenon I’ve been noticing lately is new or not, but man, there are unpublished papers everywhere. Conference papers, working papers, executive reports, etc., are posted all over scholars’ web sites, their profiles, departmental sites, and then frequently logged in Google Scholar and/or touted in news releases. They may or may not make an academic splash, though there are certainly a number of recent conference papers in political science and mass comm that used the web to gain a lot of notice, but news organizations and commentators don’t operate with the same filters as academics do. This University of Washington study (including one author who is a former colleague of mine at Wisconsin) is a good example. It was linked by Talking Points Memo, in a way that seems fairly typical of how unpublished work is disseminated through blogs and other online political commentary. Its conclusion makes the paper appealing, as it seems to inject some empirical evidence into the debate over whether “Twitter revolutions” really have anything to do with Twitter, which has already gone back and forth in the press.

At the same time, it’s a study that hasn’t gone through the peer review process. Maybe it’s applying theory in an unconventional way, or maybe there’s something odd about the data, or maybe it’s exactly right (and I should note I’m only highlighting this study because it’s the most recent example I’ve come across — there are dozens more). But the widespread availability of unreviewed research presents a twist to the science news cycle model, which relies on carefully considered and reviewed conclusions to be reined in. If we’re inserting more and more research from earlier stages of the process into our discussions of public policy, current events, etc., we could see major challenges to the peer review model.

Share this post:

    Tags: , ,

    • Candy

      You make a very good point. This also is happening in non-academic settings, as nurse researchers are forgoing the arduous task of submitting, rewriting, resubmitting and being rejected by general and specialty journals. They’re also turning to websites to publish data that may or may not be valid and increasingly using health system PR or PR bots to disseminate their research. In this case, likely as in others, there is more research being done than there is space for it to be published, with publication schedules set months in advance. This is especially true for young nurse researchers, who want to share what they’ve learned with other nurses and see the peer review process as unnecessarily taxing, or simply unnecessary. This has been highlighted by the exponential growth in the number of posters submitted to specialty nursing conferences. While some are rehashes of older work, many purport to be actual research when in fact they are data collected from a small number of patients over a short period of time and not balanced against any other data.

      Who’s at fault? The peer-reviewed journals with their unwavering requirements that appeal to fewer and fewer young nurses? The nurses themselves who submit something and call it research? Actual nurse researchers who haven’t made attempts to embrace either the younger nurses or the new technology? It’s anybody’s call, but I fear the evidence in evidence-based nursing is the real loser, and patient care may suffer as a result.