Peer review is an important part of the scientific discourse and often significantly improves the quality and accuracy of publications. However, it’s usually hidden from view: at most journals, the review process is done behind the scenes, with anonymous reviewers, and only the end result – the final article – is available to readers. What happens when this process is opened up?
At F1000Research, articles are published before peer review once they have passed an internal quality check, which is focused on readability, ethical issues, data availability and other basic criteria. Invited peer review starts after that and is completely open, so readers can follow the process and discussion around the article: peer review reports are posted alongside the article as soon as they come in, together with the reviewer’s name. Authors can upload one or more revised versions of the article and directly respond to the reviewers, who in turn can provide more comments on the revisions.
Towards more transparency
The concept of open peer review is not new and can take many different forms: The BMJ started non-anonymous peer review as early as 1999. The medical BMC-series journals soon took it a step further by revealing the names of the reviewers to their authors during the peer review and posting the referees’ names and their reports on the articles that ended up published. Many other publishers have since responded to demands for more transparency and opened up the peer review, sometimes making it compulsory, sometimes optional for the referees to be named.
There are often concerns that open peer review may result in reviewers being less critical than anonymous reviewers, although this is not what a Randomised Controlled Trial by the BMJ showed: reviewers who knew that their names would be given to the authors were be no more likely to recommend publication than reviewers who knew they could make their recommendation anonymously. Being asked to sign their referee report, especially if the review is public on the article page, may in fact encourage reviewers to be constructive and more objective, and reviewers may actually pay more attention to the quality of their review in an open peer review process.
We have certainly seen very critical, but constructive and thoughtful referees on F1000Research. In fact, some reviews end up longer than the article! When Michael McCarthy thought he saw a crucial error in an article he reviewed, he wanted to make sure that his critique was clear; he included a full analysis – including some software code – to get his point across. Because all referees’ reports in F1000Research are given a unique DOI, McCarthy’s analysis can now be cited in its own right.
Full credit for peer review
For reviewers, open review means that they can take credit for all the hard work they put into peer review and how they have helped improve the paper. Critically reading articles and writing a constructive report takes a lot of time and effort, but with traditional ‘closed’ peer review only the authors and editors will be aware of the referees’ contribution. Many researchers have started listing their activities as referees on Publons, where they can get endorsement from their peers for their peer review contributions.
Formal and standardised acknowledgement for peer review activities will also come from an important development this week: F1000Research and other partners of ORCID and the Consortia Advancing Standards in Research Administration Information (CASRAI) are starting to implement a ‘peer-review citation standard’ for collecting, storing, and exchanging peer review information. Researchers can now reference review activities (for journals, funders, conferences or institutions) on their ORCID record. In F1000Research’s case referees will be able to post a citation of their report with a single click via a link we’ll provide in an email.
Post-publication peer review
By carrying out the formal peer review (by invited experts) openly after publication, the wider scientific community can greatly benefit from the discussion between referees and authors. Traditionally, articles are published without the review comments, and presented as the final word on the topic. Authors often complain that in order to get the article accepted, they have to yield to referees’ requests and interpretation with which they don’t necessarily fully agree. With post-publication open peer review, in the way operated by F1000Research, the debate between the authors and the invited experts forms an integral part of the publication. Where controversy arises, readers can decide for themselves which interpretation (if any!) they agree with – on the basis of the arguments presented and perhaps by taking into account the expertise of the authors and referees. For example, in an F1000Research article about ‘self medication’ of bumblebees, eminent entomologist Marla Spivak adds her own opinion about the authors’ interpretation. With closed peer review, only the authors and editor would have seen her comments, whereas here, possible alternative explanations or limitations of the study can be seen (and cited) by readers too.
Problems with published (and usually peer reviewed) studies are now discussed on social media, but this form of ‘crowd-sourced’ peer review is often not linked with the actual study, leaving readers of the article often unaware for long periods of any controversies. A group of bioinformaticians created a stir recently when they raised concerns on various social media channels about a specific PNAS paper that has come out of the mouse ENCODE project. In order to make the discussion part of the formal (PubMed-indexed) literature, they published a reanalysis in F1000Research. One of the referee reports was read by more than a thousand people, all in the context of the article (and the original ENCODE paper that was under scrutiny). The referee’s request for further explanations encouraged a lively discussion with readers, authors and referees commenting further on the article. These additional comments don’t form part of the official (invited) peer review process, which ultimately determines whether a paper is listed by PubMed and other indexing services, but they do help provide a more complete picture of the research.
Importantly, post-publication peer review enables publishing in versions and makes it possible for authors to respond (with a new version or by Commenting) not just to the referees but also to suggestions raised by readers who have left Comments on the actual article or if discussions spring up elsewhere.
Ultimately, post-publication peer review at F1000Research means that articles become ‘living’ documents of the ongoing research process and the scientific debate surrounding them. By making the reviewers’ reports public and integrating them in the actual article (they are also included in the PDF and on PubMed Central, for example), the peer review adds value that can be more widely appreciated while giving the reviewers the full credit they deserve for their contribution.