• Skip to primary navigation
  • Skip to main content
  • Skip to primary sidebar

Connecting Researchers and Research

Sign in/Register
  • About
        • Our Mission
          • 2025 Vision
          • ORCID Trust
          • Annual Reports
        • Team
          • ORCID Board
          • ORCID Team
          • Work with Us!
        • Services
          • Member Portal
          • Annual data files
          • Member API
          • Public API
          • ORCID Registry
        • Governance
          • Bylaws
          • Board Elections
        • Policies
          • Privacy Policy
          • Dispute Procedures
          • Public Client Terms of Service
          • Open Source Project License
          • Public Data File Use Policy
          • Terms of Use
          • Brand Guidelines
  • For Researchers
        • Benefits for Researchers
        • Researcher FAQ
        • Video Tutorials
        • Sign in / Register
        • Get Help
  • Membership
        • Membership Benefits
          • Benefits for Funders
          • Benefits for Publishers
          • Benefits for Research Organizations
          • Benefits for Research Resources
        • Get Membership
        • Member List
        • ORCID Map
        • Membership Comparison
          • Basic Membership
          • Premium Membership
          • ORCID Consortia
        • ORCID Consortia
          • Consortia Agreement
          • Consortia Onboarding Checklist
          • Roles and Responsibilities of ORCID Consortia
  • Documentation
        • Features
          • Member Portal
          • Member API
          • Public API
          • ORCID Registry
          • Annual Data Files
        • Workflows
          • Journal Articles
          • Employment
          • Peer Review
          • Funder and Grants
          • View More
        • Integration Guide
          • Getting Started with Your Integration
          • Sandbox Testing Server
          • Registering a Member API Client
          • Integration and API FAQ
          • View More
        • API Tutorials
          • Get an Authenticated ORCID iD
          • Read Data on a Record
          • Add and Update Data on an ORCID record
          • Hands On with the ORCID API
          • View More
  • Resources
        • ORCID Community
        • Community Programs
          • Certified Service Providers
          • ORCID API Users Group
          • Historical Task Forces, Working Groups, and Steering Groups
        • Get Involved
          • Community Groups
          • Developers
          • Give Feedback
          • ORCID API Users Group
        • Member Resources
          • ORCID Enabled Systems
          • Publishers Open Letter
          • Funders Open Letter
          • Standard Member Agreement
          • Outreach Resources
          • Register a Sandbox API Client
          • Register a Production API Client
  • News & Events
        • News
          • ORCID News
          • Member News
          • Consortia News
          • Integration News
          • Blog
          • Release Notes
        • Events
          • Events Calendar
          • Webinars

Opening the Peer Review Process

October 1, 2015 By Michaela Torkar

Peer review is an important part of the scientific discourse and often significantly improves the quality and accuracy of publications. However, it’s usually hidden from view: at most journals, the review process is done behind the scenes, with anonymous reviewers, and only the end result – the final article – is available to readers. What happens when this process is opened up?

At F1000Research, articles are published before peer review once they have passed an internal quality check, which is focused on readability, ethical issues, data availability and other basic criteria. Invited peer review starts after that and is completely open, so readers can follow the process and discussion around the article: peer review reports are posted alongside the article as soon as they come in, together with the reviewer’s name. Authors can upload one or more revised versions of the article and directly respond to the reviewers, who in turn can provide more comments on the revisions.

Towards more transparency

The concept of open peer review is not new and can take many different forms: The BMJ started non-anonymous peer review as early as 1999. The medical BMC-series journals soon took it a step further by revealing the names of the reviewers to their authors during the peer review and posting the referees’ names and their reports on the articles that ended up published. Many other publishers have since responded to demands for more transparency and opened up the peer review, sometimes making it compulsory, sometimes optional for the referees to be named. 

There are often concerns that open peer review may result in reviewers being less critical than anonymous reviewers, although this is not what a Randomised Controlled Trial by the BMJ showed: reviewers who knew that their names would be given to the authors were be no more likely to recommend publication than reviewers who knew they could make their recommendation anonymously. Being asked to sign their referee report, especially if the review is public on the article page, may in fact encourage reviewers to be constructive and more objective, and reviewers may actually pay more attention to the quality of their review in an open peer review process.

We have certainly seen very critical, but constructive and thoughtful referees on F1000Research. In fact, some reviews end up longer than the article! When Michael McCarthy thought he saw a crucial error in an article he reviewed, he wanted to make sure that his critique was clear; he included a full analysis – including some software code – to get his point across. Because all referees’ reports in F1000Research are given a unique DOI, McCarthy’s analysis can now be cited in its own right.

Full credit for peer review

For reviewers, open review means that they can take credit for all the hard work they put into peer review and how they have helped improve the paper. Critically reading articles and writing a constructive report takes a lot of time and effort, but with traditional ‘closed’ peer review only the authors and editors will be aware of the referees’ contribution. Many researchers have started listing their activities as referees on Publons, where they can get endorsement from their peers for their peer review contributions.

Formal and standardised acknowledgement for peer review activities will also come from an important development this week: F1000Research and other partners of ORCID and the Consortia Advancing Standards in Research Administration Information (CASRAI) are starting to implement a ‘peer-review citation standard’ for collecting, storing, and exchanging peer review information. Researchers can now reference review activities (for journals, funders, conferences or institutions) on their ORCID record.  In F1000Research’s case referees will be able to post a citation of their report with a single click via a link we’ll provide in an email.

Post-publication peer review

By carrying out the formal peer review (by invited experts) openly after publication, the wider scientific community can greatly benefit from the discussion between referees and authors. Traditionally, articles are published without the review comments, and presented as the final word on the topic. Authors often complain that in order to get the article accepted, they have to yield to referees’ requests and interpretation with which they don’t necessarily fully agree. With post-publication open peer review, in the way operated by F1000Research, the debate between the authors and the invited experts forms an integral part of the publication. Where controversy arises, readers can decide for themselves which interpretation (if any!) they agree with – on the basis of the arguments presented and perhaps by taking into account the expertise of the authors and referees.   For example, in an F1000Research article about ‘self medication’ of bumblebees, eminent entomologist Marla Spivak adds her own opinion about the authors’ interpretation. With closed peer review, only the authors and editor would have seen her comments, whereas here, possible alternative explanations or limitations of the study can be seen (and cited) by readers too.

Problems with published (and usually peer reviewed) studies are now discussed on social media, but this form of ‘crowd-sourced’ peer review is often not linked with the actual study, leaving readers of the article often unaware for long periods of any controversies.  A group of bioinformaticians created a stir recently when they raised concerns on various social media channels about a specific PNAS paper that has come out of the mouse ENCODE project. In order to make the discussion part of the formal (PubMed-indexed) literature, they published a reanalysis in F1000Research. One of the referee reports was read by more than a thousand people, all in the context of the article (and the original ENCODE paper that was under scrutiny). The referee’s request for further explanations encouraged a lively discussion with readers, authors and referees commenting further on the article. These additional comments don’t form part of the official (invited) peer review process, which ultimately determines whether a paper is listed by PubMed and other indexing services, but they do help provide a more complete picture of the research.

Importantly, post-publication peer review enables publishing in versions and makes it possible for authors to respond (with a new version or by Commenting) not just to the referees but also to suggestions raised by readers who have left Comments on the actual article or if discussions spring up elsewhere.

Ultimately, post-publication peer review at F1000Research means that articles become ‘living’ documents of the ongoing research process and the scientific debate surrounding them. By making the reviewers’ reports public and integrating them in the actual article (they are also included in the PDF and on PubMed Central, for example), the peer review adds value that can be more widely appreciated while giving the reviewers the full credit they deserve for their contribution.

 

 

 

 

Blog

Filed Under: News

Primary Sidebar

Search

Sign up for blog updates

We will only use your email to notify you when we have new blog posts. You can unsubscribe at any time. See our Privacy Policy for more information.

Check your inbox or spam folder to confirm your subscription.

Recent Posts

  • 2020: A Look Back As We Venture Forward
  • New Integration – GIST
  • New Integration – University of Victoria
  • New Integration – Vidatum Technologies
  • New Integration – Mendel University in Brno

Blog Posts by Category

  • Consortia News (39)
  • Integration News (48)
  • Member News (30)
  • News (429)
  • ORCID News (192)
  • Release Notes (74)
ORCID logo

CC0 The text of this website is published under a CC0 license Images and marks are subject to copyright and trademark protection.

  • About ORCID
  • Privacy Policy
  • Terms of Use
  • Accessibility Statement
  • Contact us
  • Dispute procedures
  • Brand Guidelines
ORCID uses cookies to improve your experience and to help us understand how you use our websites. Learn more about how we use cookies. Dismiss