Summarizing ORCID Record Data to Help Maintain Integrity in Scholarly Publishing

Submission of suspected fake research papers has been a growing problem for some time, and shows no sign of slowing down. This growing problem undermines the integrity of the scholarly communication system and also puts enormous pressure on many stakeholders in the process, including editors of scholarly publications as well as those tasked with investigating academic misconduct at publishers, research institutions, and funders. Many efforts are currently underway across the industry to help ensure journal article submissions meet basic research integrity criteria.

While there is no one approach that is going to entirely solve the problem, ORCID believes that the validated information in ORCID records about the author has an important role to play.

While we carefully record the provenance of each item in an ORCID record, i.e. who made the assertion and when, currently it can be quite time-consuming for editors to hunt through this information to discover and assess the relevance of each piece of data to their integrity checks.

After talking with a number of our members, researchers, and others across the community, we believe that this process could be made easier and quicker by providing an easy-to-understand summary of the trust markers in each record, making it easier for editors to identify submissions that need further follow up to validate the bona fides of the authors. Read on to learn more about how we worked with publishers to develop these new record summaries, which we are piloting next month.

Staying ahead of paper mill problem

Recently, the Committee on Publication Ethics (COPE) and STM undertook a study to understand the scale of the problem of paper mills. Paper mills research report and recommendations highlights that “continued investment in tools and systems to detect potential paper mill manuscripts at submission, with consistent and shared guidance in the use of these tools” is required to tackle the problem.

The information found in ORCID records can be used as one such tool, but it’s not always obvious how to easily find or interpret the relevant data. Many users don’t even realize we have provenance information for every item in a researcher’s record that clearly states whether it was added by the researcher themselves or by one of our member organizations, and what that means in terms of the author’s scholarly bona fides. It’s easy to assume that only ORCID records with lots of data are useful, but even sparsely populated ones have value if you know what data you’re looking for and it’s presented in a usable format.

Community problems require community solutions

When we realized that there is the potential for the data in ORCID records to help with detecting potential paper mill submissions, we reached out to a number of our member organizations with a proposal to support the use ORCID data directly in the publishing workflow, presenting the available trust markers in a summarized way to help editors with research integrity checks. We are pleased that so many publishers and vendors are keen to work with us to help us refine our understanding of which information in ORCID records is the most helpful in their editorial and review processes.

We are also aware that increased use of ORCID record data in integrity checks may potentially have unintended negative consequences for researchers whose records may be less complete, for example early career researchers or those from countries where the uptake of ORCID is lower. Therefore, we also consulted with our ORCID Researcher Advisory Council (ORAC), who appreciated our proposal to make sure that the record summary data is also available in the public view of ORCID records to increase transparency around the use of this information, and that it should be subject to researchers’ ORCID privacy settings.

Piloting record summaries with publishers and vendors

As a result of our discussions with publishers, vendors, and researchers we developed an initial record summary prototype, which we will be piloting this fall. We’re hoping to make it easier for editors to find and understand information within ORCID records and surface the trust markers that can help them make decisions about the trustworthiness of an ORCID record.

Record summaries can be embedded directly within other applications, such as manuscript submission systems, putting the information directly in front of editors and reviewers. In addition, the data can also be retrieved directly via our API for use within other tools.

Here’s an example of a prototype record summary:

Note that there is no giant green tick here. ORCID doesn’t make value judgements about the data contained within records, nor is it our role to do so. However, we believe that it is our responsibility as a scholarly infrastructure provider and trusted hub of information about researchers to make the data that we hold available to the community as widely as possible, and in as clear a format as possible, so that consumers of ORCID data can make their own judgments. We believe ORCID record summaries will be very useful in quickly summarizing the previous track record of established researchers, however note that in order to avoid the biases outlined above, an “empty” summary should not be taken as a sign of a problematic submission, but rather as a signal that the publisher may need to take additional steps in order to “know their author” and be comfortable that the submission is legitimate. ORCID record summaries are simply intended to make it easier to use the information already in ORCID records when reviewing submissions, and we expect it to become one tool among many that are employed by publishers in coming years to restore confidence in the integrity of the scholarly record.

Thank you to our trust summary working group

We would like to thank the organizations in our trust summary working group for their contribution so far, without the valuable information provided over the last six months we would not be in the position to start our pilot this fall.

We would also like to thank the members of the ORCID Researcher Advisory Council for their valuable input on the topic.

Join our Pilot Partners

Special thanks to our pilot partners who will be working with us over the next several months to integrate this use case in their workflows.

• Wiley
• Aries
• KriyaDocs
• The Royal Society
• Elsevier

If you would like to join our list of pilot partners then please reach out to your ORCID Engagement Lead or contact [email protected]. We would be more than happy to meet with you and get you up to speed with the program.