• Skip to primary navigation
  • Skip to main content
  • Skip to primary sidebar

Connecting Researchers and Research

Sign in/Register
Privacy & Cookies: This site uses cookies. By continuing to use this website, you agree to their use.
To find out more, including how to control cookies, see here: Privacy Policy
  • About
        • Our Mission
          • 2025 Vision
          • ORCID Trust
          • Annual Reports
        • Team
          • ORCID Board
          • ORCID Team
          • Work with Us!
        • Services
          • Member Portal
          • Annual data files
          • Member API
          • Public API
          • ORCID Registry
        • Governance
          • Bylaws
          • Board Elections
        • Policies
          • Privacy Policy
          • Dispute Procedures
          • Public Client Terms of Service
          • Open Source Project License
          • Public Data File Use Policy
          • Terms of Use
          • Brand Guidelines
  • For Researchers
        • Benefits for Researchers
        • Researcher FAQ
        • Video Tutorials
        • Your ORCID Record
        • Get Help
  • Membership
        • Membership Benefits
          • Benefits for Funders
          • Benefits for Publishers
          • Benefits for Research Organizations
          • Benefits for Research Resources
        • Get Membership
        • Member List
        • ORCID Map
        • Membership Comparison
          • Basic Membership
          • Premium Membership
          • ORCID Consortia
        • ORCID Consortia
          • Consortia Agreement
          • Consortia Onboarding Checklist
          • Roles and Responsibilities of ORCID Consortia
  • Documentation
        • Features
          • Member Portal
          • Member API
          • Public API
          • ORCID Registry
          • Annual Data Files
        • Workflows
          • Journal Articles
          • Employment
          • Peer Review
          • Funder and Grants
          • View More
        • Integration Guide
          • Getting Started with Your Integration
          • Sandbox Testing Server
          • Registering a Member API Client
          • Integration and API FAQ
          • View More
        • API Tutorials
          • Get an Authenticated ORCID iD
          • Read Data on a Record
          • Add and Update Data on an ORCID record
          • Hands On with the ORCID API
          • View More
        • Member Portal
          • Member Portal Organization Admin Guide
          • Member Portal Email Guide
          • Member Portal Beta Program Guide
  • Resources
        • ORCID Community
        • Community Programs
          • Certified Service Providers
          • ORCID API Users Group
          • Historical Task Forces, Working Groups, and Steering Groups
        • Get Involved
          • Community Groups
          • Developers
          • Give Feedback
          • ORCID API Users Group
        • Member Resources
          • ORCID Enabled Systems
          • Publishers Open Letter
          • Funders Open Letter
          • Standard Member Agreement
          • Outreach Resources
          • Register a Sandbox API Client
          • Register a Production API Client
  • News & Events
        • News
          • ORCID News
          • Member News
          • Consortia News
          • Integration News
          • Blog
          • Product Updates
        • Events
          • Events Calendar
          • Webinars

ORCID at Scale: Improving our own Infrastructure

July 18, 2019 By Will Simpson

Are you interested in learning about how we host the ORCID Registry and APIs? Would like to know how we deal with high availability, scalability, and recovery in the event of a disaster? If so, then this post is for you! 

We handle eight million page views each month on the ORCID Registry, but the bulk of our traffic is on the APIs, which currently receive over 100 million hits per month. One of our core strategies is to invest in developing a robust information infrastructure, so we need to be confident that the technology we use to support this usage is reliable and secure.

The Registry and the rest of the website on orcid.org are routed through a Content Delivery Network (CDN) — a cloud service provider that has 150+ datacenters around the world. When your browser connects to orcid.org, the static parts of the site are served from a local datacenter near you, to enable faster load times.

The CDN has some other useful features, such as protection against distributed denial of service (DDoS) attacks, and real-time security scanning against hacking threats.

The Registry pages are hosted at our main datacenter, where traffic is load-balanced across a cluster of app servers, while the Registry data are stored in a cluster of three powerful database servers, on encrypted file systems. One is a master database, where updates are made and two are replica servers, which receive a copy of the data in real time. The replica servers are used for most of the “read” operations of the Registry and APIs, but are also hot standby servers meaning they can be promoted to be the master in the event of a failure.

We have an assortment of other servers supporting the production system, which shuffle data around to build search indexes, keep an up to date dump of public data in a different data center, and run scheduled tasks such as email reminders.

We automatically backup the database twice daily, encrypt the dump, and push it to another cloud service provider on a different continent so that, in the event of a disaster at our main datacenter, we can use the database backup to restore the system. We regularly test that this process is working using a temporary offline server.

This is a solid base. However, ORCiD keeps growing and we are increasingly relied upon as part of the research information infrastructure. So we need to do more to ensure the community can continue to depend on us.

What would we like to improve?

We’d like to have app servers and database replicas in multiple locations, so that we don’t have to rely on the somewhat lengthy database restore process, or lose data since the last backup. We’d like to be able to provision new servers in a matter of minutes, rather than hours, in case of sudden increase in demand.

We are considering separating the most critical parts of the system such as registration, sign in, and authorization to an isolated system, and would also like to ensure that public API traffic problems do not impact the Registry and Member APIs.

And we’d like a more flexible architecture using industry standard technologies such as Docker containers and Kubernetes, which will help us to make the improvements mentioned above.

Let us know what you think about our plans! How do we compare with your own organization and other services you rely on? Is there more we could or should be doing? Do you have any advice for us based on your own experience? Contact us with your input and feedback!

Blog

Filed Under: News, ORCID News

Primary Sidebar

Search

Blog Posts by Category

  • Consortia News (39)
  • Integration News (50)
  • Member News (30)
  • News (434)
  • ORCID News (196)
  • Product Updates (75)

Recent Posts

  • Restructuring Engagement to Optimize Member Support
  • OpenAIRE EXPLORE Makes it Simple to Add Works to Your ORCID Record
  • 2021 Product: Our Progress to Date and Future Plans
  • Use Your ORCID Record to Build Your Career: An Interview with Dr. Adriana Romero-Olivares
  • ORCID and Morressier

Sign up for blog updates

We will only use your email to notify you when we have new blog posts. You can unsubscribe at any time. See our Privacy Policy for more information.

Check your inbox or spam folder to confirm your subscription.

ORCID logo

CC0 The text of this website is published under a CC0 license Images and marks are subject to copyright and trademark protection.

  • About ORCID
  • Privacy Policy
  • Terms of Use
  • Accessibility Statement
  • Contact us
  • Dispute procedures
  • Brand Guidelines