Programs on Crossref

Investing $4.9 million of our surplus in rebuilding the Crossref system

Helena Cousijn — Wed, 12 Aug 2026 00:00:00 +0000

The infrastructure that registers and distributes Crossref metadata has been running and growing for more than two decades. It now does considerably more than it was built to do, and it shows: two large, tightly coupled monoliths (Content System and REST API), where a change in one place risks breaking something in another.

In its July meeting, the Crossref board approved the use of Crossref surplus funds for a three-year project to accelerate the redesign of the Crossref system. Our plan is to use $1.6 million a year for three years (~USD $4.9M in total) from our surplus funds, which will help us set up a modern infrastructure consisting of separate services for the different functionalities that the Crossref system offers.

This infusion of time-limited resources will augment our existing team. We will continue to maintain the current system, add small improvements, fix bugs, and make schema changes. We’ve mapped out our time and capacity to ensure that we can keep the current system active and robust while building the new system at the same time. In the past, when we’ve attempted large redesign initiatives, we’ve done a code freeze on the existing system—we won’t be doing that this time. 25,000+ members and thousands of downstream services rely on the system daily, so we will continue to provide time and energy to support that while building for the future. As always, you can view our public roadmap to see progress with what we’re working on.

Background

Over the years, the board, leadership team, and staff have discussed the need to address long-term technical debt in the Crossref system and the need to move from a monolithic codebase to a more modern, manageable, resilient technology that recreates the Crossref system. We are aiming to do this through a set of connected services and functions that better meet our mission and enable the Research Nexus, our goal of a rich and reusable open network of relationships connecting research organisations, people, things, and actions.

Our colleagues Paul Davis and Sara Bowman described some of the current challenges and plans earlier this year in Hit refresh: redesigning our technical infrastructure. We migrated from a closed-source database to PostgreSQL in 2024, moved out of a physical data centre into the cloud in August 2025, and have steadily been deprecating older tools not fit for purpose, such as Event Data, Co-access and ‘old’ Metadata Manager while rebuilding replacements. That was some of the groundwork; now we’re rebuilding the centre.

The two largest components of the current architecture are the Content System (CS) and the REST API. Both function as monoliths. They use mono-repositories, and their internal components are tightly coupled. Over time, they have accumulated complexity, substantially expanding beyond their original purpose. Each of them is now responsible for many different functions, with different needs and characteristics. This monolithic and tightly coupled architecture makes it difficult to maintain, scale, or expand our system. Adding new functionality often requires workarounds, which further increases complexity and adds technical debt.

As a not-for-profit, any surplus we generate has three possible destinations: reserves, paying down debt, or reinvestment in the mission. We met our goal of holding twelve months of operating costs in reserve in 2023, and we carry no debt, so reinvestment is where the rest belongs. We aim to generate surpluses to ensure that we are operating within our means, but we don’t need to accumulate them.

Some of that reinvestment goes outside Crossref; this year we’ve supported other community initiatives and infrastructure that people rely on, totalling around $500,000. A large proportion of the rest should go where members will feel it.

Reinvestment also has to be the right shape. Under the Principles of Open Scholarly Infrastructure, time-limited funds are used only for time-limited activities: day-to-day operations are covered by day-to-day revenue, and one-off funds go to one-off projects. A three-year rebuild, staffed by fixed-term roles embedded in existing teams, meets that test.

What will we rebuild?

At the end of 2024, we introduced a new structure with Crossref work happening within three cross-functional programs: Co-creation and Community Trends (CCT), Contributing to the Research Nexus (CRN), and Open and Sustainable Operations (OSO). The redesign of the Crossref system will also happen within these three programs. The approach we’ve taken is that we’ve identified the different functionalities of the current system, and divided these between the three programs. In the tables below, you can see which services will be rebuilt within each program.

Co-creation and Community Trends (CCT)

Name of component	What is it?	What does it do?	What will change with the rebuild?
Internal APIs for UIs	Purpose-built backends to serve various user interfaces. This is an architectural approach that we have recently begun using to build the new Metadata Manager record registration tool.	Take data from sources such as the Postgres database or the data lake and provide it in a form that user interfaces can use.	This approach will allow us to decouple existing interfaces from the legacy system codebase and build new ones as part of the other projects listed below.
Member self-service	A new interface for members to perform routine tasks related to their member account and user management.	Currently, these tasks are performed in highly manual processes by the Membership team. In some cases, there are semi-manual processes in place, such as with title transfers, where members can fill out a request form on our website.	A self-service portal powered by the new authentication and authorisation service (mentioned below) will lighten the workload of the Membership team. For example, we estimate that about 15% of member support tickets in 2025 represented contact updates which had to be handled manually.
New Admin tool	A user interface for members to perform routine tasks related to their DOI records and metadata submissions.	Members can sign in and upload metadata submissions or queries in XML format, as well as view details about their past submissions. The current admin tool lives at doi.crossref.org.	We will replace the current Admin tool with an interface that integrates with the new authentication and authorisation service and uses interstitial APIs for its functionality. The new interface will also be more accessible and usable.
ORCID Auto-update	A workflow without a user interface or authentication with the Crossref system.	Auto-update pushes metadata about works in the Crossref database to the ORCID API and asks researchers’ permission for Crossref to push further updates and additions.	The main goal of rebuilding our ORCID Auto-update is to make it a standalone service, less entwined with the content system. We will also tweak functionality that we have been unable to make in recent years.
Reports	A series of tables and dashboards presenting metadata from the Crossref database, including DOI resolutions, conflict reports, error reports, and participation reports.	Most report interfaces are openly available via the Crossref website. They are used mostly by members to self-serve information about their metadata on the member, prefix, or title level.	We will change the data source of these reports to the data lake and unify their overall architecture. On the frontend side, there is much scope for improving the usability and accessibility of these interfaces for greater analysis by members and the community.

Contributing to the Research Nexus (CRN)

Name of component	What is it?	What does it do?	What will change with the rebuild?
Funder matching	Matching member-submitted funder metadata to a funder identifier.	For works with funders and no funder identifier, match the funder name to a funder registry entry.	Build a new matching architecture and match funder names to ROR identifiers. Matches are added directly to the REST API as asserted by Crossref.
Preprint matching	For registered journal articles, check if there is a corresponding preprint it can be linked to.	Looks for an exact match between the title and authors. Currently notifies the preprint member via email who is obliged to add the relationship to their metadata (which Crossref reciprocates from journal article records).	Research and test a new method for matching. Add matches directly to the REST API, marked as asserted by Crossref.
Reference matching	For references deposited without a DOI, look for a corresponding Crossref DOI and add those to the metadata (see our recent update on the scale of this service)	Several different processes are used, depending on the completeness of the deposited metadata. Matching is carried out during the deposit process and added to the XML record.	Research and test a new and further improved method for matching. Add matches to the REST API, marked as asserted by Crossref.
XML API	A public API to deliver metadata in XML format.	Use a DOI to look up metadata or construct an XML query.	Reimplement separately from the deposit system. Re-evaluate the features.
Metadata enrichment	Supplement member-submitted metadata with other information held by Crossref.	Crossref metadata records have `information` added during deposit, including member information, citation counts, and relationships.	Review current processes and identify those that can be separated from metadata submission. Where possible, build standalone services that create enrichments, preparing for additional third-party data sources.

Open and Sustainable Operations (OSO)

Name of component	What is it?	What does it do?	What will change with the rebuild?
Authentication & Authorisation	An identity management system	Authenticate and authorise users to access the different Crossref tools and services.	Improve security, allow for member and sponsor self-service to manage users and permissions across their teams and vendors.
Billing	A service to generate invoices and reports based on new record registrations per quarter.	Generate a .csv of billable submissions and upload to our accounting system to generate invoices.	Reduce technical debt and remove code from the legacy system; allow for point-in-time billing.
Notifications	A service to send emails or callbacks to members as new DOI records are registered or existing ones updated.	Transactional notifications on submission status. Additional notifications when works are cited by other works.	Reduce/eliminate the number of emails sent, reduce technical debt, remove code from the legacy system.
Handle system	Update and look up DOI in the Global Handle Registry.	Registers the DOI string and associated landing page URL for redirection (DOI resolution).	Implement better fallback and error handling to make connections with the Global Handle Registry more resilient.
Deposit system	System that receives XML deposits from members.	Processes deposits and makes them available to other systems, like the REST API.	Separate out the different components of the deposit system.

The three-year timeline refers to the duration of the additional investment that is now being made. We don’t expect all of the rebuild work to be complete within three years. And system maintenance and improvement is never done. The additional resources will accelerate our ability to address these projects and get them substantially complete or meaningfully underway within three years. We will start with the things that are easiest to separate out of the current system, and work our way through it. The deposit system will be the last part we’ll tackle— after the initial 3-year period. Exactly how we’ll break out or couple together its components will depend somewhat on how the rest of the work in all three programs proceeds.

Additionally, there will be work in the OSO program to pull out deprecated/unused code from the content system as other programs build new free-standing services.

Of course, an end goal is to rebuild all of this in open code repositories, and eventually turn our amber-coloured ‘open source’ to green in our biannual POSI audits.

What does this new investment and approach mean for our community?

Having seen the long list of things we’re planning to do, you may be wondering how this will impact you. Here are some things you should know:

You’ll be able to use our services throughout this project. Part of the reason why we are rebuilding different functions one by one is that we don’t want to interrupt the services we offer to our growing membership and users. The current services will continue to run until the new ones are in place, and we’ll have clear transition plans and documentation in cases where you need to update your workflows.

We won’t just rebuild like-for-like, but we’ll make improvements when we rebuild services. We want to continue to provide the functionality we currently offer to our members, but we also realise there are things that could be better. The clearest example of this is member self-service, which will be one of the main priorities for the CCT program next year. For example, as mentioned above, in 2025, about 15% of member support tickets were contact updates: changes members should be able to make themselves in seconds, but which our membership team handles by hand; that is the kind of thing that will improve life for both members and staff.

We are planning to deprecate some older/less-used services along the way, but we’ll give you plenty of notice. Crossref is probably running more services than you realise! We’re therefore evaluating all our services as part of the redesign, and keeping only the ones that are actively used and really add value for the community. If we decide to retire a service, we will contact all users so they have enough time to transition to an alternative service. Everything we’ve deprecated so far is also itemised on our website.

Community input will be a big part of the process. To ensure the services we offer meet your needs, we will need to keep talking with you. Our community managers and UX researchers will be reaching out to users of different services to understand use cases and ensure these continue to be met within the new system.

This blog is the first tagged ‘system-redesign’ so follow this category for ongoing updates. If you have feedback or questions, please head to the dedicated community forum space for system redesign and ask us in the open.

Two halves of the same decision

Later this week we’ll also announce the latest round of fee changes resulting from our Resourcing Crossref for Future Sustainability (RCFS) work. What we charge, and what we do with what we collect, are two halves of the same decision; fees are to come down at the same time as we commit the largest single investment in our infrastructure we have ever made. Both follow from the same principle: we set fees to cover the cost of running the infrastructure, and when we hold more than we need, it goes back to the community and into the longevity of the scholarly record.

Our recent position paper on PIDs in research infrastructure policy argues that the value of an identifier lies in the metadata and services around it, and in whether those can be maintained over the long term. This approach to futureproofing open scholarly infrastructure is an example of that maintenance in action.

Acknowledgements

We would like to thank Sara Bowman, Carlos del Ojo Elias, Bharath Govindarajan, Stewart Houten, Martyn Rittman, Lena Stoll and Patrick Vale for their work on the system investment proposal and contributions to this blog post.

The programs approach: our experiences during the first quarter of 2025

Helena Cousijn — Tue, 08 Apr 2025 00:00:00 +0000

At the end of last year, we were excited to announce our renewed commitment to community and the launch of three cross-functional programs to guide and accelerate our work. We introduced this new approach to work towards better cross-team alignment, shared responsibility, improved communication and learning, and make more progress on the things members need.

In line with the Crossref strategic agenda, the three programs focus on:

Co-creation and Community Trends (CCT): This program is responsible for interfaces such as reports/dashboards, record registration interfaces, connections and collaborations such as Open Funder Registry, ROR, ORCID auto-update, as well as OJS and other partner integrations. This program also includes the Crossref website and any front-end interfaces to support other programs. It includes initiatives aimed at upholding the integrity of the scholarly record and our tools in this area, such as Crossmark and retraction/correction tooling, and Similarity Check for text comparisons.
Contributing to the Research Nexus (CRN): This program manages and oversees all activities relating to contributing to the Research Nexus. A lot of the work in this program revolves around our REST API, but also includes our other APIs, incorporating external data sources like Retraction Watch and Event Data, building out metadata matching services with the new data science team, supporting the community of metadata users with API sprints and more modern options for retrieving metadata based on usage and need.
Open and Sustainable Operations (OSO): This program manages and oversees all activities related to making our operations more open, transparent, and sustainable. This program focuses on supporting and strengthening the core functions our members rely on and enabling future growth. It includes metadata deposit and processing, most apps for e.g. managing titles, authentication, and architectural and infrastructural projects like moving from the data centre to the AWS cloud service. This program also includes modernising our operations in general, which is not just technology but also finance and human resources, so projects like membership process automation, financial analyses, and business system integrations.

The approach we are taking is to support the work within the programs through (internal) cross-functional steering groups. Led by three program leads (who share updates on their programs below), three program steering groups meet regularly to discuss the topics and work that fall within the scope of each program. The steering groups consist of representatives from all teams within Crossref, which means every steering group has people from the community team, membership team, technical team, data science team, and operations and finance team, bringing all the perspectives and expertise needed to prioritise the next steps for Crossref and fostering broad knowledge sharing and shared responsibility.

Although the whole organisation contributes to these programs, they are coordinated by the Programs and Services team. The team was formed towards the end of 2024, and on the 1st of February, Helena Cousijn joined Crossref in the new role of Director of Programs and Services. Helena has a background in both product management and community engagement and is very excited to help Crossref shape the programs approach and work with all teams across the organisation to drive the strategic agenda forward!

If you’d like to keep an eye on the work that is happening within each program, you can find more information on the Crossref productboard.

Co-creation and Community Trends (CCT)

The mission of the CCT program is to build and foster relationships with our community and other services and organisations within it, so that Crossref can meet and anticipate community needs. Curiosity and listening are at the core of how we co-create to tackle emerging challenges, develop best practices, and explore new ideas for building the Research Nexus. We want our work to benefit all of Crossref’s diverse stakeholders - from our own colleagues and members to underrepresented communities in the wider scholarly ecosystem.

In the first quarter of 2025, our focus areas have been:

Improvements to our new record registration form for journal articles, which already supports grants, and was launched in beta for articles in 2024. For example, the form now has a built-in reference deposit feature. Join the conversation on the community forum for updates and feedback on this new helper tool.
Running a series of multilingual metadata health check webinars. There are more of these coming up throughout Q2, so it’s not too late to sign up for one if you are interested.
Integrating with Rogue Scholar to automate the assignment of DOIs to, and the archiving of, posts on this very blog.
Planning for the inaugural Crossref Metadata Awards - join our upcoming community call on 7 May to find out what this is all about.

In the coming months, we are hoping to tackle the following:

Kick off a project to review the information architecture of this website and look into how we can make our documentation and related information more helpful and easier to navigate.
Expand the record registration form for journal articles to allow easy editing of previously submitted records. This will allow us to sunset the long-deprecated Metadata Manager tool, as was first announced in 2021.
Begin building new record registration forms for more work types. Watch this space.
Explore options for supporting the integration of additional software systems with Crossref, building on our existing approach with OJS plugins, with a focus on open-source tools relied upon by our members for registering metadata.
Restore faceted search on Crossref Metadata Search. This feature was disabled in 2022 following intermittent performance issues. We believe recent improvements to Metadata Search will allow us to bring some filters back, although we will need to start small so as not to overload our systems with these more complex queries.

Contributing to the Research Nexus (CRN)

The research nexus is a rich and reusable open network that represents scholarly activity. It consists of connections between research organisations, people, things, and actions; it’s an evolving model of the scholarly record that the global community can build on forever for the benefit of society.

Our metadata is already a contribution to the research nexus, however, there is much more we would like to do. Our next steps will be to consolidate our existing data and services, and build the technical capacity, partnerships, and knowledge to enhance our contribution with new relationships. Some parts of our data storage and workflows don’t yet have the flexibility to fully capture all types of research objects and how they are connected.

To support this process, the main priorities in the program are:

Collaborate with our community. We want to get to know users of our metadata better and work more collaboratively alongside them. Also, we seek partners to contribute new data sources that will enhance our metadata with additional relationships.
Share the research nexus vision. We know that we aren’t alone in developing the research nexus, so we will reach out to others with a similar vision and identify where we have common goals.
Maintain our technology. We have already identified technical improvements we can make to our REST API, and we need to keep on top of monitoring and fixing bugs. We also need to build capacity for new types of data and relationships. Our other endpoints, such as the XML API and forwardlinks (for citations), need maintenance and are likely to be affected by a planned redesign of our core architecture.
Building a new matching service. Identifying relationships between metadata records is a key part of the research nexus. We have already improved reference matching over the years, and we’re looking to implement funding, affiliations, and version matching next. We’ve carried out research on several types of matching and are looking at building a new service to handle it in production.

In the first quarter of 2025, we’ve been working on:

Schema changes, making the first significant updates to our schema for several years, including adding the capacity for depositing ROR IDs for funding organisations in funding metadata.
Delivering Retraction Watch retractions via the REST API, integrated with member-supplied retraction/correction data.
Getting the community involved and understanding needs, planning a sprint and various workshops.
Plenty of under-the-hood updates to the REST API, and more significant upgrades to come later this year.

Next up, we will:

Plan and build out the new matching service.
Improve representation of some metadata in the REST API, including Crossref members, journals, and typed citations such as data citations.
Update the grants schema to extend the award types and respond to new funder member requests
Add contributor roles to the schema, including CRediT.
Ask our community about metadata retrieval, including the various APIs and the Metadata Plus subscription service.
Upgrade elements of the REST API and optimise the underlying technical infrastructure.

Open and Sustainable Operations (OSO)

The OSO program is centered on transparency and sustainability of our technical systems and our business and people operations. We focus on maintaining critical systems and operations and ensuring their security, addressing technical and operational debt, and controlling or reducing costs - to Crossref, our community, or the environment. We’re always keen to tackle projects to automate repetitive and manual tasks – of which we have many – and pay down technical debt, being as open and transparent as possible along the way.

Our most recently completed work includes:

Moving from Oracle to an open-source database, PostgreSQL. This work aligns with the POSI principles and sets us up for a more robust, reliable, and modern infrastructure.
Implementing metadata schema changes for deposit submission and processing, so we can now accept ROR IDs in funding metadata, as well as the changes in latest schema version (5.4.0) which includes the new ability to label references with a type (such as dataset, software, blog post, article, etc.).
Automating parts of the process to keep Sponsor information on our website up to date and make it easier to search, so our community can find relevant and accurate information about our Sponsors and how to work with them, and our membership team spends less time keeping the website current.

Ongoing work in our program includes:

Moving from a physical data center into the cloud (AWS). The PostgreSQL migration was the first step needed to enable our move to the cloud, which will allow us to operate more sustainably and efficiently.
Automating new member setup in our systems, which is largely a manual process now.

And coming up are:

Making changes in our core system to accept the upcoming 5.5 metadata schema version.
Extracting billing code from our main codebase, to set up as its own service. This will allow us to simplify our code and make it easier to maintain. We’ll also be implementing the changes to billing enacted as part of the Resourcing Crossref for Future Sustainability program (TBD!).
Holding a “systems workshop” in April, to understand how our current system(s) are and aren’t meeting staff, member, and community needs, and how we might go about building the open, sustainable Crossref system of the future.

What have we learned so far?

Internal communication

One of the reasons to implement a programs approach was to improve internal communication across the organisation. With all teams being represented on all steering groups, everyone is in the loop when decisions are taken. We see that this way, people feel more connected to the strategic agenda and, importantly, the ‘why’ is clearer to people. It is easier to get perspectives from across the organisation because contributing to these conversations is now part of people’s day jobs and so it’s easier to ask for their time. We are still looking to improve how we facilitate group discussions and decision-making to ensure we make the most of the program steering groups.

Planning and delivery

Working closely with people from across the organisation has helped with more effective planning. A closer collaboration between program leads and developers makes the delivery of new features and functionality more accurate and predictable. With the community and support teams also being part of the conversation, they can plan related comms and support/documentation efforts in a timely manner. So far, it has also been easier to get more things delivered. We have some big projects coming up this year that will be a good test for the programs approach!

Cross-cutting topics

The implementation of a cross-functional approach facilitates discussions around cross-cutting topics, but also leads to the question of how cross-cutting topics fit within a specific program! Maybe you already noticed that work on metadata schema 5.4 and the planned work on 5.5 is included under both Contributing to the Research Nexus and Open and Sustainable Operations in the update above. Because metadata development impacts many of our systems, work was needed within all programs to enable these changes - the input, the output and the interfaces. Later this year, we’re planning to share some visuals that better explain which projects sit with which program and how we deal with cross-cutting topics.

Alignment

One of the most important things for the approach to be successful is that people are bought in and willing to participate and communicate. For cross-organisational alignment, a culture needs to be in place (or developed) where people are willing to collaborate and be open and transparent about their work. In a practical sense, we are still looking at how we can better align our code bases with the current programs so that it is easier to develop the relevant expertise within the programs.

We hope to see many of you at our upcoming community call on 7 May. Please register to join as we discuss some of the work included in this update.

A progress update and a renewed commitment to community

Ginny Hendricks — Thu, 12 Dec 2024 00:00:00 +0000

Looking back over 2024, we wanted to reflect on where we are in meeting our goals, and report on the progress and plans that affect you - our community of 21,000 organisational members as well as the vast number of research initiatives and scientific bodies that rely on Crossref metadata.

In this post, we will give an update on our roadmap, including what is completed, underway, and up next, and a bit about what’s paused and why. We’ll describe how we have been making resourcing and prioritisation decisions, including a revised management structure, and introduce new cross-functional program groups to collectively take the work forward more effectively.

It’s important to acknowledge that Crossref has evolved significantly from just five years ago - our member count has more than doubled from 10,000 to 21,000 organisations since 2019 and they include all kinds of organisations such as funders, universities, government bodies, NGOs, and of course scholar- and library-led publishers. The smaller organisations now collectively contribute the majority of Crossref funding. We’ve gone from 100 million records to 160 million in five years, and our metadata is retrieved more than 2 billion times monthly, quadrupling what it was five years ago.

It’s within this context that we’ve spent quite a lot of time thinking about scalability, how we collect and process feedback and contributions from many organisations, how to automate our operations, and refining the plans for the next few years.

Our strategic agenda remains the same

A few times a year we update the strategy page where there is a quadrant of projects showing what’s completed, in progress, up next, and in planning/ideas - for each strategic theme. We also link from there to our live public roadmap which shows more specifics about individual projects, including projected timelines, and is updated more frequently.

If you’ve been watching the strategy page, checking in on the public roadmap or this blog, or joining webinars and annual meetings, you’ll know that we’ve had some longstanding plans to—among other things—reduce technical debt, rebuild our metadata management system, move to the cloud, modernise our schema, support multiple languages, and partner with multiple data sources to build the Research Nexus.

You’ve heard us talk about these initiatives a lot, but you’ve not seen particularly swift action.

Moving the work forward more effectively

Earlier this year, it became clear that our almost three-year project to build a new relationships API had not worked out. The project, dubbed ‘manifold’, was to initially deliver data citations, and eventually replace our central metadata system, but what was prototyped didn’t scale, even with a subset of our metadata. We weren’t confident enough about the project’s timeline or costs to justifiably continue investing further time and resources.

Meanwhile, we’d barely scratched the surface of our aim to pay down technical and operational debt, and we’d also been neglecting to keep the live system up to date with the numerous metadata changes that have been queued up, waiting to be implemented.

We knew the manifold project was ambitious – our system has grown in complexity over the years. We were trying to rebuild the car while driving it (our system needed to continue to operate and be maintained by our team) while trying to design a new approach to manage the many relationships between 160+ million database records. In the years we worked on this project, we learned a lot that will inform future plans for a large system redesign.

In March this year, we decided to pause the manifold project. We apologised to our community partners for not delivering the promised data<->literature matches they hoped to use. They were frustrated but thankfully understanding.

We then resolved to focus on backend infrastructural changes, conduct cross-training so that all of our staff would become familiar with current in-use systems instead of greenfield tech (for now), and start to make a dent in the backlog of bugs and long-promised schema updates in our mainstream services.

We’re happy to report some movement on these things and some milestones that have been achieved in these areas in recent months.

Fostering a happy and dedicated team

Any kind of work can only happen when our staff are in a good place, feeling supported and comfortable to question things, and well-equipped with information, purpose, and clear priorities. In June, when the whole staff met up in person, we had some really good conversations about culture, communication, and about sharing responsibilities. Some people ran birds-of-a-feather sessions to explore the issues that had been keeping them up at night, such as authentication/security, and rebuilding the Crossref System (CS), and the team also co-created a set of prioritisation drivers that are now in use within our roadmap and planning processes.

Taking on feedback from the all-staff meeting and then the July board meeting, we thought strategically about the organisational structure Crossref would need over the next few years to reflect the growth in scope and size, and fulfil its longer term goals. We have long had an ambitious agenda but realised we didn’t yet have the capacity to do it all. So we came to the conclusion that we needed an updated team and management structure to take us through the next phase of our development.

The structural changes were concluded at the end of November. They included:

Moving Technology under Operations, since Technology—though a vital enabler—still works in service to our mission and in support of our community, just like other operational things like board governance and finance.
Reframing product development as Programs and Services, and reducing our workstreams from five product portfolios to three programs. We formed cross-team steering groups around clearly articulated program areas (more on those below).
Broadening the leadership to include an Executive team and an extended Director team, and forming a Senior Management Team (SMT). These changes ensure that the collective responsibility for Crossref now rests on a wider group of experts who can back each other up and share the risk and the knowledge, rather than on just a few individuals.
We started recruiting for directors for two new leadership positions. We’ll welcome a new Director of Programs and Services and a new Director of Technology in the new year.
Evolving the strategic initiatives team into a data science team, integrating research & development functions throughout all teams and with the SMT taking collective responsibility for strategic initiatives.

Unfortunately, with the shift in approach for product development and by sharing responsibility for strategic initiatives and research among the wider team, we made the difficult decision that four positions would no longer work within the new structure.

A new approach: joined-up initiatives and cross-functional programs

Research has always been an important role for Crossref, but as this function had been annexed from our regular work, it became hard to coordinate strategic initiatives across the wider organisation. In recent years we inadvertently created more technical debt for ourselves, i.e., built multiple prototype tools without plans for adoption or moving them into production. Strategic initiatives, by their nature, need thorough research and high-level alignment, so we made such initiatives—things like Resourcing Crossref for Future Sustainability (RCFS) and improving the Integrity of the Scholarly record (ISR)—the responsibility of the whole senior management team.

Some useful research had been conducted, but we were never in a position to act on any of it. Particularly promising work has been in the field of metadata matching, and with the growth in the community reliance on our metadata, and attention on data quality rightly increasing, we decided to create a new data science team to be dedicated to this work, led by Dominika Tkaczyk.

We had also struggled with a traditional product management approach since all our tools and activities are interconnected, and we found we were trying to do too many things at once but not all of them very effectively. We also acknowledged that product management comes from the commercial e.g. retail world and therefore is designed to help companies sell/upsell, which is not our goal. So we looked to other approaches more suitable to mission-based nonprofits.

Introducing three programs

We have introduced cross-functional program management in order to work towards the following:

better cross-team alignment
shared responsibility
improve communication and learning
make more progress on the things members need.

Supporting the strategic theme of co-creation, a new program, facilitated by Program Lead Lena Stoll, now manages and oversees all activities around co-creation and community trends. A cross-team steering group just began meeting regularly and will be responsible for interfaces such as reports/dashboards, record registration interfaces, connections and collaborations such as Open Funder Registry, ROR, ORCID auto-update, as well as OJS and other partner integrations. This program also includes the Crossref website and any front-end things to support other programs. And it includes ISR (the integrity of the scholarly record) and our tools in this area such as Crossmark and retraction/correction tooling, and Similarity Check for text comparisons.

Supporting the strategic theme of complete and global metadata and relationships, a new program, facilitated by Program Lead Martyn Rittman, now manages and oversees all activities relating to contributing to the Research Nexus. Working particularly closely with the metadata team, led by Patricia Feeney, this program addresses how metadata is modelled, used, enriched, and extended. Work includes our APIs, incorporating external data sources like Retraction Watch and Event Data, building out metadata matching services with the new data science team, supporting the community of metadata users with API sprints and more modern options for retrieving metadata based on usage and need.

Supporting the strategic theme of open and sustainable operations and keeping to the POSI framework, a new program, facilitated by Program Lead Sara Bowman, now manages and oversees all activities relating to making our operations more open, transparent, and sustainable. This program focuses on supporting and strengthening the core functions our members rely on and enabling future growth. It includes metadata deposit and processing, most apps for e.g. managing titles, authentication, and architectural and infrastructural projects like moving from the data centre to the AWS cloud service. This program also includes modernising our operations in general, which is not just technology but also finance and human resources, so projects like membership process automation, fee modelling and financial analyses, and business system integrations.

The Programs will start to be reflected across our website and in our communications from next year.

What are Crossref’s new prioritisation drivers?

These are the drivers that our ~40 staff co-created in June that are guiding decisions about the priorities on our roadmap. New ideas will be evaluated in the following areas:

Encourage participation from new or under-represented communities
Respond to and lead trends in scholarly communications
Benefit the greatest number of members and users
Reflect on how the community works with each other and allow members to self-serve
Expand to support and connect relevant resource types and metadata fields
Make it easier to create and update metadata
Enhance metadata for completeness and accuracy
Make it easier to retrieve and use metadata
Automate repetitive/manual tasks
Address technical and operational debt
Maintain critical systems and operations and ensure their security
Control or reduce costs - to Crossref, our community, or the environment

We’re happy to report that the changes made this year have resulted in a productive last few months of the year. As reported in our annual meeting, here is the progress update.

What’s paused

A relationships API endpoint and, therefore, a specific data citation feed
Manifold, the three-year effort to modernise our tech stack
Most of the strategic initiatives prototypes that can’t yet be scaled, such as Labs API and Labs reports

What’s recently completed

We succeeded in moving the entire Crossref corpus to an open-source database, PostgreSQL
Fixed numerous REST API data quality issues and lots of troublesome bugs
Schema development - support for ROR as a Funder identifier is live and currently in testing
We automated some very manual membership and billing processes, saving hundreds of staff hours a year
Released a new form for journal article record registration, building on the grant registration form
Upgraded Participation Reports to include Affiliations and ROR IDs
Launched a new API Learning Hub

Since the rest of the community stops for no Crossref product roadmap issue, we also progressed a number of community and governance initiatives:

The Grant Linking System (GLS) reached 5 years with over 40 funders joining Crossref and registering over 130,000 grants and awards, including use of facilities and projects
Our research for Resourcing Crossref for Future Sustainability (RCFS) with the Membership & Fees Committee is going well, and we’ll have new fee proposals for review in 2025
The integrity of the Scholarly Record (ISR) conversations have deepened, and we’ve formed strong relationships with editorial experts and research integrity sleuths, who are getting up to speed on our metadata, and we’re working with some sleuthing consultants to change our processes to handle deceptive member behaviour such as paper mills, cloned journals, and citation manipulation. The new data science team plays a role here, along with membership and governance.

What’s currently in focus

In our efforts to do less but do it more effectively, we have two current priorities:

Get out of the physical data centre and into the cloud.
Develop Schema 5.4.

These two projects are underway, involving lots of communication and learning. Since we haven’t released any schema updates in many years, all our staff are learning for the first time how a metadata schema model is interpreted in a systemic way, learning about the structure of research objects, and honing the process as they go. We’ve high hopes we’ll be in a position to release continuous metadata schema versions and catch up on the backlog over the coming years.

What’s next

Continuous metadata development, with contributor roles up next
Retraction Watch data integrated into the REST API so users have a single source of retraction/correction data
Upgraded preprint matching and notifications
Modelling more equitable fees through the RCFS projects
Piloting a non-voting membership category

Once we’re fully in the cloud and in the groove of metadata updates, and with the support of newly-hired technology and program directors joining in the new year, we’ll turn our attention to rebuilding the central metadata system that we call the Crossref System, or “CS” and report more on this next year.

So that was our summary of 2024 and an indication of what’s coming in 2025 and beyond; sorry it’s so long, and thanks for reading this far! Next year we’ll get back to more regular updates as the strategic agenda and the programs progress.