An Open Approach to Documenting Community Metadata Enrichment Projects

Why document enrichment projects openly?

Enrichment projects are routinely run by institutions and publishers to improve the quality, completeness, and interoperability of the metadata associated with scholarly works and other research outputs such as datasets and software. The curation approach and rigorous work involved are often unknown to the broader community because they’re not openly documented and shared. Nevertheless, we trust these institutions and publishers have taken due care as it’s in their best interests to look after the metadata. But does this same level of trust exist when a broader network of community members do the curation work? And what do consumers of community-enriched metadata expect to see and access in order to trust its quality? As community conveners, how can we best share enrichment project work so that collaboration opportunities are identified early on? These are some of the questions we asked ourselves as we started setting up the COMET pilot projects last year.

COMET represents diverse stakeholders and aims to enable broad participation in metadata enrichment practices and evaluation, so it was important to the organising team that the pilots were accessible to a wide audience, provided easy access to the project outputs, and gave sufficient context for the community to provide feedback. Working from the foundational principle of ‘trust through transparency’, the COMET Project Doc was conceived as a tool that supported our community-focused practice, openly shared the project work as it progressed, and could develop into a trust and provenance record over time.

You can view the Project Doc for each COMET project at cometadata.org/projects.

Enabling repeatable, rigorous processes

We’ve created the COMET Project Doc Template for the community to reuse and adapt as needed. This document is aimed at teams producing or curating metadata for scholarly outputs, whether that’s an open science office at a university, an editorial department at a publishing house, a repository or CRIS team, or an infrastructure provider working across many records. Teams can use this template to:

• Set up a new enrichment project and invite input before the work begins
• Document completed projects retrospectively, so the methods and outputs can be found and reused
• Coordinate work across organisations, where curation is done by people outside the institution that holds the records.

Walk-through of the COMET Project Doc Template

Here we describe each main section of the Project Doc and its recommended use.

Introduction

The introduction section aims to:

• State the purpose of the document
• Make it clear how readers can participate
• Provide the expected citation for the project
• Highlight where readers can access the completed project outputs
• State funding and support for the project.

Project overview

The project overview is a plain language introduction to the project, aimed at a broad audience. It answers the questions:

• What does the project aim to achieve?
• Why does it matter?
• How will the project be conducted?
• What outputs will be produced?
• Who stands to benefit from the project?

Project status

This section provides updates on the key project phases or milestones. It answers the questions:

• What’s been done so far?
• What’s in progress?
• What’s up next?

Project collaborators

This section identifies the organisations and individuals involved in the project and states these collaborators’ roles. It’s a simple collection point for creator and contributor metadata that will be used when registering the DOI for the project. (In future iterations, this section could rather be a linked spreadsheet to both collect structured information and keep all project information in one place.)

Project outputs

This section lists the project’s outputs, with URLs or DOIs depending on the phase of the project. It should include any output that helps the community understand, trust, and replicate the project. For example:

• ‘Results type’ outputs, such as enriched datasets and model code
• ‘Production type’ outputs that the team developed during the project, such as guidelines for metadata extraction and parsing from scholarly works.

Project report

This section provides a comprehensive report on the completed project, mirroring the expectations set in the ‘Project overview’. It is structured to include:

• The project summary
• The project background
• The project methodology
• The project results and how they were validated
• Any recommended next steps to take the benefits of the project further.

Project evaluation

This section captures the project team’s evaluation of the project, summarising the outcomes of their evaluation meeting. It aims to:

• Assess areas for improvement
• Provide guidance for community members wishing to use or adapt the enrichment methods or run similar projects.

Recommended practice

A tool is only useful when it’s coupled with effective practice. Below we outline how we used the COMET Project Doc while running the pilot projects.

At project start

• Conduct a kick-off meeting with the project team to validate the objectives, resources needed and expected timeline.
• Complete the ‘Enrichment Project Kick-off’ document with the team asynchronously.
• Once finalised, share it publicly. Give ’editor’ access to all project collaborators and ‘commenter’ access to anyone with the link.

During the project

• Hold project review meetings at regular intervals.
• Keep the Project Doc up to date.
• Share updates with your community, highlighting key milestones and opportunities for community feedback.

At project end

• Conduct a project evaluation meeting with the project team to gain insights into the evaluation questions posed in the template.
• Complete the ‘Enrichment Project Report’ document with the team asynchronously.
• Copy the completed document’s content into the publicly shared Project Doc and invite community feedback.
• After any community feedback is incorporated into the document, ensure the project, its documentation, and its outputs have PIDs associated, then archive the document in a repository such as Zenodo.

Try it and share your improvements

The Project Doc has already changed in response to community feedback, most evident in the ‘Project evaluation’ section. In the first pilot, ‘Match Preprints to Published Articles’, it asked the project team a wide set of questions and took considerable time to complete. Through iterative use and feedback, the current version, as used in the most recently completed pilot, ‘Improve Funding Metadata’, hones in on what community members really need to understand from the project team: what did you learn? What guidance do you have for others conducting similar projects?

The COMET Project Doc will keep changing as more enrichment projects are run in the open. The template is open for feedback: if you use it, tell us what was missing, what didn’t fit your workflow, and what your team would change so that we can improve it for wider use. Contact us at info.cometadata@gmail.com to share your feedback.