FSD has tested the capability-maturity approach as part of its work in the EOSC-Nordic project’s Work Package 4 on FAIR data. To facilitate the assessment, the FAIRsFAIR provided us with an as-of-yet unpublished user guide.
Assessment revealed things to improve
We went through all the 16 CoreTrustSeal requirements and assessed our practices against each requirement, metric and indicator. This involved assigning each requirement a capability-maturity tier from five tiers: 1. Initial, 2. Managed, 3. Defined, 4. Quantitatively managed and 5. Optimising. In addition, we reviewed the evidence available to support the selected tier, set a target tier for the next review, and assessed whether we covered the FAIR indicators and metrics mapped to each requirement.
Image source: Hervé L'Hours, Ilona von Stein, Jerry deVries, Linas Cepinskas, Joy Davidson, Patricia Herterich, Robert Huber, & Benjamin Jacob Mathers. (2021). M4.3 CoreTrustSeal+FAIRenabling, Capability and Maturity (1.0). Zenodo. https://doi.org/10.5281/zenodo.5346822
In our assessment, two requirements, R05 Organisational Infrastructure and R10 Preservation, were at tier 4 and FSD achieved the third tier, “defined”, on most requirements. With a median of 3, FSD is doing quite well, as FAIRsFAIR M4.3 points out that “the current recommendation is that a capability level of 2. Managed across all Requirements should be sufficient to demonstrate CoreTrustSeal compliance and FAIRenabling” and that “[l]evels of 3 and above … are highly desirable”.
However, there is always room for improvement. We assigned tier 2 to three requirements, R03 Continuity of Access, R08 Appraisal, and identification under R13 Discovery & Identification. We set tier 3 as our target for these requirements before our CoreTrustSeal renewal application in 2023. In addition, we set the target tier of 4 or above for four other requirements by the next assessment.
The assessment was a useful exercise that allowed us to identify several aspects to improve in our practices and take stock of additional evidence to include in our next CoreTrustSeal application. We also believe we were able to provide useful feedback for improving the approach.
Some expertise is required
We found that a good way to conduct the assessment is to bring together a small group of experts on various repository aspects (e.g., administrative, archival services, technical) and work on a table of the requirements by assessing the capability-maturity level, target and evidence. The process took us two afternoon meetings and some desk research.
The assessment seems especially useful to more experienced repositories that have perhaps already obtained a certification and performed automated FAIR evaluations of their metadata records. The assessment is significantly facilitated by having prior knowledge of CoreTrustSeal requirements, FAIR principles, FAIRsFAIR metrics and RDA indicators.
The capability-maturity approach appears to fall somewhere between a full CoreTrustSeal self-assessment and FAIR evaluations. We found the idea of maturity levels promising, although we had some challenges in finding the correct one for some requirements. The approach brings in some specificity and concreteness to the more general requirements of the CoreTrustSeal.
We noticed that in some cases, the FAIR principles and metrics do not match very well with the CoreTrustSeal requirements, or they do so only partially. This is no fault of the mapping itself but demonstrates that the two have different use cases. They cannot be perfectly aligned but are nonetheless complementary. The emerging capability-maturity approach is a welcome addition to the tools available to assess the FAIRness and trustworthiness of a digital repository.
More information:
» FAIRsFAIR project
» EOSC-Nordic
» FAIRsFAIR WP4: M4.3 CoreTrustSeal+FAIRenabling, Capability and Maturity
» CoreTrustSeal
Henri Ala-Lahti
Information Specialist
firstname.surname [at] tuni.fi
Ei kommentteja:
Lähetä kommentti