Monitoring and accessing the uptake of Open Research practices is an activity identified by the higher education sector as urgent and necessary, but are our current systems and infrastructure equipped to support this? The UK Reproducibility Network’s Open Research Programme (UKRN ORP) member institutions were involved in a pilot to address the feasibility of current systems, and the reliability and usefulness of recent advances in machine learning. The Open Research Indicators Pilot was sector-led and explored how institutions and solution providers could work together to develop, test, and evaluate prototype machine learning solutions to provide valid, reliable, and ethical indicators for Open Research.
This presentation will explore the findings from the pilot that explored the Openness of data by addressing the inherent challenges and limitations of monitoring and assessing data within a research landscape that still prioritises traditional publishing outputs. This means datasets can only be monitored in Data Availability Statements, which do not have consistent language, terminology, or templates, and are often not included in the structured metadata of a publication. Prototyping innovative machine learning tools uncovered an uncomfortable truth: we cannot reliably and accurately extract inconsistent data from uncontrolled vocabularies. This presentation is a call to action to return to our roots as record-keeping professionals; to monitor and assess open research practices, institutions, publishers, and funders must work together to agree on metadata standards and consistent language to improve the reliability and ease with which researchers cite, curate, publish, and ultimately, preserve research data.

In this presentation we report on our work combining Machine Learning (ML) and human-centered research methods to identify gender biased language in archival catalogue metadata descriptions. Though the ML community has achieved significant advances in language modeling, state-of-the-art models still have social biases encoded within them, the implications of which are under-explored. ML researchers and practitioners have focused more on minimizing and removing bias than on understanding and managing bias. Adopting pre-trained ML models thus risks harm to marginalized communities that information and heritage professionals have been working to better represent. Motivated by information and heritage professionals' desire to manage bias, as well as the scalability and efficiency that ML offers, we investigate a new use case for ML: identifying bias.

This work was funded by a Research and Innovation Grant from The National Archives, building upon previous research with the University of Edinburgh's Archive. There, we created the first text classification models trained on a Gallery, Library, Archive, or Museum (GLAM) catalogue’s metadata descriptions, designed to identify gender biased language. Most recently, we applied the models to descriptions of archives in Newcastle University’s Special Collections. The models and our mixed-methods approach to their development and evaluation offer practical insights on the capabilities and limitations of ML for information and heritage professionals interested in leveraging this technology in their workflows. We publish our code and a tutorial for replicating our approach, aiming to support critical cataloging and inform ongoing improvements to description practices in archives and the wider GLAM sector.

This paper explores the conference theme from the perspective of how innovation and imagination are aligned with creativity. There is currently considerable professional discourse on the impact of Artificial Intelligence on archives. This paper provides a counterbalance in reflecting upon the ongoing need for human centred activity in the archive, in particular, the role that creativity can play in developing and delivering services.

In 2024 Louise Ray began cataloguing the archive of Sir Ken Robinson (1950-2020) https://www.sirkenrobinson.com/about-sir-ken/. Sir Ken Robinson was a leading international authority on creativity and chaired the UK National Advisory Committee on Creative and Cultural Education that produced the influential report, 'All Our Futures: Creativity, Culture and Education'. This paper will draw on Robinson’s approach to creativity in education, life, work, and leadership to consider what the archive profession can learn from his ideas. Starting from the personal perspective of how his publications; 'Out of Our Minds: The Power of Being Creative', 'The Element' and 'Finding Your Element' have informed her professional practice, Louise will explore what can be learnt from Robinson, as well as from the archives of creative individuals and organisations, in considering the extent to which creativity might guide future archival practice.

Whilst this presentation may not have the impact of Sir Ken Robinson’s renowned 2006 TED Talk, currently standing at over 78 million views on YouTube, it seeks to offer up some ideas for celebrating creative thinking in an archival setting and why this should be embraced within our professional advocacy.