Beyond open research: understanding what matters for quality and reliability

Abstract

What really matters in research? What practices and behaviours lead to the highest quality? How can we recognise and incentivise those practices? These are some of the questions we hope to answer at Imperial College London in a pioneering research culture project entitled ‘Beyond Open Research: Understanding What Matters for Quality and Reliability.’ Our vision is a research culture that: recognises the public interest in high-quality, reliable research, and incentivises the behaviours that lead to it. Activities to support a positive research culture are distributed across many functional areas of the university, some exhibiting best practice, some unaware and most unrecognised. Moreover, it’s not clear what practices engender quality/reliability. In this phase, we unpick these challenges in data science, research data management and research software engineering. There are two ways the project goes Beyond Open Research. Firstly, recognising that openness doesn’t by itself imbue research with quality, we address gaps in scrutiny of data, software and AI. Secondly, we examine the inputs, practices and outputs that matter – to researchers, staff and the public – to refocus recognition towards quality and reliability. Through a participatory approach engaging research, academic, clinical and professional staff and the public, we’re: identifying the status of Open Research within Imperial exploring the challenges, questions and concerns of researchers and staff in engaging with responsible research practices around research data, software and AI co-creating practical, stakeholder-led recommendations to embed transparency, reliability and trustworthiness. Semi-structured interviews with experts will enrich the workshop findings, and a network of postgraduate Research Software Champions is raising the profile of software and development best practice among researchers. To the best of our knowledge, this ambitious approach has never been attempted before at a British university. We have a rare opportunity to try to break free from the culture of publish-or-perish, in the interests of researchers and society. We present the context and ambition of our vision, explore the value of meaningful engagement and share interim findings.