Beyond Impact

Beyond Impact is an Open Society Foundations funded project that aims to facilitate a conversation between researchers, their funders, and developers about what we mean by the “impact” of research and how we can make its measurement more reliable, more useful, and more accepted by the research community. The project will run a workshop that will be of interest to funders, research institutions, researchers, data-holders, publishers and developers. We aim to identify specific use-cases and then build prototype demonstrators to address them over the course of the three day workshop.

For Developers

The world of research data and particularly data about the research process presents great opportunities for developers with experience in visualisation, data aggregation and analysis. The opportunity exists to build a set of tools that will assist researchers, institutions, and research funders in maximising the wider positive impact of their research. Bringing the skills of the wider development community to bear in this space has a significant potential to make research more effective and efficient.

For Providers

There are a wide range of publishers, data aggregators and other commercial tool providers who can provide either information or tools that will support the framework that delivers both diverse and reliable measures of the use and re-use of research outputs. Providers will have the opportunity to showcase work in development and to interact directly with the researchers, funders, and institutional managers that will be involved in the specification and development of the next generation of research assessment systems.

For Researchers and Managers

Researchers and their managers in today’s competitive environment need to be able to demonstrate the impact of their research. Research managers in turn are concerned about tracking and valuing an increasingly diverse range of research outputs. Placing tools in the hands of researchers that enable them to track the use and impact of their own research outputs the project will provide an opportunity to re-engage with the impact agenda and provide a common ground to demonstrate and validate the impact of their work with research managers.

For Funders

Funders today are more and more conscious of the need to demonstrate the impact of work they fund. Moving beyond simple paper counting to measurement of the wide range of downstream uses is a significant challenge. The workshop will provide a venue for funders to articulate their needs and desires in being able to track and quantify the use and re-use of funded research. The development sprint will provide examples of tools that track and measure research impact and identify where funders can make such development easier.

Workshop Report Published

Thanks to input and comments from a wide range of people we have now got the workshop report into a shape that is appropriate for wider publication. The report is published in two forms. The first is searchable and viewable online but is not available for editing. This is the formally published version. In addition to this there is an editable version which remains available for any improvements and comments to be left. The report isn’t the end of the story and there are no doubt opportunities for it to be improved so feel free to leave further comments or make changes that can be incorporated into future versions.

A citation by any other name smells awful

“A rose by any other name would smell as sweet” is a quotation by William Shakespeare from his play Romeo and Juliet to suggest that the names of things do not matter, only what things are. The Beyond Impact workshop has looked at mechanisms for citations of “non-traditional” research outputs (physical and virtual; persistent, evolving and ephemeral) and it is clear that their names, their identifiers, really do matter.

By providing better guidance that spans best practice in citations for production (generation of identifiers) and consumption (style guides, use of identifiers), we are able to lay the foundation for a more robust system of metrics that encourages more micro-publication, and lets us construct better measures of a researcher’s influence and impact.

Total Impact – building a tool to understand ‘my’ influence

An overarching theme of the Beyond Impact workshop has been to get past the existing standard definition of ‘impact’ to look at other research outputs and measures. The problems with existing measures are well known but there has been limited movement at a broad level to really move beyond these and look at newer forms of measurement of impact such as blogs, presentations, datasets etc.

The workshop has developed the concept of an enhanced personal impact dashboard, where the researcher can input a list of research outputs to create a ‘live CV’ with a number of indicators associated with each. From this, one could create an aggregate score, providing a ‘Total Impact’ for that individual. This could lead to a much broader measure of impact than is easy to achieve today and provide an idea of what an individual’s output has accumulated in terms of ‘externalised mindshare’.

The role and position of Thomson Reuters

This piece was contributed by Laure Haak of Discovery Logic, a business of Thomson Reuters – Thomson Reuters is the world’s leading source of intelligent information for businesses and professionals. We combine industry expertise with innovative technology to deliver critical information to leading decision makers in the financial, legal, tax and accounting, healthcare and science and media markets, powered by the world’s most trusted news organization. With headquarters in New York and major operations in London and Eagan, Minnesota, Thomson Reuters employs 55,000 people and operates in over 100 countries.

Research outputs, money, and stories: The view from BBSRC and EPSRC

Between them the BBSRC and EPSRC fund a significant proportion of the natural sciences research in the UK spreading from engineering, physics, and computer science through biochemistry, to ecology and agriculture. Sue Smart is head of evaluation at EPSRC and Mari Williams leads the parallel group at BBSRC. In conversation with both of them some important points came through that are worth discussion.

Software Impact – the differences from datasets

In the research world, we’re often told that the the only way for our work to have a measurable impact is through the citation of publications based on our work. The mechanisms for this are in place, the stakeholders understand the framework, and everyone is happy… or so we’re led to think. However, the status […]

Some suggestions on vocabulary

One of the things that has become clear in discussions with people leading up to the workshop is that the language we use to discuss research impact is a barrier in its own right. Confusion over terminology and interpretations of the agenda behind terminology are a significant source of problems. To help deal with this […]

John Green: Imperial College London

John Green is an Honourary Research Associate at Imperial College London and has implemented a range of the research management systems that are in place there. From 2004 until 2010 he was responsible at a university-wide level for the co-ordination of activities which span faculties and/or service departments. He led major projects involving IT, restructuring, […]

A Draft Framework for Research Impact Data Standards

Introduction Organizations that support important research activity, both funders and hosts, must report on the impact of that activity; particularly if they are to maintain support momentum in an environment that is competitive for available resources. Two factors make impact reporting increasingly difficult. First, excellence in research activity means a multiplicity of disciplines, institutions and […]

Draft Workshop Programme

This is the current working draft of the Workshop Agenda. This remains negotiable right up to the point of the workshop. I have spoken to a number of attendees about leading one of the breakout sessions and am looking for a few more participants to take that role on. Feel free to get in contact […]

A chat with David Kryl of the NIHR

The National Institute for Health Research provides the health research system that supports clinical and translational research in the NHS. NIHR funds research projects and research faculty as well as providing the infrastructure and systems required to monitor, support and optimize the systems that turn research outcomes into changes in clinical practice. David Kryl is Head of Business Intelligence for NIHR.

The personal impact dashboard

One of the arguments that the Beyond Impact project proposal made was that funders might make researcher engagement with the impact agenda more appealing if researchers felt they had stake in defining what the important impacts of their research were. In conversation with David Baker from CASRAI he brought up an idea that could both help to engage researchers, and assist institutions and funders in gather more information on research outputs and their impact; the personal impact dashboard.