On Tuesday 22 January I attended an online seminar organised by Elsevier’s Research Trends, a magazine interested in scientific trends based on bibliometric analysis. With the complete title of “The Individual and Scholarly Networks: A two-part seminar on Building Networks and Evaluating Network Relationships” the sessions were broadcast online from New York, Oxford and Amsterdam, and featured the live presentations of researchers from various disciplines in different geographical locations on both sides of the Atlantic. The programme can be found here.

The event was chaired by Mike Taylor from Elsevier Labs. I live-tweeted most of the event and as a means to document the Twitter discussion I used Storify to collect the #scholnet tweets:  Part 1 is here and Part 2 is here.

Part 1: Building Networks

Presenters from part 1 of The Individual and Scholarly Networks (screenshot)

Part 1 provided important insights into open data, open access, research quality evaluation in a context of “information overload” and alt-metrics. It featured presentations by Professor Jeremy Frey (Head of Physical Chemistry, University of Southampton), Gregg Gordon (CEO and President, SSRN) and Dr William Gunn (Head of Academic Outreach, Mendeley).

Some key points I took with me from the first part were:

Frey: Openness, Methods, Context

• Open data is as important as access: “one needs to know what process was undertaken.”
• The methods to obtain data are as important as the data provided.
• Context must be conveyed with the data. Metadata in advance, not after publication.
• “The surprises can often be the most important.”
• “Highly-linked” e-notebooks are a good way of recording and collaborating, communication challenges can be overcome by most people.
• How scientific research is changing: reconstructing experiments, sharing spaces, open experiments, interaction changing radically.

Gordon: Information Over-abundance and Strategic Access

• SSRN gets 70 thousand submissions each year and got 11 million downloads in 2012.
• According to SSRN “Information over-abundance is a problem.”
• Alt-metrics can help us filter and find the research we need.
• Not only open access, but “strategic access.”

Gunn: The Importance of Communities

• The Internet was designed for scholarly communication. It has not yet reached its potential.
• Asking answers in public leads to sharing knowledge, which in turns leads to more questions and haring.
• “We don’t want a Facebook for scientists because there’s already a Facebook.”
• Communities will only grow if they enable something that works better for online collaboration.
• Communities have to be open if they are to thrive.

Part 2: Evaluating Network Relationships

Presenters from part 2 of The Individual and Scholarly Networks (screenshot)

Part 2 of the seminar explored the related areas of online academic presence/identity, alt-metrics, impact and the evaluation of research. It featured presentations by Dr Gudmundur Thorisson (Research Associate, ORCID and University of Iceland), Kelli Barr (Graduate Research Assistant, Center for Study of Interdisciplinarity,
University of North Texas) and Dr Heather Piwowar (Duke University Postdoc; ImpactStory). Below, some key points I wrote down.

Thorisson: If Your Mum Can Be Online, So Can You!

• ORCID is a means to disambiguate authors and contributors through a persistent unique identifier.
• ORCID seeks to work as “a springboard for less Net-savvy academics  to get connected on the Web and become part of the network.”
• “If my mum could do it, so could scholars.”

Barr : What is Scholarly Evaluation For?

• We need to define the goal for research evaluation by discovering its underlying values.
• What is the goal of research evaluation? “The goal is accountability.”
• Accountability is a call for a more democratic system of knowledge production and a concern for the alt-metrics community.
• Many metrics are a reaction to the shortcomings of peer review.
• Is it really the case the sciences have higher impact than the humanities or social sciences?
• To whom are researchers being hold accountable when they are being evaluated?
• What do we value when we count citations vs. measures that reflect public relevance?

Piwowar: Impact Flavours

• There is a need to move beyond the journal impact factor to promote more effective scholarly communication.
• Need for a “a much broader impact palate” with more “flavours.”
• Impact has more “flavours” than just the one-dimensional citations impact factor of a published journal article.
• There are many alt-metrics tools available.
• The provenance of the data alt-metrics provide means to be understood in context.
• When looking at alt-metrics it is important to provide and apply context to establish relative importance/scale of a given measurement.

The seminar concluded with a final discussion featuring all the presenters and Research Trends editor-in-chief Henk Moed and Gali Halevi from the Research Trends editorial board. The conversation addressed concerns around the political autonomy of research in the context of government evaluation and the difficulties of measuring any kind of societal impact.

The seminar’s format, employing an online conference service, effectively combined live video of the speakers (with their slides being controlled from a central webcasting point) and used Twitter and platform-specific comments to encourage and gather audience participation.

With the exception of some minor technical problems with the audio at some points, the webcasting worked very well, and the real-time engagement on Twitter (under the #scholnet hashtag) was very active, providing a lively, critical and collegial stream of specialised feedback.

The discussion, in general, was heavily focused on anxieties around alt-metrics and open access publishing that we have seen repeated again and again elsewhere (issues around reputation, legitimacy, viability, quantitative versus qualitative assessment, relationships between altmetrics and bibliometrics, etc.). Nevertheless, the seminar offered an opportunity for individuals around the world to connect in an online space in real time to debate shared scholarly interests: online networking, alt-metrics and the new forms in which research is taking place and is being assessed.

–

NB. This post represents the personal experience of the author and does not in any way intend to convey a definitive version of the proceedings.