Google Scholar and Microsoft Academic are upping the stakes in the burgeoning industry of scientific metrics. These free, widely-available online platforms allow users to search for academics or academic scholarship and also compute citation metrics, such as the H- and g-index, that track the influence of the work and its author on that particular field. At the moment, well-kept citation data is largely confined to the major publishing firms – Web of Science (run by Thompson Scientific) and Scopus (Elsevier) being the gold standards – but Google and Microsoft are beginning to give them a run for their money.
But it’s too soon to say whether the democratizing power of technological advancement will actually improve the metrics themselves; if we take higher education as an example, then the lesson seems to be that democratization and decentralizing expertise comes with some serious consequences. The proliferation of specialized metrics among ranks of non-experts (the term smells of snobbery, I know) and academic policy makers has resulted in academic incentive schemes that reward (albeit unintentionally) highly productive researchers, not quality research.
Our department has struggled with the issue of how to account for itself to the administration in a way that respects the unique contributions of each department member and the humanistic nature of the department itself. My concern with making these metrics widely available is that their complexities, shortcomings, and proper implementation are lost in translation from specialized tool to general product. An evaluation always involves judgment-making, but good and fair judgment-making is impossible unless one understands the nature of the thing being judged. It is dangerously easy to misinterpret numbers such as these – and perhaps therein lies their allure.
Striking a harmonious balance between the cult of expertise and democratization is tricky, and exponentially more so once the Internet is involved.