• Subscribe

Tracking the success of e-infrastructures at ISGC2013

Read more about ISGC on the GridCast blog.

The first two speakers at March's International Symposium on Grids and Clouds 2013 (ISGC2013) were Steven Newhouse, director of European Grid Infrastructure (EGI) and Daniel Katz, program director in the Division of Advanced Cyberinfrastructure at the US National Science Foundation (NSF). Together, they provided perspectives from both sides of the Atlantic Ocean on ways to measure the success of e-infrastructures, including the people and services that contribute to their development. Steven Newhouse introduced some of the metrics currently being used to determine the impact of e-infrastructures in Europe from projects such as e-FISCAL, e.inventory and ERINA+ .

So how do developers and architects of software and e-infrastructures know when their work is successful? And, more importantly, how does the outside world (especially funders) recognise exactly who collaborated and contributed to each and every scientific output? For the developer of an open source physics simulation, possible metrics could include recording downloads and users, as well as tracking paper citations. However, Katz suggests that what could be more valuable but harder to monitor are the number of citations each paper that cited your software receives. He emphasized the need to develop a system where credit is given to those involved in building the software and data sets.

The NSF has proposed a number of solutions to this conundrum. One recommendation is that all products - including software, data sets and workflows, are registered. Next, a credit map is developed. The input would be the weighted list of contributors (both people and products) with the DOI as the final output. This could lead to transitive credit enabling developers to gather evidence for their applications' widespread usage. Product usage could also be automatically recorded by simply incorporating a general usage code into each software package. Commercial tools are generally tracked by the money they generate, but this doesn't help academics understand what tools were used for what outcomes, says Katz. Tokens could be provided as part of science grants. Users could then distribute tokens either during or after using the tools.

At the moment, it is largely up to e-infrastructures to not only demonstrate an active user base, but to monitor the quality of service, track their citations and understand usage patterns. Alexandre Bonvin described the ways in which WeNMR is keeping track of how their services have contributed to progress in structural biology. In 2012, there was a significant output from the project with 36 publications acknowledging WeNMR. Their analysis found the use of their e-infrastructure to be both application/portal and discipline dependent, with a large fraction of users seeming to only use specific tools for rather limited lifetimes (approximately 0.5 to 1 years). Bonvin suggests that e-infrastructures should only expect a fraction of potential users to actually use it at any given time.

Unfortunately, as researchers don't always acknowledge the full array of tools that helped them with their work, developing an effective reward system for those developers and architects working behind the scenes will be vitally important going forward.

Read more: Measuring the success of European e-infrastructures

This article was first published on the GridCast blog. You can read the original version, here.

Join the conversation

Do you have story ideas or something to contribute? Let us know!

Copyright © 2023 Science Node ™  |  Privacy Notice  |  Sitemap

Disclaimer: While Science Node ™ does its best to provide complete and up-to-date information, it does not warrant that the information is error-free and disclaims all liability with respect to results from the use of the information.

Republish

We encourage you to republish this article online and in print, it’s free under our creative commons attribution license, but please follow some simple guidelines:
  1. You have to credit our authors.
  2. You have to credit ScienceNode.org — where possible include our logo with a link back to the original article.
  3. You can simply run the first few lines of the article and then add: “Read the full article on ScienceNode.org” containing a link back to the original article.
  4. The easiest way to get the article on your site is to embed the code below.