Whether you call it a science gateway, portal, or hub, online access to data, computational tools, and resources can be invaluable to researchers.
Nevertheless, funding for the ongoing development and maintenance of gateways is far from guaranteed. This prompts the question, "What makes one science gateway more successful than another?"
Our paper presents some preliminary results from the first stage of an applied research study of science gateways. Our goal for the project was to understand why some projects change the fundamental ways that a community conducts its research while other projects do not.
The study consists of a series of cross-disciplinary focus groups designed to identify the social and technical enablers of and barriers to successful projects. The first day-long focus group included three idea-generation exercises focusing on enablers of success, methods for working through barriers, and external forces influencing sustainability. The resulting paper draws some initial conclusions from the first focus group session, conducted in June 2010.
There were several recurring themes that emerged from the session: funding, project goals, tools, community engagement, and rewards and recognition. Many of these themes have implications with no straightforward resolution due to the trade-offs inherent in choosing one approach versus another or the challenges of achieving certain standards.
Funding: Development versus operations
Development timelines-and the funding associated with them-were a source of debate across the focus group. A complication for gateways is that technology and goals can change so quickly that something envisioned at the beginning of a multi-year project may need significant changes midway through. Participants widely agreed that it is useful to divide the development timeline into periods with synchronized funding, each with its own funding stream. Many found that funding the initial stages of a project can be considerably easier than funding the more operational or maintenance phases.
Project Goals: Research versus production
A second thematic tension was the contrast between research and production. Most gateways are funded as research projects, yet they require the construction and maintenance of infrastructure in order to support the research productivity of their target community. This tension between research and production manifests itself in several ways. For example, if both the developers and the end users are expecting to use the gateway to promote research interests, there may be a conflict. It may benefit gateway programs to be split into components, each with its own success criteria.
Tools: Standardized versus open-source versus custom
Participants expressed several different viewpoints related to the construction of gateways, specifically who should build them and how. Because most science and engineering gateways are housed in academic settings, students are plentiful as programmers. Likewise, the budget constraints of academic settings favor using inexpensive tools and techniques (such as open-source software) for building gateways. The question is whether such methods are ultimately more cost effective and whether they jeopardize the success of the projects. Several participants cited the use of professional software developers, a well-developed software engineering process, and industry-grade software as being critical to their success.
Community engagement: Delivering what the users want
Not one of the participants disputed the value and necessity of building gateways that are focused on usability and reliability. Although gateway development begins with real user experiences, developers must be prepared to adapt. Retaining a focus on content is important as well, as some participants have found that it can be easy to alienate researchers.
Rewards and recognition: Traditional versus new
A final tension is how to reconcile traditional, established forms of academic recognition with the need for new incentives to motivate gateway development. Gateway work enables research over the longer-term, but it is a very different type of research product and does not fit the usual template for evaluation. We need to identify what metrics we should use to measure success. Metrics can also help funding agencies evaluate the sustainability of a gateway and whether it merits additional funding. One goal of our study is to provide useful evaluation criteria to support longer-term funding decisions.
A few ideas emerged from our first focus group that are worth noting. One of these was the idea of forming a gateway consortium. Such a consortium could provide a software repository with tagging and reviewing opportunities as well as a venue to capture lessons learned. The notion was supported enthusiastically by participants.
The need for stable software foundations was also raised as an important issue; the software packages that underlie many gateways are themselves research projects, which could be completely redesigned at any time in the name of research. Software that is rarely altered, or that remains backwards-compatible when altered, could be invaluable to gateway developers.
Finally, to better understand what constitutes the success of a gateway, we discussed the idea of involving social scientists and monitoring a wide variety of metrics. Social scientists may be able to provide a perspective on usability that developers lack; metrics likewise could provide a greater understanding by yielding data on how people interact with the site, where they run into difficulty, and so forth.
ACKNOWLEDGEMENT: This material is based upon work supported by the National Science Foundation under Grant No. OCI-0948476. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation.