When the staff of the European Middleware Initiative sent out their first user survey, they only expected to receive 50 or so responses. Instead, said Alberto Di Meglio, project director of EMI, they received over 243 as of January 20, almost a full month before the survey closed.
Like a flock of birds in flight, all grid middleware can work as one unit, while each keeps its independence. Image courtesy Asif Akbar, stockexchng
For Di Meglio, the volume of responses was a pleasant surprise. Most responses came from people who described themselves as grid experts, who use the grid for 64-bit processing - the higher end of the scale.
The respondents' overall message was clear: the grid is indispensable for their jobs, but could be improved.
To avoid the possibility of bias, Di Meglio had randomized the survey by sending it to various grid communities and projects supported by the Seventh Framework Program of the European Commission, such as WLCG and EGI.
Di Meglio thinks the reason the survey drew so many responses was that his questions were specific, focused, concrete, and relevant to users' daily work: What exactly do you use middleware for? On which operating systems? If the grid disappeared tomorrow, would you be able to do your work?
There may be another reason for the high response rate: "At this moment, we are seeing a transition from grids to clouds or something in-between," said Di Meglio. "Therefore, users want their opinions to be heard during this transition period."
The responses also showed something else. "This demonstrates that grid technologies are still heavily in use," commented Florida Estrella, EMI deputy project director.
The survey's timing was important, because Di Meglio and his team are planning for the future development of grid middleware. Once full survey results are in, a baseline can be established for grid developers to prioritize their strategies. EGI project director Steven Newhouse highlighted "The challenge for EGI and EMI is to now drive the development of the EMI software to improve the experience and functionality offered to end-users across all communities".
These improvements include increasing the output of the grid, and reducing the effort and time it takes to learn and use middleware.
Coherence vs complexity
One possible reason for the steep learning curve may be that, there are currently four major EMI European middleware providers (gLite, ARC, UNICORE and dCache), each with their own interface. This situation can be frustrating for new users, who need to learn different commands and tools depending upon which middleware they use.
To fix this, the EC tasked the EMI project with combining these four grid middlewares to form a single more intuitive and user-friendly middleware, removing duplications and redundancies. EMI's goal is to provide users with a coherent set of tools and hide the grid's complexity.
EMI's vision for grid middleware. Image courtesy Alberto Di Meglio
Grid middleware has not yet reached a level of critical mass where open standards - such as those that enable the World Wide Web to function so well - are widely used. The web's underlying common set of protocols, such as 'http,' allow a user to move freely from one website to another, without having to interact with the internet's complexity. Grid middleware should function similarly, said Di Meglio. The challenge for EMI is to determine a set of best practices that the various middlewares can use.
Di Meglio concludes: "The important point is not whether grids should move to clouds for example, but that a collaborative approach should be made to develop long-lasting open standards for distributed computing and data-management middleware. Users should be able to move their processing jobs around different grids without worrying about learning a new submission interface."
"So, scientists can just do science," said Di Meglio.
--The full survey results will be available at the EMI website.
