Wolfgang Gentzsch, co-founder of the UberCloud HPC experiment and advisor to EUDAT speaks exclusively to iSGTW about the challenges facing cloud computing in the run up to this month's Cloudscape V conference in Brussels, Belgium (27-28 February).
Perhaps you could start by telling us why sustainability, particularly in relation to clouds, is so important for research?
Sustainability in the context of cloud infrastructures means the long-term maintenance of their services for a wider user and customer community. Researchers who previously have used their own limited resources are benefiting tremendously from clouds: for example, by avoiding long waiting times for their compute jobs; enjoying on-demand availability of resources; experiencing no peak-demand bottlenecks; having the best-suited resources always available at their fingertips; not having to deal with cumbersome procurements; and having no need to maintain their own expensive IT infrastructure. Clouds serve larger communities and, therefore, sustainability is key.
And what do you think the main challenges to achieving this sustainability are?
As with any EU-funded project in both the past and the present, the huge challenge arises that the project's deliverables should be maintained after the end of the project, when funding dries out. But where should the money come from to continue providing the novel infrastructure, software, or services to a wider community. Sure, the community could pay for these services, but where should that money come from? I see two major ways (and several derivatives) for this: First, to pay for it themselves, the users of these novel services could use the money they previously spent on existing (or precursor) services, which they then successively discontinue and replace by the new and better services. Second, one could find new user communities, which will also benefit from these novel services, for example industries.
Certainly, there are more challenges for building sustainable clouds; in fact, to guarantee sustainability of the whole infrastructure, we have to ensure sustainability of individual areas, such as technology, operations, expertise, communities, collaborations, and the eco-political landscape. But, since the early days of grid computing, we have learned how to tackle many of these challenges, and the 'Top 10 Rules for Building a Sustainable Grid', which I defined in an article for the Open Grid Forum (OGF) in 2007, are also mostly true for cloud research infrastructures.
How well do you think Europe is doing at overcoming these challenges, particularly compared to other regions, such as the US and Asia?
Europe has a long history of orchestrating challenging research across all EU member states and communities, at least going back 30 years, to the early Esprit project in 1983. Since then, thousands of innovative technology projects have been performed, with great opportunities for research and industry, strengthening Europe's competitiveness in the world. These concerted efforts of the best and brightest minds in Europe have led to an eco-system of continuous innovation, research, development, and deployment, which makes tackling any challenge a relatively common endeavor.
Is federation of clouds more difficult to achieve in Europe, compared with the US for example, due to different legal frameworks existing from country to country?
A cloud federation is the deployment and management of multiple public and/or private cloud computing services to match research or business needs. When using multiple clouds for your workflow you obviously face even more challenges than with using just one cloud or cloud service; the most outstanding one being interoperability which has been addressed by OGF's OCCI working group, the SIENA project and others, and some of them will be discussed at CloudScape V in Brussels. So, here again we are collecting a lot of experience regarding how to cope with the challenges which are typically European, i.e. multi-cultural, multi-lingual, and multi-legal frameworks. So I would say, yes, federation of clouds is more challenging in Europe. But Europe, because of its concerted efforts, is used to coping with all kinds of challenges.
How about industry? What role does this play in driving cloud development forward for the research community?
Industry, in my opinion, has had an undisputed head start. While in the early 2000s research was strongly focused on grid computing - for good reasons, such as collaboration and resource sharing - large IT firms like Sun Microsystems started big virtualization projects like N1 in 2001 which later became fundamental for business clouds. In the beginning, this was more a great technology effort. But this changed as soon as virtualization technology was widely available and companies were able to easily build compute power plants, thus offering computing on demand. Especially in the case of industry, which always cares deeply about the economics, this new cloud computing trend opened new opportunities for introducing greater dynamics into all layers of the business process, in R&D, in production, and in making marketing and sales more flexible and adjustable instantly to customer needs.
Only recently have we seen research clouds starting to bloom, like Helix Nebula, for example. Now, the research community can greatly benefit from the tremendous effort from industry in cloud computing in the past few years allowing for fast moving forward, often with a mix of grid and cloud infrastructure.
The motto of this year's Cloudscape event is 'cloud for savings, cloud for quality'. Is this what it's really all about then: researchers feel that - with government budgets tight - they will be able to do more for less with the cloud? Or is there more to it than that?
Oh, it is much more than that. The cost factor is certainly an important part, when considering the immense cost of acquiring and continuously maintaining the latest and greatest IT equipment. And yes, the 'quality' factor is important as well, allowing for a much more in-depth exploration of new products and services. In addition, as I mentioned previously, with cloud computing being part of your infrastructure, you are able to create much more dynamic processes and workflows, reducing time to result or time to market, and reacting flexibly to changing customer requirements or markets. These are all very important factors for strengthening the competitiveness of a company, a research team, and of all Europe, eventually.
I believe you'll also be giving a presentation on The UberCloud HPC Experiment, which you're leading. Can you tell us a little about the project?
The UberCloud HPC Experiment aims at exploring the end-to-end process for SMEs to access and use remote computing resources - 'HPC as a Service' - in remote datacenters and in clouds. Burak Yenier and I have come up with 22 steps describing in detail this end-to-end process. In round one, from August to November last year, we formed 25 teams from the 160 participating organizations around the world, each team consisting of the industry end-user and their application, the software provider, the resource provider, and an HPC Expert to help all team mates get software and applications onto the remote computing resource and perform the manufacturing simulation, closely following these 22 steps. For the summary report, we have selected 15 use cases, and I will present some of them and the lessons learned at the CloudScape V conference in February. The UberCloud HPC Experiment is now in round two (from December to the end of March) and has over 330 organizations registered. We are currently very busy forming 58 teams (so far) and continuing to explore 'HPC as a Service'.
What have the main challenges for the project been so far?
Because of the complexity of HPC in the cloud, each team is facing its own individual challenges and some of them are reaching out to industry end-users, who are ready and willing to engage in HPC (especially HPC in the cloud). The experiment, however, is usually not the highest priority of our participants' day jobs, meaning delays are quite common. About half of our participants also want to remain anonymous, for various reasons, including internal policies and processes, fear of failure, etc.
HPC is complex and at times it requires multiple experts to satisfy an end-user's requirement. Optimal matching of participants is critical to a team's success and resource providers may often have to deal with issues relating to internal policies, legal hurdles, and licenses. Learning the many different access and usage processes (there are, unfortunately, no standards) of the different hardware and software providers can also take many days and process automation on behalf of our providers varies greatly - some have focused on automation, while others haven't.
Finally, are there any other sessions you are particularly looking forward to at Cloudscape V?
Absolutely. I'm looking forward to the keynote from my friend Dennis Gannon from Microsoft on 'Cloud4Science - Empowering the research community' and another keynote focusing on the economic benefits and strategic role of cloud adoption. Konstantinos Glinos's session on sustainable clouds for research should also be very interesting. In addition, there are going to be sessions on sustainability, security, collaboration between industry and research, open collaborative models, open data, and big data (this is especially valuable for my contribution to the EUDAT project). Finally, there will also be sessions on cloud for new government services, an expert panel on standards and interoperability, and a few talks aimed at guiding SMEs to the cloud. Altogether, this CloudScape V conference is a must for newcomers and for experts.