iSGTW is now Science Node Learn more about our evolution

  • Subscribe

iSGTW Feature - GridFTP moves your data and your news

Feature - GridFTP moves your data and your news

Cluster-to-cluster data movement with GridFTP using up to 64 nodes on each end
of TeraGrid's 30 Gbit/s wide area network between Urbana, IL and San Diego, CA.

Images courtesy of Raj Kettimuthu (plot) and Corky and Holly Siegel (duck).

GridFTP, a data transfer protocol optimized for high-bandwidth wide-area networks, handles an average of more than 2.5 million data transfers a day.

The Large Hadron Collider, the Southern California Earthquake Center, the Relativistic Heavy Ion Collider, the Laser Interferometer Gravitational Wave Observatory, the European Space Agency, the Disaster Recovery Center in Japan and even the British Broadcasting Corporation use it.

Based on the old workhorse FTP, GridFTP supports reliable and restartable data transfers and provides extensions for high-performance operation and security. It is a specification (meaning anyone can write code to implement it) for which standards are defined within the Open Grid Forum. Globus Alliance provides a reference implementation, and frequent upgrades keep it current and competitive.

To overcome the performance limitations of TCP (a veteran core transmission protocol of the Internet protocol suite), GridFTP supports parallel TCP streams over high-speed wide-area network links and allows users to set optimal TCP buffer size for a transfer. These features offer roughly an order of magnitude (a factor of ten) improvement in performance over the standard FTP. Globus GridFTP also operates on UDT, a newer transfer protocol designed for extremely high speed networks.

GridFTP supports coordinated data transfer using multiple computer nodes at both the source and destination, adding another order of magnitude improvement in performance for network links that support much higher data rates than individual nodes at either end. With this feature, GridFTP has delivered over 25 Gb/s on a 30 Gb/s TeraGrid link using 32 nodes at both ends.

Raj Kettimuthu

Image courtesy of ANL.

The GridFTP server in the Globus Toolkit version 4.2, to be released in summer 2008, sports some new features.

For security, Secure Shell (SSH) is now on the menu, as an alternative to Grid Security Infrastructure, the de facto authentication standard for grid computing. The pipelining feature lets the client send requests while previous requests are still completing, and thus reduces overhead and speeds up the transfer of many small files. The new version also supports multicasting (one-to-many transfers) and GridFTP overlay routing, in which GridFTP servers can be used as intermediate nodes to route data.

-Raj Kettimuthu, Argonne National Laboratory and University of Chicago, technology coordinator for Globus GridFTP

Join the conversation

Do you have story ideas or something to contribute?
Let us know!

Copyright © 2015 Science Node ™  |  Privacy Notice  |  Sitemap

Disclaimer: While Science Node ™ does its best to provide complete and up-to-date information, it does not warrant that the information is error-free and disclaims all liability with respect to results from the use of the information.


We encourage you to republish this article online and in print, it’s free under our creative commons attribution license, but please follow some simple guidelines:
  1. You have to credit our authors.
  2. You have to credit — where possible include our logo with a link back to the original article.
  3. You can simply run the first few lines of the article and then add: “Read the full article on” containing a link back to the original article.
  4. The easiest way to get the article on your site is to embed the code below.