The universe is a vast and mysterious place, but we are beginning to understand it better thanks to some powerful technology. Scientists around the world are using supercomputers to simulate how the big bang led to the formation of galaxies, such as our own Milky Way. A new project sponsored by three of the US Department of Energy's (DOE's) National Labs will enable scientists to study this vastness - with a new cosmological simulation analysis toolbox - in greater detail.
Modeling the universe with a computer is a highly complex task. To simulate the evolution of galaxies, scientists look to supercomputers for help. Simulations that produce galaxies also produce extreme amounts of data - each dataset could potentially require hundreds of terabytes of storage. Many different scientific analyses and processing sequences are carried out with each data set, making it impractical to rerun simulations for each new study. Efficient storage and sharing of data among scientists is paramount.
Fermi National Accelerator Laboratory (Fermilab), near Chicago, Illinois, US, is developing a partnership with Argonne and Lawrence Berkeley National Laboratories to develop a state-of-the art, cosmological simulation analysis toolbox. The partnership seeks to take advantage of the DOE's investments in supercomputers and high-performance computing codes.
"The three labs are developing an open platform, web-based front end that will enable the scientific community to download, transfer, manipulate, search, and record simulation data," says Robert Roser, head of Fermilab's scientific computing division. "Scientists will be able to upload and share applications, as well as carry out complex computational analyses."
The team is enhancing existing high-performance computing, high-energy physics, and cosmology-specific software systems to handle the large datasets of galaxy-formation simulations. Team members are also benefiting from expertise they've gained by working on the big data challenges posed by particle physics experiments at the Large Hadron Collider at CERN, near Geneva, Switzerland.
"This is an exciting project for Fermilab, Argonne, and Berkeley Labs. Large-scale simulations of cosmological structure formation are key discovery tools in the department's Cosmic Frontier program," Roser says. "Not only will this new project provide important tools for Cosmic Frontier scientists and the many institutions involved in this research, but it will also serve as a prototype for a successful big data software project spanning many groups and communities."
Roser will speak next in October at the Big Data Conference in Chicago, IL, US.