• Subscribe

LIGO and OSG peer into the Dark Energy Camera

Speed read
  • LIGO's gravitational wave detection has sparked a hunt for the wave's source. 
  • Dark Energy Camera (DECam) joins the hunt, sending cosmic images from Chile to scientists at Fermilab.
  • The Open Science Grid enables rapid processing of the dense DECam images.

On September 14, 2015, gravitational waves were directly observed for the first time by both detectors of the Laser Interferometer Gravitational-wave Observatory (LIGO). The detection confirmed Einstein’s proposal in his general theory of relativity. Now, scientists are seeking the wave source.

(Check out our coverage of the historic LIGO announcement here.)

Seeking the source

While LIGO, funded by the US National Science Foundation (NSF), can pick out the general direction of the source of gravitational waves, it can’t identify the exact location. So, LIGO scientists are coordinating their measurements with observations made by the Dark Energy Camera (DECam) on the Blanco Telescope at the Cerro Tololo Inter-American Observatory in Chile.

<strong>Dark eyes.</strong> The Dark Energy Camera (DECam), ready to observe 300 million galaxies and discover thousands of supernovae. Scientists at Fermilab are looking through this camera to find the source of the gravitational waves detected by LIGO. Courtesy Reidar Hahn, Fermilab.

Scientists at Fermilab and other institutions in the Dark Energy Survey use the camera to understand dark energy — a force scientists believe is helping the universe expand. A subset of members known as the Dark Energy Survey-Gravitational Wave (DES-GW) group are using the camera and the Open Science Grid (OSG) to build on LIGO’s groundbreaking findings.

“Our focus primarily is the search for dark energy,” says Marcelle Soares-Santos, associate scientist at the US Department of Energy’s Fermilab. “Since we have experience detecting things through magnetic emissions, we coordinated with LIGO to find a source that we would find useful in our own research. Unfortunately this time we did not see anything, but we are now much better prepared when LIGO becomes active again later this year.”

How to find a needle in a universe-sized haystack

The area of sky DES-GW members observe is very large — 700 square degrees of sky, or about 2,800 times the size of the full moon — and requires rapid image processing. That’s where the OSG comes in. Without OSG, Soares-Santos says they couldn’t keep up.

“For this event, we had something like 4-5,000 jobs. We must break every image down into smaller parts and process them in parallel on the OSG. It is critical to get our results fast — within 24 hours.”

Dark Energy Camera

  • DECam field of view so large that one image records 20 times the size of the moon as seen from Earth. The camera can see light from up to eight billion light years away, and captures more than 100,000 galaxies in each digital image. DECam takes about 400 of these pictures every night, each about 1 GB.
  • DECam relays these images via microwave link to the Universidad de La Serena in La Serena, Chile. From there, they are sent via national research and education network Internet2 to the US National Center for Supercomputer Applications (NCSA) in Champaign, Illinois. The NCSA reduces and corrects the images, identifies and catalogs stars and galaxies, and stores everything in a database.

Scientists then analyze their observations with a spectrograph — which is expensive — so it’s important to narrow the choices down to only a few candidates. “At first, our turnaround time was not very fast, but thanks to our close partnership with the computing side here at Fermilab, now it is. We have great confidence that when LIGO observations start again in early August, we will be ready and hopefully see something.”

Kenneth Herner, an application developer and systems analyst at Fermilab, is one of those key partners on the computing side. He makes sure the DES group has as many resources as they need and devotes part of his time to OSG.

“Opportunistic OSG resources really help with the computing needs and the time crunch,” says Herner. “When we submit jobs, we get the first resources that meet the requirements no matter where they may be. We use the CernVM File System to pull in a code repository over HTTP to a local cache on a worker node. It only pulls down what it needs as it needs it. We don’t have to configure each OSG site — it just works. All OSG sites then look the same and all the site has to do is mount a repository.”

In preparation for the LIGO partnership, Herner’s group prepared a code pipeline and made sure everything would work. The LIGO alert came on the 14th. “We had to wait on the telescope—and on top of that an earthquake in Chile,” says Herner. “We worked our plan, checked our code, transferred images from Chile up to the US, and submitted our jobs.”

"We couldn’t do it without the OSG" ~ Marcelle Soares-Santos

Almost all the jobs ran at Fermilab, but Herner says they could have gone anywhere on the OSG. “This was our shakedown cruise,” said Herner. “The first event used about 15,000 CPU hours for a full pass over all nights, but with multiple passes and preprocessing it was over 25,000 hours.” Without OSG resources, the group would have taken Fermilab computing resources away from other experiments, he says.

Observing the sources of these gravitational waves will tell Soares-Santos how systems work and give her and her colleagues deeper insight into the physics. “It is quite challenging to observe these events,” says Soares-Santos.

“We have to be quick to respond to see them. We have to be on the spot sooner and it is the computing that makes that possible. We couldn’t do it without the OSG because of the volume of data. We must have massive parallel computing and quick turnaround and hopefully next time we will see something exciting.”

Read the original version of this OSG science highlight here. 

Read more about the Dark Energy Camera here, and learn about the DECam's Legacy Survey here.

Join the conversation

Do you have story ideas or something to contribute?
Let us know!

Copyright © 2017 Science Node ™  |  Privacy Notice  |  Sitemap

Disclaimer: While Science Node ™ does its best to provide complete and up-to-date information, it does not warrant that the information is error-free and disclaims all liability with respect to results from the use of the information.


We encourage you to republish this article online and in print, it’s free under our creative commons attribution license, but please follow some simple guidelines:
  1. You have to credit our authors.
  2. You have to credit ScienceNode.org — where possible include our logo with a link back to the original article.
  3. You can simply run the first few lines of the article and then add: “Read the full article on ScienceNode.org” containing a link back to the original article.
  4. The easiest way to get the article on your site is to embed the code below.