- Forecasting weather patterns that cause extreme events like heat waves remains a challenge
- Advanced numerical models that run on powerful supercomputers extend to only six days
- Applying deep learning to pattern recognition can improve cost and accuracy of forecasts
Weather forecasting played a crucial role in winning the Second World War.
D-Day, the largest seaborne invasion in history, relied heavily on weather conditions. June 5, chosen by Supreme Allied Commander General Dwight Eisenhower to be D-Day, was the first date in a narrow three-day window with the necessary weather conditions.
However, the weather on D-Day was far from ideal, and the operation had to be delayed 24 hours, until June 6, 1944. According to military planners and meteorologists, all other dates considered would have failed.
In addition to defeating Nazis, accurate weather predictions are important for planning our day-to-day activities. Farmers need weather information to help them plan for the planting and harvesting of crops. By law, planes aren't allowed to fly without first obtaining a weather briefing. The same goes for ships at sea.
However, extreme weather events such as extended hot and cold spells that can produce deadly heat waves and winter storms are entirely different. They can have dire impacts on public health, the environment, and the economy.
Forecasting the weather patterns that cause extreme weather events is challenging despite decades of efforts and advances in numerical weather prediction (NWP). Modern forecasts use mathematical models of the atmosphere and oceans to predict the weather based on current weather conditions.
Even with the increasing power of today's supercomputers, the forecasting skill of numerical weather models extends to only about six days, although there is some dependence on location, season, and type of weather pattern.
Persistent weather patterns that often drive extreme events are particularly hard to forecast. Improving the forecast of such events using NWP requires using higher resolution models and running more simulations. But these demand enormous computational resources.
Pedram Hassanzadeh of Rice University, and his PhD students Ashesh Chattopadhyay and Ebrahim Nabizadeh, recently introduced a data-driven framework that: 1) formulates extreme weather prediction as a pattern recognition problem, and 2) employs state-of-the-art deep learning techniques.
The advantage of a data-driven framework is that once trained on observational and/or high-resolution numerical model data, it can provide relatively accurate predictions at very little computational cost.
"Generally, the numerical weather models do a good job predicting weather, but they still have some difficulties with extreme weather," Hassanzadeh said. "We're trying to do extreme weather prediction in a very different way."
As a proof-of-concept demonstration, Hassanzadeh and team predicted heat waves and cold spells over North America using limited information about the atmospheric circulation at an altitude of around five kilometers, and in some cases, the surface temperature a few days earlier.
The results of their demonstration suggest that extreme weather prediction can be done as a pattern recognition problem, particularly enabled by recent advances in deep learning.
"We found that because the relative position of weather patterns play a key role in their evolution, using a more advanced deep learning method that tracks the relative position of features improves the accuracy and is also more robust when we don't have a large amount of data for training," Hassanzadeh said.
Interestingly, pattern matching is the way people started doing weather prediction before and during the Second World War. In that era, people barely scratched the surface of what is possible today.
During that time, people did weather prediction by looking through catalogs of weather patterns and pattern matching—this is called analog forecasting. But meteorologists abandoned this approach after World War II once computers became more widely available.
The analog technique is a complex way of making a forecast, requiring the forecaster to remember a previous weather event that is expected to be mimicked by an upcoming event. But there is rarely a perfect analog for a future event.
"In this paper, we show that with deep learning you can do analog forecasting with very complicated weather data — there's a lot of promise in this approach," Hassanzadeh said.
To obtain their results, the researchers analyzed large data sets and employed machine learning codes on supercomputers at the Texas Advanced Computing Center (TACC) and the Pittsburgh Supercomputing Center. In addition, they used data that had already been produced by supercomputers at the National Center for Atmospheric Research as input for the deep learning models.
"Our work would not have been possible without XSEDE (Extreme Science and Engineering Discovery Environment)'s computing resources," Hassanzadeh said. "Stampede2, Wrangler, and Bridges enabled us to do this work. We have supplemental systems at Rice, but Stampede2 is the main supercomputing resource that my group uses, and Bridges enables us to efficiently work with very large datasets."
According to Hassanzadeh, a growing number of people in the weather and climate community are interested in how deep learning can help improve climate and weather modeling.
"I think we're showing people that this approach works," he said. "The next step for my group is to see if deep learning can be more accurate than the operational numerical weather models used for day-to-day weather forecasts.”
He continues, “We may be able to train the neural networks using observational data, and it might work better and more accurately than what you get from the numerical weather models for predicting extreme events. We're going to focus on predictions with longer lead times, where the numerical models perform poorly. If it works, it will be a huge advance in weather prediction."