- First 3D high resolution blood flow model takes prize.
- Sequoia supercomputer simulates full human circulatory system to 10 microns.
- Higher resolution models lead to better heart pumps and lives saved.
Visualizing smaller and smaller components of the human body is one advantage supercomputers bring to scientists. Take Duke University biomedical engineering professor Amanda Randles, for instance. With some heavy lifting from the Blue Gene Sequoia supercomputer, Randles has introduced Harvey, the first 3D high-resolution model of the full human circulatory system.
Computer simulations help scientists understand factors that affect blood flow throughout the body. Models can also lead to better surgical options and inform the design of better drug delivery methods.
But to achieve real scientific discovery, bigger and faster computers are required. In the case of modeling the human circulatory system with its peculiar and complex geometry, higher resolution and larger scales call for high-performance computing. To extend the reach of vascular models, Randle looked to the supercomputers at the Lawrence Livermore National Laboratory.
“Supercomputer simulations allow us to model the circulatory system at unprecedented fidelity,” says Randles. “It’s an exciting research area because we’re creating patient-specific blood flow models that can better inform clinical decision making.”
Until now, most circulatory models focused on blood flow in specific regions of the body -- the heart or head. Randles' team was the first to successfully model the entire human circulatory system, receiving the best paper award at the 2015 International Conference on Computational Science.
Supercomputer simulations allow us to model the circulatory system at unprecedented fidelity. ~Amanda Randles.
And whereas previous research had managed an impressive one billion fluid nodes, Randles’ team shatters that mark with a 9 μm (micron) simulation that juggles over 500 billion fluid nodes. This record helped place them as a finalist for the ACM Gordon Bell prize for achievement in high-performance computing.
An impressive effort, yet Randles' team couldn’t have managed it without first scaling the heights of the Blue Gene supercomputer. Operating at a theoretical peak of 20 petaFLOPS, Sequoia enabled Randles’ group to perform simulations at a 10 μm resolution. (Human hair is closer to 100 μm; thickness of paper, about 180 μm.)
“Dealing with memory bandwidth, load balancing, and footprint limitations of current supercomputers is a constant struggle,” says Randles. “This work taught us how to leverage the unique hardware of systems like Blue Gene to push simulations to unprecedented scales.”
Randles is working with David Frakes of Arizona State University to validate these simulations. Frakes is 3D printing subsections of the human vasculature to compare the results of the simulation to a known outcome. Complex vessel geometries also need to reflect what happens under different flow conditions.
“Modeling each patient-specific case physically is often time and resource prohibitive in the clinical environment," says Frakes. “Simulation gives us a way to answer important pre-operative questions that just couldn't be answered at scale any other way.”
With these higher resolution models, Randles sees great potential for insight into the mechanisms that drive disease progression and clinicians' ability to assist with treatment planning. Being able to simulate blood flow effects before ever entering the operating room will be a great benefit.
Simulations at this resolution will yield more realistic vascular models and better heart-mimicking pumps, which will save human lives. The holy grail Randles seeks is to simulate individual bodies suspended within the blood stream – such as red blood cells or circulating tumor cells.
Once again, supercomputers matter. The computational tools available to the modern scientist are circumventing the restrictions of previous eras.