Visualizing and Exploring Findings as Highly Accurate Simulations

Most science research programs that run on high-performance computers like the IBM Blue Gene/P Intrepid at the Argonne Leadership Computing Facility (ALCF) generate enormous quantities of data that represent the results of their calculations. But scientists can also use the ALCF to visualize, explore and communicate their findings as highly accurate simulations and often beautiful images. Argonne operates the ALCF for the DOE Office of Science as part of the larger DOE Leadership Computing Facility strategy. DOE leads the world in providing the most capable civilian supercomputers for science.

The ALCF's ability to visualize such enormous quantities of data is made possible by one of the world's largest graphics processing units (GPU). Nicknamed Eureka, this installation of NVIDIA Quadro Plex S4 external GPUs allows researchers to better understand the data they produce with Intrepid at the ALCF. The powerful installation provides more than 111 teraflops and more than 3.2 terabytes of RAM.

"Eureka provides a vital link between simulation and analysis by allowing scientists to probe and interrogate their data in an interactive manner," said Argonne computational scientist Paul Fischer. Since Eureka and Intrepid share a disk, there is no need to move data between machines. " Eureka dramatically reduces the amount of time needed to create these hugely complex visualizations, while greatly boosting their quality."

The ALCF's Intrepid provides resources for the U.S. Department of Energy's (DOE) Innovative and Novel Computational Impact on Theory and Experiment (INCITE) program, which supports computationally intensive projects from industry, scientific researchers and research organizations.

Using software developed both at Argonne and externally, computer scientists have visualized data with Eureka for DOE INCITE projects focusing on turbulent thermal transport in sodium-cooled nuclear reactor cores, cardiac rhythm disorders and Type Ia supernovae, which are among the brightest and most powerful exploding stars in the universe.

"Eureka delivers a quantum leap in visual compute density, enabling breakthrough levels of productivity and capability in visualization and data analysis," said Craig Dunwoody, CEO of GraphStream, Inc. in Belmont, Calif., the supplier of scalable computer systems that provided Eureka.

Eureka incorporates four high-end graphics cards and places them in a configuration known as a "pizza box." Because the cards are packed so closely together, this configuration helps to reduce the complicated power and cooling issues associated with the graphics cards. Eureka needs only four racks to hold the same number of cards that previous configurations required more than 10 racks to accommodate.

The heart of Eureka's data-management system contains a nine-switch complex that supports up to 2,048 connections, each of which simultaneously exchanges data at roughly 1 billion bytes per second. The storage system consists of a bank of more than 10,000 disk drives that will send and receive data from the Blue Gene/P's more than 100,000 processors. Altogether, this system can deliver nearly 80 billion bytes per second to and from the disk—the equivalent of transferring the content of 100 full CDs every second.