A Scalable In Transit Solution for Comprehensive Exploration of Simulation Data

As simulations produce more data than available disk space on supercomputers, many simulations are employing in situ analysis and visualization to reduce the amount of data that needs to be stored. While in situ visualization offers potential for substantial data reduction, its efficacy is hindered by the need for a priori knowledge. First, we need to know what visualization parameters to use to highlight features of interest. Second, we do not know ahead of time how much resources will be needed to run the in situ workflows, e.g. how many compute nodes will be needed for in situ work. In this work, we present SeerX, a lightweight, scalable in-transit in situ service that supports dynamic resource allocation and lossy compression of 3D simulation data. SeerX enables multiple simulations to offload analysis to a shared, elastic service infrastructure without MPI synchronization.