Particle tracking in large-scale numerical simulations of turbulent flows presents one of the major bottlenecks in parallel performance and scaling efficiency. Here, we describe a particle tracking algorithm for large-scale parallel pseudo-spectral simulations of turbulence which scales well up to billions of tracer particles on modern high-performance computing architectures. We summarize the standard parallel methods used to solve the fluid equations in our hybrid MPI/OpenMP implementation. As the main focus, we describe the implementation of the particle tracking algorithm and document its computational performance. To address the extensive inter-process communication required by particle tracking, we introduce a task-based approach to overlap point-to-point communications with computations, thereby enabling improved resource utilization. We characterize the computational cost as a function of the number of particles tracked and compare it with the flow field computation, showing that the cost of particle tracking is very small for typical applications.
翻译:在大规模数字模拟中对动荡流进行粒子跟踪是平行性能和规模化效率方面的一个主要瓶颈。 在这里, 我们描述一个大规模平行的平行伪光谱风暴模拟的粒子跟踪算法, 该算法在现代高性能计算结构中将高达数十亿微粒的微粒缩成比例。 我们总结了我们混合的MPI/ OpenMP 实施过程中用于解决流体方程的标准平行方法。 作为主要重点, 我们描述粒子跟踪算法的实施情况并记录其计算性能。 为了解决粒子跟踪所需的广泛的进程间通信问题, 我们采用了基于任务的方法, 将点对点通信与计算重叠, 从而能够改善资源的利用。 我们将计算成本描述为所跟踪的粒子数量的一个函数, 并将其与流动场计算进行比较, 表明对于典型应用而言, 粒子跟踪的成本非常小 。