Speaker
Description
Particle-in-Cell (PIC) Monte Carlo (MC) simulations of the plasma edge play an important role in magnetic confinement fusion research, providing insights into plasma-surface interactions and plasma, impurity and neutral particle transport, relevant to both present and future fusion devices. As plasma simulations continue to increase in size and complexity, efficiently exploiting modern heterogeneous supercomputing systems presents significant challenges, including data movement, load imbalance, scalability, and resilience. This talk presents recent developments in advancing the Berkeley Innsbruck Tbilisi 1D3V (BIT1) PIC MC code [1,2] towards exascale computing through a portable hybrid MPI+OpenMP implementation capable of leveraging large-scale multi-GPU systems across both NVIDIA and AMD architectures. The work focuses on practical aspects of code development and optimisation, including GPU offloading strategies, overlapping communication and computation, memory management approaches, particle load balancing, and scalable checkpoint/restart capabilities. The integration of openPMD and ADIOS2 for standardised high-performance I/O is also discussed, enabling parallel I/O [3], enhanced diagnostics [4], in-memory data streaming [5], and in-situ analysis and visualisation workflows [4,5]. Performance and scalability results from leading pre-exascale and exascale systems, including Frontier, LUMI-G and MareNostrum 5 ACC are presented, together with lessons learned from profiling [6], porting [7,8], and enabling portability [9]. The talk concludes with ongoing efforts to improve resilience and workflow efficiency [10], together with future research extending hybrid BIT1 to Intel GPUs towards exascale, targeting Aurora and Europe's first exascale system, JUPITER Booster, to assess portability and performance across NVIDIA, AMD, and Intel GPUs and identify any remaining architecture-specific limitations at scale.
References
[1] D. Tskhakaya, et al., “Optimization of PIC codes by improved memory management,” Journal of Computational Physics, vol. 225, no. 1, pp. 829–839, 2007. doi:https://doi.org/10.1016/j.jcp.2007.01.002
[2] D. Tskhakaya, et al., “PIC/MC code BIT1 for plasma simulations on hpc,” in 2010 18th Euromicro, pp. 476–481, IEEE, 2010. doi:https://doi.org/10.1109/PDP.2010.47
[3] J.J. Williams, et al., "Enabling high-throughput parallel I/O in particle-in-cell Monte Carlo simulations with OpenPMD and darshan I/O monitoring." in 2024 IEEE International Conference on Cluster Computing Workshops (CLUSTER Workshops), IEEE, 2024. doi:https://doi.org/10.1109/CLUSTERWorkshops61563.2024.00022
[4] J.J. Williams, et al., “Understanding the Impact of OpenPMD on BIT1, a Particle-in-Cell Monte Carlo Code, Through Instrumentation, Monitoring, and In-Situ Analysis,” in European Conference on Parallel Processing, pp. 214–226, Springer Nature Switzerland, 2024. doi:https://doi.org/10.1007/978-3-031-90200-0_18
[5] J.J. Williams, et al., “Integrating High Performance In-Memory Data Streaming and In-Situ Visualization in Hybrid MPI+ OpenMP PIC MC Simulations Towards Exascale,” The International Journal of High Performance Computing Applications, p. 10943420251409229, 2025. doi:https://doi.org/10.1177/10943420251409229
[6] J.J. Williams, et al., “Leveraging HPC Profiling and Tracing Tools to Understand the Performance of Particle-in-Cell Monte Carlo Simulations,” in European Conference on Parallel Processing, pp. 123–134, Springer Nature Switzerland, 2023. doi:https://doi.org/10.1007/978-3-031-50684-0_10
[7] J.J. Williams, et al., “Optimizing BIT1, a Particle-in-Cell Monte Carlo Code, with OpenMP/OpenACC and GPU Acceleration,” in International Conference on Computational Science, pp. 316–330, Springer Nature Switzerland, 2024. doi:https://doi.org/10.1007/978-3-031-63749-0_22
[8] J.J. Williams, et al., “Accelerating Particle-in-Cell Monte Carlo Simulations with MPI, OpenMP/OpenACC and Asynchronous Multi-GPU Programming,” Journal of Computational Science, vol. 88, p. 102590, 2025. doi:https://doi.org/10.1016/j.jocs.2025.102590
[9] J.J. Williams, et al., “Multi-GPU Hybrid Particle-in-Cell Monte Carlo Simulations for Exascale Computing Systems,” in International Conference on Computational Science, pp. 32–47, Springer Nature Switzerland, 2026. doi:https://doi.org/10.1007/978-3-032-29921-5_3
[10] J.J. Williams, et al., “High-Performance Resilient Multi-GPU Hybrid Particle-in-Cell Monte Carlo Simulations at Scale,” in Euro-Par 2026 Workshops (BIGHPC), Accepted For Publication, 2026