Advancing Cloud Modelling in HANAMI: Microphysics and HPC Optimisation of UWLCM

 

By Piotr Dziekan (University of Warsaw)

 

 

As part of the HANAMI project, the University of Warsaw is collaborating with researchers from the University of Hyogo, Japan, to advance high-fidelity cloud simulations and improve the computational efficiency of atmospheric models.

 

The cooperation focuses on the continued development of the University of Warsaw Lagrangian Cloud Model (UWLCM). The UWLCM employs an Eulerian-Lagrangian approach, coupling a traditional fluid-flow solver with the Super-Droplet Method (SDM) to simulate cloud microphysics at a highly resolved scale.

 

Over the past few months, our joint actions have focused on expanding the physical accuracy of the microphysics scheme, enhancing numerical efficiency, and porting our code to run on Japan’s Fugaku supercomputer.

 

In this article, we highlight three key areas of progress: the implementation of ice microphysics, the development of adaptive time-stepping for condensation and deposition, and the deployment of UWLCM on Fugaku.

 

 

Implementing Ice Microphysics

 

Prior to the HANAMI project, UWLCM was limited to warm-cloud (liquid-only) microphysics. A major goal of our current collaboration is to introduce a comprehensive ice microphysics scheme into the model. Building on the theoretical framework developed by our Japanese colleagues, we have successfully completed the first phase of this upgrade, which involved implementing the code to simulate spherical ice particles.

 

However, accurately representing the radiative and microphysical properties of mixed-phase and ice clouds ultimately requires moving beyond spherical approximations. We are now actively working on the next phase: tracking the shape of ice crystals by modelling them as oblate spheroids (Shima et al., 2020). By explicitly tracking the multidimensional growth histories and shape parameters of individual ice particles, we will be able to simulate differential fall velocities, collision-coalescence efficiencies, and habit-dependent growth rates with a much higher level of physical detail.

 

 

Adaptive Time-Stepping for Condensation and Deposition

 

One of the core numerical challenges in particle-based microphysics is the stiffness of the equations governing phase changes. Modelling condensation and deposition often requires a time step that is an order of magnitude shorter than the one used for air dynamics. Rather than forcing the entire model to operate with such a short time step, we use a technique known as substepping, in which multiple phase-change time steps are performed within a single dynamics time step.

 

In UWLCM, the number of substeps is the same for all super-droplets (which are computational proxies for real droplets). However, the phase relaxation timescales for condensation and deposition can vary with local supersaturation levels and the amount of aerosol dissolved within a droplet. To take advantage of this, within the HANAMI project, we are developing an adaptive time-stepping method for phase-change calculations. It works by setting the number of substeps for each super-droplet individually, taking into account the droplet’s characteristics and its environment. This has the potential to improve computational performance, since it requires fewer substeps, and the model’s ease of use, because the number of substeps won’t have to be determined by trial and error.

 

By reducing the number of unnecessary substeps, this approach can improve computational performance and make the model easier to configure.

 

Development tests of the adaptive substepping algorithm are done in a two-dimensional simulation of a thermal cloud forming in a rising bubble of humid air. A snapshot of the modelled domain, in Figure 1, shows the benefits of substep adaptivity.

 

A large number of substeps is required only within the cloud and in its vicinity, while in cloud-free regions, a single substep is often sufficient. Thus, computational overhead in grid cells where phase transitions are slow or inactive is drastically reduced compared to simulations with a large, fixed number of substeps.

 

 

 

 

Deploying UWLCM on the Fugaku Supercomputer

 

High-resolution Large-Eddy Simulation (LES) and SDM modelling are highly computationally expensive. UWLCM has traditionally been optimised for hybrid HPC architectures, leveraging standard multi-core CPUs coupled with GPU accelerators to handle the large-scale parallel workload involved in tracking billions of super-droplets.

 

As part of the HANAMI project‘s integration with Japanese supercomputing infrastructure, we are currently deploying UWLCM on the Fugaku supercomputer.

 

Fugaku presents a fundamentally different hardware paradigm for our code since it is entirely CPU-based and uses a specific Tofu-D interconnect. Transitioning from a CPU+GPU execution model to a massively parallel, purely CPU-driven environment is a significant software engineering challenge.

 

We are in the process of setting up a Spack environment on Fugaku that will contain all UWLCM dependencies. The next step is to evaluate the scalability of UWLCM across multiple nodes. This work will not only allow UWLCM users to run simulations on Fugaku, but will also drive UWLCM optimisations for CPU-only clusters.

 

 

These developments demonstrate how the HANAMI project is advancing both the scientific capabilities and computational performance of UWLCM. By combining new microphysics features, adaptive numerical methods, and large-scale deployment on Fugaku, the collaboration is helping to enable more accurate and efficient cloud simulations for atmospheric research.