Speeding Up Simulations with Parallel Processing in EFDC+: A Tutorial and Benchmarks using MPI and OpenMP

Environmental Fluid Dynamics Code Plus (EFDC+) is a powerful modeling tool widely used for hydrodynamic simulations. However, large and complex models can take significant computational time, sometimes making simulations impractical. To address this, EFDC+ now employs parallel processing capabilities using MPI (Message Passing Interface) combined with OpenMP (Open Multi-Processing), significantly accelerating model simulations. In this tutorial, we discuss how to harness the full potential of parallel processing and showcase its dramatic performance improvements through benchmarks.

Understanding Parallel Processing: MPI and OpenMP

Parallel processing distributes computational workloads across multiple processors or cores, significantly reducing simulation run times. EFDC+ leverages two primary methods:

  • MPI (Message Passing Interface): A standardized method used for parallel processing that divides computational tasks across multiple processors, enabling simulations to run on clusters of interconnected computers.
  • OpenMP: A simpler parallelization method that uses multiple threads within a single processor or node to distribute computational tasks.

EFDC+ combines these two approaches into a hybrid MPI/OpenMP system, effectively harnessing both methods' strengths.

Chesapeake Bay Elevation Grid

EFDC+ Domain Decomposition Approach

EFDC+ utilizes a domain decomposition approach, dividing the simulation domain into multiple smaller subdomains. Each subdomain is processed in parallel, with MPI handling the communication between these subdomains. This approach ensures that computations remain efficient by balancing the workload evenly among processors, minimizing communication overhead, and thus maximizing computational speed.

  • Load Balancing: Distributing computational tasks evenly among subdomains.
  • Ghost Cells: Cells at subdomain boundaries that exchange data to ensure continuity across subdomains.
  • Adaptive Time-Stepping: Synchronizing time steps across subdomains by choosing the minimum timestep among all subdomains.

Benchmarking with the Chesapeake Bay Model

To demonstrate the performance improvements offered by parallel processing, EFDC+ conducted benchmarks using a detailed hydrodynamic model of Chesapeake Bay:

  • Model Setup: Nearly 800,000 computational cells, simulating temperature, salinity, and hydrodynamics.
  • MPI/OpenMP Hybrid Performance: Achieved a nearly 17-fold speedup with 32 cores compared to the previous OpenMP-only approach, which could only achieve about a 2.5-fold speedup.
  • Strong Scaling Study: Conducted using Amazon Web Services (AWS) cluster up to 96 cores, resulting in a maximum speedup of approximately 25 times compared to single-core execution.

These benchmarks clearly illustrate the dramatic performance gains possible with EFDC+’s parallel processing capabilities.

Best Practices for Achieving Optimal Performance

To fully benefit from parallel processing, EFDC+ users should consider the following guidelines:

  • Optimal Subdomain Configuration: Utilize EFDC Explorer’s automated tools to balance computational loads effectively across subdomains.
  • Computational Environment Setup: Leverage high-speed network interfaces and powerful computing resources, such as those available on cloud platforms like AWS.
  • Monitoring and Adjustments: Regularly analyze runtime logs to fine-tune subdomain and thread settings for maximum efficiency.

We highly recommend our 2021 webinar about the MPI + OpenMP features of EFDC+ and how to make the most effective use of them:

We also provide a more detailed discussion of the MPI domain decomposition feature and the Chesapeake Bay example in our EFDC+ MPI White Paper available here:

Conclusion

EFDC+'s MPI and OpenMP hybrid parallelization approach offers transformative performance improvements, enabling faster, more practical modeling of large and complex environmental systems. By effectively employing domain decomposition and parallel processing, EFDC+ empowers users to tackle significantly more ambitious simulations with greatly reduced computational times.

Explore EFDC+ today and experience the benefits of accelerated environmental fluid dynamics simulations through advanced parallel processing.

Talk to the experts

Paul Craig

Tom Mathis

Operations Manager

← Back to Blog