MPI on Matilda
Overview
Message Passing Interface (MPI) applications are designed to permit the spawning of, and communications between, parallel processes run across multiple nodes using distributed memory. In serial or multi-threaded OpenMP applications, processes utilize shared memory in a common address space on a single node. When MPI is used to spawn parallel processes across multiple nodes, the memory space of each of the individual nodes contains a portion of the information used by those processes - thus, that memory address space is distributed. Message passing is needed to exchange information between those distributed memory spaces and the processing cores controlling them in a way that allows all of the parallel tasks to be coordinated/controlled in a manner conducive to orderly task completion.
OpenMP Model |
MPI Model |
|
|
In order to utilize MPI, an application must be written in a manner that leverages an MPI Library. Code is added to the application which utilizes various features of the selected MPI Library to spawn, control, and terminate parallel processes. MPI Libraries not only supply functionality that can be linked to application code at compile time, but they also provide runtime "launchers" and as well as runtime configuration controls.
Matilda MPI Capabilities
MPI Libraries
Matilda currently offers two of the most commonly used MPI Libraries:
OpenMPI - An open source MPI standards-conforming implementation of MPI developed by the OpenMPI Project, and optimized for High-Performance Computing (HPC) environments.
MPICH - The MPI reference-standard implementation of MPI developed by the MPI Forum.
Hardware
Matilda's hardware compliment has been designed with MPI in-mind. Important features include Mellanox/NVIDIA Infiniband networking between nodes, and a DDN Lustre parallel filesystem (/scratch). This configuration offers Remote Direct Memory Access (RDMA), which permits communications between distributed memory spaces on different nodes without involving the operating system. RDMA provides significant speed improvements, and can substantially reduce the load on CPUs, thus saving cycles for computational (rather than I/O) tasks.
The diagram below compares sockets-based communication using TCP, with RDMA communications using the Verbs API:
* Source: teimori.net
MPI Tips
For more information on best practices for the use of MPI on Matilda, please refer to the specific Matilda documentation on OpenMPI and MPICH.
CategoryHPC