Skip navigation
NASA Logo, National Aeronautics and Space Administration
Modeling Guru Banner
Currently Being Moderated

PoDS and MPI Tasks

VERSION 3  Click to view document history
Created on: Aug 27, 2013 12:57 PM by Jules Kouatchou - Last Modified:  Dec 17, 2015 2:22 PM by Jules Kouatchou

PoDS is an application that allows users to execute independent tasks concurrently across nodes on multicore clusters. It does not require any knowledge of the individual tasks and does not make any assumptions about the underlying application. As a matter of fact, the tasks to be executed can be from different applications. It can be seen as a task parallelism tool where concurrent independent jobs are executed in parallel.

 

PoDS was initially designed to run serial tasks but it can also be used on parallel (MPI or OpenMP instance) tasks as long as the following conditions are satisfied:

 

  • Only one task is assigned to any vailable node
  • Each parallel task runs within the assigned node only (no communication across nodes)
  • Each parallel task may use all the cores within the node

 

To illustrate how we can submit MPI tasks with PoDS, we use two mpi4py scripts: trapezoidal_rule.py (compute an integral using the trapezoidal rule) and matrix_vector_product.py (matrix-vector multiplication). We create two tasks:

 

trap.sh

#! /bin/csh -f

source /usr/share/modules/init/csh
module purge
module load other/comp/gcc-4.9.2-sp3
module load other/mpi/mvapich2-2.1/gcc-4.9.2-sp3
module load lib/mkl-15.0.2.164
module load other/SIVO-PyD/spd_1.24.0_gcc-4.9.2-sp3_mkl-15.0.2.164_mvapich2-2.1


setenv workDir /directory/where/the/script/is/located/

cd $workDir

mpirun -np 12 trapezoidal_rule.py 0.0 5.0 1000000

 

and prod.sh

#! /bin/csh -f

source /usr/share/modules/init/csh
module purge
module load other/comp/gcc-4.9.2-sp3
module load other/mpi/mvapich2-2.1/gcc-4.9.2-sp3
module load lib/mkl-15.0.2.164
module load other/SIVO-PyD/spd_1.24.0_gcc-4.9.2-sp3_mkl-15.0.2.164_mvapich2-2.1

setenv workDir /directory/where/the/script/is/located/

cd $workDir

mpirun -np 12 matrix_vector_product.py

 

The PoDS execution file (named myExecutable) is then:

 

/directory/where/the/tasks/are/located/trap.sh
/directory/where/the/tasks/are/located/prod.sh

 

The PBS script to submit the PoDS job looks like:

 

#!/bin/csh -f

#SBATCH --time=00:30:00
#SBATCH --job-name=pods_mpi
#SBATCH --nodes=4
#SBATCH --constraint=hasw
#SBATCH -A XXXXX


umask 022

source /usr/share/modules/init/csh
module purge
module load other/comp/gcc-4.9.2-sp3
module load other/mpi/mvapich2-2.1/gcc-4.9.2-sp3
module load lib/mkl-15.0.2.164
module load other/SIVO-PyD/spd_1.24.0_gcc-4.9.2-sp3_mkl-15.0.2.164_mvapich2-2.1

cd $PBS_O_WORKDIR

/usr/local/other/pods/pods.py myExecutable 1

 

The number 1 after myExecutable is used to request one task per node.

 

In trap.sh and prod.sh presented above, we could have used MPI executables (instead of mpi4py scripts) generated by Fortran or C.

Comments (0)
USAGov logo NASA Logo - nasa.gov