Skip navigation
NASA Logo, National Aeronautics and Space Administration
Currently Being Moderated

GFS Benchmark How To

VERSION 10  Click to view document history
Created on: Sep 23, 2009 5:26 PM by Rahman Syed - Last Modified:  May 6, 2010 10:22 AM by Rahman Syed

Instructions:

 

Download software

 

1.)  Download NOAA's GFS benchmark (ccs.rfi.gfs.tar.gz (1631 MB)) from http://www.ncep.noaa.gov/ccs/benchmark/

Untar this archive into a location of your choosing; this document will refer to this location as <GFS_BENCHMARK_DIR>

 

2.)  Download Jeff Whitaker's Linux port of GFS from the SVN repository located here: http://code.google.com/p/ncepgfs/ with the following command:

 svn checkout http://ncepgfs.googlecode.com/svn/trunk/ ncepgfs-read-only  

This document will refer to this location as <GFS_LINUX_PORT_DIR>

 

3.)  Download ESMF v2.2.2 from NCAR:  http://www.earthsystemmodeling.org/esmf_releases/non_public/ESMF_2_2_2/reg/ESMF_ Framework_Reg.html

 

Build ESMF

 

1.)  Set your module environment.  Known working setup:

module purge; module load comp/intel-10.1.021 lib/mkl-10.0.5.025 mpi/impi-3.2.011

 

2.)  Untar esmf_2_2_2_src.tar.gz into <GFS_LINUX_PORT_DIR>/lib

 

3.)  Edit <GFS_LINUX_PORT_DIR>/buildesmf.sh by changing ESMF_COMM to "intelmpi"

 

4.)  Unset the environment variable "MPI_HOME" (unsetenv MPI_HOME)

 

5.)  Run <GFS_LINUX_PORT_DIR>/buildesmf.sh

 

6.)  Note the final location of include and lib directories inside lib/esmf (may look like <GFS_LINUX_PORT_DIR>/lib/esmf/mod/modO/Linux.intel.64.default and <GFS_LINUX_PORT_DIR>/lib/esmf/lib/libO/Linux.intel.64.default)

 

Build GFS

 

1.)  Edit <GFS_LINUX_PORT_DIR>/buildgfs.sh by changing the following lines:

 

export FCS=ifort
export FCP=mpiifort
export CC=icc
export ESMFINC=../lib/esmf/mod/modO/Linux.intel.64.default
export ESMFLIB="-lstdc++ -limf -lm -lcxa -lunwind -lrt -ldl -threads -L../lib/esmf/lib/libO/Linux.intel.64.default -lesmf -lnetcdf_stubs -L/usr/local/intel/mpi/3.2.011/lib64 -lmpi -lmpiif"

 

Make sure to use the location of relevant ESMF directories noted in step 6 under "Build ESMF".

 

2.)  Run <GFS_LINUX_PORT_DIR>/buildgfs.sh

 

Run GFS

 

1.)  Link <GFS_LINUX_PORT_DIR>/bin/global_fcst to <GFS_BENCHMARK_DIR>/sorc/gfs/f.applic.x

 

2.)  Edit <GFS_BENCHMARK_DIR>/sorc/scripts/x.190.port.hours

#PBS -l walltime=1:00:00
#PBS -l select=4:ncpus=8
#PBS -N GFS
#PBS -W group_list=<YOUR_GROUP>

source /usr/share/modules/init/ksh
module purge
module load comp/intel-10.1.021 lib/mkl-10.0.5.025 mpi/impi-3.2.011

Also remember to set "BASE" to the <GFS_BENCHMARK_DIR> directory, and "run_dir" to a place where you'd like to have each run saved.

 

On line 117, change the content from "./fxxx  >out 2>time.$$" to "mpirun -np 16 ./fxxx  >out 2>time.$$" or with whatever parameters your MPI library requires to execute an MPI application.

 

NOTE: You can change the duration of the run by altering the variable FHMAX.  The default is 240 hours, but only a 24 hr run is required to validate GFS so set FHMAX to 24.

 

3.)  Submit the script to PBS with "qsub x.190.port.hours"

 

4.)  You can watch output with the command:

tail -f <GFS_RUN_DIR>/out

 

5.)  The "out" file may report ESMF errors at the beginning of the run, like " Error Happened When Running the ESMF_ConfigGetAttribute-IDATE1_IMPORT rc = 39" however this is not a problem.  The model may crash at the end during ESMF_Finalize, but this is a known issue.

Build and run Diff Tool

 

1.)  Build the diff tool with the command:

cd <GFS_BENCHMARK_DIR>/verify
ifort -convert big_endian -o rms.190.x rms.diff.190.f

 

2.)  Edit rms.script.190 by changing file1 to the SIG.F24 located in <GFS_BENCHMARK_DIR>/results/bench.baseline.prod/T190.hours.  Change file2 to the SIG.F24 located in your <GFS_RUN_DIR>.

 

3.)  Execute rms.script.190

 

4.)  Examine the output found in rms.diff.190.out.  Ensure that it fits the criteria listed in <GFS_BENCHMARK_DIR>/verify/README

Comments (0)

Bookmarked By (0)

More Like This

  • Retrieving data ...
USAGov logo NASA Logo - nasa.gov