Skip navigation
NASA Logo, National Aeronautics and Space Administration
Modeling Guru Banner

Manage categories


Create and manage categories in Climate@Home. Removing a category will not remove content.

Categories in Climate@Home
Add a new category (0 remaining)

Manage Announcements


Create and manage announcements in Climate@Home. Try to limit the announcements to keep them useful.

Announcements in Climate@Home
Subject Author Date Actions

Blog Posts

8 Posts

Here is the promised document describing how to create your own TinyCore/MicroCore image. Also attached is the TinyCore/MicroCore package that I've created that contains gcc and gfortran 4.4.1 which is needed to build and run the model.


I've quickly tried to get Model E to run with more than one processor within the VM. While I have been able to compile and build, the build fails to execute. This is most likely a result of me not fully understanding everything necessary to run it with multiple-cores. I'll probably spend some more time on that aspect in the coming weeks.


There was a slight configuration change that required a small change to the file for compilation. The file specifies the cpp link as /lib/cpp. However, for TinyCore and MicroCore there is no /lib/cpp link. Instead, cpp is located in /usr/local/bin (this is how the packages are "mounted"). Once that was adjusted, the model compiled and ran (using the two additional files data files Carlos mentioned).


The updated E4M20 rundeck appears to be exercising some additional code. The native run time and the VM run time have all increased by 20-30 seconds for a day long run vs. the E1M20 deck.


Mac OSX Lepoard Core 2 Duo (serial): 124.25 seconds

VM on same platform: 134.89 seconds


modelE code update

Posted by Carlos Cruz Oct 29, 2009

Shawn and I have updated to the latest modelE code as stored in the trunk of the GISS repository. We would have preferred to use code from a "blessed" tag to have a calibrated reference but such tags are not available.


The latest code does include an updated rundeck, E4M20, which is a low-resoluton version of the atmosphere-only AR5 model (but without interactive chemistry or aerosols). We will use this rundeck and the latest code to perform reproducibilty experiments where we will compare the model output from three different platforms (all intel-based) including our virtual box-based system (running tinyCore Linux).


NOTE: Before today we had been running modelE with the E1M20 rundeck. Preliminary runs on DISCOVER and on my MAC laptop show that the new rundeck, E4M20, does require two additional files: soil_textures_top30cm and soilcarb_top30cm_nmaps_4x5bin.dat.


Bad gfortran 4.2

It turns out you have to be careful what version of gfortran you use to build ModelE, irrespective of the operating system.


The latest XCode tools for Mac OSX Leopard comes with gcc/gfortran version 4.2. If you use this version and build ModelE, it will apparently run fine until the end of the first month. At that point, the model attempts to read some unformatted binary files (in the case of E1M20, a sea ice file) and promptly fails. This happened on OSX as well as the TinyCore/MicroCore VMs. Depending on your OS, the IOSTAT return code may vary. On OSX, it was -1 (EOF) while on TinyCore/MicroCore it was 17 (operation not permitted on direct access file).


This was discovered due to my co-worker Carlos having gcc/gfortran 4.3 on his native machine. With the exact same rundeck, source, and data files his monthly run would complete without error, while mine continued to fail. After grabbing the latest version of gcc/gfortran for OSX Leopard (version 4.4.1), the model ran fine.


What about TinyCore?

For TinyCore/MicroCore there is no gfortran package available other than version 4.2 (there is a a later version gcc, but only for c/c++). This meant downloading the source and building a new version. In order to build, you need to have binutils and gcc installed.


Once the source is extracted, you need to create a directory that will be the build directory (do not build in the same directory, it will not work). Go into you're new build directory and run configure with these options:


     ../configure --prefix=/your/build/directory --enable-languages=c,fortran --enable-checking=release --disable-bootstrap


Other configuration options MIGHT work, but this was the set that would finally build without errors.


After the build is complete, you can go to the tce directory on your persistent storage (typically /mnt/hda1) and delete the gcc and gfortran packages. In your .ashrc or .profile file, be sure to add your build directory's bin to your path. At this point, reboot the VM. When it comes up again, go to a shell and check the gfortran version.


Some Quirks of TinyCore

When TinyCore starts, it starts from a compressed image file that contains unalterable files (unless you re-master your own distribution). Also, in the default environment certain environment variables don't seem to work like one would expect. In this instance, the shared libraries of the custom built gcc/gfortran aren't visible. Setting LD_LIBRARY_PATH appears ignored, and altering etc/ld-* files will only work for a single sesssion as they are forgotten on every reboot.


So if you're building using shared libraries, you may run into issues running tthe executables. There are standard *nix methods of getting around this, or you can build statically. Another option is creating a TinyCore package (a tczl file) which will tell TinyCore to load any libraries you specify.


Why TinyCore?

Posted by Shawn Freeman Oct 20, 2009

Both Carlos and I have mentioned using TinyCore Linux for building our virtual images. However, we have not specified why we are using TinyCore.


Eventually, these virtual images will be distributed to end user machines to execute Model E so the idea is to minimize any downloading to the absolute minimum necessary. Any of the mainstream Linux versions contains hundreds of megabytes even in a stripped down distribution. Some other Linux distributions have been made small, but they use older kernels. Others don't have any decent package management system which means building everything from source, which can either be really easy or very painful depending on any number of factors.


TinyCore is 10 MB, uses the latest Linux core, and even has a simple package system with 100s of pre-built packages, including common development tools like gcc and gfortran. Setting up a TinyCore virtual machine in VirtualBox takes just a few minutes, and setting up a TinyCore image for compiling and running model E can be done in less than half an hour (a document will be posted soon on how to do this).


The final distributed image will use the MiniCore version of TinyCore, which eliminates the GUI packages from the image and bring the total OS down to 6MB. With a statically linked modelE executable, that brings the initial download size to less than 20MB. Any data files/restarts needed by the model will be downloaded dynamically based on the run deck (this will vary). The virtual image will expand as needed up to a pre-defined limit for getting data and storing data as the model runs (the results will be uploaded to a server in the background).


Virtualization and ModelE

Posted by Shawn Freeman Oct 20, 2009

For the Climate@Home effort, we are applying virtualization to run ModelE to eventually deploy across the Internet so anyone can help out in useful climate research.


As Carlos has mentioned here, we are using VirtualBox and a very small Linux distribution called TinyCore. One the most important tests of this approach is achieving bitwise reproducibility of results.


For the tests, the virtual machine was created and run on several different machines:

  • A Windows XP box with an AMD Athlon 3400+ processor.
  • A Windows XP box with an Intel Pentium III 800 MHz processor.
  • A Windows XP box with an Intel Q6600 2.7GHz quad core processor.
  • A Mac OSX Leopard box with an Intel Core 2 Duo 2.4 GHz processor


The model was configured with a basic deck based on the E1M20 deck and runs were done for an hour, one day, and one month. The results were compared using the CMPE002P tool provided with ModelE. The executable was compiled using gfortran with -O3 optimizations.


The results showed bitwise identical results for all pentium based machines going back to the Pentium III family, regardless of the of the underlying system configuration or OS.


However, there were differences between the Intel and AMD based machines, though they were small precision differences. This is to be expected, as all mathematical operations must eventually be run on the native CPU, and there are known differences between AMD and Intel implementations.


Based on these results, bitwise reproducibility can be expected across members of the same processor family but not across processor families. In other words, you can expect a Pentium III and a Core 2 Duo running a virtual machine to produce the same results, but not a Pentium III and an AMD Athlon. So using a virtual machine based paradigm an implementer can either choose to limit the participants to one processor family, use all processor families and mix results, or use all processor families and store the results separated by processor family.


In the case of Climate@Home, the results from Intel based and AMD based machines will most likely be stored separately.


Some other observations from these experiments:


  • The gfortran compiler is significantly better at producing optimized code than g95. The model ran about 40% faster when compiled with gfortran.
  • Virtualization overhead also dropped when using gfortran. With g95 we were seeing a 15-20% performance decrease when comparing against native runtimes. With gfortran, this was 10% or less. This may be the result of better IO handling in gfortran.

modelE on Tiny Core Linux

Posted by Carlos Cruz Oct 13, 2009

Tiny Core Linux ( is a very small (10 MB) minimal Linux GUI Desktop. It is based on Linux 2.6 kernel, Busybox, Tiny X, and Fltk. The core runs entirely in ram and boots very quickly.


We installed VirtualBox ( on our MacBook Pro systems and installed TC within it. Preparing the TC environment for modelE installation required that other software packages be installed: base_dev, binutils, make, perl, g95 and gfortran. Obviously given our experiences under OSX (leopard) we want to test both g95 and gfortran compilers under TC.


Our preliminary findings under TC show that model performance is worse under TC than under OSX using g95 (generally a 15-20% degradation).


modelE on a MacBook Pro

Posted by Carlos Cruz Oct 13, 2009

modelE can be built on a MAC with very little effort.  Assuming you have g95 installed:


set "NETCDF = " in .modelErc (assumes netCDF 3.6.3 is installed in /usr/local)


add "-cpp" to g95 command (


And these is the software modelE used during the build:


[Macintosh-6] decks > g95 --version
G95 (GCC 4.0.3 (g95 0.92!) May  2 2008)


[Macintosh-6] decks > cpp --version
i686-apple-darwin9-gcc-4.0.1 (GCC) 4.0.1 (Apple Inc. build 5490)


[Macintosh-6] decks > gmake --version
GNU Make 3.81


If you are using gfortran, first you need to create the corresponding .mk file. It is similar to the but it should be called These are the settings we had to change:


< F90 = gfortran
> F90 = g95 -cpp

< FFLAGS = -O1 -fconvert=big-endian -xf77-cpp-input -frecord-marker=4
< F90FLAGS = -O1 -ffree-form -fconvert=big-endian -xf95-cpp-input -frecord-marker=4
> FFLAGS = -fno-second-underscore -O0 -fendian=big
> F90FLAGS = -fno-second-underscore -O0 -ffree-form -fendian=big

< FFLAGS += -ffpe-trap=invalid,zero,overflow


The option -frecord-marker=4 is needed by gfortran since by default gfortran uses 8 bytes.




[Macintosh-6] config > gfortran --version
GNU Fortran (GCC) 4.3.0 20080125 (experimental)
Copyright (C) 2007 Free Software Foundation, Inc.


These are some timings for a 1-day modelE run (E1M20 rundeck):


g95 running on OSX with -O3:         145 seconds main loop.
gfortran running on OSX with -O3:   100 seconds main loop.


NOTE: For gfortran if you are running leopard or later you need to install xCode 3.x


ADDENDUM (6/9/2010)


I have recently built modelE on a MacBook Pro (Intel Core i7) running the snow-leopard OS (10.6.3) using gfortran v4.4 (and netCDF 4.1.1).

When using the latest modelE code (code base 06-09-2010), a  is included and the code compiles and runs as-is. However one must make sure to use gfortran v 4.4 as the model does not compile under gfortran v 4.2 (the version I had available from Xcode).



USAGov logo NASA Logo -