Howto setup Mac OSX Lion with Python and Fortran dev tools

Notes for how to setup Python and Fortran development tools on Mac OSX Lion

Deprecated – click here for updated page for Yosemite version

Created by Trond Kristiansen on 06.03.2012 (updated 07.03.2012)

This page contains my own notes for how to install a variety of packages that I use on a regular basis on Intel Mac computers. I used to compile all packages from source, but maintaining all of the libraries while wanting to update the OSX turned out to be difficult (see my old page). I have therefore simplified the installation slightly when I upgraded to Lion, using the Enthought Python Distribution. Since I also use Fortran extensively I still have to compile HDF5, ZLIB, UDUNITS, and NetCDF4 with Fortran support, but that is much easier than maintaining all of the Python libraries as well. Please leave a comment if you have suggestions for improvements!

Step 1:
Install Lion
Download and upgrade to Lion OSX using the App store.

Step 2:
Install Xcode
Download Xcode from Apple Developer

Step 3:
Install command tools for Xcode
Download command line tools for Xcode from Apple Developer. The package includes necessary header files.

Step 4:
Install Enthought Python Distribution
Download and install the EPD. In my case I paid $199 for a full license that enables me to install all of the libraries and packages i could ever want. There is also a free version of EPD which contains all of the base libraries and provides a great start.

Step 5>
Install Intel Fortran Compiler
I bought the Intel XE Fortran Compiler from Intel which is great for speed and also comes with Xcode integration. After installation add these lines to your .bash_profile

source /opt/intel/bin/compilervars.sh intel64

Also, to enable f2py to function properly with the compiler add the following to .bash_profile

export DYLD_LIBRARY_PATH="/opt/intel/composer_xe_2011_sp1/lib/intel64:$DYLD_LIBRARY_PATH"

Step 6:
Install ZLIB
Zlib is used for compression/decompression in reading/writing netcdf4 files. The package can be downloaded here and installs by default to /usr/local.

./configure
make
make check
sudo make install

After installation you want to add this to your .profile (or .bash_profile) file:

if [ -z "${ZLIBDIR}" ]
then
ZLIBDIR="/usr/local"; export ZLIBDIR
else
ZLIBDIR="/usr/local:${ZLIBDIR}"; export ZLIBDIR
fi
export LIB=/usr/local/lib:$LIB
export INCLUDE=/usr/local/include:$INCLUDE

Step 7:
Howto get F2PY to work for 64-bit python
To get f2py to compile together with 64-bit fortran Intel compiler, you have to edit the following file in disutils (notice two options:

sudo chmod a+r+w+x /Library/Frameworks/EPD64.framework/Versions/7.2/lib/python2.7/site-packages/numpy/distutils/fcompiler/intel.py

Edit the line that contains “linker_so” for intelem (Intel 64-bit Fortran compilers) so that it looks like this:

'linker_so' : ['',"-O0","-shared-intel","-bundle",'-dynamiclib', '-Wl,-undefined,dynamic_lookup', '-Wl,-framework,Python'],

Then you can compile Fortran modules that can be imported into Python using Intel Fortran compiler with the following option in f2py:

f2py --verbose --fcompiler=intelem -c -m barotropic barotropic.f90

The compiled module can be imported to python and called using import barotropic

I experienced several problems where I received the confusing error message:

Fatal Python error: Interpreter not initialized (version mismatch?)

This was caused by the Fortran compiler (I believe) linking with the incorrect Python framework.

otool -L barotropic.so
./barotropic.so (compatibility version 0.0.0, current version 0.0.0)
/System/Library/Frameworks/Python.framework/Versions/2.7/Python (compatibility version 2.7.0, current version 2.7.1)
libifport.dylib (compatibility version 1.0.0, current version 1.11.0)
...

Here, you can see that the program is linking with the system Python framework and not the Enthought Python Distribution framework which is what I wanted. I found a solution to this problem here that worked perfectly for me. Thanks to Arlindo da Silva.

cd  /System/Library/Frameworks/Python.framework/Versions
sudo mv Current Current.old
sudo ln -s /Library/Frameworks/EPD64.framework/Versions/7.2 Current

Step 8:
Install UDUNITS 
Udunits is used by NetCDF4 an HDF5 which will be installed later.

./configure
make
sudo make install

The default configuration installs UDUNITS (currently version 2.1.4) into /usr/local/share.

Step 9:
Install HDF5
Download HDF5 here: http://www.hdfgroup.org/HDF5/release/obtain5.html

I compiled HDF5 (version HDF5 version 1.8.8) in static mode for use with Fortran (--enable-fortran) and NetCDF4. If you want to compile HDF5 as a shared library you can not create Fortran libraries and have to use the option --disable-fortran.

./configure --with-zlib=/usr/local --enable-fortran --prefix=/usr/local/ --target=x86_64-apple-darwin -build=x86_64-apple-darwin --host=x86_64-apple-darwin --enable-shared --disable-production
make
make check
sudo make install

Next add these lines to your .profile so that when we install netcdf4 HDF5 will be visible to the configurator:

export HDF5_LIB_DIR="/usr/local/lib"
If [ -z "${LD_LIBRARY_PATH}" ]
then
LD_LIBRARY_PATH="/usr/local/lib"; export LD_LIBRARY_PATH
else
LD_LIBRARY_PATH="/usr/local/lib:${LD_LIBRARY_PATH}"; export LD_LIBRARY_PATH
fi
if [ -z "${LIBDIR}" ]
then
LIBDIR="/usr/local/lib"; export LIBDIR
else
LIBDIR="/usr/local/lib:${LIBDIR}"; export LIBDIR
fi
export INCLUDE=/usr/local/lib:$Include
export PATH=/usr/local/bin:$PATH
export HDF5_LIB_DIR=/usr/local/lib
export HDF5DIR=/usr/local
export HDF5_DIR=/usr/local
export CPPFLAGS="-I$HDF5_DIR/include -I/usr/local/include"
export LDFLAGS="-L$HDF5_DIR/lib -L/usr/local/lib"

Step 10:
Install NETCDF
NetCDF source files can be downloaded from here. As of the writing of this, latest stable version is v4.1.3. This version was particularly difficult to get to compile in 64 bit mode (at least for me) and it took me several hours to make sure that both HDF 1.8.8 was compiled as 64 bit and that it could be linked to the compilation of the NetCDF4. I had to set several environmental variables to get this to work, including setting the GNU GCC compileer to version 4.6 (as described here):

export FC=ifort
export F90=ifort
export F77=ifort
export FCFLAGS=-m64
export FFLAGS=-m64

Next, I still had problems with the compilation until I found a tip at the Unidata website that suggested a solution to my problem. The trick to get NetCDF4+HDF5+Intel XE composer to work together was a simple statement:

 lt_cv_ld_force_load=no

. The following commands enabled me to compile and load NetCDF4 using Intel Fortran X:

FC=ifort lt_cv_ld_force_load=no CXX="" CPPFLAGS="-I/usr/local/include" LDFLAGS="-L/usr/local/lib" ./configure --enable-f90 --prefix=/usr/local/ --target=x86_64-apple-darwin -build=x86_64-apple-darwin --host=x86_64-apple–darwin --enable-netcdf4 --enable-shared
make
make check
make install

After installation of netCDF4 add this to your .profile:

export NETCDFHOME=/usr/local
export NETCDF_PREFIX=/usr/local
export NETCDF_LIB=/usr/local/lib
export NETCDF4_ROOT=/usr/local
export NETCDF4_DIR=/usr/local
export NETCDF_INC=/usr/local/include
export PATH=/usr/local/bin:$PATH