Installation on Ubuntu

Today is 3/29/2010 - These are notes on how to install instructions for flooxs on 64-bit Ubuntu 9.10 Karmic Koala. These instructions are intended for linux/unix beginners. You may email me direct questions about installing flooxs on Ubuntu at nrowsey at ufl.edu, but please try these instructions first, or google if it is an ubuntu question.

Get Linux - and check out basic concepts page

Here's a great and easy website for Ubuntu. They even have show me how buttons that expose easy-to-follow tutorials, as well as a section that explains dual booting with Windows. Use the "desktop" edition even if you have a laptop. The distinctions in editions are in case you have a netbook (which needs a smaller operating system that will work on the less-powerful processors that netbooks have) and servers.

If you've never used Linux / Unix before, you might find the following flooxs installation instructions confusing. I've started a page to explain some basic unix concepts that held me up as I was learning and trying to install flooxs.

Get Your Flooxs!

1. Download the software and put it on your hard drive in a directory called flooxs

       $ mkdir ~/flooxs
       - go to [1], download the latest linux executable file (get the one that says linux)

2. Unpack the flooxs file

       Double click -> Export

This should make a directory called rel2008. Put it in your new flooxs folder. Inside rel2008 you should have these folders:

       $ cd ~/flooxs/rel2008
       $ ls
       arch  bin  help  lib  Params  src  support  TclLib  Test

To install flooxs, we first must install several supporting programs (such as math or graphing packages) in the support directory. Then, we will go into the src (short for "source") directory and compile flooxs from there.

Get PETSC math package, with BLAS, LAPACK and MPI

In this section, we will download, configure, and compile three supporting math packages that flooxs uses to solve matrix problems, as well as a compiler that these math packages use.

1. First, set some environment variables so flooxs knows where to look for libraries:

       $ cd ~/flooxs/rel2008
       $ export FLXSHOME=`pwd`

2. Download the latest PETSc package from [2]. When I did this step, the latest package was 3.0.0-p11. You may have better or worse luck using older or newer packages. (Note: the PETSc version that comes with your flooxs 2008 download does not have the capability of downloading BLAS and LAPACK automatically, so if you choose the version already in your $FLXSHOME/support directory you will have to install and link blas and lapack yourself). Do not use archive manager to unpack this. Instead, we will unpack manually from the command line.

       - put the downloaded file in $FLXSHOME/support
       $ cd $FLXSHOME/support
       $ gunzip -c petsc-3.0.0-p11.tar.gz | tar -xof -
       $ cd petsc-3.0.0-p11
       $ export PETSC_DIR=$PWD

3. Find out how to Configure PETSc, with BLAS, LAPACK, and MPI. Check out example configure files in $PETSC_DIR/config/examples/*.py. For example, I am running linux, and my processor is AMD 64-bit, so I looked at the example file named linux-gnu-amd64.py.

       $ cd $PETSC_DIR/config/examples/
       $ ls
       aix5.1.0.0-64.py      cygwin-borland.py        linux-gnu-nagf90.py
       aix5.1.0.0.py         cygwin-ms.py             linux-gnu.py
       asterix32.py          cygwin.py                linux-gnu-superlu.py
       asterix32-sun.py      freebsd5.1-c89.py        linux-gnu-uni.py
       asterix64-c89.py      freebsd5.1.py            linux-gnu-valgrind.py
       asterix64-openmpi.py  linux-gnu-amd64.py       linux-rhAS3-gcc32-opt.py
       asterix64.py          linux-gnu-blocksolve.py  linux-rhAS3-intel81-opt.py
       asterix64-sun.py      linux-gnu-cxx-sieve.py   macosx-gnu-g95.py
       bgl-gnu-opt.py        linux-gnu-gcc-absoft.py  macosx-gnu.py
       bgl-ibm-cxx-opt.py    linux-gnu-gcc-ifc.py     sicortex.py
       bgl-ibm-debug.py      linux-gnu-gcc-pgf90.py   solaris2.9-lam.py
       bgl-ibm-opt.py        linux-gnu-ia64-intel.py  solaris2.9.py
       bgp-ibm-opt.py        linux-gnu-ia64.py        solaris-64.py
       cray-x1.py            linux-gnu-intel.py       solaris-gnu.py
       cray-xt3.py           linux-gnu-ml.py          solaris-uni.py

This file sounded like my architecture, so I opened it with a text editor and took a look at what was inside:

       $ gedit linux-gnu-amd64.py &

This is what I saw:

               #!/usr/bin/env python

                       configure_options = [
                         '--download-mpich=1',
			  '--download-mpich-pm=gforker',
			  '--download-f-blas-lapack=1',
			  '--download-prometheus=1',
			  '--download-parmetis=1',
			  '--with-debugging=0'
			  ]

			if __name__ == '__main__':
			  import sys,os
			  sys.path.insert(0,os.path.abspath('config'))
			  import configure
			  configure.petsc_configure(configure_options)
			  

From the installation instructions on the PETSc website, I saw that, if I wanted to download BLAS, LAPACK, and the MPI compilers that PETSc needs, I should add these lines to the list in the above file:

			  '--download-f-blas-lapack=1',
			  '--download-mpich=1'

Also, the prometheus and metis packages are not used in flooxs, and can cause problems, so I knew I had to delete these two lines:

			  '--download-prometheus=1',
			  '--download-parmetis=1',

Below explains in more detail how I did this.

4. Edit your PETSc configure file: First, save a backup:

       $ cd $PETSC_DIR/config/examples
       $ cp linux-gnu-amd64.py linux-gnu-amd64.py.bk

Edit linux-gnu-amd64.py (or the version that matched your machine) to include the BLAS/LAPACK/MPICH install options:

       $ gedit linux-gnu-amd64.py &

configure_options = [ '--download-mpich=1', '--download-mpich-pm=gforker', '--download-f-blas-lapack=1', '--with-debugging=0', '--with-cc=gcc', '--with-fc=gfortran', '--download-f-blas-lapack=1', ]

			if __name__ == '__main__':
			  import sys,os
			  sys.path.insert(0,os.path.abspath('config'))
			  import configure
			  configure.petsc_configure(configure_options)

5. Configure PETSc with BLAS, LAPACK, and MPI You will need gfortran and gcc compilers to configure and compile PETSc. You can get this using "apt-get," Ubuntu's Advanced Packaging Tool, which is easy to use from the command line:

       $ sudo apt-get install build-essential
       $ sudo apt-get install gfortran
       $ sudo apt-get install gcc ;#(you might already have this one, that's ok)

Now we are ready to configure. Here's instructions so far from beginning to end of a PETSc configure, including the setting of some environment variables:

       $ cd ~/flooxs/rel2008
       $ export FLXSHOME=`pwd`
       $ cd $FLXSHOME/support/petsc-3.0.0-p11
       $ export PETSC_DIR=$PWD
       $ export PETSC_ARCH=linux-gnu-amd64
       $ ./config/examples/linux-gnu-amd64.py
       $ make all
       $ make test

Note: it is okay to ignore this warning:

       WARNING! Compiling PETSc with no debugging, this should only be done for timing 
       and production runs. All development should be done when configured using
       --with-debugging=1                                                

You should NOT get any errors. If you have NOT gotten any errors, then you can move on to the next step. If you HAVE gotten errors, you need to go over the directions again, maybe do some reading or googling, and try again until you do not get an errors.

Get tk, tcl, and blt

You will be so happy because this one is easy. If you get these as root (using sudo), the Ubuntu package manager will put the relevant files in your /usr/bin directory. You will also need to get the development packages associated with tk, tcl, and blt, which just have a -dev on them. I recommend getting the outdated 8.3 versions of tcl and tk so that your old blt code will be compatible.

       $ sudo apt-get install tk8.3 tcl8.3
       $ sudo apt-get install tk8.3-dev tcl8.3-dev

Unfortunately there is some controversy over the ./bltwish executable in later versions of blt (the easy, apt-gettable one). If you figure out how to make flooxs work without the bltwish (e.g. by using the ./wish executable with some finagling, or you find a 64-bit patch) please let me know. I achieved blt success by apt-getting the above versions 8.3 of tcl and tk, and then compiling the blt version that came with flooxs in the support directory. So, after you have tcl and tk and their dev packages, type these on your command line to configure, compile, and install blt:

       $ cd $FLXSHOME/support/blt2.4y
       $ ./configure --with-tcl=/usr/lib/tcl8.3 --with-tk=/usr/lib/tk8.3
       $ make
       $ sudo make install

Your blt should be ready! You can test by

       $ cd /usr/bin
       $ ./bltwish

If a graph window pops up, then you installed blt correctly. Next, we need to edit some flooxs files in the src directory (if you want, you can save a backup first like you did with some of the PETSc files):

       $ cd $FLXSHOME/src
       $ gedit FLOO* &

This should open 3 files: FLOODS.models, FLOOPS.models, and FLOOXS.models. Do a ctrl+f (find) for bltwish. In each of these files you will see the lines (around line 23):

       #set local [exec arch]
       set local ppc
       if { [string compare $local sun4] == 0 } {
           set GraphWindow /opt/blt2.3/bin/bltwish
           set WebViewer /usr/X11R6/bin/netscape
       } elseif { [string compare $local ppc] == 0 } {
           set GraphWindow $FLXSHOME/bin/bltwish
           set WebViewer /usr/local/bin/dillo
       } elseif { [string compare $local i686] == 0 } {
           set GraphWindow $FLXSHOME/bin/bltwish
           set WebViewer /usr/bin/netscape
       }

Wherever it says "$FLXSHOME/bin/bltwish" you need to change it to "/usr/bin/bltwish", because that is where your bltwish executable is. (Do this in all 3 files).

       #set local [exec arch]
       set local ppc
       if { [string compare $local sun4] == 0 } {
           set GraphWindow /opt/blt2.3/bin/bltwish
           set WebViewer /usr/X11R6/bin/netscape
       } elseif { [string compare $local ppc] == 0 } {
           set GraphWindow /usr/bin/bltwish
           set WebViewer /usr/local/bin/dillo
       } elseif { [string compare $local i686] == 0 } {
           set GraphWindow /usr/bin/bltwish
           set WebViewer /usr/bin/netscape
       }

SuperLU

Here some documentation on the SuperLU Debian packages: [3]. Type this on the command line to get them:

       $ sudo apt-get install libsuperlu3
       $ sudo apt-get install libsuperlu3-dev

UMFpack

Ok, there is no easy package for this one. There used to be, but that version is too old. We can just use the version in the $FLXSHOME/support directory, (even though 5.2 is technically the newest), so if you're looking for a good time without me, go ahead and download [4] and follow the instructions in their readme (good luck, let me know how it turns out). Here's what I did (for more detailed instructions, read the UMFPACK/README.txt).

       $ cd $FLXSHOME/support/UMFPACKv5.0/UMFPACK
       $ gedit UF/config/UFconfig.mk &

The instructions say: "To enable an option of the form "# OPTION = ...", edit this file and delete the "#" in the first column of the option you wish to use." I uncommented the 64 bit stuff because I knew I had a 64 bit system. Then I attempted to compile with:

       $ cd $FLXSHOME/support/UMFPACKv5.0/UMFPACK
       $ make

I had some errors while compiling, first with xerbla. I'm not sure, but it looked to me like there was a typo in their library name (a "c" instead of an "x"). In the UFconfig.mk file, they suggested:

       # XERBLA = ../../UFconfig/xerbla/libxerbla.a

but in that directory, there is only libcerbla.a, so I changed that line to:

       XERBLA = ../../UFconfig/xerbla/libcerbla.a 

Then, I had a second error about "can't find -lgoto64", which is in UFconfig.mk for blas 64 bit. So, I just commented that line out... and UMFpack compiled with no errors. Flooxs worked fine later, so this seemed to be ok. There is one last step before moving on:

       $ make clean

If you find my methods suspect, see the User Guide (Doc/UserGuide.pdf), or ../UFconfig/UFconfig.mk for more details, including options for compiling in 64-bit mode.

Edit FLOOXS Library Paths in your Arch File

In this section, we will edit a long list of library paths contained in your arch file. These paths tell flooxs where you have installed all the packages from above. First, let's find the file with all these paths:

       $ cd $FLXSHOME/arch
       $ cp linux64 linux64_bk ;# (makes a backup)
       $ gedit linux64 &

This part is painstakingly annoying. For *each* path in this file, I had to navigate to that directory in an old, working flooxs directory, find what files were there, and then use

       $ find . -name "*file*" -print

to see where the new ones were. Most of the time you are looking for .h or .a files, which are headers and library files respectively.

Here is my completed arch file that resulted in a successful compile and future operation of flooxs: linux64. You can see I commented out the old paths with #, and put my new paths underneath.

Compile Flooxs

If you get warnings, probably this is okay. Errors will never be okay. In general, you should resolve any errors in the "make depend" before trying to make. When you get an error, scroll up to see what directory you were in, check and see what file the error is in, and what line number it is giving you. Most likely if you get errors you simply did not set your paths correctly in your arch file.

Flooxs can be compiled in two ways. The first is with cmake:

       $ cd $FLXSHOME
       $ mkdir release && cd release
       $ cmake .. && make -j8
       $ sudo make install

The second is with:

       $ cd $FLXSHOME/src
       $ make depend ARCH=linux64
       $ make ARCH=linux64

Do a CVS update if you want

If you are one of Dr. Law's students and you have a login to our computers, you can do a CVS update to get the latest version of the flooxs code. Don't worry if you cannot do this. The 2008 version works fine.

First, get cvs (concurrent versioning system) for your Ubuntu:

       $ sudo apt-get install cvs
       $ cd $FLXSHOME/src/CVS (contains 3 files)
       $ ls
       $ gedit Root

Replace line there with:

       :ext:<username>@swamp2.tec.ufl.edu:/home/killebrew0/flooxs/cur/cvsroot

where <username> is YOUR username. You will have to do this in each of the subdirectories too. Each subdirectory has a CVS directory with Root in it. You can use recursive grep to find these directories:

       $ cd $FLXSHOME/src
       $ grep -r swamp2 .

This last command uses the grep program (grep) to recursively search (-r) for the pattern (swamp2) in the directory you are in (.) which because of the previous line is ($FLXSHOME/src). You will have to fix the "Root" file in each of the directories that this command prints out to have your username. You can "man grep" to find other options that will print prettier output, and you can also write your own script that will go in and replace your username automatically. Go ahead and post here if you make a script.

When you're done with that, then you can do the CVS update:

       $ cd $FLXSHOME/src
       $ cvs update
       -log into swamp2 with your passwd

If you do a cvs update, there may be some conflicts with the new version that the cvs program was not intelligent enough to know how to merge. This is normal. As the intelligent user, you must resolve these conflicts. But how, you ask? Files with conflicts will have conflict markers in them. Conflict markers look like:

               <<<<<<< filename
               your lines
               =======
               other person's lines
               >>>>>> current version"

In this case, you want the other person's line, as this is Dr. Law's new version, and not your old version. Of course, you can always make a backup first. For example, I used this method to find the conflicts, then fixed them by hand:

       $ find $FLXSHOME -name "<<<<<" -print
       $ cd $FLXSHOME/src
       $ cp Makefile Makefile_bk
       $ gedit Makefile &

       $ cd math
       $ gedit Makefile &
       $ gedit blas.h & # conflict here too

some other errors I got and how I resolved them

guess: seems dsp_defs.h is old, and slu_ddefs.h is new better one. Go through the files in the math directory and replace dsp_defs.h with slu_ddefs.h. Hopefully, that will do the trick.

       $ cd src/math
       $ grep -r "dsp_defs.h" .
               ./Superlu.h:#include "dsp_defs.h"
       $ gedit Superlu.h &

- replace with #include "slu_ddefs.h" Where should SUPERLUDIR point? "It looks like it should point to the home directory of the install of SuperLU, so in your case it doesn't really work. I think you need to edit the Makefile in the math directory. There is an INCL definition that uses SUPERLUDIR. Change that to -I/usr/include/super..... - whatever your directory location.

       - error:
       -I/home/nicrawfolerd/flooxs/rel2008/support/UMFPACKv5.0/AMD/Include Sles.cc
       Sles.cc: In member function ‘virtual int SlesMatrix::FinishLoad()’:
       Sles.cc:55: error: ‘MAT_ROWS_SORTED’ was not declared in this scope
       Sles.cc: In member function ‘virtual void SlesPrecond::NewMatrix(Matrix&)’:
       Sles.cc:111: error: ‘PCFactorSetMatOrdering’ was not declared in this scope
       make[1]: *** [/home/nicrawfolerd/flooxs/rel2008/lib/linux64/math.a(Sles.o)] Error 1
       make[1]: Leaving directory `/home/nicrawfolerd/flooxs/rel2008/src/math'
       make: *** [libs] Error 1

"I'm not sure why, but I also know we don't use sles very often. Comment out the offending lines and let's see if we can get further."

       so: line 55:

// ierr = MatSetOption(mat, MAT_ROWS_SORTED); //comment out to help compile?

       line 111: has comment below it:
       // ierr = PCFactorSetMatOrdering(pc, matord);
       ///// Newer versions of PETSC might work with this - great! I have a newer version of PETSC!!
       // ierr = PCFactorSetMatOrderingType(pc, matord);

but this didn't work so I commented that last one out too. sles seems to be a problem. check SLESDIR in arch/linux64 file:

       - in old slesdir is: 
       nicrawfolerd@bluebox:~/flooxs/rel2008/support/petsc-2.3.2-p7/lib/linux-gnu-c-debug$ ls
       libpetsc.a         libpetscdm.a   libpetscmat.a   libpetscts.a
       libpetsccontrib.a  libpetscksp.a  libpetscsnes.a  libpetscvec.a
       - new slesdir is?:
       nicrawfolerd@bluebox:~/flooxs/rel2008/support/petsc-3.0.0-p11/linux-gnu-amd64/lib$ ls
       libfblas.a    libmpich.a     libpetsc.a         libpetscmat.a   libprometheus.a
       libflapack.a  libmpichcxx.a  libpetsccontrib.a  libpetscsnes.a  libpromfei.a
       libfmpich.a   libmpichf90.a  libpetscdm.a       libpetscts.a    pkgconfig
       libmetis.a    libparmetis.a  libpetscksp.a      libpetscvec.a
       - MPILIB has:
       -old:
       nicrawfolerd@bluebox:~/flooxs/rel2008/support/petsc-2.3.2-p7/externalpackages/mpich2-1.0.4p1/lib$ ls
       libfmpich.a  libmpich.a  libmpichcxx.a  libmpichf90.a  pkglist  st2m5gyL
       -new:
       nicrawfolerd@bluebox:~/flooxs/rel2008/support/petsc-3.0.0-p11/externalpackages/mpich2-1.0.8/lib$ ls
       pkglist

These are not the same except for package list. I changed paths for MPILIB and MPIDIR (see arch/linux64 file), but got the same error. Finally, I recompiled petsc without prometheus and metis (edit linux-gnu-amd64.py, etc), and this got rid of the prometheus and metis erros, but still have 2 errors:

       /home/nicrawfolerd/flooxs/rel2008/support/petsc-3.0.0-p11/linux-gnu-amd64/lib/libpetscdm.a(ao.o):
               In function `AOApplicationToPetscIS':
               ao.c:(.text+0x296): undefined reference to `ISStrideToGeneral'
       /home/nicrawfolerd/flooxs/rel2008/support/petsc-3.0.0-p11/linux-gnu-amd64/lib/libpetscdm.a(ao.o):
       In function `AOPetscToApplicationIS':
       ao.c:(.text+0x467): undefined reference to `ISStrideToGeneral'
       collect2: ld returned 1 exit status
       Command exited with non-zero status 1

This seems to be a problem with libpetscdm.a(ao.o). So, I looked in this file (ao.o) and commented out all lines with "ISStrideToGeneral". Flooxs compiled with no errors. However, the graphing was broken... turns out there's some issues with ./bltwish on newer blt versions. I solved this by downgrading my tcl/tk to v8.3, and updated the tcl/tk/blt sections in this wiki to reflect that change on 1/1/2010.