Installation on Ubuntu

Today is 3/29/2010 - These are notes on how to install instructions for flooxs on 64-bit Ubuntu 9.10 Karmic Koala. These instructions are intended for linux/unix beginners. You may email me direct questions about installing flooxs on Ubuntu at nrowsey at ufl.edu, but please try these instructions first. Right now the formatting of this text is not that great. I'll learn more about how to format on this wiki and fix this soon.

Get Your Flooxs!

1. Download the software and put it on your hard drive in a directory called flooxs

       $ mkdir ~/flooxs
       - go to [1], download the latest linux executable file (get the one that says linux)

2. Unpack the flooxs file

       Double click -> Export

This should make a directory called rel2008. Put it in your new flooxs folder. Inside rel2008 you should have these folders:

       $ cd ~/flooxs/rel2008
       $ ls
       arch  bin  help  lib  Params  src  support  TclLib  Test

To install flooxs, we first must install several supporting programs (such as math or graphing packages) in the support directory. Then, we will go into the src (short for "source") directory and compile flooxs from there.

Get PETSC math package, with BLAS, LAPACK and MPI

In this section, we will download, configure, and compile three supporting math packages that flooxs uses to solve matrix problems.

1. First, set some environment variables so flooxs knows where to look for libraries:

       $ cd ~/flooxs/rel2008
       $ export FLXSHOME=`pwd`

2. Download the latest PETSc package from [2]. When I did this step, the latest package was 3.0.0-p11. You may have better or worse luck using older or newer packages. (Note: the petsc version that comes with your flooxs 2008 download does not have the capability of downloading blas and lapack automatically, so if you choose the version already in your FLXSHOME/support directory you will have to install and link blas and lapack yourself). Do not use archive manager to unpack this. Instead, we will unpack manually from the command line.

       - put the downloaded file in $FLXSHOME/support
       $ cd $FLXSHOME/support
       $ gunzip -c petsc-3.0.0-p11.tar.gz | tar -xof -
       $ cd petsc-3.0.0-p11
       $ export PETSC_DIR=$PWD

3. Find out how to Configure PETSc, with BLAS, LAPACK, and MPI. Check out example configure files in $PETSC_DIR/config/examples/*.py. For example, I am running linux, and my processor is AMD 64-bit, so I looked at the example file named linux-gnu-amd64.py.

       $ cd $PETSC_DIR/config/examples/
       $ ls
       aix5.1.0.0-64.py      cygwin-borland.py        linux-gnu-nagf90.py
       aix5.1.0.0.py         cygwin-ms.py             linux-gnu.py
       asterix32.py          cygwin.py                linux-gnu-superlu.py
       asterix32-sun.py      freebsd5.1-c89.py        linux-gnu-uni.py
       asterix64-c89.py      freebsd5.1.py            linux-gnu-valgrind.py
       asterix64-openmpi.py  linux-gnu-amd64.py       linux-rhAS3-gcc32-opt.py
       asterix64.py          linux-gnu-blocksolve.py  linux-rhAS3-intel81-opt.py
       asterix64-sun.py      linux-gnu-cxx-sieve.py   macosx-gnu-g95.py
       bgl-gnu-opt.py        linux-gnu-gcc-absoft.py  macosx-gnu.py
       bgl-ibm-cxx-opt.py    linux-gnu-gcc-ifc.py     sicortex.py
       bgl-ibm-debug.py      linux-gnu-gcc-pgf90.py   solaris2.9-lam.py
       bgl-ibm-opt.py        linux-gnu-ia64-intel.py  solaris2.9.py
       bgp-ibm-opt.py        linux-gnu-ia64.py        solaris-64.py
       cray-x1.py            linux-gnu-intel.py       solaris-gnu.py
       cray-xt3.py           linux-gnu-ml.py          solaris-uni.py
       $ gedit linux-gnu-amd64.py &

This is what I saw inside the file that sounded like my architecture:

               #!/usr/bin/env python

                       configure_options = [
                         '--download-mpich=1',
			  '--download-mpich-pm=gforker',
			  '--download-f-blas-lapack=1',
			  '--download-prometheus=1',
			  '--download-parmetis=1',
			  '--with-debugging=0'
			  ]

			if __name__ == '__main__':
			  import sys,os
			  sys.path.insert(0,os.path.abspath('config'))
			  import configure
			  configure.petsc_configure(configure_options)
			  

From the installation instructions on the petsc website, I saw that, if I wanted to download blas, lapack, and the MPI compilers that petsc needs, I should add these lines to the list in the above file:

			  '--download-f-blas-lapack=1',
			  '--download-mpich=1'

Also, the prometheus and metis packages are not used in flooxs, and can cause problems, so I knew I had to delete these two lines:

			  '--download-prometheus=1',
			  '--download-parmetis=1',

Below explains in more detail how I did this.

4. Edit your PETSc configure file: First, save a backup:

       $ cd $PETSC_DIR/config/examples
       $ cp linux-gnu-amd64.py linux-gnu-amd64.py.bk

Edit linux-gnu-amd64.py or the version that matched your machine to include the BLAS/LAPACK/MPICH install options:

       $ gedit linux-gnu-amd64.py &

configure_options = [ '--download-mpich=1', '--download-mpich-pm=gforker', '--download-f-blas-lapack=1', '--with-debugging=0', '--with-cc=gcc', '--with-fc=gfortran', '--download-f-blas-lapack=1', ]

			if __name__ == '__main__':
			  import sys,os
			  sys.path.insert(0,os.path.abspath('config'))
			  import configure
			  configure.petsc_configure(configure_options)

5. Configure PETSc with BLAS, LAPACK, and MPI You will need gfortran and gcc compilers to configure and compile petsc. You can get this using the Ubuntu package manager:

       $ sudo apt-get install build-essential
       $ sudo apt-get install gfortran
       $ sudo apt-get install gcc ;#(you might already have this one)

Now we are ready to configure. Here's instructions so far from beginning to end of a petsc configure, including the setting of some environment variables:

       $ cd ~/flooxs/rel2008
       $ export FLXSHOME=`pwd`
       $ cd $FLXSHOME/support/petsc-3.0.0-p11
       $ export PETSC_DIR=$PWD
       $ export PETSC_ARCH=linux-gnu-amd64
       $ ./config/examples/linux-gnu-amd64.py
       $ make all
       $ test all

Note: it is okay to ignore this warning:

       WARNING! Compiling PETSc with no debugging, this should only be done for timing 
       and production runs. All development should be done when configured using
       --with-debugging=1                                                

You should NOT get any errors. If you have not gotten any errors, you can move on to the next step.

Get tk, tcl, and blt

You will be so happy because this one is easy. If you get these as root, the Ubuntu package manager will put the relevant files in your /usr/bin. You will also need to get the development packages, which just have a -dev on them.

       $ sudo apt-get install tk tcl blt

SuperLU

Here some documentation on the superlu Debian packages: [3]. Type this on the command line to get them:

       $ sudo apt-get install libsuperlu3
       $ sudo apt-get install libsuperlu3-dev

UMFpack

Ok, there is no easy package for this one. There used to be, but that version is too old. We can just use the version in the $FLXSHOME/support directory that

- UMFpack -couldn't find headers for UMFpack, so tried $ sudo apt-get install libsuitesparse-dev -give up and compile from source? http://linux.softpedia.com/progDownload/UMFPACK-Download-35440.html - dr. law has v5.0, so why bother to compile the "new" one, 5.2? $ cd $FLXSHOME/support/UMFPACKv5.0/UMFPACK - Edit the Makefile to select the system of your choice and then edit that file. $ gedit Makefile & - select "include Make.linux" - nothing to select!! just compile everything $ gedit Make.linux & -You'll need to again indicate the location of the BLAS lib, and you do not need to compile a version for matlab - dr. law's advice was not helpful: try the UMFPACK/README.txt

"GNUmakefile a nice Makefile, for GNU make" "Makefile an ugly Unix Makefile (for older make's)"

"To configure, edit the UFconfig/UFconfig.mk file (otherwise, you may get warnings that the BLAS (dgemm, etc) are not found). $ gedit UF/config/UFconfig.mk -To enable an option of the form "# OPTION = ...", edit this file and # delete the "#" in the first column of the option you wish to use. - uncommented the 64 bit stuff... $ cd to UMFPACK $ make - had some errors while compiling, first with xerbla, seems to be typo in the name of the library? -they suggest: # XERBLA = ../../UFconfig/xerbla/libxerbla.a -but in that directory is only libcerbla.a, so changed: XERBLA = ../../UFconfig/xerbla/libcerbla.a -typo? 3.4.2010 - had a second err about "can't find -lgoto64", which is in UFconfig.mk for blas 64 bit. I just commented it out... and it compiled. we'll see if it works with floods compile... but first:

$ make clean

-See the User Guide (Doc/UserGuide.pdf), or ../UFconfig/UFconfig.mk for more details (including options for compiling in 64-bit mode).

   		- see if the flooxs make will do the umf make as well, since the directories are there.

- no, wait, then you won't have the right path for blas in umf :( make doesn't work anyway...

11. On to FLOOPS/FLOODS - edit library paths

Edit FLOOXS Library Paths in your Arch File

- Edit your library paths $ cd $FLXSHOME/arch $ cp linux64 linux64_bk ;# (makes a backup) $ gedit linux64 & - you might need to update locations for your specific versions of the support libraries.

THIS IS FRIKIN HARD :(:(:( for *each* old path name, you have to navigate to that dir in an old flooxs directory, find what files are there, and then find . -name "*file*" -print those files to see where your new ones are. Most of the time they will be .h or .a files.

- TCL/TK/BLT $ find / -name tcl.h - only found it in FLXSHOME/include and FLXSHOME/support/tcl.?.? - note, .h probably means "header"

looked in /usr/include, saw a superlu folder, but no grep on tk or tcl. so tried $ sudo apt-get install tk-dev -seems to have gotten a lot of stuff - I don't know if that was necessary, or if you just needed ls -a, there is a tk and a tcl.8.4 folder with .h's in them

- ls -a | grep blt gives nothing $ sudo apt-get install blt-dev - now I get a grep blt!! and a blt.h, which is in the /usr/include - note: I think "-dev" stands for development

- Final Guess: TCL_INCLUDE = /usr/include/tcl8.4 TK_INCLUDE = /usr/include/tk BLT_INCLUDE = /usr/include 3.3.2010

- X11 $ xterm & ;# works so it looks like you already have X11

-Final Guess X_INCLUDE = /usr/X11 MASTER_INCLUDE = -I$(TCL_INCLUDE) -I$(TK_INCLUDE) -I$(BLT_INCLUDE) -Dlinux -DLINUX X_LIB = -L/usr/lib64/X11 -lX11

- SuperLU -Final Guess SUPERLUDIR = /usr/include/superlu SUPERLIB = /usr/lib64/libsuperlu.a

---

Compile Floops/Floods

- make depend was ok - make had LOTS of errors :(:(

Do a CVS update (how to do this...)

$ sudo apt-get install cvs - look in $FLXSHOME/src/CVS (contains 3 files) $ gedit Root -replace line there with: :ext:nrowsey@swamp2.tec.ufl.edu:/home/killebrew0/flooxs/cur/cvsroot $ cd $FLXSHOME/src $ cvs update -log into swamp2 with your passwd

-ok problem with flooxs make depend: Makefile:25: *** commands commence before first target. Stop.

-ok your version was not the same as Dr. Law's new version, and cvs did a merge: "Files with conflicts will have conflict markers in them. Conflict markers look like: <<<<<<< filename your lines ======= other person's lines >>>>>>> current version"

- The error above is telling me that I should look in the Makefile near line 25. I looked there, saw the conflict, then decided to delete my lines, and keep the "other person's" bc the other person is Dr. Law, but I'll make a backup first

$ cd $FLXSHOME/src $ cp Makefile Makefile_bk $ gedit Makefile &

- error in src/math dir near line 12, same kind so, $ cd math $ gedit Makefile & $ gedit blas.h & # conflict here too 3/5/2010 - seems dsp_defs.h is old, and slu_ddefs.h is new better one. Go through the files in the math directory and replace dsp_defs.h with slu_ddefs.h. Hopefully, that will do the trick. $ cd src/math $ grep -r "dsp_defs.h" . ./Superlu.h:#include "dsp_defs.h" $ gedit Superlu.h & - replace with #include "slu_ddefs.h"

- Where should SUPERLUDIR point? "It looks like it should point to the home directory of the install of SuperLU, so in your case it doesn't really work. I think you need to edit the Makefile in the math directory. There is an INCL definition that uses SUPERLUDIR. Change that to -I/usr/include/super..... - whatever your directory location.

- error: -I/home/nicrawfolerd/flooxs/rel2008/support/UMFPACKv5.0/AMD/Include Sles.cc Sles.cc: In member function ‘virtual int SlesMatrix::FinishLoad()’: Sles.cc:55: error: ‘MAT_ROWS_SORTED’ was not declared in this scope Sles.cc: In member function ‘virtual void SlesPrecond::NewMatrix(Matrix&)’: Sles.cc:111: error: ‘PCFactorSetMatOrdering’ was not declared in this scope make[1]: *** [/home/nicrawfolerd/flooxs/rel2008/lib/linux64/math.a(Sles.o)] Error 1 make[1]: Leaving directory `/home/nicrawfolerd/flooxs/rel2008/src/math' make: *** [libs] Error 1

"I'm not sure why, but I also know we don't use sles very often. Comment out the offending lines and let's see if we can get further."

so: line 55: // ierr = MatSetOption(mat, MAT_ROWS_SORTED); //comment out to help compile?

line 111: has comment below it: // ierr = PCFactorSetMatOrdering(pc, matord);

///// Newer versions of PETSC might work with this - great! I have a newer version of PETSC!! // ierr = PCFactorSetMatOrderingType(pc, matord); - this didn't work so I commented it out too

3/11/2010 - sles seems to be a problem. check SLESDIR in arch/linux64 file: - in old slesdir is: nicrawfolerd@bluebox:~/flooxs/rel2008/support/petsc-2.3.2-p7/lib/linux-gnu-c-debug$ ls libpetsc.a libpetscdm.a libpetscmat.a libpetscts.a libpetsccontrib.a libpetscksp.a libpetscsnes.a libpetscvec.a - new slesdir is?: nicrawfolerd@bluebox:~/flooxs/rel2008/support/petsc-3.0.0-p11/linux-gnu-amd64/lib$ ls libfblas.a libmpich.a libpetsc.a libpetscmat.a libprometheus.a libflapack.a libmpichcxx.a libpetsccontrib.a libpetscsnes.a libpromfei.a libfmpich.a libmpichf90.a libpetscdm.a libpetscts.a pkgconfig libmetis.a libparmetis.a libpetscksp.a libpetscvec.a - seems to have all the same files and more - see email to dr. law - MPILIB has: -old: nicrawfolerd@bluebox:~/flooxs/rel2008/support/petsc-2.3.2-p7/externalpackages/mpich2-1.0.4p1/lib$ ls libfmpich.a libmpich.a libmpichcxx.a libmpichf90.a pkglist st2m5gyL -new: nicrawfolerd@bluebox:~/flooxs/rel2008/support/petsc-3.0.0-p11/externalpackages/mpich2-1.0.8/lib$ ls pkglist

- ew - not the same except for package list... - changed paths for MPILIB and MPIDIR (see arch/linux64 file) -same error - tried to recompile petsc without prometheus and metis (edit linux-gnu-amd64.py, etc) - ok, this got rid of the prometheus and metis erros, but still have 2 errors: /home/nicrawfolerd/flooxs/rel2008/support/petsc-3.0.0-p11/linux-gnu-amd64/lib/libpetscdm.a(ao.o): In function `AOApplicationToPetscIS': ao.c:(.text+0x296): undefined reference to `ISStrideToGeneral'

/home/nicrawfolerd/flooxs/rel2008/support/petsc-3.0.0-p11/linux-gnu-amd64/lib/libpetscdm.a(ao.o): In function `AOPetscToApplicationIS': ao.c:(.text+0x467): undefined reference to `ISStrideToGeneral' collect2: ld returned 1 exit status Command exited with non-zero status 1

- seems to be a problem with libpetscdm.a(ao.o) whatever ao.o means - I looke in this file, commented out all lines with "ISStrideToGeneral" - floods compiled - I can run my files, but the graphing is broken

- fix BLT??