Skip to content

Latest commit

 

History

History
276 lines (216 loc) · 10.8 KB

HACKING.md

File metadata and controls

276 lines (216 loc) · 10.8 KB
Raw

Open MPI Hacking / Developer's Guide

Overview

This file is here for those who are building/exploring OMPI in its source code form, most likely through a developer's tree (i.e., a Git clone).

Obtaining Open MPI

Open MPI is available from many distributions, however some users prefer to obtain it directly from the Open MPI community via prepackaged tarball (see: https://www.open-mpi.org/software/ompi/). The Open MPI tarball includes manpages, and openpmix and openprrte components, along with an auto-generated configure script.

Some developers prefer to obtain Open MPI by directly cloning it from https://github.com/open-mpi/ompi. It is recommended that users who choose to clone the source directly, use the git clone flag --recurse-submodules, to also obtain the openpmix, and openprrte.

Regardless of how openpmix and openprrte are obtained, the configure logic in Open MPI v5.0+ prefer externally installed components. Please see configure --help for more details.

Developer Builds: Compiler Pickyness by Default

If you are building Open MPI from a Git clone (i.e., there is a .git directory in your build tree), the default build includes extra compiler pickyness, which will result in more compiler warnings than in non-developer builds. Getting these extra compiler warnings is helpful to Open MPI developers in making the code base as clean as possible.

Developers can disable this picky-by-default behavior by using the --disable-picky configure option. Also note that extra-picky compiles do not happen automatically when you do a VPATH build (e.g., if .git is in your source tree, but not in your build tree).

Prior versions of Open MPI would automatically activate a lot of (performance-reducing) debugging code by default if .git was found in your build tree. This is no longer true. You can manually enable these (performance-reducing) debugging features in the Open MPI code base with these configure options:

  • --enable-debug
  • --enable-mem-debug
  • --enable-mem-profile

NOTE: These options are really only relevant to those who are developing Open MPI itself. They are not generally helpful for debugging general MPI applications.

Use of GNU Autoconf, Automake, and Libtool (and m4)

You need to read/care about this section ONLY if you are building from a developer's tree (i.e., a Git clone of the Open MPI source tree). If you have an Open MPI distribution tarball, the contents of this section are optional -- you can (and probably should) skip reading this section.

If you are building Open MPI from a developer's tree, you must first install fairly recent versions of the GNU tools Autoconf, Automake, and Libtool (and possibly GNU m4, because recent versions of Autoconf have specific GNU m4 version requirements). The specific versions required depend on if you are using the Git master branch or a release branch (and which release branch you are using). The specific versions can be found here.

You can check what versions of the autotools you have installed with the following:

shell$ m4 --version
shell$ autoconf --version
shell$ automake --version
shell$ libtoolize --version

Required version levels for all the OMPI releases can be found here.

To strengthen the above point: the core Open MPI developers typically use very, very recent versions of the GNU tools. There are known bugs in older versions of the GNU tools that Open MPI no longer compensates for (it seemed senseless to indefinitely support patches for ancient versions of Autoconf, for example). You WILL have problems if you do not use recent versions of the GNU tools.

NOTE: On MacOS/X, the default libtool program is different than the GNU libtool. You must download and install the GNU version (e.g., via MacPorts, Homebrew, or some other mechanism).

If you need newer versions, you are strongly encouraged to heed the following advice:

  1. Unless your OS distribution has easy-to-use binary installations, the sources can be can be downloaded from:

    NOTE: It is certainly easiest to download/build/install all four of these tools together. But note that Open MPI has no specific m4 requirements; it is only listed here because Autoconf requires minimum versions of GNU m4. Hence, you may or may not need to actually install a new version of GNU m4. That being said, if you are confused or don't know, just install the latest GNU m4 with the rest of the GNU Autotools and everything will work out fine.

  2. Build and install the tools in the following order:

    1. m4
    2. Autoconf
    3. Automake
    4. Libtool

  3. You MUST install the last three tools (Autoconf, Automake, Libtool) into the same prefix directory. These three tools are somewhat inter-related, and if they're going to be used together, they MUST share a common installation prefix.

    You can install m4 anywhere as long as it can be found in the path; it may be convenient to install it in the same prefix as the other three. Or you can use any recent-enough m4 that is in your path.

    1. It is strongly encouraged that you do not install your new versions over the OS-installed versions. This could cause other things on your system to break. Instead, install into $HOME/local, or /usr/local, or wherever else you tend to install "local" kinds of software.

    2. In doing so, be sure to prefix your $path with the directory where they are installed. For example, if you install into $HOME/local, you may want to edit your shell startup file (.bashrc, .cshrc, .tcshrc, etc.) to have something like:

      # For bash/sh:
export PATH=$HOME/local/bin:$PATH
# For csh/tcsh:
set path = ($HOME/local/bin $path)

    3. Ensure to set your $PATH BEFORE you configure/build/install the four packages.

  4. All four packages require two simple commands to build and install (where PREFIX is the prefix discussed in 3, above).

    shell$ cd <m4 directory>
shell$ ./configure --prefix=PREFIX
shell$ make; make install

    NOTE: If you are using the csh or tcsh shells, be sure to run the rehash command after you install each package.

    shell$ cd <autoconf directory>
shell$ ./configure --prefix=PREFIX
shell$ make; make install

    NOTE: If you are using the csh or tcsh shells, be sure to run the rehash command after you install each package.

    shell$ cd <automake directory>
shell$ ./configure --prefix=PREFIX
shell$ make; make install

    NOTE: If you are using the csh or tcsh shells, be sure to run the rehash command after you install each package.

    shell$ cd <libtool directory>
shell$ ./configure --prefix=PREFIX
shell$ make; make install

    NOTE: If you are using the csh or tcsh shells, be sure to run the rehash command after you install each package.

    m4, Autoconf and Automake build and install very quickly; Libtool will take a minute or two.

  5. You can now run OMPI's top-level autogen.pl script. This script will invoke the GNU Autoconf, Automake, and Libtool commands in the proper order and setup to run OMPI's top-level configure script.

    Running autogen.pl may take a few minutes, depending on your system. It's not very exciting to watch. 😄

    If you have a multi-processor system, enabling the multi-threaded behavior in Automake 1.11 (or newer) can result in autogen.pl running faster. Do this by setting the AUTOMAKE_JOBS environment variable to the number of processors (threads) that you want it to use before invoking autogen.pl. For example (you can again put this in your shell startup files):

     # For bash/sh:
 export AUTOMAKE_JOBS=4
 # For csh/tcsh:
 set AUTOMAKE_JOBS 4

    1. You generally need to run autogen.pl whenever the top-level file configure.ac changes, or any files in the config/ or <project>/config/ directories change (these directories are where a lot of "include" files for Open MPI's configure script live).

    2. You do NOT need to re-run autogen.pl if you modify a Makefile.am.

Use of Flex

Flex is used during the compilation of a developer's checkout (it is not used to build official distribution tarballs). Other flavors of lex are not supported: given the choice of making parsing code portable between all flavors of lex and doing more interesting work on Open MPI, we greatly prefer the latter.

Note that no testing has been performed to see what the minimum version of Flex is required by Open MPI. We suggest that you use v2.5.35 at the earliest.

NOTE: Windows developer builds of Open MPI require Flex version 2.5.35. Specifically, we know that v2.5.35 works and 2.5.4a does not. We have not tested to figure out exactly what the minimum required flex version is on Windows; we suggest that you use 2.5.35 at the earliest. It is for this reason that the contrib/dist/make_dist_tarball script checks for a Windows-friendly version of Flex before continuing.

For now, Open MPI will allow developer builds with Flex 2.5.4. This is primarily motivated by the fact that RedHat/Centos 5 ships with Flex 2.5.4. It is likely that someday Open MPI developer builds will require Flex version >=2.5.35.

Note that the flex-generated code generates some compiler warnings on some platforms, but the warnings do not seem to be consistent or uniform on all platforms, compilers, and flex versions. As such, we have done little to try to remove those warnings.

If you do not have Flex installed, see the Flex Github repository.

Use of Pandoc

Similar to prior sections, you need to read/care about this section ONLY if you are building from a developer's tree (i.e., a Git clone of the Open MPI source tree). If you have an Open MPI distribution tarball, the contents of this section are optional -- you can (and probably should) skip reading this section.

The Pandoc tool is used to generate Open MPI's man pages. Specifically: Open MPI's man pages are written in Markdown; Pandoc is the tool that converts that Markdown to nroff (i.e., the format of man pages).

You must have Pandoc >=v1.12 when building Open MPI from a developer's tree. If configure cannot find Pandoc >=v1.12, it will abort.

If you need to install Pandoc, check your operating system-provided packages (to include MacOS Homebrew and MacPorts). The Pandoc project web site itself also offers binaries for their releases.