Documentation & Tutorials

Building Guide

Introduction

This guide gives directions for building Xyce from the source code. It is recommended that anyone intending to build Xyce be familiar with editing text files and compiling programs on their target computer. Prior experience using command line tools like configure, cmake and make is recommended.

Overview

Installing Xyce from source requires the following general steps:

It is also recommended that after building Xyce one run the complete regression test suite. After building and testing, the code may be installed.

Most libraries that Xyce requires are available through system package managers; they are also all available in source form, which you can compile yourself. Since the majority of the work in installing Xyce involves installing the prerequisite libraries, most of this page is focused on these prerequisites.

Build Directories

It is important to note that our recommendations for building Trilinos and Xyce make use of "out-of-source" builds. This means that the executables and/or libraries are built in a directory completely separate from the directory containing the source code. This is accomplished by creating an empty directory anywhere in the file system of your computer (the "build directory"), and using appropriate arguments in "configure" scripts to tell the build system where to find the source code (the "source directory").

Doing builds in directories separate from the source code keeps the original source tree pristine, and allows you to build many different versions (specifically, both serial and parallel versions) without changing the source tree. It also permits sharing a single source tree over a networked filesystem, and building the code for multiple operating systems without copying the source to every machine. This process is described in detail in the generic "INSTALL" file included at the top level of the Xyce source tree; this file is part of the GNU Automake system, and is not specific to Xyce.

Upgrading from a previous version of Xyce

If you have already compiled an earlier version of Xyce, then you are likely ready to build the current version of Xyce. The prerequisite libraries are generally the same for this release as they were for prior versions.  One notable change in the current release is that Xyce no longer requires UMFPACK to build, nor do parallel builds of Xyce require ParMETIS --- this is necessary to build Xyce only if Trilinos was built with these features explicitly enabled, and our configure script automatically detects whether or not Trilinos requires them.  All of our guidance in the following sections has dropped reference to these libraries.

While Trilinos 12.12.1 is the latest version to be tested with Xyce 6.10, it is not necessary to upgrade to this version of Trilinos if you already have Trilinos built for a previous Xyce installation. Xyce 6.10 is known to build against all versions of Trilinos between 12.4.2 and 12.12.1.  

Xyce 6.10 has NOT been fully tested with versions of Trilinos released after 12.12.1, and we cannot guarantee that Xyce 6.10 will run well with later versions, nor can we be sure Xyce will even build against such versions.

If you plan to upgrade any components before installing Xyce 6.10, please review the sections below, as Open MPI 1.10 and Trilinos 12.x need special treatment.

Prerequisites

Building Xyce requires a computer with a modern C++ compiler. Some of the libraries that Xyce uses also require a Fortran compiler. The Xyce development team has successfully used the Gnu Compiler Collection (gcc), the Intel Compiler Collection and Clang. Other compilers may work, but we have not tested any others. For building and running Xyce in parallel, an MPI compiler front-end is needed, such as Open MPI or MPICH.

The Automake system used by Xyce assumes a basic Unix-like command line system. Windows users may use the Cygwin system for this purpose. The Xyce team does not support non-Unix development environments such as Visual Studio; some users have worked out how to get Xyce compiled in these environments, but the team can only support the build types described here.

Xyce depends on many external libraries to supply enhanced parsing, mathematical algorithms and parallel communication. Some of the required libraries may already be present on your system, and many can be obtained with package managers. Occasionally, some will have to be built from source. All must be installed prior to building Xyce.

The required libraries for Xyce are listed in the table below. All of the libraries are either freely available, or have equivalents that are open source.  Please note the minimum version numbers listed.  If you try to use versions earlier than those indicated, you will likely see failures.  If shown, maximum version numbers indicate the latest version that has been tested with Xyce.

Licensing Note: Please be sure to understand the licenses associated with the various libraries. Most of the licenses are common and should be acceptable to most users. Inclusion of some non-open-source libraries could limit your rights to distribute binaries you produce.

Name Version Notes and Download URL
BLAS May be included in your compiler's libraries, or may require separate package installation; most systems have a version of BLAS in their package management systems. ATLAS may be used to provide optimized versions of BLAS and LAPACK. BLAS source code may be downloaded from http://www.netlib.org/blas if the package is otherwise unavailable.
LAPACK May be included in your compiler's libraries, or may require separate package installation; most systems have a version of LAPACK in their package management systems. ATLAS may be used to provide optimized versions of BLAS and LAPACK. LAPACK source code may be downloaded from http://www.netlib.org/lapack if the package is otherwise unavailable.
bison 2.3-3.1 Required for the chemical reaction parser. Included in most operating systems and package management systems. Download source from http://www.gnu.org/software/bison/ and compile if it is otherwise unavailable on your system. 
flex 2.5.34 or later Required for the chemical reaction parser. Included in most operating systems and package management systems. Download source from https://github.com/westes/flex and compile if otherwise unavailable on your system.
FFT package

FFTW 3.x

OR

Intel Math Kernel Library

Required for Harmonic Balance analysis. Xyce is set up to use either the Intel Math Kernel Library or FFTW3 for this purpose. FFTW3 is available in most Unix-like operating systems' package management systems. See http://software.intel.com or http://www.fftw.org. Xyce is not compatible with versions of FFTW in the obsolete 2.x series.
SuiteSparse 2.1.1 or later Xyce requires AMD, which is part of the SuiteSparse package. If otherwise unavailable, download from http://faculty.cse.tamu.edu/davis/suitesparse.html.
OpenMPI 1.8.8 and later

This is required for a parallel build of Xyce. Many package management systems provide Open MPI, and it is available from source at http://www.open-mpi.org. Note that some package management systems provide later versions of Open MPI.

We have observed infrequent, random hangs of Xyce on exit with Open MPI 1.10 on some operating systems. When this happens, Xyce has generally exited properly, but mpirun hangs.  If you observe that mpirun is hanging when Xyce exits, we recommend you fall back to Open MPI 1.8.8 rather than the one provided by the package management system. (This will likely require building Open MPI from source.) MPICH (http://www.mpich.org) is also an alternative, but the Xyce team does not build or test against this toolset.

We have observed some segmentation faults inside the Zoltan library on some operating systems with pre-built OpenMPI 2.1.x and OpenMPI 3.x packages.  See the note below under "Install prerequisite libraries."

Trilinos 12.4.2-12.12.1 Xyce requires Trilinos for many linear algebra and solver services. When building the serial version of Xyce one must build Trilinos without MPI support enabled. When building Trilinos for the parallel version of Xyce, then Trilinos must be built with MPI support enabled. Download from: http://trilinos.org. NOTE: Trilinos versions after 12.12.1 have not been tested with Xyce 6.10 and may not work properly.

Important Note Regarding Trilinos and C++11

Trilinos contains many packages that require a C++11-compatible compiler for full functionality. If your system has a C++11 compatible compiler, these features will be compiled into Trilinos by default. Some versions of Trilinos can be built with an older, C++11-incompatible compiler by addition of an option at the time CMake is invoked.

Trilinos 12.4.x must be compiled with a C++11 compatible compiler (with gcc, the version must be 4.7 or above; but 5.1 or above is recommended).

Trilinos versions 12.6.x-12.12.1 can be compiled with a C++11 incompatible compiler if C++11 support is explicitly disabled in the Trilinos cmake script (using "-DTrilinos_ENABLE_CXX11=OFF"). If a C++11 compatible compiler is not available on your system, you must use a version later than 12.6.x and no more recent than 12.12.1.

If you are building a version of Trilinos with C++11 enabled, you must also build Xyce with C++11 enabled. This is generally done by adding "-std=c++11" to the CXXFLAGS variable when configuring Xyce. If Trilinos is built with C++11 enabled and "-std=c++11" is not added to the CXXFLAGS when configuring Xyce, the result will be numerous compilation errors. Xyce does not at this time use any C++11 features or any of the features of Trilinos that require C++11, and for this reason we generally recommend that Trilinos be built without C++11 support as specified above. Our example CMake invocations for Trilinos will therefore all contain the option to disable C++11 support, even if the compiler has this capability.

Important Note Regarding ParMETIS and UMFPACK

Previous releases of Xyce required UMFPACK for all builds and ParMETIS for parallel builds. As of release 6.8, these libraries are now optional, and are searched for by Xyce's configure script ONLY if it is detected that Trilinos has been built with these libraries enabled. We are no longer recommending that Trilinos be built with these libraries enabled, and all of our recommended configurations below have had reference to these libraries removed.

Building Prerequisite Libraries from Source Code

For the best instructions on building the libraries listed in the table above, see the documentation included with those libraries. On many systems, these libraries may be available as pre-compiled packages, and are installable with a package management system such as apt-get, yum, port or brew. Where a package is available for your system, it is usually preferable to use it; the package maintainer will have taken care of special porting issues associated with your system, and should also provide timely updates as those libraries have new releases.

While prebuilt packages can save one time, other considerations may require that one build from source code, instead. If you choose to do so, keep the following information in mind.

It is useful to keep the libraries needed for Xyce in a consistent location. This makes it easier to tell Xyce's configure script where to find all the external libraries. Additionally, since some libraries require MPI to be enabled, it makes sense to keep separate library directories for MPI and non-MPI enabled code. Thus, we suggest you make two directories such as XyceLibs/Serial and XyceLibs/Parallel. Within these directories you should make lib and include directories. For the sake of consistency we will use $HOME/XyceLibs as the prototypical name for this directory in all our examples below.

Installing Prerequisite Libraries

Except for Trilinos, and sometimes ParMETIS, the libraries required for building Xyce are available in popular system package repositories, and are thus easily installed. The Xyce team recommends using these prepackaged libraries as much as possible.

Here we enumerate the packages known to exist on systems the Xyce team has built on. Other systems probably have similarly named packages. The systems for which we provide specific guidance are:

Where a system does not provide a package for a given library, you will have to build that library from source. Guidelines for building required libraries from source are given in the final section below.

Note that at this time no system provides packages for versions of Trilinos that are compatible with Xyce, so it will always need to be built from source. While some systems do provide packages for Trilinos in package management systems, these versions are either too old to be used or are not configured to have all the essential package options required by Xyce, and we do not recommend using them even if you happen to find one for your system.

Note concerning OpenMPI:

It has come to our attention that some Linux distributions have broken versions of OpenMPI in their package repositories.  On these systems, OpenMPI has been compiled with the "--enable-heterogeneous" option, which causes it to perform certain communications operations incorrectly, making it non-compliant with the MPI standard.   Building Xyce with these broken OpenMPI libraries will result in segmentation faults on some problems (including 6 test cases in the Xyce regression suite).

Ubuntu ---beginning with release 17.10 and continuing through release 18.04 LTS--- is one of these, and there may be others.  On such systems it is essential not to use the packaged OpenMPI to build Xyce.

You can determine whether your system's install of OpenMPI is broken in this way by running a small test program that is present in the src/test/MPITest directory of the Xyce source tree.  This program is run automatically as part of parallel testing of Xyce, but that would require that you build and test Xyce first; waiting until then to find out that your OpenMPI install is broken is not the best approach.

If you plan to build Xyce in parallel and wish to pre-test your OpenMPI package, you can copy the file src/test/MPITest/testBUG967.c out of the Xyce source tree, then compile and run it yourself:

    mpicc -o testBUG967 testBUG967.c
    mpirun -np 2 ./testBUG967

On some systems you might have to provide full path names to the mpicc and mpirun commands.

If this produces any output with the word "BAD", your OpenMPI install is broken and cannot be used.  You must instead build and install OpenMPI from source instead of relying on prepackaged binaries from your operating system's repository.  You are also advised to open a bug report on your operating system's issue tracker if one does not already exist, and ask that the package be rebuilt without "--enable-heterogeneous" in the future.  If a bug already exists in their issue tracker for the problem, add your name to the bug as an interested party rather than opening a duplicate bug report.

 

Linux

There are many different "distros" of Linux, and the Xyce team cannot provide precise guidance for building on all of them. We provide here guidance for building on two popular classes of Linux distributions.

RedHat Enterprise Linux (RHEL) and CentOS

These systems have all the prerequisite libraries available in the RPM repository. The following packages are required:

  • A version of gcc, g++ and gfortran (see the Trilinos note)
  • blas-devel
  • blas
  • cmake (required to build Trilinos)
  • lapack-devel
  • lapack
  • bison
  • flex
  • fftw-devel
  • fftw
  • suitesparse-devel
  • suitesparse

Note that on versions of these operating systems older than RHEL 6/CentOS 6, the fftw and suitesparse libraries might require use of an additional repository. On RHEL 6 and CentOS 6 they require no special additions.

Install these packages using sudo yum install on each package name. In most cases, the -devel package will also cause the other package without "-devel" to be installed, so it is not strictly necessary to list all of the above.

If you are building the parallel version of Xyce you will also need:

  • openmpi
  • openmpi-devel

Debian Linux and Variants

All required packages are available in these systems' default repositories.

  • gcc (see the Trilinos note)
  • g++
  • gfortran
  • make
  • cmake
  • bison
  • flex
  • libfl-dev
  • libfftw3-dev
  • libsuitesparse-dev
  • libblas-dev
  • liblapack-dev
  • libtool

If you are building the parallel version of Xyce you also need:

  • libopenmpi-dev
  • openmpi-bin

Install these packages using sudo apt-get install, or a graphical package manager. The "-dev" packages will also pull additional packages in as dependencies.

NOTE:

As of release 17.10 of Ubuntu and continuing with release 18.04, the packaged version of OpenMPI in the package repositories has been compiled with the "--enable-heterogeneous" option, which breaks MPI's standard compliance and causes Xyce to fail on some problems.  This package cannot be used as is until it is rebuilt and repackaged without that option.  There is an open bug report for the issue in Ubuntu's issue tracker:  https://bugs.launchpad.net/ubuntu/+source/openmpi/+bug/1731938.   This same problem may be present in other operating systems that derive from Debian. 

If you are running a version of Ubuntu that has this issue, the only workaround is to uninstall the OpenMPI package and build OpenMPI from source yourself, without the "--enable-heterogeneous" option.

It is possible that this issue could get resolved in the Ubuntu package after the time of this writing.  You should check the status of the linked bug report before assuming that this issue is still a problem.  The note in the section above on "Installing prerequisite libraries" also provides a quick way of testing whether an installed version of OpenMPI will work with Xyce.

Mac OS X

Building Xyce on Mac OS X will require that you obtain the Xcode package from Apple, and you must also install the XCode Command Line Tools. In addition to XCode, you also need a package manager such as MacPorts (http://www.macports.org), Fink (http://fink.thetis.ig42.org/), or Homebrew (http://brew.sh). Install one of these package managers according to the instructions provided on its web site.

It is not necessary to install blas and lapack, as XCode has those available in a system library. Bison and flex are provided by the XCode command line tools. However, in some OS X versions, the flex and Bison supplied by XCode do not work with Xyce; therefore, we recommend using the versions installed by one of the package managers. Before proceding, be sure to read the Trilinos note.

Fink

For Fink, install the following packages:

  • a version of gcc (such as gcc6)
  • fftw
  • suitesparse
  • cmake (for Trilinos)
  • flex
  • bison

For parallel installations, also install

  • openmpi

Install packages using sudo fink install on each package.

MacPorts

For MacPorts, install:

  • a version of gcc (such as gcc6)
  • fftw-3
  • SuiteSparse
  • cmake (for Trilinos)
  • flex
  • bison

Install packages using sudo port install on each package.

For parallel installations, also install

  • the appropriate version of Open MPI (such as openmpi-gcc6)

Homebrew

For Homebrew install the following formulae:

  • fftw
  • a version of gcc (such as gcc, or gcc@6)
  • homebrew/science/suite-sparse
  • cmake (for Trilinos)
  • flex
  • bison

For parallel installations, also install

  • openmpi

Install packages using brew install on each package.

FreeBSD

All libraries required by Xyce are available in the FreeBSD ports system. Before proceding, be sure to read the Trilinos note.

For the serial version of Xyce, install the following ports:

  • lang/gcc48 or later  (you need this even if you are building with the system "clang" compiler suite, because you still need gfortran from ports)
  • math/blas
  • math/lapack
  • math/suitesparse
  • math/fftw3
  • devel/bison
  • devel/cmake (needed for Trilinos)
  • textproc/flex

Install each of these using the standard FreeBSD ports system. For example, to install GCC 6, run the following commands as root:

cd /usr/ports/lang/gcc6
make install clean

The ports system will fetch the source code for gcc6 and compile it on your system, the install the binary.

You may prefer to use binary packages rather than the ports system, in which case you may accomplish the same install as root with:

pkg install gcc6

If you are building the parallel version of Xyce, you also require:

  • net/openmpi

You may also use net/openmpi2 or net/openmpi3 instead.

Note:

The FreeBSD ports of openmpi2 and openmpi3 were built with the inappropriate "--enable-heterogeneous" option until 25 September 2018, at which time the package was fixed.  If your ports directory is updated after that date, you will have no problem.  If your ports directory has not been updated, you must first update it according to the FreeBSD handbook instructions, after which you can install a version of OpenMPI 2 or 3 that works with Xyce.  If you are using "pkg" to install binary packages instead, you must check that the package you are installing has a version no earlier than openmpi2-2.1.5_1 or openmpi3-3.1.2_1  (the "_1" on these packages indicates the update that fixed the build error in the ports system; openmpi2-2.1.5 and openmpi3-3.1.2 were broken).

The net/openmpi package (OpenMPI 1.10.x) has never had the "--enable-heterogeneous" option in its build instructions, and has always been usable with Xyce.

Incompatibility note:

On 4 Oct 2018, the maintainer of the "math/arpack-ng" package made MPICH an unconditional dependency, which means that as of that date, any port that requires arpack-ng will now force MPICH to be installed --- this includes math/octave.  Unlike the OpenMPI ports, which install into directories that don't intefere with each other, MPICH installs its libraries directly into /usr/local/lib, which interferes with all the OpenMPI ports at link time (the MPICH libraries are found instead of the OpenMPI libraries in /usr/local/mpi/openmpi<version>/lib).  This makes it very difficult to build Xyce with OpenMPI while MPICH is installed.   It is possible to build Xyce and Trilinos with MPICH, but we have seen some evidence that MPICH is substantially slower on some systems than OpenMPI on the same machine.  We have not yet found a workaround to make Xyce build properly with OpenMPI in the presence of an MPICH port install, and therefore recommend that you remove MPICH if it is installed (which will, of course, force all other packages that depend on MPICH to be uninstalled).

Possible issues on FreeBSD 11:

Until November 2018, FreeBSD was only being built routinely on FreeBSD 10.4, which reached end of life on 31 Oct 2018.  After upgrading to FreeBSD 11.2, we have found on one AMD-based system running FreeBSD 11.2 that Xyce built with OpenMPI fails with a segmentation fault deep inside one of the system "jemalloc" functions on 13 of the test cases in the test suite when run on two processors --- this appears to be a threading issue in the system jemalloc, and not a bug in Xyce.  The problem disappears if only 1 or 3 processors are used instead, or if the program is invoked through valgrind (trying to detect memory errors).  On the other hand, on a different FreeBSD 11.2 system with Intel processors but otherwise identically installed, no such problem showed up.  It is too early to say anything definitive about this issue.

Cygwin

For Windows systems, the Xyce project team uses the Cygwin (http://www.cygwin.com/) system to provide a Unix-like environment with Unix-style tools. While it is possible to build Xyce using native Windows tools, the team does not support that approach in this release.

Install Cygwin according to its instructions. You will then need to install a series of packages in addition to the base Cygwin packages. Before proceding, be sure to read the Trilinos note.

Add:

  • gcc-core
  • gcc-g++
  • gcc-fortran
  • make
  • cmake (needed for configuring Trilinos)
  • lapack, liblapack0, liblapack-devel
  • libsuitesparseconfig-devel
  • libamd-devel
  • bison
  • flex
  • fftw and libfftw3-devel
  • libtool
  • m4

Many of these packages will automatically install additional packages that they require.

The Xyce team has never attempted to build the parallel version of Xyce on Windows. While Cygwin does provide a version of openmpi, it is our understanding that ParMETIS does not currently build easily on Cygwin. Now that Xyce no longer requires ParMETIS, it may be possible to get it working in parallel on Cygwin. If you succeed in doing so, the Xyce team would be interested to hear about your success.

Installing Trilinos

Introduction

Trilinos is an extensive set of numerical packages for a wide range of computational problems. This section will provide an overview of building Trilinos for Xyce. However, for detailed questions on Trilinos or its build system please see the Trilinos getting started page at https://trilinos.org/getting-started/ for places to start your inquiry.

This section assumes you have installed all prerequisite libraries from packages according to the instructions in the previous section, or have built missing libraries from source.

Important Note Regarding Trilinos 12.4.x: Trilinos 12.4.x must be compiled with a C++11 compatible compiler (with gcc, the version must be 4.7 or above; but 5.1 or above is recommended).  Trilinos versions 12.6.x through 12.12.1 can be compiled with a C++11 incompatible compiler if C++11 support is explicitly disabled in the Trilinos cmake script (using "-DTrilinos_ENABLE_CXX11=OFF").  If a C++11 compatible compiler is not available on your system, you must use Trilinos version 12.6.x.  If you DO build Trilinos wtih C++11 enabled, you MUST build Xyce with C++ enabled, too.  See the system-specific sections on building Xyce for details on the latter.

First, download a version of Trilinos no later than 12.12.1 from http://trilinos.org/, and unpack the source code. Xyce 6.10 is not guaranteed to build properly with versions of Trilinos later than this.

Because of the size and complexity of Trilinos, it is recommended to build the necessary packages outside the source directories. For example, you could create two "build" directories, called trilinosSerial and trilinosParallel for the serial and parallel builds, respectively.

Trilinos has many packages that can be built, only some of which are required by Xyce. The packages and compilation options are specified using command line arguments for CMake. In the next sections we will provide examples of CMake invocations for typical serial and parallel builds of Xyce. It is important to note that any variation from the scripts given here could lead to Xyce compilation failures due to uncaptured library dependencies.

Building Trilinos involves only a few steps:

  • create an empty build directory and "cd" to it.
  • invoke CMake using a build script specifying all the required parameters.
  • make
  • make install

Because multiple versions of Trilinos might need to be built to support both serial and parallel builds of Xyce, we do not recommend installing Trilinos directly into system library and header directories (e.g. /usr/lib or /usr/include). Our instructions for building Trilinos will therefore be set up for installation in a user-specified, non-standard location, $HOME/XyceLibs/Serial and $HOME/XyceLibs/Parallel.

NOTE:  Please note that in the examples below we have explicitly disabled building of the Kokkos package of Trilinos.  At this time we have determined that on some systems the interdependencies introduced when this library is built can cause link failures in Xyce (usually due to issues related to the pthread library).  Xyce does not currently use Kokkos, and so the best solution to these link errors is to exclude that package from your Trilinos build. Versions of Trilinos prior to 11.14 did not automatically build Kokkos; but 11.14 and later versions will, unless it is explicitly disabled. If you have previously built an older version of Trilinos for prior releases of Xyce, don't use the older cmake line: it must  be augmented with the extra line that disables Kokkos, especially if you find that you suddenly get link errors from Xyce related to Kokkos or pthreads. 

Building Trilinos for Serial Xyce

Serial Xyce requires a build of Trilinos that contains the NOX, LOCA, Epetra, EpetraExt, Ifpack, Isorropia, AztecOO, Belos, Teuchos, Sacado and Amesos packages. Amesos must be configured to use KLU. EpetraExt must be configured with BTF (block triangular factorization), experimental features, and graph reordering enabled. Trilinos must also be told where to find the AMD libraries and headers.

After downloading a version of Trilinos no later than 12.12.1 from http://trilinos.org, you will have a "tarball" of Trilinos source code. We will assume for this example that the file ended up in $HOME/Downloads. Decide where you want to unpack it. For example, we will use the directory $HOME/Trilinos12.12. The commands below extract the tarball in that directory.

mkdir $HOME/Trilinos12.12
cd $HOME/Trilinos12.12
tar xzf $HOME/Downloads/trilinos-12.12.1-Source.tar.gz

The CMake invocation below is an example that configures a version of Trilinos with the packages and options required by a serial version of Xyce.  It is very likely that Xyce will not build at all if you do not include at least these options when you build Trilinos.

There are several things to note about this example:

  • This text should be put into a file that will be used as a shell script. This file should be placed in an otherwise empty directory (which we will call the "build directory"). We will use the name "reconfigure" as the name of this script.
  • Set the variable "SRCDIR" to have the full path of the location of the Trilinos source code. If you followed the example above, this would be $HOME/Trilinos12.12/trilinos-12.12.1-Source. This variable is used in the example invocation to specify to CMake how to find the source code.
  • The example script makes extensive use of the shell continuation character "\" that tells the shell to ignore the following newline. The entire CMake invocation is really meant to be a single line, but for clarity we have divided it up using this continuation character. There must be NO characters on the line following the "\", not even a space. There must also be a space before the "\". A common source of problems is having spaces after this continuation character—in that case, the shell ignores the space after the "\" and does NOT ignore the newline at the end of the line.
  • There must be no space before or after the equals sign on these lines.
  • The script specifies which C, C++, and Fortran compilers to use in the lines "-DCMAKE_C_COMPILER", "-DCMAKE_CXX_COMPILER", and "-DCMAKE_Fortran_COMPILER" lines.
    • On most Linux systems, the C, C++ and Fortran compilers are named gcc, g++ and gfortran. Your system may be different and may have version number suffixes on the commands.
    • With Fink on OSX, compiler names have a version suffix (e.g., gcc5, g++5 and gfortran5, though version numbers may differ from these examples on your system)
    • With MacPorts on OSX, the compiler names have additional suffixes (e.g., gcc-mp-5, g++-mp-5, and gfortran-mp-5, though specific version numbers and suffixes may differ from these examples on your system)
    • Consult your system documentation for appropriate settings if your system is not covered in the examples above.
  • Flags for each compiler are specified in the "-DCMAKE_CXX_FLAGS", "-DCMAKE_C_FLAGS", and "-DCMAKE_Fortran_FLAGS" lines. In this example, we are using the same values for all three compilers; these are set in the "FLAGS=" line that defines a shell variable, and then all three compiler flags simply use that variable.
  • Set the variable "ARCHDIR" to the location where you want the Trilinos libraries and headers to be installed. In this example, we use $HOME/XyceLibs/Serial. The install process will make "include" and "lib" subdirectories.
  • We have assumed that AMD is installed in /usr/lib and its associated headers are installed in /usr/include/suitesparse, which is where they are installed by the suitesparse packages on 32-bit Linux systems. If they are installed elsewhere, modify the "-DAMD_LIBRARY_DIRS" and "-D_TPL_AMD_INCLUDE_DIRS" settings.
    • On 64-bit linux, the libraries are in /usr/lib64/ instead.
    • On OS X using Fink, the libraries are in /sw/lib and theh headers are in /sw/include/suitesparse.
    • On OS X using MacPorts, the libraries are in /opt/local/lib and the headers are in /opt/local/include.
    • On OS X using Homebrew, the libraries are in /usr/local/lib and the headers are in /usr/local/include.
    • On FreeBSD the libraries are in /usr/local/lib and the headers are in /usr/local/include/suitesparse.
    • On Cygwin, the libraries are in /usr/lib and headers are in /usr/include/suitesparse just as for Linux.
  • If you set "-DTrilinos_ENABLE_CXX11=ON" on Cygwin, Trilinos will fail to build due with an error about "gettimeofday" not being defined.  One user has reported that a further build failure with errors related to "strdup" will also occur, at least in recent versions of Cygwin.  Both of these errors can be worked around by adding "-D_BSD_SOURCE -D_GNU_SOURCE" to the CXXFLAGS, or by disabling C++11 support ("-DTrilinos_ENABLE_CXX11=OFF").

Once the script is created and customized, change it to an executable script and execute it using the commands:

chmod u+x reconfigure
./reconfigure

If the script completes without error, Trilinos is configured for building.  Build it and install it with the commands:

make
make install

Example "reconfigure" script (cmake invocation):

#!/bin/sh
SRCDIR=$HOME/Trilinos12.12/trilinos-12.12.1-Source
ARCHDIR=$HOME/XyceLibs/Serial
FLAGS="-O3 -fPIC"
cmake \
-G "Unix Makefiles" \
-DCMAKE_C_COMPILER=gcc \
-DCMAKE_CXX_COMPILER=g++ \
-DCMAKE_Fortran_COMPILER=gfortran \
-DCMAKE_CXX_FLAGS="$FLAGS" \
-DCMAKE_C_FLAGS="$FLAGS" \
-DCMAKE_Fortran_FLAGS="$FLAGS" \
-DCMAKE_INSTALL_PREFIX=$ARCHDIR \
-DCMAKE_MAKE_PROGRAM="make" \
-DTrilinos_ENABLE_NOX=ON \
-DNOX_ENABLE_LOCA=ON \
-DTrilinos_ENABLE_EpetraExt=ON \
-DEpetraExt_BUILD_BTF=ON \
-DEpetraExt_BUILD_EXPERIMENTAL=ON \
-DEpetraExt_BUILD_GRAPH_REORDERINGS=ON \
-DTrilinos_ENABLE_TrilinosCouplings=ON \
-DTrilinos_ENABLE_Ifpack=ON \
-DTrilinos_ENABLE_Isorropia=ON \
-DTrilinos_ENABLE_AztecOO=ON \
-DTrilinos_ENABLE_Belos=ON \
-DTrilinos_ENABLE_Teuchos=ON \
-DTeuchos_ENABLE_COMPLEX=ON \
-DTrilinos_ENABLE_Amesos=ON \
-DAmesos_ENABLE_KLU=ON \
-DTrilinos_ENABLE_Sacado=ON \
-DTrilinos_ENABLE_Kokkos=OFF \
-DTrilinos_ENABLE_ALL_OPTIONAL_PACKAGES=OFF \
-DTrilinos_ENABLE_CXX11=OFF \
-DTPL_ENABLE_AMD=ON \
-DAMD_LIBRARY_DIRS="/usr/lib" \
-DTPL_AMD_INCLUDE_DIRS="/usr/include/suitesparse" \
-DTPL_ENABLE_BLAS=ON \
-DTPL_ENABLE_LAPACK=ON \
$SRCDIR

Building Trilinos for Parallel Xyce

Building Trilinos for use in Parallel Xyce requires enabling some additional Trilinos packages (Zoltan and ShyLU). It also requires using the compiler wrappers provided by your system's MPI package (ex. Open MPI) instead of the compilers (gcc, g++, gfortran) directly. These wrappers are typically called "mpicc," "mpic++", and "mpif77" for the C, C++, and Fortran compilers, respectively, but your system may install them with different names or special paths.  "mpicxx" is a common alternate name for the C++ wrapper.  It is also common on systems with case-sensitive file systems to install something called "mpiCC" for the C++ compiler, but we caution against relying on this --- you might find yourself building later on a system with a case-insensitive file system, and "mpiCC" will actually be the same as "mpicc", i.e. the C compiler instead of C++.  This will result in very confusing behavior.

The CMake invocation below is an example invocation that would be appropriate on a Linux system that provides AMD through SuiteSparse. The notes in the serial build instructions also apply to this CMake invocation.

Once the script is created and customized, run it. If it completes without error, Trilinos is configured for building. Build with make and install with make install.

System specific notes:

  • On OS X using MacPorts, the OpenMPI wrappers may have additional suffixes such as mpicc-openmpi-gcc5, mpicxx-openmpi-gcc5, and mpif77-openmpi-gcc5.  Other OS X package managers may give these wrappers different names.

Example "reconfigure" script for Parallel Trilinos:

#!/bin/sh
SRCDIR=$HOME/Trilinos12.12/trilinos-12.12.1-Source
ARCHDIR=$HOME/XyceLibs/Parallel
FLAGS="-O3 -fPIC"
cmake \
-G "Unix Makefiles" \
-DCMAKE_C_COMPILER=mpicc \
-DCMAKE_CXX_COMPILER=mpic++ \
-DCMAKE_Fortran_COMPILER=mpif77 \
-DCMAKE_CXX_FLAGS="$FLAGS" \
-DCMAKE_C_FLAGS="$FLAGS" \
-DCMAKE_Fortran_FLAGS="$FLAGS" \
-DCMAKE_INSTALL_PREFIX=$ARCHDIR \
-DCMAKE_MAKE_PROGRAM="make" \
-DTrilinos_ENABLE_NOX=ON \
-DNOX_ENABLE_LOCA=ON \
-DTrilinos_ENABLE_EpetraExt=ON \
-DEpetraExt_BUILD_BTF=ON \
-DEpetraExt_BUILD_EXPERIMENTAL=ON \
-DEpetraExt_BUILD_GRAPH_REORDERINGS=ON \
-DTrilinos_ENABLE_TrilinosCouplings=ON \
-DTrilinos_ENABLE_Ifpack=ON \
-DTrilinos_ENABLE_ShyLU=ON \
-DTrilinos_ENABLE_Isorropia=ON \
-DTrilinos_ENABLE_AztecOO=ON \
-DTrilinos_ENABLE_Belos=ON \
-DTrilinos_ENABLE_Teuchos=ON \
-DTeuchos_ENABLE_COMPLEX=ON \
-DTrilinos_ENABLE_Amesos=ON \
-DAmesos_ENABLE_KLU=ON \
-DTrilinos_ENABLE_Sacado=ON \
-DTrilinos_ENABLE_Kokkos=OFF \
-DTrilinos_ENABLE_Zoltan=ON \
-DTrilinos_ENABLE_ALL_OPTIONAL_PACKAGES=OFF \
-DTrilinos_ENABLE_CXX11=OFF \
-DTPL_ENABLE_AMD=ON \
-DAMD_LIBRARY_DIRS="/usr/lib" \
-DTPL_AMD_INCLUDE_DIRS="/usr/include/suitesparse" \
-DTPL_ENABLE_BLAS=ON \
-DTPL_ENABLE_LAPACK=ON \
-DTPL_ENABLE_MPI=ON \
$SRCDIR

Building Xyce

Xyce is built using the standard GNU Autoconf/Automake configure system. Most of the work getting Xyce configured involves getting configure to find the required libraries and headers. For Trilinos built according to the directions in this guide, this is primarily accomplished by passing the "ARCHDIR" parameter in to configure. It may also be necessary to tell configure the location of additional libraries and headers through the LDFLAGS and CPPFLAGS variables.  You also may need to specify which compiler to use, or any special libraries it should link in.

The Xyce team recommends compiling Xyce "out-of-source." This is accomplished by creating an empty directory (the "build directory"), making that directory the current working directory, and invoking the Xyce configure script with a complete path name and appropriate options. Once configure is complete, Xyce can be built with "make" and installed with "make install." The complete process looks like this:

mkdir my_build
cd my_build
/path/to/Xyce/configure [configure options]
make
sudo make install

It is only necessary to use "sudo" with "make install" if you do not have write privileges to the directory into which Xyce will be installed.  The default install location for Xyce will be into subdirectories of /usr/local.  This may be changed by adding the "--prefix=" option to configure and providing a prefix to use instead of "/usr/local".  We recommend using a non-default prefix for Xyce installs, so that you can have multiple serial and parallel versions of Xyce installed to different locations.  If you use the default prefix, each install would overwrite files from previous installs.

For simplicity and repeatability, it is often best to put the full configure invocation into a file and execute that file as a shell script rather than typing it directly.

We recommend that the test suite be run in between the "make" and "make install" steps, to assure that your build is working properly. Consult the Running the Xyce Regression Test Suite page for instructions on running these tests.

The remaining sections below detail the configure options most likely to be required. A full list of configure's options may be obtained by typing:

/path/to/Xyce/configure --help

In the first section below, we describe the basic options that can be used to control configure. The section after that provides example configure invocations for selected systems the Xyce team has built on.

NOTE:  Some open-source users have noticed that the Xyce source tree contains a set of "CMakeLists.txt" files and have tried to use CMake instead of autoconf to build Xyce.  These are used by the Xyce team for special builds of Xyce on Windows in a specific environment, and they are not maintained for use on other systems.   The Xyce team does not provide support for CMake; the autoconf build is our only supported process.

Special note for building on small-memory computer systems

Building Xyce can require a great deal of memory and disk space when compiled with full optimization. Four gigabytes of RAM should be sufficient, but more is better. This can be a problem when attempting to build on very small systems, such as Raspberry Pi. The ideal solution for such a system, resulting in fully optimized, fully capable builds of Xyce, would involve cross-compiling Xyce on a larger system with the smaller system as target. This solution can be hard to realize, and the Xyce team can't help you with it.

However, if you need to build Xyce on a system without sufficient memory, there are ways to reduce the memory usage at compile time. Doing so will result in a less optimized build of Xyce, but may be the only path forward for users of very small systems.

If you find your Xyce build process aborting due to insufficient memory, you can try the following changes, roughly ordered by the severity of their impact on the final build:

  • Reduce the optimization level of models derived from Verilog-A sources
    Xyce contains a number of modern device models that have been translated from Verilog-A to C++. These models make heavy use of the Sacado package from Trilinos to perform automatic differentiation. Sacado is a heavily templated library, and optimization of these models using very aggressive optimization levels such as "-O3" can consume a tremendous amount of RAM. To deal with this, the build system provides a means of reducing the optimization level of models contained in the ADMS subdirectory of the DeviceModelPKG directory. This is done by setting the "ADMS_CXXFLAGS" variable at configure time to something that differs from the normal CXXFLAGS used for the rest of the code. For example, if you have CXXFLAGS="-O3" on your configure line, you can force the ADMS directory to use a different optimization level by adding ADMS_CXXFLAGS="-O1" to the configure line. Most of the code will be compiled with the -O3 optimization level, and the ADMS devices will be compiled with only -O1. This change is often enough to get Xyce compiled on systems with around 2GB of RAM.
  • Turn off analytic sensitivity derivative calculation for Verilog-A-derived devices
    Xyce supports sensitivity analysis using analytic derivatives if the models have that capability.  If models do not provide analytic derivatives, Xyce will use finite differences to compute the derivatives needed for sensitivity analysis.  Finite differences will be less accurate than analytic derivatives.  Unfortunately, providing these analytic derivatives in models derived from Verilog-A requires increased dependence on Sacado templates, and drastically increases memory usage at build time. It also significantly increases compile time as the compiler struggles to optimize these complex templates. If simple reduction of compiler optimization level is not sufficient to get Xyce built on your small-memory system, you can try disabling analytic derivatives for these models. This is done by adding --disable-adms_sensitivities to your configure command line. The result will be that all the code that computes analytic sensitivities will be ifdeffed out, and the compilation will generally proceed much faster and with much lower memory use. This will have no impact on you if you are not using sensitivity analysis using these models, but may have impact on accuracy and speed if you are using this analysis.
    This method has been successfully applied to get Xyce to build on a Raspberry Pi 2, which has only 1GB of RAM.
  • Reduce the optimization level for all of Xyce
    If neither of the above approaches reduces the memory usage at compile time enough to get Xyce to build on your system, and the compilation aborts due to memory exhaustion in something other than the ADMS subdirectory, you might have to reduce optimization level for everything. This is done by inserting the appropriate optimization level flag into CXXFLAGS on the configure line. Doing this will definitely impact performance of the code. It will not help at all if the compilation is aborting while compiling the ADMS subdirectory and the previous two approaches didn't help. In that case, your only option is to build on a machine with more memory.

Generic Configure Options

Specifying compilers and helper programs

"configure" usually tries to find the C, C++, and Fortran compilers from a well-known list of standard compiler names. If your system does not have one of the standard-named compilers, or you wish to use some other compiler, you can specify them with the configure variables CC, CXX and F77. It is important to use the same compilers to compile Xyce that you used to compile the Trilinos libraries. Additional variables also allow you to specify particular versions of helper programs, like the Flex lexical analyzer generator (using the LEX variable). For example:

/path/to/Xyce/configure \
CC=strangeC \
CXX=strangeC++ \
F77=strangeF77 \
LEX=/usr/local/bin/flex

Here we have used the shell's continuation mechanism, and placed a backslash at the end of each line other than the last. It is important that no spaces appear after the backslashes.  There must also be a space before the "\".

Specifying Xyce-specific library locations

We recommend installing Trilinos, built especially according to the directions in this guide, into a separate directory.  In the earlier section on building Trilinos, we instructed you to use a variable called "ARCHDIR" in your CMake invocation for Trilinos.  Xyce's configure script also has a variable called "ARCHDIR" that can be used not only to find these libraries and headers, but also some extra files that help determine interdependencies between Trilinos packages and link in all required Trilinos libraries.  Simply use the same install directory you gave to the Trilinos CMake script (ARCHDIR) as the value of Xyce's configure ARCHDIR:

/path/to/Xyce/configure ARCHDIR=$HOME/XyceLibs/Serial [other configure options]

This will result in $HOME/XyceLibs/Serial/lib being added to LDFLAGS and $HOME/XyceLibs/Serial/include being added to CPPFLAGS, as well as ensuring the interdependent Trilinos packages are properly linked.

We make this recommendation because Trilinos must be built differently for parallel builds of Xyce than it must be for serial builds.  Keeping these two builds in completely separate installation directories simplifies the process of having both versions of Xyce built on the same system.

Specifying compiler flags

Each compiler has a configure variable for passing it parameters. These variables are CXXFLAGS for C++ compiler flags, CFLAGS for C compiler flags, and FFLAGS for Fortran compiler flags:

/path/to/Xyce/configure \
CC=gcc6 \
CXX=g++6 \
F77=gfortran6 \
LEX=/usr/local/bin/flex \
CXXFLAGS="-O3" \
CFLAGS="-O3"

Specifying alternate installation directory

By default, configure will set up the Xyce build process so that "make install" installs Xyce components into subdirectories /usr/local: the executable will go in /usr/local/bin, associated libraries into /usr/local/lib, and header files into /usr/local/include.  This behavior can be changed by adding the "--prefix" option to configure:

path/to/Xyce/configure \
--prefix=/opt/Xyce \
...

The example above will cause "make install" to install into /opt/Xyce/bin, /opt/Xyce/lib, and /opt/Xyce/include.

If you do not have root privileges on your system, this procedure can allow you to install Xyce into a directory that you own instead.  Otherwise, use "sudo" with "make install" to install Xyce into system directories.

Specifying additional library search directories

If you have prerequisite libraries that are installed in any directory that the linker will not search by default, you can direct configure to search them by adding "-L" flags to the variable "LDFLAGS":

/path/to/Xyce/configure \
LDFLAGS="-L/some/random/directory/lib -L/home/me/archdir/ZipChip2000/lib"

Specifying additional include search directories

It is sometimes the case that system packages install headers in subdirectories of /usr/include or /usr/local/include. The C preprocessor (and therefore configure) will not search for include files outside standard locations. If header files are installed in any directory that configure will not search by default, you can direct configure to search them by adding "-I" flags to the variable "CPPFLAGS":

/path/to/Xyce/configure \
CPPFLAGS="-I/usr/local/include/obscurelib"

Example configure invocations for selected systems

Here we present example invocations of configure for selected systems.

Because multiple versions of Trilinos might need to be built to support both serial and parallel builds of Xyce, we do not recommend installing Trilinos directly into system library and header directories (/usr/lib, /usr/include, etc.). Therefore, in each example we have assumed that Trilinos (and any other handbuilt libraries) have been installed into a directory called $HOME/XyceLibs/Serial/lib, and their associated headers into $HOME/XyceLibs/Serial/include. In the case of Trilinos, that can be accomplished by setting "ARCHDIR" to $HOME/XyceLibs/Serial in the CMake invocation script.

The configure invocations below are presented separately for serial and parallel builds.

Serial builds:

Parallel builds

Serial Builds

Linux

If all packages have been installed as recommended, configuring on Linux is fairly simple. This is because the package manager installs all of its packages in /usr/lib and /usr/include, and these are searched by configure and the compilers with no extra options required.

The example configure invocation below should be appropriate for any distribution of Linux, though it has only been tested specifically on Red Hat, Ubuntu, and Linux Mint variants.  Minor modifications may be necessary on other distros.

/path/to/Xyce/configure \
CXXFLAGS="-O3" \
ARCHDIR="$HOME/XyceLibs/Serial" \
CPPFLAGS="-I/usr/include/suitesparse"

Here, we have not had to specify anything more than the extra directories to search for SuiteSparse headers, and the location into which Trilinos was installed.

NOTE: If you are using Trilinos 12.4.x, or are building Trilinos 12.6.3  through 12.12.1 with C++11 enabled, you will need to add "-std=c++11" to CXXFLAGS.

Mac OS X

If you have installed XCode, Fink, and all the Fink packages we have recommended, and installed Trilinos in /home/me/XyceLibs/Serial (per our example), building Xyce on Mac OS X in serial can be accomplished with the following configure invocation:

/path/to/Xyce/configure \
ARCHDIR="$HOME/XyceLibs/Serial" \
CXXFLAGS="-O3" \
LDFLAGS="-Wl,-framework,Accelerate -L/sw/lib" \
CPPFLAGS="-I/sw/include/suitesparse -I/sw/include" \
CXX=g++6 \
CC=gcc6 \
F77=gfortran6

For a MacPorts installation, use the following:

/path/to/Xyce/configure \
ARCHDIR="$HOME/XyceLibs/Serial" \
CXXFLAGS="-O3" \
LDFLAGS="-Wl,-framework,Accelerate -L/opt/local/lib" \
CPPFLAGS="-I/opt/local/include" \
CXX=g++-mp-6 \
CC=gcc-mp-6 \
F77=gfortran-mp-6

Finally, if you are using homebrew as your package manager, you can configure Xyce with:

/path/to/Xyce/configure \
ARCHDIR=$HOME/XyceLibs/Serial \
CXXFLAGS="-O3" \
LDFLAGS="-Wl,-framework,Accelerate -L/usr/local/lib -L/usr/local/opt/flex/lib" \
CPPFLAGS="-I/usr/local/include -I/usr/local/opt/flex/include" \
LEX=/usr/local/opt/flex/bin/flex \
CC=/usr/local/bin/gcc \
CXX=/usr/local/bin/g++ \
F77=/usr/local/bin/gfortran

The -Wl,-framework,Accelerate option in LDFLAGS tells configure to link in the optimized libraries provided by XCode. These libraries include BLAS and LAPACK, so when configure starts to look for those routines it will find them without having to search for libraries like libblas or liblapack.

While we have used gcc in the above examples, the system clang compiler could also be used to compile Xyce (assuming it was used to compile Trilinos as well). If clang is used to build Xyce then gfortran must still be installed and used, since clang does not have a native Fortran compiler. (If clang is used, replace Wl,-framework,Accelerate with -framework Accelerate.)

NOTE: If you are using Trilinos 12.4.x, or are building Trilinos 12.6.3  through 12.12.1 with C++11 enabled, you will need to add "-std=c++11" to CXXFLAGS.

NOTE: While numerous Xyce developers have built the code with exactly the configure options shown above, it has been observed that in some combinations of package managers and XCode versions the build process has problems finding the Fortran versions of BLAS and LAPACK. When this happens, there will be a large number of link errors at the end of the build, reporting "undefined symbols for architecture" referenced by functions in Teuchos (a Trilinos package). Should this occur, one can add the --disable-fortran_test option to the configure invocation to make it use C versions of BLAS and LAPACK library functions instead. Re-run configure with this option added and recompile.

 

FreeBSD

If all libraries and packages have been installed as described above, serial Xyce can be configured on FreeBSD with the following configure invocation.

/path/to/Xyce/configure \
CXXFLAGS="-O3" \
LEX=/usr/local/bin/flex \
ARCHDIR="$HOME/XyceLibs/Serial" \
LDFLAGS="-L/usr/local/lib -Wl,-rpath=/usr/local/lib/gcc6" \
CPPFLAGS="-I/usr/local/include" \
CXX=clang++ \
CC=clang \
F77=gfortran6

 

NOTE: If you are using Trilinos 12.4.x, or are building Trilinos 12.6.3  through 12.12.1 with C++11 enabled, you will need to add "-std=c++11" to CXXFLAGS.

IF YOU HAVE BUILT TRILINOS WITH C++11 SUPPORT: If you built Trilinos with C++11 support enabled (despite our suggestion not to), then on FreeBSD you not only have to add "-std=c++11" to CXXFLAGS, but also "-stdlib=libc++". 

In this example, we have used the system Clang compiler.  Because the system contains no Fortran compiler, it is necessary to specify an "rpath" in the CXXFLAGS so that the resulting binary will always reference the correct Fortran standard library installed by the unversioned "gcc" port (which at the time of this writing installed GCC 6.4, and provides the "gfortran6" command).

You could also have used gcc6 and g++6 as the C and C++ compilers here (assuming you had also built Trilinos with this compiler).  In this case, the rpath in CXXFLAGS is still needed due to the incompatibility of the GCC 6 standard C++ library with the system C++ library.

Cygwin

If all libraries and packages have been installed as described above, the following configure invocation will configure Xyce to be built under Cygwin:

/path/to/Xyce/configure \
ARCHDIR="$HOME/XyceLibs/Serial" \
CXXFLAGS="-O2" \
CPPFLAGS="-D_BSD_SOURCE" \
CXX=g++ \
CC=gcc

NOTE: If you are using Trilinos 12.4.x, or are building Trilinos 12.6.3  through 12.12.1 with C++11 enabled, you will need to add "-std=c++11" to CXXFLAGS.

NOTE: The "-D_BSD_SOURCE"  in CPPFLAGS might not always be required, but the Xyce team has observed that it is necessary when building with C++11, otherwise you will see errors about "M_PI" not being defined.

While it is tempting to let configure guess the C++ and C compilers as with Linux, doing so causes problems in the link stage, when for some reason libtool is invoked using the C compiler as linker instead of C++ (it appears that the reason for this is that configure finds "CC" as a valid C++ compiler, but when this name is passed to libtool it is interpreted to mean "C mode"). This results in numerous inexplicable "undefined references" to standard C++ library functions. Explicitly specifying g++ and gcc as the compilers avoids this problem.

NOTE: if you are creating a "reconfigure" script file containing the configure invocation above, it is essential that you do so using a Unix-style editor under Cygwin (vim, nano, emacs, etc.) and not a Windows text editor such as Notepad. Notepad will create a file using DOS style line breaks (""CR/LF"), and this will confuse the bash shell when running the script, and the error messages it emits will not indicate the true source of the problem.

Parallel Builds

Most of the complexity of building Xyce in parallel on any platform is in getting the third-party libraries built. If you have all the required packages for Trilinos built, and have properly built Trilinos using MPI compiler wrapper scripts, then configuring Xyce is usually just a matter of providing the same wrapper scripts for compilers and adjusting the configure line to point at the parallel versions of libraries.

In addition to naming the MPI compiler wrappers and identifying library directories, one must add --enable-mpi to the configure options.

Linux

The example configure invocation below should be appropriate for any distribution of Linux, though it has only been tested specifically on Red Hat, Ubuntu, and Linux Mint variants.  Minor modifications may be necessary on other distros.


/path/to/Xyce/configure \ CXXFLAGS="-O3" \
ARCHDIR="$HOME/XyceLibs/Parallel" \ CPPFLAGS="-I/usr/include/suitesparse" \ --enable-mpi \ CXX=mpicxx \ CC=mpicc \ F77=mpif77

NOTE: If you are using Trilinos 12.4.x, or are building Trilinos 12.6.3  through 12.12.1 with C++11 enabled, you will need to add "-std=c++11" to CXXFLAGS.

Mac OS X

Fink

/path/to/Xyce/configure \
CXXFLAGS="-O3" \
ARCHDIR="$HOME/XyceLibs/Parallel" \ LDFLAGS="-Wl,-framework,Accelerate -L/sw/lib" \ CPPFLAGS="-I/sw/include/suitesparse -I/sw/include" \ --enable-mpi \ CXX=mpicxx \ CC=mpicc \ F77=mpif77

NOTE: If you are using Trilinos 12.4.x, or are building Trilinos 12.6.3  through 12.12.1 with C++11 enabled, you will need to add "-std=c++11" to CXXFLAGS.

MacPorts

/path/to/Xyce/configure \
CXXFLAGS="-O3" \
ARCHDIR="$HOME/XyceLibs/Parallel" \
LDFLAGS="-Wl,-framework,Accelerate -L/opt/local/lib" \
CPPFLAGS="-I/opt/local/include" \
--enable-mpi \
CXX=mpicxx-openmpi-gcc6 \
CC=mpicc-openmpi-gcc6 \
F77=mpif77-openmpi-gcc6

NOTE: If you are using Trilinos 12.4.x, or are building Trilinos 12.6.3  through 12.12.1 with C++11 enabled, you will need to add "-std=c++11" to CXXFLAGS.

Homebrew

Note that, with Homebrew, Open MPI wrapps the native clang compiler.
$HOME/Downloads/Xyce/configure \
CXXFLAGS="-O3" \
ARCHDIR="$HOME/XyceLibs/Parallel" \
LDFLAGS="-Wl,-framework,Accelerate -L/usr/local/lib -L/usr/local/opt/flex/lib" \
CPPFLAGS="-I/usr/local/include -I/usr/local/opt/flex/include" \
LEX=/usr/local/opt/flex/bin/flex \
--enable-mpi \
CXX=/usr/local/bin/mpicxx \
CC=/usr/local/bin/mpicc \
F77=/usr/local/bin/mpif77

NOTE: If you are using Trilinos 12.4.x, or are building Trilinos 12.6.3  through 12.12.1 with C++11 enabled, you will need to add "-std=c++11" to CXXFLAGS.

FreeBSD

/path/to/Xyce/configure \
CXXFLAGS="-O3 -Wl,-rpath=/usr/local/lib/gcc6" \
LEX=/usr/local/bin/flex \
ARCHDIR="$HOME/XyceLibs/Parallel" \
LDFLAGS="-L/usr/local/lib" \
CPPFLAGS="-I/usr/local/include" \
--enable-mpi \
CXX=/usr/local/mpi/openmpi/bin/mpicxx \
CC=/usr/local/mpi/openmpi/bin/mpicc \
F77=/usr/local/mpi/openmpi/bin/mpif77

NOTE: If you are using Trilinos 12.4.x, or are building Trilinos 12.6.3  through 12.12.1 with C++11 enabled, you will need to add "-std=c++11" to CXXFLAGS.

Note the required "rpath" is still there for FreeBSD, because the MPI wrappers will have the same gfortran shared library dependence that the serial build does.  It is assumed here that you have installed gfortran from the GCC 6 compiler collection prior to building OpenMPI, and that your OpenMPI is built with the default compiler your version of FreeBSD will choose for them (which will be generally be clang on FreeBSD 9 and later).

Usage Tracking Code in Xyce

If you browse the Xyce source code, you will see that beginning with version 6.2 Xyce has code to enable usage tracking.  Some users may be concerned about this, so this section documents the code, how to enable it, how to disable it, and what it does.

Usage tracking was implemented in Xyce so that we could track our internal users' use of the code to better serve them, and to report to management on how heavily our software is being used in-house.

Usage tracking is disabled by default

The usage tracking code in Xyce is not compiled in unless specifically requested, and when a URL for usage data collection provided.  The open-source binary that we provide on the external web site does not have usage tracking enabled.

Only binaries intended strictly for internal use have usage tracking turned on.  In these binaries, "Xyce -capabilities" will report "Metrics reporting via cURL".

What usage data is collected?

No open source binaries have usage tracking turned on, and usage tracking is not built by default.  No usage data is collected from these builds.

If your binary is one that was intended for purely internal use at Sandia, "Xyce -capabilities" will report "Metrics reporting via cURL," and each run of Xyce will result in a small packet of data about your run sent to an internal web server.  This web server is inside Sandia's firewall and cannot receive data from outside Sandia.

The data packet contains

  • Your user ID.
  • The name of the machine on which you are running,
  • The hardware type (x86, amd64, etc.),
  • Your OS (Windows, Linux, etc.) as reported by a system call,
  • The version number of Xyce you are running,
  • The total count of devices in your circuit,
  • A breakdown of the number of devices of each type in your circuit,
  • Counts of operations as reported in the console output summary (number of linear solves, number of time steps, number of residual and jacobian loads, etc.),
  • The start and end times of the run.

Your user ID is used to determine your Org number and Sandia UID by the web server that collects the data, and this information is added to the packet.

Turning off usage reporting

If you are running a binary that includes usage reporting and you wish to opt out of this reporting, set the environment variable "XYCE_NO_TRACKING" to any value.  If this environment variable is set, Xyce will skip sending the usage data.  Remember that no open source builds have this feature enabled, and that you can check if the feature is enabled with "Xyce -capabilities".

On Unix-like systems

export XYCE_NO_TRACKING="anything at all"

On Windows

set XYCE_NO_TRACKING="anything"

If you are running Xyce at Sandia, we ask that you only opt out of usage reporting if absolutely necessary.  We will use this information to document how widely Xyce is used inside Sandia to management. 

Turning on usage tracking

The usage tracking URL at Sandia is inaccessible outside Sandia, and therefore we cannot collect information from open source users.

If you are building Xyce for institutional use and wish to collect usage data yourself, you can turn on usage tracking easily.  You must provide your own web server and collection script.  If you are a Sandia user building Xyce from source, please contact the Xyce team and ask for the URL to use to enable usage tracking.  Counting uses of Xyce inside Sandia is important to the Xyce team.

  • To turn on usage tracking, you must have the cURL library installed on your system.  Most Linux systems have this package available in their package repositories.  You will require both the library package and the "development" package to build Xyce with this feature.  On Windows, you must build cURL from source.  See the cURL web site for details if building cURL from source.
  • add "--enable-curl" to your configure options when building Xyce
  • set the TRACKING_URL variable to the URL of your tracking script on the configure command line:
    • /path/to/configure [...options...] --enable-curl  TRACKING_URL="http://mysite.org/directory/cgi-bin/trackingscript.pl" [...more options...]

The data sent to the tracking URL is a single line of JSON-encoded data. 

A trivial script that simply accepts the data and stores it in a file can be used, or you can write a more elaborate one yourself:

#!/usr/bin/env perl
use CGI;

my $query=CGI->new;
my %params=$query->Vars;
my $json_text=$params{POSTDATA};
open(STATSFILE,">>/path/to/writable/statsfile");
print STATSFILE "$json_text\n";
close(STATSFILE);

print $query->header(),$query->start_html("Gotcha");
print $query->h1("got it");
# This line is useful for debugging, but if the run has a lot of data to report
# might create an unreasonably large response.  Comment out for production.
#print $query->p($json_text);
print $query->end_html();