There are three modes in which MPQC can be used in developing software implementation of existing and new methods:
- native: you develop within the MPQC source tree directly;
- plug-in: you plug your code into MPQC to extend its functionality, e.g. by adding new classes to MPQC;
- plug-out: you plug MPQC libraries into your project, e.g. by using MPQC classes and functions in your code.
These methods are demonstrated and discussed below.
- Developing code within MPQC
- Extending MPQC functionality
- Using MPQC as an external library in a separate project
Developing code within MPQC
This approach is useful when you want to extend and replace existing classes in MPQC, e.g. you want to add a new Wavefunction class that implements a new or existing electronic structure method.
The pros of this approach are:
- the user can take advantage of the MPQC infrastructure, e.g. the build system and test harness;
- this approach makes it possible to eventually incorporate the new code into the MPQC master source repository for release and maintenance by the core MPQC team.
The workflow for developing code within MPQC is based on pull requests, albeit it differs for core and non-core developers. Since MPQC is hosted on GitHub, we by default assume everyone will use the GitHub-provided mechanisms for working with pull requests. For more info see GitHub Help.
- Core MPQC developers (e.g. anyone with write access to the private MPQC repo) are encouraged to use the shared repository model, i.e. create a feature branch (and create a pull request immediately), add/modify code, mark the pull request "ready to merge" when done, and respond to feedback (if any) before merging.
- Others can use the pull-and-fork model.
When you start working on a feature, it is recommended to immediately create a pull request. There are several reasons to do this.
- This will advertise your work to the rest of the team and avoid duplicate effort by keeping everyone abreast of what's in the pipeline.
- Others will be able to contribute to your work by pushing to your feature branch.
- This will enable Continuous Integration tests to be done on your feature branch.
You will want to periodically (i.e., often) sync with the changes in the main repo by merging in changes via the pull request mechanism or by direct git merge. When you are ready to merge your changes into the main repo create a pull request for code review and incorporation.
If you are a core MPQC developer or want to contribute your code to MPQC refer to Contrubutor Guide for more recommended practices.
Extending MPQC functionality
It is also possible to extend the functionality of the MPQC executable by adding new classes. The plug-in style approach allows you to keep your source separate from the rest of the MPQC.
To use this approach it is necessary to compile MPQC libraries and install them to the location specifies by the CMAKE_INSTALL_PREFIX
CMake variable. It is recommended that the MPQC is validated before installation.
MP2 Implementation Example
To illustrate how to use the plug-in approach let's consider a simple example in which we extend the MPQC executable with a new implementation of MP2 energy. Although MPQC has several implementations of MP2 already, they all assume canonical orbitals, and whereas our new formulation will not make this assumption by solving the MP1 amplitude equations iteratively.
MP2 Implementation Example: Source
This example code illustrates a complete MP2 energy implementation using the MPQC Toolkit. The source file can be found at doc/dev/examples/mp2/mp2.cpp
in the MPQC source directory. Note that the source file does not contain the main()
function. Instead the MP2 class defined here will be linked into the MPQC executable and will become usable just like any other existing Wavefunction class in MPQC: you will be able to use it to compute gradients by finite differences, etc.
To compile the new class and link into the MPQC executable see the MP2 Implementation Example: CMakeLists.txt section.
MP2 Implementation Example: CMakeLists.txt
Although it is possible to build our example manually, or use Make, it is recommended to use CMake. This is because KitWare's CMake tool is used to configure, build, and install MPQC itself. Installing MPQC will export key information about the MPQC libraries and other prerequisites (e.g., TiledArray and MADWorld runtime) that is necessary to compile the code against them (e.g., the interlibrary dependencies, the compiler and linker flags, etc.). This exported info then makes it incredibly easy to use CMake to build our example in a few lines of CMake code. This file, located at doc/dev/examples/mp2/CMakeLists.txt
in the MPQC build directory, was generated from the doc/dev/examples/mp2/CMakeLists.txt.in
template located in the MPQC source directory.
The CMakeLists.txt file illustrates how to compile the mp2.cpp
source and use it to create the mp2
executable by linking against MPQC libraries. Note that the MPQC library names do not appear explicitly: instead we use their CMake target MPQCmain
.
MP2 Implementation Example: Input
This input JSON file can be used with the program illustrated in the MP2 Implementation Example: Source section.
Using MPQC as an external library in a separate project
To use MPQC from a standalone project it is necessary to compile MPQC libraries and install them to the location specifies by the CMAKE_INSTALL_PREFIX
CMake variable. It is recommended that the MPQC is validated before installation.
To use the MPQC code your program must do the following before using any nontrivial MPQC functionality:
- initialize the MADWorld parallel runtime + miscellaneous TiledArray runtime components (e.g. CUDA streams, memory pools), by calling
TA::initialize
() ; this will initialize the MPI runtime by calling MPI_Init_thread ; - initialize the MPQC package and the dependent components (e.g., the Libint2 Gaussian integral library) by calling mpqc::initialize();
Here's a brief example:
Note that there are more elaborate use cases that we do not discuss here, e.g., it is possible to initialize MADWorld using a communicator other than MPI_COMM_WORLD
; please see the documentation for TA::initialize()
.
After the MPQC functionality is no longer needed, you can release resources by calling mpqc::finalize() . This is not necessary if you are about to exit the program: the MPQC resources are released automatically when the program exits. Note that it is possible to call mpqc::initialize() again after calling mpqc::finalize().
After the MADWorld runtime is no longer needed, it is recommended to call TA::finalize
() . You should not call TA::initialize
() again after calling TA::finalize
() because the MPI Standard does not specify whether the MPI libraries must support more than one initialize/finalize cycle.
Note that the most common use case is to initialize MADworld+TiledArray+MPQC all in one shot; for this purpose there is an ::mpqc::finalize(argc,argv)
overload that should be used as demonstrated here:
AOInts Implementation Example
To illustrate how to use the plug-out approach let's consider a simple example in which we use MPQC functionality to compute 2-body Coulomb integrals with brakets constisting of 2, 3, and 4 atomic orbitals (in the quantum chemistry context these are referred to as 2-, 3-, and 4-center electron repulsion integrals). The 2- and 3-index integrals are used to approximate the 4-center integrals by density fitting, which involves some simple tensor algebra.
AOInts Implementation Example: Source
This example code illustrates how to construct TiledArray tensors of AO integrals using the MPQC Toolkit. The source file can be found at doc/dev/examples/aoints/aoints.cpp
in the MPQC source directory. Note that the source file does contain the main()
function which explicitly includes all steps needed to initialize MPQC.
To compile the new class and link into the MPQC executable see the AOInts Implementation Example: CMakeLists.txt section.
AOInts Implementation Example: CMakeLists.txt
Just like with the MP2 example, it is recommended to use CMake to build this example. See file doc/dev/examples/aoints/CMakeLists.txt
located in the MPQC build directory that was generated from the doc/dev/examples/aoints/CMakeLists.txt.in
template located in the MPQC source directory.
The CMakeLists.txt file illustrates how to compile the aoints.cpp
source and use it to create the aoints
executable by linking against MPQC libraries. Note that the MPQC library names do not appear explicitly: instead we use their CMake target libmpqc
(unlike the CMake target MPQCmain
that we used previously, libmpqc
does not include MPQC's main()
function ).
AOInts Implementation Example: Input
This input JSON file can be used with the program illustrated in the AOInts Implementation Example: Source section.
The contents of this file are used to construct a KeyVal object used to construct MPQC infrastructure classes; since it is possible to construct KeyVal objects programmatically, it is not necessary to use input files, but here we use it for simplicity. It is, for example, possible to change the basis sets in this file without needing to recompile the aoints
program.
Additional notes on development within and with MPQC.
Reducing compile time
Due to the design of C++, compiling C++ programs can be time- and resource-intensive. It is a common misconception that you are working when your code is being compiled. Some improvements are on the way (e.g. Modules in C++20) but for the foreseeable future reducing compilation resource requirements falls on the developer, by designing for compile times and by utilizing special build techniques. Below you will find some tips on how to the compile times of MPQC and MPQC-dependent codes.
Generic recommendations for reducing compile times
- Use RAM disk to compile as well as for IDE (e.g. CLion) to keep its wares. See https://github.com/zafarella/OSX-RAMDisk for a fancy script that will move caches for various apps (including CLion) or just google ‘ramdisk macos’. To tell the compiler to use RAM disk you need to set the environment variable
TMPDIR
(Preferences->Build,Execution,Deployment->CMake then edit Environment) to point to a directory in ramdisk, e.g. setTMPDIR
to/Users/yourname/ramdisk/Clion
- Use Ninja (instead of GNU Make) to speed up builds. It may not speed up the template heavy C++ code in MPQC by much, but its ability to accurately compute prerequisites for targets is key to making the next idea work well. If you use the CLion IDE, note that Ninja (and other non-Make) generator support was introduced in CLion 2019.3!
- Use a faster compiler. Clang is often faster than GCC at compiling code (but this != "produces faster code"). You can even squeeze out top performance from your C++ compiler by recompiling it with profile-guided optimizations.
- Set up a compile farm.
These are just few examples; for a more thorough discussion please see http://www.bitsnbites.eu/faster-c-builds/ .
MPQC-specific recommendations for reducing compile times
- When developing within MPQC it is recommended to make a unit test for the feature you are working on, and then compile that unit test only. Currently MPQC’s unit tests are all integrated into a single executable
unit_tests
, so you want to trim the list of source files included in that executable; edit CMake variableutests_src
intests/unit/CMakeLists.txt
. - If to test your work you cannot unit test and you need to run the
mpqc
executable, unless you must run a feature that integrates almost everything in MPQC (e.g. CCSD-F12) you can still drastically reduce the time it takes to buildmpqc
by trimming down the MPQC feature list. This is particularly important when you work on heavily reused features (i.e. you are modifying files that are included by many other files). To trim it set theMPQC_FEATURES
CMake cache variable to the minimum list of features you need, e.g., to only compile mean-field methods add-DMPQC_FEATURES="lcao_scf"
to the CMake command line. Figuring out the shortestMPQC_FEATURES
that you need may require some experimentation, but it will be worth it; seesrc/bin/mpqc/CMakeLists.txt
for more details. N.B. It seems that even with a trimmed-downMPQC_FEATURES
all libraries are compiled the first time you build MPQC, unless you use Ninja, which can properly deduce the correct list of dependencies for thempqc
executable. So it is another reason to use Ninja. - Ask cmake to enable unity builds for MPQC (and any other components built as part of MPQC) by giving it
-DCMAKE_UNITY_BUILD=ON
. Unity builds are especially recommended for complete* (as opposed to incremental) builds of MPQC (e.g. if you are working on low-level features that touch most of the code so that almost all MPQC libraries are rebuilt every time); if you only touch a particular top-level source file unity builds may actually make incremental recompilation slower. Also, unity builds will stress your compiler so stick with Clang if you want to use them; variableCMAKE_UNITY_BUILD_BATCH_SIZE
can be used to control the granularity of the unity targets to manage the resource requirements. - Choose your build target wisely. Most of the time there is no need to build the executable to test the latest rounds of changes, it's sufficient to build a library that has .cpp files that use your feature. E.g. when changing something in
lcao/scf
I would just buildMPQClcao_scf-obj
target to test it. Note the-obj
suffix: this ensures that transitive dependencies are ignored. This is important when you are tweaking something in the core of MPQC and you just want whether it breaks the simplest wavefunction class (e.g.RHF
). Compiling the object SCF library builds only the SCF-related files, and avoids rebuilding libraries thatMPQClcao_scf
depends on (i.e. all libraries inutil
andmath
, and most libraries inchemistry
).