This file documents how to build the Engine both using Docker and from the command line. It assumes a Linux operating system and the bash shell.

We strongly recommend building with Docker for deployment, since this provides a repeatable and consistent environment, and includes all prerequisites (other than Docker itself).

For development, it is often convenient to build on the command line, since this provides a faster edit/compile/debug loop.

The Dockerfiles that support the Engine can be found under docker/mrp (the mrp namespace stands for "Meeting Recorder Project and is used in several places in the code to distinguish parts of the "core" or "upstream" Kaldi project from Engine-related extensions added by Mod9). The primary files we will cover here build Docker images for the build environment, runtime environment, models, and the Engine itself.

Some of the Dockerfiles depend on other base images already existing. By default, these dependent images would be expected to reside on Google Cloud Repository (GCR). Also in the same directory are files used to build images on Google Cloud Build (GCB) when code changes are pushed to a repository. This document does not cover GCB/GCR continuous integration.

Instead, this document will describe the process for building local images in the appropriate order so that dependent images are built before the images that depend on them.

All the Dockerfiles described in this document start with Dockerfile.engine. They can be followed with a suffix describing their function (e.g. Dockerfile.engine-builder). Although the images produced from these Dockerfiles can be named arbitrarily, our convention when building locally is to name the image e.g. local/engine-builder.

Note that the first comment line in each Dockerfile refers to images residing on GCR. For example, Dockerfile.engine-builder is related to gcr.io/remeeting/kaldi:engine-builder. Although you can use a similar convention if you wish, the steps below assume you are building locally and perhaps pushing to private remote Docker repository.

IMPORTANT: due to the proprietary nature of this software, it is not permissible to push images to a public Docker repository (e.g. docker.io or Docker Hub), unless you are the owner of the mod9/asr public repository hosting carefully licensed Engine images for evaluation purposes.

This image contains various tools and external libraries that are needed to build later images. This includes the gcc compiler suite, GNU build tools, memory allocation libraries, math libraries, etc. It also builds various tools that are needed by Kaldi (e.g. OpenFST) and extra tools that Mod9 incorporated into the Engine (e.g. Boost and TensorFlow). Note that most of these tools are only used during the build process, and are not copied into the final Engine image.

This builder image is based on CentOS 7. Note that the versions of Linux (CentOS 7) and the compiler (gcc-11) were selected for maximum compatibility with older Linux distributions. CentOS 7 uses the oldest version of standard C/C++ libraries that is compatible with a version of gcc that supports C++17. If you have full control over the OS in which the Engine will ultimately run, you may consider upgrading the build environment to use more modern standard libraries -- but these Engine builds will not be backwards-compatible to run on older Linux distributions.

To build this image, navigate to the top level of a clean copy of the Kaldi fork containing the Engine source. There should be files COPYING, INSTALL, README.md, src, tools etc. in this directory. It's important that the copy has not been used for building on the command line (that is, that you have not tried to build or install the copy using other methods). A sanity check that the directory is clean is to run du -hs . while in the directory. It should return about 30M. If this Docker build context is larger, then it may include artifacts from prior command-line builds that may at best slow down the build, or at worst corrupt it. (To clean up, try cd src && make -j distclean; or better yet, delete and reinstall a fresh copy of the source code.)

Next, build the image with the command:

docker build -t local/engine-builder \
             -f docker/mrp/Dockerfile.engine-builder .

By default, this will use all available CPUs. If you have limited memory, you may need to reduce the number of CPUs used during the build. You can override the number of CPUs with a build argument. For example, to limit the number of simultaneous jobs to 10:

docker build --build-arg MAKE_JOBS=10 \
             -t local/engine-builder \
             -f docker/mrp/Dockerfile.engine-builder .

This image contains Mod9's ASR models. It copies the models from a URL, which by default points to a publicly readable Amazon S3 bucket. To build this image:

docker build -t local/engine-models \
             -f docker/mrp/Dockerfile.engine-models .

You can also build other model images (e.g. vosk, zamia) using similar commands, or construct your own models image by following the instructions in Dockerfile.engine-models.

It is worth noting that the final Docker image for the Engine is very intentionally structured so that these models are in the innermost layers. Because models are infrequently updated, this provides a significant advantage in terms of layer caching because updated versions of the Engine software result in a docker pull command reusing the model layers and only pulling the outermost layers with the updated software.

The runtime includes the OS environment and tools required to run the services included in the Engine. This includes Ubuntu 22.04, an Apache server (for the documentation, REST interfaces, etc.), a Python environment, and so on. You must also specify which builder image and models image will be used to build the runtime image.

docker build --build-arg BUILDER_IMAGE=local/engine-builder \
             --build-arg MODELS_IMAGE=local/engine-models \
             -t local/engine-runtime \
             -f docker/mrp/Dockerfile.engine-runtime .

The final Engine image performs the C++ compilation and library linking, and also uses the previously built images to produce an image that includes everything you need to run the Engine and associated tools (e.g. REST wrapper, documentation, etc.). To build the Engine image:

docker build --build-arg BUILDER_IMAGE=local/engine-builder \
             --build-arg RUNTIME_IMAGE=local/engine-runtime \
             -t local/engine \
             -f docker/mrp/Dockerfile.engine .

As with Dockerfile.engine-builder, this command will by default use all available CPUs, which could be problematic when building on a system with limited memory. You can restrict the parallelism of this step with --build-arg MAKE_JOBS=10, for example.

Now that you have a final image, you can run the Engine as easily as:

docker run -it --rm -p 9900:9900 local/engine

Note that this locally build image named local/engine should be essentially the same as the publicly hosted remote Docker repository image named mod9/asr. Note also that the image's default command (but not its entrypoint) is the engine binary. To request non-default command-line arguments, be sure to specify the engine command explicitly, e.g.:

docker run -it --rm local/engine engine --version

See the deployment documentation (e.g. https://mod9.io/deployment) for more information.

The easiest way to extract the Engine executable is to create a temporary Engine container and then use docker cp to copy it out. For example:

docker create --name=tmp-engine local/engine

Then, in another window:

docker cp tmp-engine:/opt/mod9-asr/bin/engine .

It may be helpful to also copy out the models from tmp-engine:/opt/mod9-asr/models.

Lastly, remember to clean up:

docker rm tmp-engine

Although we strongly recommend using a Docker image for building the Engine for deployment, it's often better to use the command line for development. This section describes how to set up your environment and build the Engine. It assumes you're on Linux using the bash shell and have a clean directory tree containing the forked Kaldi source code. The top level directory should contain files COPYING, README.md, etc., and directories tools, src, etc.

The ASR Engine uses the same directory structure and build system as Kaldi. If you're familiar with the Kaldi build process, the only difference is a few extra dependencies installed under the tools directory and building the mrpboostbin directory containing the Engine itself.

To install the extra dependencies, navigate to the tools directory and run the command extras/check_dependencies.sh. It will list the tools you need to install if they're not already installed. To build the Engine itself, navigate to the src directory, configure the build appropriately, and then run make -C mrpboostbin engine.

For those unfamiliar with Kaldi, the following sections provide step-by-step instructions.

There are a number of dependencies that you must install to build Kaldi and the Engine code. These include build tools (such as a compiler) and various libraries (such as OpenFST). Many dependencies are installed using scripts provided with the source code, but some (such as a compiler) are installed using your Linux distribution's installation system.

To check it the required build tools are installed, run the following from the top level directory:

cd tools
extras/check_dependencies.sh

This will report any missing system dependencies, perhaps reporting a line such as:

extras/check_dependencies.sh: Some prerequisites are missing; install
them using the command:

Copy and paste the line directly after this to install missing system tools (such as a compiler).

Next, there are a number of scripts under tools/extras to install other packages required either by Kaldi or by the Engine. The script extras/check_dependencies.sh should list them. You should copy/paste the commands for each of these dependencies. (If installing Tensorflow interactively, you should probably accept the suggested defaults.)

The dependencies checked by the above steps are not exhaustive; it is assumed you have a fairly fully featured Linux distribution installed. If you get command not found errors, it's usually easy to correct by searching for how to install the reported tool using your distribution's installation tools. It may also be helpful to refer to docker/mrp/Dockerfile.engine-builder for an example that sets up the build environment in a Centos/Fedora/RHEL Linux distribution (i.e. using the yum package manager). Mod9 generally uses an Ubuntu LTS release for local development.

Kaldi is a WFST-based ASR toolkit that critically depends on the OpenFST library. Mod9's fork of Kaldi upgrades this dependency to OpenFST 1.8.1, which requires C++17; the upstream Kaldi project uses an older version of OpenFST, along with C++14.

To install the required OpenFST library, navigate to the tools directory, and run:

make -j openfst

This will use all available cores to build openfst. If you want to run on fewer CPUs (e.g. if you don't have a lot of memory), you can run e.g.:

make -j 10 openfst

This will run on 10 cores rather than all available cores.

Once all the prerequisites are installed and built, the next step is to configure the source tree for Kaldi and the Engine. This uses the GNU autoconf package with a custom configure script.

To configure the source tree, navigate to the top-level src directory, and run:

cd src
./configure --static --use-cuda=no --enable-jemalloc=yes

This uses default values for most arguments, and our recommended settings to:

• generate statically linked executables;
• disable CUDA support;
• enable a recommended memory allocator.

See the following subsections for a description of our recommended settings and other arguments you may want to specify. Run ./configure --help for a full list of all supported commands.

The Engine does not support CUDA. However, since the Engine uses the Kaldi infrastructure, and some Kaldi tools (notably training) use CUDA, the default is to include CUDA support. This will fail if you do not have the proper CUDA development tools already installed. We strongly recommend configuring the build environment without CUDA support. If you want to enable CUDA support, change the argument to --use-cuda=yes.

The build system can either produce self-contained executables (known as a static build), or the executables can point to shared libraries (known as a dynamic build). For deployment, we recommend a static build, since version skew between libraries and executables can cause difficulty in producing replicable builds. For development, it can be faster to use dynamic builds, though this introduces complexity and is only recommended if you're familiar with Linux shared libraries.

The ./configure script accepts several arguments related to static vs. dynamic builds, but we recommend using --static as shown above, or, if you want dynamic builds, using --shared in place of --static.

The default memory allocator on Linux is known to be inefficient. The Engine supports several replacement allocation libraries. The following flags to ./configure enable various memory allocation libraries. Call ./configure --help for more information on them.

--enable-jemalloc=yes
--enable-mimalloc=yes
--use-tcmalloc-mod9=yes

You should pass only one of the above. If you pass none of them, the standard malloc for your compiler will be used. Although each of the libraries has strengths and weaknesses, we recommend jemalloc as a good compromise on speed, memory overhead, and features. The Dockerfile build described above uses jemalloc by default.

By default, the Engine executable includes a custom license. That is, the license reported by the Engine assumes a custom license agreement has been negotiated and agreed to between the user of the Engine and Mod9. The other option is to compile with an evaluation license, which allows for 45 days of use (though this is not checked or enforced). You can change from the default custom license to the evaluation license by adding --license-type=evaluation to the arguments of ./configure.

The Engine can be configured to expire at midnight Dec 31 the year after the year the Engine was built. That is, if the Engine was built on Apr 26 2022, then it expires on Dec 31 2023. You can configure the system to include an expiration date by adding --mod9-expiration=yes to the arguments of ./configure.

The options to ./configure described in the previous sections include the most important options you can pass to ./configure. For a full list of options, call ./configure --help. The description of options unique to the Engine (rather than to Kaldi itself) are prefixed with MOD9:.

Note that the ./configure script also reads certain environment variables. When developing locally, it may be helpful to set CXXFLAGS=-Og for example, to override the default of -O3 and enable faster builds at a slight decrease in runtime performance. Alternatively, it may be necessary to set CXXFLAGS='-O3 -fno-omit-frame-pointer' to enable profiling with perf.

Once ./configure has run successfully, run the following commands to build the source:

make -j -C mrpboostbin engine

This will use all available CPUs. If you want to limit the number of CPUs (e.g. if you don't have a lot of memory), you can specify the number of concurrent jobs after -j, e.g.:

make -j 10 -C mrpboostbin engine

Note that this will only build the engine executable in mrpboostbin, which is generally what you want. However, you can also build all of the executables in mrpboostbin, including the mrp-crypt tool, by calling e.g. make -j 10 mrpboostbin. The executables in the rest of Kaldi are built with the usual make all (this excludes the Engine, under the make ext target).

If you change any of the source code, calling make -j 10 mrpboostbin will rebuild the changed files, the files that depend on the changed files, and the Engine. If you add new source code (and change the Makefiles appropriately), you will need to call e.g. make -j 10 depend so the build system can track which source files depend on which.

You can run make -j clean to remove all intermediate files, and make -j distclean will generally clean up the entire source tree.

Once the build finishes successfully, the executable will be in src/mrpboostbin/engine. Assuming models are installed in the default location /opt/mod9-asr/models, you can start the Engine with:

cd mrpboostbin
./engine

See the deployment documentation (e.g. https://mod9.io/deployment) for more details.

Note that running engine --version will report metadata describing the build configuration.