Running FastSurfer in a container

This page explains the commands that run FastSurfer in our Apptainer (Singularity) or Docker images: the flags of the container, and how FastSurfer’s own flags work inside it. To get an image, see the installation page of your system (Linux, Windows, Intel Macs); the Examples show complete commands for common tasks.

Containerization tools like Apptainer (Singularity) or Docker provide several advantages. Most importantly, they allow for exactly the same setup across different machines and even data centers and compute clusters. They thus increase reproducibility by reducing software differences between evaluations. Additionally, errors and unexpected behavior are easier to track down, since developers can reproduce the setup much more easily. Finally, containers provide a security advantage, because they can only access data that is explicitly shared with them, which reduces the risk of both data theft and data encryption attacks. This strategy is also called sandboxing.

A container command has two parts: the options of the container (<singularity_flags> or <docker_flags>), which define access to data and hardware such as graphics cards, and the options of FastSurfer (<fastsurfer_flags>), which the container passes on to run_fastsurfer.sh. It is useful to think of the image as a “hard drive” and the container as a “simulated computer inside the computer”.

Using Apptainer (or Singularity)

Apptainer is the open-source continuation of Singularity, and both run the same images. Apptainer also installs the singularity command, so the commands below use singularity and work with either. Linux explains how to build the image (a .sif file).

To run FastSurfer on a subject with the Apptainer image and GPU access, execute:

Note

This documents the development version 2.6.0. Official Docker images only exist for releases, so the commands below use the latest release, 2.5.4, for its default CUDA version, 12.8. Images of 2.5.4 for other CUDA versions, ROCm and CPU are available on Docker Hub. To run the development version, build your own image.

freesurfer_license=${freesurfer_license:-/path/to/your/freesurfer/license_file}
singularity exec --nv \
                 --no-mount home,cwd -e \
                 -B $HOME/my_mri_data \
                 -B $HOME/my_fastsurfer_analysis \
                 -B $freesurfer_license \
                 $HOME/my_singularity_images/fastsurfer-cu128-v2.5.4.sif \
                 /fastsurfer/run_fastsurfer.sh \
                 --fs_license $freesurfer_license \
                 --t1 $HOME/my_mri_data/subjectX/t1_weighted.nii.gz \
                 --sid subjectX --sd $HOME/my_fastsurfer_analysis \
                 --3T --threads 4

Without a GPU, use the CPU image and leave out --nv.

Apptainer flags

  • --nv: gives the container access to NVIDIA GPUs. Leave it out to run FastSurfer on the CPU.

  • --no-mount home,cwd: tells Apptainer not to mount the home directory or the current working directory inside the container (see below).

  • -e: does not pass the environment variables of the host into the container.

  • -B <host_dir>: shares a directory or file of the host with the container. Only paths listed here are available to FastSurfer, so these mount your data, the output directory and the FreeSurfer license file. Inside the container, they are visible under the same paths as on the host. With -B <host_dir>:<container_dir>, <host_dir>/<file_name> is visible inside the container as <container_dir>/<file_name> instead.

Mounting Home and Current Working Directory

Do not mount the user home directory into the Apptainer container as the home directory.

Why? If the user inside the Apptainer container has access to a user directory, settings from that directory might bleed into the FastSurfer pipeline.

How? Apptainer mounts the home directory by default. To avoid this, specify --no-mount home,cwd. Additionally, the -e flag ensures that no environment variables are passed from the host system into the container.

Without the home directory, Apptainer may warn:

WARNING: Error changing the container working directory. Using '/' instead:
  chdir /home/***: no such file or directory

You can ignore this warning, since / as the working directory does not cause any issues, or specify a different working directory with --cwd <directory>, for example --cwd /fastsurfer.

Using Docker

Linux and Windows explain how to download the image. To run FastSurfer on a subject with the Docker image and GPU access, execute:

Note

This documents the development version 2.6.0. Official Docker images only exist for releases, so the commands below use the latest release, 2.5.4, for its default CUDA version, 12.8. Images of 2.5.4 for other CUDA versions, ROCm and CPU are available on Docker Hub. To run the development version, build your own image.

freesurfer_license=${freesurfer_license:-/path/to/your/freesurfer/license_file}
docker run --gpus all -v $HOME/my_mri_data:$HOME/my_mri_data \
           -v $HOME/my_fastsurfer_analysis:$HOME/my_fastsurfer_analysis \
           -v $freesurfer_license:$freesurfer_license \
           --rm --user $(id -u):$(id -g) deepmi/fastsurfer:cu128-v2.5.4 \
           --fs_license $freesurfer_license \
           --t1 $HOME/my_mri_data/subjectX/t1_weighted.nii.gz \
           --sid subjectX --sd $HOME/my_fastsurfer_analysis \
           --3T --threads 4

Without a GPU, use the CPU image and leave out --gpus all. AMD GPUs need additional flags.

Docker flags

  • --gpus: gives the container access to GPUs, and selects which ones. In the example above, all makes every GPU available to FastSurfer in the container. To use a single one (e.g. GPU 0), set --gpus device=0. To use several specific GPUs (e.g. GPU 0, 1 and 3), use --gpus "device=0,1,3". Leave it out to run FastSurfer on the CPU.

  • -v: defines which data is shared between the host system and the container, and how. By default, nothing is shared, so -v explicitly shares a directory or file. It follows the format -v <host_dir>:<container_dir>:<options>. In its simplest form, <host_dir> and <container_dir> are the same, so paths inside the container are the same as on the host. :<options> may be left out, or :ro makes the files read-only for the container. Share the input files, the output folder (subjects directory) and the FreeSurfer license.

  • --user $(id -u):$(id -g): the user and group the container runs as, which decides the access to files (required). $(id -u) and $(id -g) give your user and group IDs, so all generated files belong to you. Without this flag, FastSurfer exits with a message asking you to map your host user. Running the container as root (--user 0:0) is strongly discouraged and must be combined with the FastSurfer flag --allow_root.

  • --rm: removes the container once the analysis has finished (optional, but recommended).

  • -d: runs the container in detached mode, so you return to the shell without screen output (optional).

  • --group-add <group_list>: if additional user groups are required to access files, add them with --group-add <group_id>[,...] or --group-add $(id -G <group_name>).

Do not mount the user home directory into the Docker container as the home directory: settings from that directory might bleed into the FastSurfer pipeline. Docker does not mount the home directory by default, so unless you manually set the HOME environment variable, all should be fine.

FastSurfer flags in a container

In principle, these are the same as for run_fastsurfer.sh. The paths after --fs_license, --t1 and --sd are paths inside the container, so they have to be shared with -B (Apptainer) or -v (Docker); in the examples above, they are the same as on your system.

  • --fs_license: the path to your FreeSurfer license. In a container, it cannot be detected automatically and must be passed. The segmentation alone (--seg_only) does not need a license, see FreeSurfer license.

  • --t1: the path to the T1-weighted MRI image to analyze.

  • --sid: the subject ID (the name of the output folder).

  • --sd: the path to the output directory (the subjects directory).

  • --3T: uses the 3T atlas instead of the 1.5T atlas for the Talairach registration, for better Talairach transforms and ICV estimates (eTIV).

  • --threads: the number of threads for the segmentation and the surface reconstruction. With more than one thread, FastSurfer processes the left and right hemispheres in parallel. max uses all threads available, e.g. 16 on an 8-core system with hyperthreading.

A directory with the name specified in --sid (here subjectX) will be created in the output directory, so in this example, the output will be written to $HOME/my_fastsurfer_analysis/subjectX/. Make sure this directory does not exist yet, to avoid overwriting existing files.

All other flags are explained in the run_fastsurfer.sh documentation.

Common problems

  1. Slow processing despite a GPU.

    FastSurfer runs on the CPU if it cannot use the GPU, and the log says why: for example, the container was started without --nv or --gpus all, or the NVIDIA driver is too old for the CUDA version of the image. Then update the driver or use the image for an older CUDA version, see which image fits your GPU.

  2. FastSurfer stops with a message about the user (Docker).

    The container runs as the image’s default user, so pass --user $(id -u):$(id -g), see Docker flags.