Installation

Autosubmit is free software, licensed under GPL-3.0-or-later.

Prerequisites

Python 3.10 – 3.12 — enforced by the installed package, and pip ≥ 24.0 (check with pip -V).

These must be available on the host machine:

• bash

• git >=2.32 — check with git --version

• graphviz >=2.38 — 2.40 is known not to work; check with dot -v

• rsync

• sqlite3

• subversion

Optional, for plot:

• xdg-utils — provides xdg-open, used by autosubmit create –plot

Note

The dot -v command should list dot, pdf, png, SVG and Xlib in the device section.

Important

The host machine must be able to reach HPCs and clusters via password-less SSH. Generate a PEM-format key with ssh-keygen -t rsa -b 4096 -C "email@email.com" -m PEM.

Install Autosubmit

Autosubmit is distributed via PyPI and as source on GitHub. Choose one of the methods below. The commands use apt and assume Ubuntu 22.04 LTS or 24.04 LTS, whose default python3 falls within the supported range; adjust for other distributions.

Install with pip

Install system packages and the Autosubmit Python package:

# Update repositories
apt update

# Avoid interactive stuff
export DEBIAN_FRONTEND=noninteractive

# System dependencies
# This example is for Debian-based distributions. On Suse, RHEL, Alpine,
# Rocky and others, install the equivalent packages for your package manager.
apt install --yes --quiet \
    git \
    graphviz \
    python3 \
    python3-pip \
    rsync \
    sqlite3 \
    subversion

# Install Autosubmit from PyPI
pip3 install autosubmit

# Verify the CLI works
autosubmit -h

Then proceed to Quick setup or Full setup.

Build from source

Clone the source from GitHub and install with pip from a working copy.

git clone https://github.com/BSC-ES/autosubmit.git
cd autosubmit
pip install .

Then proceed to Quick setup or Full setup.

Verify the install

$ autosubmit -v

Output:
4.2.0

Quick setup

For a personal test with a user-level database. This creates $HOME/.autosubmitrc and puts everything under $HOME/autosubmit/.

# Quick-configure (user-level database)
autosubmit configure

# Create Autosubmit directories and database
autosubmit install

# Generate a new <EXPID>
autosubmit expid -H "local" -d "Test exp in local."

# Create the experiment structure
# Since it was a new install, the <EXPID> will be a000
autosubmit create a000

# Run the experiment
autosubmit run a000

Hint

The dialog (GUI) library is optional. Without it, autosubmit configure prompts on the CLI. Use autosubmit configure -h to see all options.

You can pass --advanced to autosubmit configure to choose different paths:

• Experiments path ($HOME/autosubmit/ by default) and database filename (autosubmit.db by default).

• Global logs — those not belonging to any experiment (default $HOME/autosubmit/logs).

• Autosubmit metadata (default $HOME/autosubmit/metadata/).

You can also configure an SMTP server and email address for notifications.

To run on a remote platform, first set up password-less SSH:

# Generate a key pair for password-less ssh. PEM format is recommended as others can cause problems
ssh-keygen -t rsa -b 4096 -C "email@email.com" -m PEM

# Copy the public key to the remote machine
ssh-copy-id -i ~/.ssh/id_rsa.pub user@remotehost

# Add your key to the ssh-agent (if encrypted)
# If not initialized, initialize it
eval `ssh-agent -s`
# Add the key
ssh-add ~/.ssh/id_rsa
# Where ~/.ssh/id_rsa is the path to your private key

Example: user-level .autosubmitrc

[database]
path = /home/<user>/autosubmit
filename = autosubmit.db

[local]
path = /home/<user>/autosubmit

[globallogs]
path = /home/<user>/autosubmit/logs

[structures]
path = /home/<user>/autosubmit/metadata/structures

[historicdb]
path = /home/<user>/autosubmit/metadata/data

[historiclog]
path = /home/<user>/autosubmit/metadata/logs

Full setup

Use this setup for production. Configuration goes in /etc/autosubmitrc, which points every user on the machine at the same experiment database, so they can see and work with each other’s experiments.

Priority order of configuration files:

AUTOSUBMIT_CONFIGURATION > $HOME/.autosubmitrc > /etc/autosubmitrc

Set the AUTOSUBMIT_CONFIGURATION environment variable to the path of an autosubmitrc file to override everything else.

Warning

If you already have $HOME/.autosubmitrc from Quick setup, delete or rename it before doing the full setup, or it will shadow /etc/autosubmitrc.

autosubmit configure

Create /etc/autosubmitrc with the sections below, or move an existing $HOME/.autosubmitrc there. Any of the locations listed above will do — /etc/autosubmitrc is the usual choice for a shared installation because every user on the machine picks it up.

Mandatory parameters

[database]
# Accessible for all users of the filesystem
path = <database_path>
# Experiment database name can be whatever.
filename = autosubmit.db

# Accessible for all users of the filesystem, can be the same as database_path
[local]
path = <experiment_path>

# Global logs, logs without <EXPID> associated.
[globallogs]
path = /home/<user>/autosubmit/logs

# This depends on your email server and can be left empty if not applicable
[mail]
smtp_server = mail.bsc.es
mail_from = automail@bsc.es

Recommended parameters

The following parameters are the Autosubmit metadata. They are not mandatory, but it is recommended to have them set up, as some of them can positively affect the Autosubmit performance.

[structures]
path = /home/<user>/autosubmit/metadata/structures

[historicdb]
path = /home/<user>/autosubmit/metadata/data

[historiclog]
path = /home/<user>/autosubmit/metadata/logs

Optional parameters

These parameters provide extra functionalities to Autosubmit.

[conf]
# Allows using a different jobs_<EXPID>.yml default template on `autosubmit expid`
jobs = <path_jobs>/jobs_<EXPID>.yml
# Allows using a different platforms_<EXPID>.yml default template on `autosubmit expid`
platforms = <path_platforms>/platforms_<EXPID>.yml

# Autosubmit API includes extra information for some Autosubmit functions. It is optional to have access to it to use Autosubmit.
[autosubmitapi]
# Autosubmit API (The API is right now only provided inside the BSC network), which enables extra features for the Autosubmit GUI
url = <url of the Autosubmit API>:<port>

# Used for controlling the traffic that comes from Autosubmit.
[hosts]
authorized = [<command1,commandN> <machine1,machineN>]
forbidden = [<command1,commandN> <machine1,machineN>]

About hosts parameters:

From 3.14+ onwards, the users can tailor Autosubmit commands to run on specific machines. Previously, only the run was affected by the deprecated whitelist parameter.

• authorized = [<command1,commandN> <machine1,machineN>] list of machines that can run given autosubmit commands. If the list is empty, all machines are allowed.

• forbidden = [<command1,commandN> <machine1,machineN>] list of machines that cannot run given autosubmit commands. If the list is empty, no machine is forbidden.

Example: BSC /etc/autosubmitrc

[database]
path = /esarchive/autosubmit
filename = ecearth.db

[local]
path = /esarchive/autosubmit

[conf]
jobs = /esarchive/autosubmit/default
platforms = /esarchive/autosubmit/default

[mail]
smtp_server = mail.bsc.es
mail_from = automail@bsc.es

[hosts]
authorized = [run bscearth000,bscesautosubmit01,bscesautosubmit02] [stats, clean, describe, check, report, dbfix, upgrade, updateversion all]
forbidden = [expid, create, recovery, delete, inspect, monitor, recovery, configure, setstatus, testcase, test, refresh, archive, unarchive bscearth000, bscesautosubmit01, bscesautosubmit02]

autosubmit install

Finally, create the directories defined in /etc/autosubmitrc and initialise a blank database:

autosubmit install

Dependencies and licenses

Python dependencies and licenses

Generated at build time from pyproject.toml.

Package	Version Spec	License
xlib	==0.21	GNU Lesser General Public License v2 or later (LGPLv2+)
bscearth.utils	<=0.5.2	MIT
requests	<=2.34.2	Apache Software License
networkx	<=3.6.1	BSD-3-Clause
portalocker	<=4.4.0	BSD-3-Clause
paramiko	>=3.5.0	LGPL-2.1
pyparsing	==3.3.3	MIT
matplotlib	<=3.10.9	Python Software Foundation License
psutil	<=7.2.2	BSD-3-Clause
py3dotplus	==1.1.0	MIT License
numpy	<3	BSD-3-Clause AND 0BSD AND MIT AND Zlib AND CC0-1.0
rocrate	==0.16.0	Apache-2.0
setproctitle	-	BSD-3-Clause
invoke	>=2.0	BSD-2-Clause
SQLAlchemy	~=2.0.23	MIT
ruamel.yaml	==0.19.1	MIT License
configobj	-	BSD License
configparser	-	MIT License