Introduction

What is Autosubmit ?

Autosubmit is a python-based workflow manager to create, manage and monitor experiments by using Computing Clusters, HPC’s and Supercomputers remotely via ssh. It has support for experiments running in more than one HPC and for different workflow configurations.
Autosubmit is currently used at Barcelona Supercomputing Centre (BSC) to run models (EC-Earth, MONARCH, NEMO, CALIOPE, HERMES...), operational toolchains (S2S4E), data-download workflows (ECMWF MARS), and many other.
Autosubmit has been used to manage models running at supercomputers in BSC, ECMWF, IC3, CESGA, EPCC, PDC and OLCF.
Autosubmit is now available via PyPi package under the terms of GNU General Public License.

Get involved or contact us:
 
Autosubmit GitLab: https://earth.bsc.es/gitlab/es/autosubmit
Autosubmit Mailing List: autosubmit@bsc.es

Why is Autosubmit needed ?

Autosubmit is the only existing tool that satisfies the following requirements from the weather and climate community:

  • Automatisation Job submission to machines and dependencies between jobs are managed by Autosubmit. No user intervention is needed.
  • Data provenance Assigns unique identifiers for each experiment and stores information about model version, experiment configuration and computing facilities used in the whole process.
  • Failure tolerance Automatic retrials and ability to rerun chunks in case of corrupted or missing data.
  • Resource management Autosubmit manages supercomputer particularities, allowing users to run their experiments in the available machine without having to adapt the code. Autosubmit also allows to submit tasks from the same experiment to different platforms.

How does Autosubmit work ?

You can find help about how to use autosubmit and a list of available commands, just executing:

autosubmit -h

Execute autosubmit <command> -h for detailed help for each command:

autosubmit expid -h

Experiment creation

To create a new experiment, run the command:

autosubmit expid -H HPCname -d Description

HPCname is the name of the main HPC platform for the experiment: it will be the default platform for the tasks. Description is a brief experiment description.

This command assigns a unique four character identifier (xxxx, names starting from a letter, the other three characters) to the experiment and creates a new folder in experiments repository with structure shown in Figure 1.

experiment folder

1 Example of an experiment directory tree.

Experiment configuration

To configure the experiment, edit expdef_xxxx.conf, jobs_xxxx.conf and platforms_xxxx.conf in the conf folder of the experiment (see contents in Figure 2).

configuration files

2 Configuration files content

After that, you are expected to run the command:

autosubmit create xxxx

This command creates the experiment project in the proj folder. The experiment project contains the scripts specified in jobs_xxxx.conf and a copy of model source code and data specified in expdef_xxxx.conf.

Experiment run

To run the experiment, just execute the command:

autosubmit run xxxx

Autosubmit will start submitting jobs to the relevant platforms (both HPC and supporting computers) by using the scripts specified in jobs_xxxx.conf. Autosubmit will substitute variables present on scripts where handlers appear in %variable_name% format. Autosubmit provides variables for current chunk, start date, member, computer configuration and more, and also will replace variables form proj_xxxx.conf.

To monitor the status of the experiment, the command:

autosubmit monitor xxxx

is available. This will plot the workflow of the experiment and the current status.

experiment plot

3 Example of monitoring plot for EC-Earth run with Autosubmit for 1 start date, 1 member and 3 chunks.