Script templates#

Autosubmit jobs require a FILE or SCRIPT property that points to or defines a script template. Script templates can be written in Bash shell, R, or Python. By default, the TYPE property of a job is set to bash. Template scripts can have any file extension, the generated script will have it replaced by .cmd.

5 Job JOB_1 with template script print_expid.sh#
JOBS:
  JOB_1:
    FILE: print_expid.sh
    PLATFORM: LOCAL
    RUNNING: once
    TYPE: bash # default

In the example above, the script template print_expid.sh file must exist in the Autosubmit Project. When you run autosubmit create or autosubmit refresh, Autosubmit will copy the Project files, including template scripts, to the experiment folder proj.

The first time the create command is run, it performs the equivalent of autosubmit refresh, updating the Autosubmit project and it will plot and try to display a new graph. The template scripts are then preprocessed and the final script is generated when an Autosubmit experiment is inspected or created.

When Autosubmit preprocesses the template script, it replaces placeholders with configuration values. A placeholder is any configuration key enclosed in %% (in-place) or in %^% (postloaded). For example, %DEFAULT.EXPID% refers to the Autosubmit configuration value specified in the DEFAULT.EXPID key of the YAML configuration file.

Warning

If you want a placeholder to remain unsubstituted, list its name (without surrounding %) under the CONFIG.SAFE_PLACEHOLDERS key in your YAML configuration. Placeholders listed there will be kept as-is (e.g. %CURRENT_PROJECT% remains %CURRENT_PROJECT%).

Example:

CONFIG:
  SAFE_PLACEHOLDERS:
    - CURRENT_PROJECT
    - ANOTHER_VAR

Assuming that one of the Autosubmit experiment configuration files contains the following:

6 Autosubmit configuration#
DEFAULT:
  EXPID: a000
  # ... other settings

and a template script used in that experiment contains the following code:

7 A template script#
#!/bin/bash

echo "The experiment ID is %DEFAULT.EXPID%"

then after the template script is preprocessed by Autosubmit, the generated script will be written in the experiment folder with the extension .cmd and it will look like this:

8 Generated script#
#!/bin/bash

# Autosubmit Header
# ...

echo "The experiment ID is a000"

# Autosubmit Tailer

Header and Tailer#

Autosubmit does not require users to modify the header and tailer used. The header contains code to set the correct locale, create supporting files for Autosubmit (e.g. <EXPID>_TEST_STAT), and functions used internally by Autosubmit (e.g. for checkpointing).

The tailer creates other supporting files (e.g. <EXPID>_TEST_COMPLETED and update <EXPID>_TEST_STAT).

For certain platforms, like Slurm for instance, Autosubmit will populate the header with options for the job scheduler. These options come from platform and job configuration values (e.g. TASKS, MEMORY, etc.) and are automatically translated by Autosubmit into the format expected by the job scheduler.

If necessary, Autosubmit allows users to customize the header and the tailer. The configuration keys EXTENDED_HEADER_PATH and EXTENDED_TAILER_PATH can be used to indicate the location of, respectively, the header and tailer scripts used by Autosubmit. The location is relative to the project root folder, and the scripts are appended after the default header and tailer scripts.

Substitute placeholders after all files have been loaded#

Autosubmit allows users to define placeholders that are substituted only after all files have been loaded. This is particularly useful when the value of a placeholder is mutable at the time of file loading.

For instance, consider the following YAML files loaded in alphabetical order (ca.yml, conf.yml, cz.yml):

ca.yml:

model:
  version: "first"

conf.yml:

other_variable: "something"
test_in_place: "%other_variable%/%^model.version%/%another_other_variable%"
test_at_the_end: "%other_variable%/%model.version%/%another_other_variable%"
another_other_variable: "something"

cz.yml:

model:
  version: "last"

If a placeholder is defined as %model.version% in conf.yml, the behavior differs based on the enclosing format:

  1. In-place `%%` Enclosing:

The placeholder’s value will correspond to the key defined in the previously loaded file or the file currently being loaded. For example, %model.version% would resolve to “first” (from ca.yml) when conf.yml is loaded.

  1. Postloaded `%^%` Enclosing:

The placeholder’s value will always correspond to the key defined in the last loaded file. For example, %^model.version% in conf.yml would resolve to “last” (from cz.yml) after all files are loaded.

In this case, the value of test_in_place in conf.yml would resolve as: “something/first/something”

And, the value of test_at_the_end would resolve as: “something/last/something”

In-line Scripting#

It is also possible to define a script directly in the yaml configuration file.

Note

If SCRIPT is defined, it will take precedence over FILE.

If you wish to run only one liner script:

9 Job JOB_IN_LINE_ONELINER that runs sleep 2#
JOBS:
  JOB_1:
    PLATFORM: LOCAL
    RUNNING: once
    TYPE: bash # default
    SCRIPT: sleep 2

In case that your script needs lines, the yaml specification allows to do so with the PIPE character (|).

10 Job JOB_IN_LINE_MULTILINE that runs hello world and sleep#
JOBS:
  JOB_1:
    PLATFORM: LOCAL
    RUNNING: once
    TYPE: bash # default
    SCRIPT: |
      echo "hello world!"
      sleep 2

Also, for debugging purposes, if PROJECT_TYPE is set to NONE (see Developing a project), the SCRIPT directive will overwrite Autosubmit’s self-contained dummy templates.

Ensuring Data Integrity with I/O Sync#

To ensure that all data generated by a job is safely written to disk, Autosubmit automatically forces an I/O sync at the end of each script’s execution. This applies to jobs running Bash, Python, or R templates. This measure prevents data loss in case of unexpected interruptions immediately after a job reports completion.

Script error handling#

Autosubmit scripts contain the header, body, and tailer. The code is executed in this order, and in the case of Bash Shell, a failure somewhere in the body or extended header may result in the rest of the script not running. That includes the tailer.

In the tailer of an Autosubmit-generated script there are important files generated: _STAT files, used by Autosubmit and API for metrics and reporting on status of jobs, and _COMPLETED files which mark when a job has successfully completed.

If the user script fails mid-way, and the tailer never produces a _COMPLETED file, that does not cause any issues as a _COMPLETED file is not expected upon failures. However, the missing _STAT file may result in incorrect values reported in Autosubmit commands and API/GUI.

For that reason, Autosubmit 4.1.16+ adopted Bash shell trap functions. The script watches for EXIT, and for the following signals:

  • SIGHUP

  • SIGINT

  • SIGQUIT

  • SIGTERM

  • SIGXCPU

  • SIGXFSZ

If a command fails in the user-provided template, the EXIT function will make sure the _STAT file is generated, and the script exits with the same return code of the failing function.

If any of the signals above happen and are captured by the script, then the trap-function will set the exit code to failure (128 + the signal value) and will write the _STAT file.

The files are not generated if the process running the script receives a SIGKILL (as that cannot be captured) or if the script receives a signal like SIGTERM, emitted by Slurm upon wall time limit (seconds before it emits a SIGKILL) but the script is executing a long-running command (e.g., sleep 10000, or compiling a model).

In these cases, neither _COMPLETED nor _STAT files are generated.