################ Script templates ################ Autosubmit jobs require a ``FILE`` or ``SCRIPT`` property that points to or defines a script template. Script templates can be written in Bash shell, R, or Python. By default, the ``TYPE`` property of a job is set to ``bash``. Template scripts can have any file extension, the generated script will have it replaced by ``.cmd``. .. code-block:: yaml :emphasize-lines: 3 :caption: Job ``JOB_1`` with template script ``print_expid.sh`` JOBS: JOB_1: FILE: print_expid.sh PLATFORM: LOCAL RUNNING: once TYPE: bash # default In the example above, the script template ``print_expid.sh`` file must exist in the Autosubmit Project. When you run ``autosubmit create`` or ``autosubmit refresh``, Autosubmit will copy the Project files, including template scripts, to the experiment folder `proj`. The first time the ``create`` command is run, it performs the equivalent of ``autosubmit refresh``, updating the Autosubmit project and it will plot and try to display a new graph. The template scripts are then preprocessed and the final script is generated when an Autosubmit experiment is :ref:`inspected ` or :doc:`created `. When Autosubmit preprocesses the template script, it replaces placeholders with configuration values. A placeholder is any configuration key enclosed in %% (in-place) or in %^% (postloaded). For example, ``%DEFAULT.EXPID%`` refers to the Autosubmit configuration value specified in the ``DEFAULT.EXPID`` key of the YAML configuration file. .. warning:: If you want a placeholder to remain unsubstituted, list its name (without surrounding ``%``) under the ``CONFIG.SAFE_PLACEHOLDERS`` key in your YAML configuration. Placeholders listed there will be kept as-is (e.g. ``%CURRENT_PROJECT%`` remains ``%CURRENT_PROJECT%``). Example: .. code-block:: yaml CONFIG: SAFE_PLACEHOLDERS: - CURRENT_PROJECT - ANOTHER_VAR Assuming that one of the Autosubmit experiment configuration files contains the following: .. code-block:: yaml :caption: Autosubmit configuration DEFAULT: EXPID: a000 # ... other settings and a template script used in that experiment contains the following code: .. code-block:: bash :caption: A template script #!/bin/bash echo "The experiment ID is %DEFAULT.EXPID%" then after the template script is preprocessed by Autosubmit, the generated script will be written in the experiment folder with the extension ``.cmd`` and it will look like this: .. code-block:: bash :caption: Generated script #!/bin/bash # Autosubmit Header # ... echo "The experiment ID is a000" # Autosubmit Tailer Header and Tailer ================= Autosubmit does not require users to modify the header and tailer used. The header contains code to set the correct locale, create supporting files for Autosubmit (e.g. ``_TEST_STAT``), and functions used internally by Autosubmit (e.g. for checkpointing). The tailer creates other supporting files (e.g. ``_TEST_COMPLETED`` and update ``_TEST_STAT``). For certain platforms, like Slurm for instance, Autosubmit will populate the header with options for the job scheduler. These options come from platform and job configuration values (e.g. ``TASKS``, ``MEMORY``, etc.) and are automatically translated by Autosubmit into the format expected by the job scheduler. If necessary, Autosubmit allows users to customize the header and the tailer. The configuration keys ``EXTENDED_HEADER_PATH`` and ``EXTENDED_TAILER_PATH`` can be used to indicate the location of, respectively, the header and tailer scripts used by Autosubmit. The location is relative to the project root folder, and the scripts are appended after the default header and tailer scripts. Substitute placeholders after all files have been loaded ======================================================== Autosubmit allows users to define placeholders that are substituted only after all files have been loaded. This is particularly useful when the value of a placeholder is mutable at the time of file loading. For instance, consider the following YAML files loaded in alphabetical order (`ca.yml`, `conf.yml`, `cz.yml`): **ca.yml**: .. code-block:: yaml model: version: "first" **conf.yml**: .. code-block:: yaml other_variable: "something" test_in_place: "%other_variable%/%^model.version%/%another_other_variable%" test_at_the_end: "%other_variable%/%model.version%/%another_other_variable%" another_other_variable: "something" **cz.yml**: .. code-block:: yaml model: version: "last" If a placeholder is defined as `%model.version%` in `conf.yml`, the behavior differs based on the enclosing format: 1. **In-place `%%` Enclosing**: The placeholder's value will correspond to the key defined in the previously loaded file or the file currently being loaded. For example, `%model.version%` would resolve to `"first"` (from `ca.yml`) when `conf.yml` is loaded. 2. **Postloaded `%^%` Enclosing**: The placeholder's value will always correspond to the key defined in the last loaded file. For example, `%^model.version%` in `conf.yml` would resolve to `"last"` (from `cz.yml`) after all files are loaded. In this case, the value of `test_in_place` in `conf.yml` would resolve as: `"something/first/something"` And, the value of `test_at_the_end` would resolve as: `"something/last/something"` In-line Scripting ================= It is also possible to define a script directly in the yaml configuration file. .. note:: If SCRIPT is defined, it will take precedence over FILE. If you wish to run only one liner script: .. code-block:: yaml :emphasize-lines: 6 :caption: Job ``JOB_IN_LINE_ONELINER`` that runs sleep 2 JOBS: JOB_1: PLATFORM: LOCAL RUNNING: once TYPE: bash # default SCRIPT: sleep 2 In case that your script needs lines, the yaml specification allows to do so with the PIPE character (|). .. code-block:: yaml :emphasize-lines: 7,8 :caption: Job ``JOB_IN_LINE_MULTILINE`` that runs hello world and sleep JOBS: JOB_1: PLATFORM: LOCAL RUNNING: once TYPE: bash # default SCRIPT: | echo "hello world!" sleep 2 Also, for debugging purposes, if PROJECT_TYPE is set to NONE (see :ref:`develproject`), the SCRIPT directive will overwrite Autosubmit's self-contained dummy templates. Ensuring Data Integrity with I/O Sync ====================================== To ensure that all data generated by a job is safely written to disk, Autosubmit automatically forces an I/O sync at the end of each script's execution. This applies to jobs running Bash, Python, or R templates. This measure prevents data loss in case of unexpected interruptions immediately after a job reports completion. Script error handling ===================== Autosubmit scripts contain the header, body, and tailer. The code is executed in this order, and in the case of Bash Shell, a failure somewhere in the body or extended header may result in the rest of the script not running. That includes the tailer. In the tailer of an Autosubmit-generated script there are important files generated: ``_STAT`` files, used by Autosubmit and API for metrics and reporting on status of jobs, and ``_COMPLETED`` files which mark when a job has successfully completed. If the user script fails mid-way, and the tailer never produces a ``_COMPLETED`` file, that does not cause any issues as a ``_COMPLETED`` file is not expected upon failures. However, the missing ``_STAT`` file may result in incorrect values reported in Autosubmit commands and API/GUI. For that reason, Autosubmit 4.1.16+ adopted `Bash shell trap functions`_. The script watches for ``EXIT``, and for the following signals: * ``SIGHUP`` * ``SIGINT`` * ``SIGQUIT`` * ``SIGTERM`` * ``SIGXCPU`` * ``SIGXFSZ`` If a command fails in the user-provided template, the ``EXIT`` function will make sure the ``_STAT`` file is generated, and the script exits with the same return code of the failing function. If any of the signals above happen and are captured by the script, then the trap-function will set the exit code to failure (``128`` + the signal value) and will write the ``_STAT`` file. The files are not generated if the process running the script receives a ``SIGKILL`` (as that cannot be captured) or if the script receives a signal like ``SIGTERM``, emitted by Slurm upon wall time limit (seconds before it emits a ``SIGKILL``) but the script is executing a long-running command (e.g., ``sleep 10000``, or compiling a model). In these cases, neither ``_COMPLETED`` nor ``_STAT`` files are generated. .. _Bash shell trap functions: https://www.gnu.org/software/bash/manual/html_node/Bourne-Shell-Builtins.html#index-trap