Getting started#

This tutorial is a starter’s guide to run a dummy experiment with Autosubmit.

Dummy experiments run workflows with inexpensive empty tasks and therefore are ideal for teaching and testing purposes.

Real experiments instead run workflows with complex tasks. To read information about how to develop parameterizable tasks for Autosubmit workflows, refer to Developing a project.

Set up password-less SSH#

Autosubmit needs to establish password-less SSH connections in order to run and monitor workflows on remote platforms.

Ensure that you have a password-less connection to all platforms you want to use in your experiment. If you are unsure how to do this, please follow these instructions:

  • Open a terminal and run ssh-keygen -t rsa -b 4096 -C "email@email.com" -m PEM.

  • Copy the resulting key to your platform of choice, via scp or ssh-copy-id.

# Generate a key pair for password-less ssh, PEM format is recommended as others can cause problems
ssh-keygen -t rsa -b 4096 -C "email@email.com" -m PEM
# Copy the public key to the remote machine
ssh-copy-id -i ~/.ssh/id_rsa.pub user@remotehost
# Add your key to ssh agent (if encrypted)
# If not initialized, initialize it
eval `ssh-agent -s`
# Add the key
ssh-add ~/.ssh/id_rsa
# Where ~/.ssh/id_rsa is the path to your private key

See also

For host keys, proxies and shared setups, see Configuration details, setup and sharing.

Create a new experiment#

Example:

autosubmit expid -dm -H "local" -d "Tutorial"

  • -dm: Generates a dummy experiment.

  • -H: Sets the principal experiment platform.

  • -d: Sets a short description for the experiment.

The output of the command will show the <EXPID> of the experiment and generate the following directory structure:

Experiment folder

Contains

conf

Experiment configuration files.

pkl

Workflow pkl files.

plot

Visualization output files.

tmp

Logs, templates and misc files.

proj

User scripts and project code. (empty)

Then, execute autosubmit create <EXPID> and Autosubmit will generate the workflow graph.

See also

For every expid option, and for how experiment IDs are allocated, see Create an experiment and Experiment IDs.

Run and monitor the experiment#

To run an experiment, use autosubmit run <EXPID>. Autosubmit runs experiments performing the following operations:

  • First, it checks the experiment configuration. If it is wrong, it won’t proceed further.

  • Second, it runs the experiment while retrieving all logs from completed or failed tasks as they run.

  • Third, it manages all the workflow steps by following the dependencies defined by the user until all jobs are in COMPLETED or FAILED status. There can be jobs left in WAITING status if their dependencies are in FAILED status.

While the experiment is running, it can be visualized via autosubmit monitor <EXPID>.

dummy

It illustrates the output of autosubmit monitor. It describes all workflow jobs’ possible status and actual status.

Concurrently, the <EXPID>/tmp gets filled with the cmd scripts generated by Autosubmit to run the local and remote tasks (in this case, they are sent and submitted to the remote platform(s)).

Autosubmit keeps logs at ASLOGS and LOG_<EXPID> folders, which are filled up with Autosubmit’s command logs and job logs.

See also

This tutorial only covers a plain run. For the full set of run options, and for restarting a workflow that stopped part-way, see Running experiments and Restart an experiment. The monitoring output and the job statuses are covered in Monitor and Check Experiments.

Accessing the logs#

The autosubmit commands such as expid, run, monitor, all may produce log files on the user’s file system. To save the user from having to navigate to the log file, or to memorize the location of these files, Autosubmit provides the autosubmit cat-log command.

$ autosubmit cat-log a000

Output:
Autosubmit v4.2.0

Command traceability:
  Command: /home/docs/checkouts/readthedocs.org/user_builds/autosubmit/envs/latest/bin/autosubmit cat-log a000
  Host: build-35117997-project-45339-autosubmit
  User: docs
  PID: 977
  Python: 3.12.13
  Platform: Linux-7.0.0-1004-aws-x86_64-with-glibc2.35
  Timezone: UTC (UTC+00:00)
  Working directory: /home/docs/checkouts/readthedocs.org/user_builds/autosubmit/checkouts/latest/docs

No logs found.

Note

The -f (--file) option is for the file type, not the file path. See the complete help and syntax with autosubmit cat-log --help for a list of supported types, depending on whether you choose a workflow log or a job log file. Note too that there is a -i (--inspect) flag in the command to tell Autosubmit you want job files generated by autosubmit inspect, instead of job files generated by autosubmit run.

See also

For log retrieval, retention and the complete cat-log syntax, see Log operations.

Inspect the experiment#

Once an experiment exists on disk, two commands help you understand what it contains.

autosubmit describe <EXPID> displays information about the experiment: its owner, location on disk, creation time, project model and branch, HPC platform, and description.

$ autosubmit describe a000

Output:
Autosubmit v4.2.0

Command traceability:
  Command: /home/docs/checkouts/readthedocs.org/user_builds/autosubmit/envs/latest/bin/autosubmit describe a000
  Host: build-35117997-project-45339-autosubmit
  User: docs
  PID: 979
  Python: 3.12.13
  Platform: Linux-7.0.0-1004-aws-x86_64-with-glibc2.35
  Timezone: UTC (UTC+00:00)
  Working directory: /home/docs/checkouts/readthedocs.org/user_builds/autosubmit/checkouts/latest/docs

YAML configuration loaded:
  /home/docs/checkouts/readthedocs.org/user_builds/autosubmit/checkouts/latest/_readthedocs/autosubmit/autosubmit/a000/conf/as_misc.yml
  /home/docs/checkouts/readthedocs.org/user_builds/autosubmit/checkouts/latest/_readthedocs/autosubmit/autosubmit/a000/conf/autosubmit_a000.yml
  /home/docs/checkouts/readthedocs.org/user_builds/autosubmit/checkouts/latest/_readthedocs/autosubmit/autosubmit/a000/conf/expdef_a000.yml
  /home/docs/checkouts/readthedocs.org/user_builds/autosubmit/checkouts/latest/_readthedocs/autosubmit/autosubmit/a000/conf/jobs_a000.yml
  /home/docs/checkouts/readthedocs.org/user_builds/autosubmit/checkouts/latest/_readthedocs/autosubmit/autosubmit/a000/conf/platforms_a000.yml

Partial configuration validated correctly

Experiment a000
  Owner:       docs
  Location:    /home/docs/checkouts/readthedocs.org/user_builds/autosubmit/checkouts/latest/_readthedocs/autosubmit/autosubmit/a000
  Created:     2026-10-13 13:19:34.494545
  Model:       Not Found
  Branch:      Not Found
  HPC:         LOCAL
  Description: Tutorial

autosubmit report <EXPID> extracts the experiment variables. It has two modes that can be used together:

  • -all writes every available parameter to <EXPID>_parameter_list_<timestamp>.txt.

  • -t <template> renders a user-supplied template, substituting %KEY% markers with values, into <EXPID>_report_<timestamp>.

Both files land in <EXPID>/tmp/ by default.

autosubmit report a000 -all
autosubmit report a000 -t conf/report-template.txt

For the template syntax and a starter template, see Monitor and Check Experiments.

See also

For the meaning of each parameter that report emits, see Variables reference. To inspect the generated scripts before submitting anything, see Inspect and debug tools.

Configuration files#

In the folder <EXPID>/conf there are different files that define the actual experiment configuration.

File

Content

expdef_<EXPID>.yml

  • It contains the default platform, the one set with -H.

  • Allows changing the start dates, members and chunks.

  • Allows changing the experiment project source (git, local, svn or dummy).

platforms_<EXPID>.yml

  • It contains the list of platforms to use in the experiment.

  • This file contains the definitions for managing clusters, fat-nodes and support computers.

  • This file must be filled up with the platform(s) configuration(s).

  • Several platforms can be defined and used in the same experiment.

jobs_<EXPID>.yml

  • It contains the tasks’ definitions in sections. Depending on the parameters, one section can generate multiple similar tasks.

  • This file must be filled up with the tasks’ definitions.

  • Several sections can be defined and used in the same experiment.

autosubmit_<EXPID>.yml

  • This file contains the definitions that impact the workflow behavior.

  • It changes workflow behavior with parameters such as job limitations, remote_dependencies and retries.

  • It extends Autosubmit functionalities with parameters such as wrappers and mail notification.

proj_<EXPID>.yml

  • This file contains the configuration used by the user scripts.

  • This file is fully customizable for the current experiment. Allows setting user parameters that will be readable by the autosubmit jobs.

See also

Every key in these files is documented in Configure experiments. For dependencies between job sections, see Defining the workflow; for grouping several jobs into a single submission, see Wrappers.

Run on a remote platform#

It is time to look into the configuration files of the dummy experiment and modify them with a remote platform to run a workflow with a few more chunks.

Open expdef_<EXPID>.yml.

$ cat /home/docs/autosubmit/a000/conf/expdef_a000.yml

Output:
cat: /home/docs/autosubmit/a000/conf/expdef_a000.yml: No such file or directory

Now open platforms_<EXPID>.yml. Note: this is an example for marenostrum4.

$ cat /home/docs/autosubmit/a000/conf/platforms_a000.yml

Output:
cat: /home/docs/autosubmit/a000/conf/platforms_a000.yml: No such file or directory

autosubmit create <EXPID> will generate the new workflow and autosubmit run <EXPID> will run the experiment with the latest changes. If you want to visualize the workflow when creating it, use the command autosubmit create <EXPID> -plt instead.

Warning

If you are using an encrypted key, you will need to add it to the ssh-agent before running the experiment. To do so, run ssh-add <path_to_key>.

See also

For the full set of platform keys, see Configure experiments. If your site maps local users to different remote users, see User mapping.

Common commands#

Command

Short description

expid

Creates a new experiment and generates a new entry in the database by giving it a serial id composed of 4 letters. In addition, it also creates the folder experiment and the basic folder structure.

create <EXPID>

Generates the experiment workflow.

run <EXPID>

Runs the experiment workflow.

monitor <EXPID>

Shows the experiment workflow structure and status.

inspect <EXPID>

Generates Autosubmit scripts and batch scripts for inspection, by processing the tasks’ templates with the experiment parameters.

refresh <EXPID>

Updates the project directory.

recovery <EXPID>

Recovers the experiment workflow obtaining the last run complete jobs.

setstatus <EXPID>

Sets one or multiple jobs status to a given value.

describe <EXPID>

Displays summary information about the experiment: owner, location, creation time, model, branch, HPC platform, and description.

report <EXPID>

Extracts the experiment parameters into a flat list (-all) or renders them through a user-supplied template (-t).

See also

For archiving, deleting, migrating and otherwise administering experiments, see Manage experiments.