Manage Experiments#
Cleaning the experiment#
This procedure allows you to save space after finalising an experiment. You must execute:
autosubmit clean <EXPID>
Options:
$ autosubmit clean -h
Output:
Autosubmit v4.2.0
Command traceability:
Command: /home/docs/checkouts/readthedocs.org/user_builds/autosubmit/envs/latest/bin/autosubmit clean -h
Host: build-35786016-project-45339-autosubmit
User: docs
PID: 1202
Python: 3.12.13
Platform: Linux-7.0.0-1004-aws-x86_64-with-glibc2.35
Timezone: UTC (UTC+00:00)
Working directory: /home/docs/checkouts/readthedocs.org/user_builds/autosubmit/checkouts/latest/docs
usage: autosubmit clean [-h] [-pr] [-p] [-s] [-v] [--profile] expid
Cleans an experiment, reducing disk space usage.
It removes the project directory and outdated plots or stats.
When it deletes plots or stats, it keeps the two newest files
(i.e. two newest statistics files, and two newest plot files).
positional arguments:
expid Experiment identifier.
options:
-h, --help Show available options and exit.
-pr, --project Clean project.
-p, --plot Keep only the two most recently created plots.
-s, --stats Keep only the most recently created statistics.
-v, --update_version Update experiment version.
--profile Print performance parameters of the execution of this
command.
Examples:
# clean an experiment.
$ autosubmit clean a000
# clean the Autosubmit project folder of an experiment.
$ autosubmit clean --project a000
# clean the plot and stat files of an experiment.
$ autosubmit clean --plot --stats a000
The -p and -s flags are used to clean our experiment
plotfolder to save disk space. Only the two latest plots will be kept. Older plots will be removed.
Example:
autosubmit clean <EXPID> -p
The -pr flag is used to clean our experiment
projlocally in order to save space (it could be particularly big).
Caution
Bear in mind that if you have not synchronized your experiment project folder with the information available on the remote repository (i.e.: commit and push any changes we may have), or in case new files are found, the clean procedure will be failing although you specify the -pr option.
Example:
autosubmit clean <EXPID> -pr
A bare copy (which occupies less space on disk) will be automatically made.
Hint
That bare clone can be always reconverted in a working clone if we want to run again the experiment by using git clone bare_clone original_clone.
Note
In addition, every time you run this command with the -pr option, it will check the commit unique identifier for the local working tree existing in the proj directory.
If that commit identifier exists, clean will register it in the expdef_<EXPID>.yml file.
Archiving an experiment#
When you archive an experiment in Autosubmit, it automatically cleans the experiment as well. This means the experiment will not be available for use, unless it is unarchived.
autosubmit archive EXPID
Options:
$ autosubmit archive -h
Output:
Autosubmit v4.2.0
Command traceability:
Command: /home/docs/checkouts/readthedocs.org/user_builds/autosubmit/envs/latest/bin/autosubmit archive -h
Host: build-35786016-project-45339-autosubmit
User: docs
PID: 1204
Python: 3.12.13
Platform: Linux-7.0.0-1004-aws-x86_64-with-glibc2.35
Timezone: UTC (UTC+00:00)
Working directory: /home/docs/checkouts/readthedocs.org/user_builds/autosubmit/checkouts/latest/docs
usage: autosubmit archive [-h] [-nclean] [-uc] [--rocrate] [-v] [--profile]
expid
Archives an experiment.
Compresses the experiment directory and moves it to the archival location.
positional arguments:
expid Experiment identifier.
options:
-h, --help Show available options and exit.
-nclean, --noclean Do not clean the experiment folder.
-uc, --uncompress Create the container without compression.
--rocrate produce an RO-Crate file
-v, --update_version Update experiment version.
--profile Print performance parameters of the execution of this
command.
The archived experiment will be stored as a tar.gz file, under
a directory named after the year of the last _COMPLETED file
date or, if no _COMPLETED job is present, it will use the year of
the date the autosubmit archive was run (e.g. for the selected
year 2023, the location will be $HOME/autosubmit/2023/<EXPID>.tar.gz).
Unarchiving an experiment#
To unarchive an experiment, use the command:
autosubmit unarchive <EXPID>
Options:
$ autosubmit unarchive -h
Output:
Autosubmit v4.2.0
Command traceability:
Command: /home/docs/checkouts/readthedocs.org/user_builds/autosubmit/envs/latest/bin/autosubmit unarchive -h
Host: build-35786016-project-45339-autosubmit
User: docs
PID: 1206
Python: 3.12.13
Platform: Linux-7.0.0-1004-aws-x86_64-with-glibc2.35
Timezone: UTC (UTC+00:00)
Working directory: /home/docs/checkouts/readthedocs.org/user_builds/autosubmit/checkouts/latest/docs
usage: autosubmit unarchive [-h] [-nclean] [-uc] [--rocrate] [-v] [--profile]
expid
Unarchives an experiment.
Decompresses the experiment archive, ensuring the experiment can be used again.
positional arguments:
expid Experiment identifier.
options:
-h, --help Show available options and exit.
-nclean, --noclean Avoid cleaning of experiment folder.
-uc, --uncompressed Extract files of the tar file without gzip
compression.
--rocrate Unarchive an RO-Crate file.
-v, --update_version Update experiment version.
--profile Print performance parameters of the execution of this
command.
Deleting the experiment#
To delete the experiment, use the command:
autosubmit delete <EXPID>
<EXPID> is the experiment identifier. You can pass a single identifier, or a comma-separated list of identifiers.
Warning
DO NOT USE THIS COMMAND IF YOU ARE NOT SURE ! It deletes the experiment from database and experiment’s folder.
Options:
$ autosubmit delete -h
Output:
Autosubmit v4.2.0
Command traceability:
Command: /home/docs/checkouts/readthedocs.org/user_builds/autosubmit/envs/latest/bin/autosubmit delete -h
Host: build-35786016-project-45339-autosubmit
User: docs
PID: 1208
Python: 3.12.13
Platform: Linux-7.0.0-1004-aws-x86_64-with-glibc2.35
Timezone: UTC (UTC+00:00)
Working directory: /home/docs/checkouts/readthedocs.org/user_builds/autosubmit/checkouts/latest/docs
usage: autosubmit delete [-h] [-f] [--profile] expid
Deletes an experiment.
Removes the experiment from the database and file system.
positional arguments:
expid Experiment identifiers separated by commas.
options:
-h, --help Show available options and exit.
-f, --force Delete experiment without confirmation.
--profile Print performance parameters of the execution of this command.
Example:
autosubmit delete <EXPID>
Warning
Be careful ! force option does not ask for your confirmation.
Synchronizing with the project’s latest changes#
Autosubmit supports directly fetching files from the repository, which could be local or remote.
In order to synchronize with remote, use the command:
autosubmit refresh <EXPID>
where <EXPID> is the experiment identifier.
It checks the experiment configuration and copies code from the original repository to project directory.
Warning
THIS WILL OVERWRITE LOCAL CHANGES!
Project directory <EXPID>/proj will be overwritten and you may loose local changes.
Options:
$ autosubmit refresh -h
Output:
Autosubmit v4.2.0
Command traceability:
Command: /home/docs/checkouts/readthedocs.org/user_builds/autosubmit/envs/latest/bin/autosubmit refresh -h
Host: build-35786016-project-45339-autosubmit
User: docs
PID: 1210
Python: 3.12.13
Platform: Linux-7.0.0-1004-aws-x86_64-with-glibc2.35
Timezone: UTC (UTC+00:00)
Working directory: /home/docs/checkouts/readthedocs.org/user_builds/autosubmit/checkouts/latest/docs
usage: autosubmit refresh [-h] [-mc] [-jc] [-v] [--profile] expid
Refreshes the project files for an experiment.
Synchronises the experiment's project files with their configured source.
When the project uses Git, the project is cloned again, replacing the local
copy and discarding any local modifications.
Use --model_conf or --jobs_conf to also overwrite the corresponding
configuration files from the refreshed project.
positional arguments:
expid Experiment identifier.
options:
-h, --help Show available options and exit.
-mc, --model_conf Overwrite model conf file.
-jc, --jobs_conf Overwrite jobs conf file.
-v, --update_version Update experiment version.
--profile Print performance parameters of the execution of this
command.
Examples:
# refresh the project files for an experiment.
$ autosubmit refresh a000
# refresh the project and overwrite the model configuration file.
$ autosubmit refresh a000 --model_conf
# refresh the project and overwrite the jobs configuration file.
$ autosubmit refresh a000 --jobs_conf
# refresh the project and overwrite both configuration files.
$ autosubmit refresh a000 --model_conf --jobs_conf
Example:
autosubmit refresh <EXPID>
Updating the experiment description#
Use the command:
autosubmit updatedescrip <EXPID> "DESCRIPTION"
EXPID is the experiment identifier.
DESCRIPTION is the new description of your experiment.
Options:
$ autosubmit updatedescrip -h
Output:
Autosubmit v4.2.0
Command traceability:
Command: /home/docs/checkouts/readthedocs.org/user_builds/autosubmit/envs/latest/bin/autosubmit updatedescrip -h
Host: build-35786016-project-45339-autosubmit
User: docs
PID: 1212
Python: 3.12.13
Platform: Linux-7.0.0-1004-aws-x86_64-with-glibc2.35
Timezone: UTC (UTC+00:00)
Working directory: /home/docs/checkouts/readthedocs.org/user_builds/autosubmit/checkouts/latest/docs
usage: autosubmit updatedescrip [-h] [-v] [--profile] expid description
Updates the description of an experiment.
positional arguments:
expid Experiment identifier.
description New description.
options:
-h, --help Show available options and exit.
-v, --update_version Update experiment version.
--profile Print performance parameters of the execution of this
command.
Examples:
# change the experiment description to "Testing"
$ autosubmit updatedescrip a000 "Testing"
Autosubmit will validate the provided data and print the results in the command line.
Example:
autosubmit a29z "Updated using Autosubmit updatedescrip"
Changing the job status#
This procedure allows you to modify the status of your jobs.
Warning
Beware that Autosubmit must be stopped to use setstatus.
Otherwise a running instance of Autosubmit, at some point, will overwrite any changes you may have done.
You must execute:
autosubmit setstatus <EXPID> <FILTER> <VALUE_TO_FILTER> -t <STATUS_FINAL> -s
By default, plots are not generated when changing status. To generate plots showing the updated job statuses, use
the -plt or --plot option.
autosubmit setstatus <EXPID> <FILTER> <VALUE_TO_FILTER> -t <STATUS_FINAL> -s -plt
Where:
FILTER |
Meaning |
Example of VALUE_TO_FILTER |
|---|---|---|
-fl |
filter by job name |
|
-fs |
filter by job status |
|
-ft |
filter by job type (and optionally split) |
|
-fc |
filter by chunk/section/split |
|
If multiple filters are provided (-fl, -fs, -ft, -fc), they will be combined as logical AND, meaning that only jobs
matching ALL specified filters will have their status changed.
Mandatory arguments:
<EXPID>: experiment identifier<FILTER> <VALUE_TO_FILTER>: at least one filter is required to select jobs (see below for filter options)-t STATUS_FINAL: target status (READY,COMPLETED,WAITING,SUSPENDED,HELD,UNKNOWN)
Optional filter arguments (combine multiple for granular selection):
-fl: space-separated list of job names
autosubmit setstatus <EXPID> -fl "<EXPID>_20101101_fc3_21_SIM <EXPID>_20111101_fc4_26_SIM" -t READY -s
-fc: chunk/section/split filter (JSON-like format, takes precedence over legacy filters)
autosubmit setstatus <EXPID> -fc "[ 19601101 [ fc1 [1] ] ]" -t READY -s
-fs: space-separated job statuses. The available statuses are:READY,COMPLETED,WAITING,SUSPENDED,HELD, andUNKNOWN.
autosubmit setstatus <EXPID> -fs FAILED -t READY -s
-ft: space-separated job types with optional split selection
autosubmit setstatus <EXPID> -ft TRANSFER -t SUSPENDED -s
autosubmit setstatus <EXPID> -ft "TRANSFER[1 2]" -t SUSPENDED -s
-ftc: [DEPRECATED] legacy chunk/type filter (use a combination of-fcand-ftinstead).
Command:
autosubmit setstatus <EXPID> -ftc "[ 19601101 [ fc0 [1 2 3 4] ] ],SIM" -t READY -s
Can be replaced with:
autosubmit setstatus <EXPID> -fc "[ 19601101 [ fc0 [1 2 3 4] ] ]" -ft SIM -t READY -s
-ftcs: [DEPRECATED] legacy chunk/type/split filter (use a combination of-fcand-ftinstead).
Command:
autosubmit setstatus <EXPID> -ftcs "[ 19601101 [ fc0 [1 2 3 4] ] ],SIM,1" -t READY -s
Can be replaced with:
autosubmit setstatus <EXPID> -fc "[ 19601101 [ fc0 [1 2 3 4] ] ]" -ft SIM -t READY -s
Options:
$ autosubmit setstatus -h
Output:
Autosubmit v4.2.0
Command traceability:
Command: /home/docs/checkouts/readthedocs.org/user_builds/autosubmit/envs/latest/bin/autosubmit setstatus -h
Host: build-35786016-project-45339-autosubmit
User: docs
PID: 1214
Python: 3.12.13
Platform: Linux-7.0.0-1004-aws-x86_64-with-glibc2.35
Timezone: UTC (UTC+00:00)
Working directory: /home/docs/checkouts/readthedocs.org/user_builds/autosubmit/checkouts/latest/docs
usage: autosubmit setstatus [-h] [-np | -plt] [-s] -t
{READY,COMPLETED,WAITING,SUSPENDED,UNKNOWN,HELD}
[-fl LIST] [-fc FILTER_CHUNKS] [-fs FILTER_STATUS]
[-ft FILTER_TYPE] [-ftc FILTER_TYPE_CHUNK]
[-ftcs FILTER_TYPE_CHUNK_SPLIT] [--hide]
[-group_by {date,member,chunk,split,automatic}]
[-expand EXPAND] [-expand_status EXPAND_STATUS]
[-nt] [-cw] [-d] [-v] [--profile]
expid
Changes the status of selected jobs in the database.
NOTE: Use --save to persist the changes to disk!
This can be used to skip completed parts of a workflow and allow downstream
jobs to run.
For example, jobs that have already been completed outside Autosubmit can be
marked as COMPLETED so that a downstream job, such as CLEAN, can run.
positional arguments:
expid Experiment identifier.
options:
-h, --help Show available options and exit.
-np, --noplot Omit plot (default).
-plt, --plot Generate plot.
-s, --save Save changes to disk.
-t {READY,COMPLETED,WAITING,SUSPENDED,UNKNOWN,HELD}, --status_final {READY,COMPLETED,WAITING,SUSPENDED,UNKNOWN,HELD}
Supply the target status.
-fl LIST, --list LIST
Supply the list of job names to be changed. Default =
"Any". LIST = "b037_20101101_fc3_21_sim
b037_20111101_fc4_26_sim".
-fc FILTER_CHUNKS, --filter_chunks FILTER_CHUNKS
Supply the list of chunks to change the status.
Default = "Any". LIST = "[ 19601101 [ fc0 [1 2 3 4]
fc1 [1] ] 19651101 [ fc0 [16-30] ] ]".
-fs FILTER_STATUS, --filter_status FILTER_STATUS
Select the status (one or more) to filter the list of
jobs.Valid values = ['Any', 'READY', 'COMPLETED',
'WAITING', 'SUSPENDED', 'FAILED', 'UNKNOWN'].
-ft FILTER_TYPE, --filter_type FILTER_TYPE
Select the job type and split to filter the list of
jobs. Default split = "Any". LIST = "LOCALJOB [5-10]
SIM".
-ftc FILTER_TYPE_CHUNK, --filter_type_chunk FILTER_TYPE_CHUNK
[Deprecated] Equivalent behaviour can be achieved by
combining -ft and -fc. Supply the list of chunks to
change the status. Default = "Any". When the member
name "all" is set, all the chunks selected from for
that member will be updated for all the members.
Example: all [1], will have as a result that the
chunks 1 for all the members will be updated. Follow
the format: "[ 19601101 [ fc0 [1 2 3 4] Any [1] ]
19651101 [ fc0 [16-30] ] ],SIM,SIM2,SIM3".
-ftcs FILTER_TYPE_CHUNK_SPLIT, --filter_type_chunk_split FILTER_TYPE_CHUNK_SPLIT
[Deprecated] Equivalent behaviour can be achieved by
combining -ft and -fc. Supply the list of chunks &
splits to change the status. Default = "Any". When the
member name "all" is set, all the chunks selected from
for that member will be updated for all the members.
Example: all [1], will have as a result that the
chunks 1 for all the members will be updated. Follow
the format: "[ 19601101 [ fc0 [1 [1 2] 2 3 4] Any [1]
] 19651101 [ fc0 [16-30] ] ],SIM,SIM2,SIM3".
--hide Hide the plot window.
-group_by {date,member,chunk,split,automatic}
Group the jobs automatically or by date, member, chunk
or split.
-expand EXPAND Supply the list of dates/members/chunks to filter the
list of jobs. Default = "Any". LIST = "[ 19601101 [
fc0 [1 2 3 4] fc1 [1] ] 19651101 [ fc0 [16-30] ] ]".
-expand_status EXPAND_STATUS
Select the statuses to be expanded.
-nt, --notransitive Disable transitive reduction.
-cw, --check_wrapper Generate possible wrapper in the current workflow.
-d, --detail Generate detailed view of changes.
-v, --update_version Update experiment version.
--profile Print performance parameters of the execution of this
command.
Examples:
# mark selected jobs as completed and save the changes.
$ autosubmit setstatus a000 --status_final COMPLETED --save
# mark specific jobs as completed.
$ autosubmit setstatus a000 --status_final COMPLETED --list "JOB1 JOB2" --save
# mark jobs in a specific status as completed.
$ autosubmit setstatus a000 --status_final COMPLETED --filter_status WAITING --save
# mark jobs as completed for specific chunks.
$ autosubmit setstatus a000 --status_final COMPLETED --filter_chunks "[ 19601101 [ fc0 [1 2 3 4] ] ]" --save
Filter precedence (when multiple chunk filters are specified):
When -fc, -ftc, and -ftcs are combined, precedence is: -fc → -ftc → -ftcs. A warning will be
logged if multiple chunk filters are detected.
Filter Examples#
Single filter, by job list:
autosubmit setstatus <EXPID> -fl "<EXPID>_20101101_fc3_21_SIM <EXPID>_20111101_fc4_26_SIM" -t READY -s
Single filter, by chunk/section/split:
autosubmit setstatus <EXPID> -fc "[ 19601101 [ fc1 [1] ] ]" -t READY -s
Single filter, by current status:
autosubmit setstatus <EXPID> -fs FAILED -t READY -s
Single filter, by job type:
autosubmit setstatus <EXPID> -ft TRANSFER -t SUSPENDED -s
Single filter, by job type and split:
autosubmit setstatus <EXPID> -ft "TRANSFER[1:5]" -t SUSPENDED -s
Multiple filters combined (AND logic, selects jobs matching ALL filters):
autosubmit setstatus <EXPID> -fc "[ 19601101 [ fc1 [1] ] ]" -ft "SIM" -t SUSPENDED -s
This selects jobs that are in both the chunk filter AND have type “SIM”.
Chunk/Section/Split Filter Details#
Format:
[ 19601101 [ fc0 [1 2 3 4] fc1 [1] ] 19651101 [ fc0 [16-30] ] ]
Date (month) range example:
autosubmit setstatus <EXPID> -fc "[ 1960(1101-1201) [ fc1 [1] ] ]" -ft "SIM" -t SUSPENDED -s
Result:
<EXPID>_19601101_fc1_1_SIM
<EXPID>_19601201_fc1_1_SIM
Date (day) range example:
autosubmit setstatus <EXPID> -fc "[ 1960(1101-1105) [ fc1 [1] ] ]" -ft "SIM" -t SUSPENDED -s
Result:
<EXPID>_19601101_fc1_1_SIM
<EXPID>_19601102_fc1_1_SIM
<EXPID>_19601103_fc1_1_SIM
<EXPID>_19601104_fc1_1_SIM
<EXPID>_19601105_fc1_1_SIM
Using the “Any” Keyword#
The keyword Any (case-insensitive) means “no restriction” in that filter:
-fl Any: all jobs (no job list restriction)-fs Any: all job statuses (no status restriction)-ft Any: all job types (no type restriction)
Example:
autosubmit setstatus <EXPID> -fs Any -fc "[ 19601101 [ fc1 [1] ] ]" -t READY -s
This changes all jobs in the chunk filter to READY, regardless of their current status.
Deprecated Filters#
Warning
The -ftc and -ftcs filters are deprecated and will be removed in future versions. Use -fc instead.
-ftc (deprecated) is similar to -fc but also accepts job types without separate -ft:
autosubmit setstatus <EXPID> -ftc "[ 19601101 [ fc0 [1 2 3 4] ] ],SIM" -t READY -s
Use -fc with -ft instead:
autosubmit setstatus <EXPID> -fc "[ 19601101 [ fc0 [1 2 3 4] ] ]" -ft SIM -t READY -s
Hint
When satisfied with your filter selections, use the parameter -s to save changes to the pkl file. In order to understand more the grouping options, which are used for visualization purposes, please check Grouping jobs.
Changing the job status without stopping Autosubmit#
This procedure allows you to modify the status of your jobs without having to stop Autosubmit.
You must create a file in <experiments_directory>/<EXPID>/status/ named:
updated_list_<EXPID>.txt
Format:
This file should have two columns: the first one has to be the job_name and the second one the status.
Options:
READY,COMPLETED,WAITING,SUSPENDED,FAILED,UNKNOWN
Example:
vi updated_list_<EXPID>.txt
<EXPID>_20101101_fc3_21_SIM READY
<EXPID>_20111101_fc4_26_SIM READY
If Autosubmit finds the above file, it will process it. You can check that the processing was OK at a given date and time, if you see that the file name has changed to:
updated_list_<EXPID>.txt_<YYYYMMDD_HHMM>
Note
A running instance of Autosubmit will check the existence of the above file after checking already submitted jobs.
It may take some time, depending on the setting SAFETYSLEEPTIME.
Note
Job names and statuses are matched case-insensitively. Malformed lines, unknown jobs/statuses and active
statuses (SUBMITTED, QUEUING, RUNNING) are skipped with a warning. When a job is active and its platform
is reachable, Autosubmit cancels it on the platform before applying the requested change.