User Guide

• Create an Experiment
  • Create new experiment
    • Configuring Default Platforms
  • Copy another experiment
  • Create a dummy experiment
  • Create a test case experiment
  • Profiling experiment creation
• Configure Experiments
  • Configuration files
  • How to add a new job
  • How to add a new heterogeneous job
  • How to configure email notifications
  • How to configure CPMIP threshold notifications
    • Example notification
  • How to add a new platform to the experiment configuration
  • How to request exclusivity or reservation
  • How to set a custom interpreter for your job
  • How to create and run only selected members
  • Remote Dependencies - Presubmission feature
• Defining the workflow
  • Simple workflow
  • Running jobs once per startdate, member or chunk
  • Dependencies
    • Dependencies with previous jobs
    • Dependencies between running levels
    • Dependencies rework
    • Start conditions
    • Job frequency
    • Job synchronize
    • Job split
      • Basic behavior
      • How split dependencies are resolved
    • Job Splits with calendar
    • Job delay
  • Loops (FOR jobs)
  • Workflow examples:
    • Example 1: How to select a specific chunk
    • Example 2: SKIPPABLE
    • Example 3: Weak dependencies
    • Example 4: Select a member
• Wrappers
  • Basic configuration
    • Wrapper parameters description
      • Type
      • Jobs_in_wrapper
      • Method
      • Extend_wallclock
      • Retrials
      • Queue
      • Export
      • Check_time_wrapper
      • Number of jobs in a wrapper({MIN/MAX}_WRAPPED{_H/_V}
      • Policy
  • Vertical wrapper
  • Horizontal wrapper
  • Vertical-horizontal wrapper
  • Horizontal-vertical wrapper
    • Advanced example: Set-up an crossdate wrapper
• Running Experiments
  • Run an experiment
    • How to run an experiment that was created with another version
    • How to run an experiment that was created with version <= 4.0.0
    • How to run only selected members
  • How to start an experiment at a given time
  • How to start an experiment after another experiment is finished
  • How to profile Autosubmit while running an experiment
  • How to prepare an experiment to run in two independent job_list. (Priority jobs, Two-step-run) (OLD METHOD)
    • Example using the old method
  • How to prepare an experiment to run in two independent job_list. (New method)
    • Simplified example using the new method
    • Example 2: Crossdate wrappers using the the new dependencies
  • How to stop the experiment
    • Examples:
  • Retries
• How to restart the experiment
  • How to recover an experiment
  • Typical workflow of recovery
  • Important options for recovery
  • Examples:
  • How to rerun a part of the experiment
• Manage Experiments
  • How to clean the experiment
  • How to archive an experiment
  • How to unarchive an experiment
  • How to delete the experiment
  • How to synchronize with the project’s latest changes
  • How to update the description of your experiment
  • How to change the job status
    • Filter Examples
    • Chunk/Section/Split Filter Details
    • Using the “Any” Keyword
    • Deprecated Filters
    • How to change the job status without stopping autosubmit
• Monitor and Check Experiments
  • How to check the experiment configuration
    • How to use check in running time:
  • How to generate cmd files
  • How to monitor an experiment
    • Possible job status
  • Grouping jobs
  • How to profile Autosubmit while monitoring an experiment
  • How to get details about the experiment
  • How to monitor job statistics
    • Console output description
    • Diagram output description
    • Summaries output description
    • Custom statistics
  • How to extract information about the experiment parameters
    • What goes into the parameter list
    • Template syntax
    • Starter template
    • Tips
• Configuration details, setup and sharing
  • Experiment configuration
  • Standard configuration structure
  • Advanced configuration structure and restrictions
  • How to create and share the configuration
    • Standard Configuration
    • Sharing a standard Configuration
    • Advanced Configuration
      • CUSTOM_CONFIG: Syntax
  • Advanced configuration - Full dummy example (reproducible)
    • Sharing an advanced configuration
• Variables reference
  • Job variables
    • Custom directives
  • Platform variables
  • Other variables
  • Performance Metrics variables
• Experiment IDs
• Provenance
  • RO-Crate
• Traceability
  • Configuration
  • Logs
  • Data
  • Version control
  • A practical example
• User Mapping
  • About
  • When to use
  • Prerequisites
  • How it works
  • How to activate it with examples
• Script templates
  • Header and Tailer
  • Substitute placeholders after all files have been loaded
  • In-line Scripting
  • Ensuring Data Integrity with I/O Sync
  • Script error handling
• User-defined metrics
  • Define your own metrics
  • Prepare your script template
• Performance
  • Available CPMIP metrics
• Log operations
  • Compressing logs from a remote job execution
  • Removing log files after transfer
• Inspect & Debug Tools
  • Inspect ( dry-run )
  • Overview
  • Filters
  • What inspect checks
    • Parameter Substitution
    • CMD Syntax Validation
  • Common Debugging Workflow

Command list

• expid Create a new experiment

• create Create specified experiment workflow

• check Check configuration for specified experiment

• describe Show details for specified experiments

• run Run specified experiment

• inspect Generate cmd files

• test Test experiment

• testcase Test case experiment

• monitor Plot specified experiment

• stats Plot statistics and summaries for specified experiment

• setstatus Sets job status for an experiment

• recovery Recover specified experiment

• clean Clean specified experiment

• refresh Refresh project directory for an experiment

• delete Delete specified experiments

• configure Configure database and path for autosubmit

• install Install database for Autosubmit on the configured folder

• archive Clean, compress and remove from the experiments’ folder a finalized experiment

• unarchive Restores an archived experiment

• migrate_exp Deprecated and removed in v4.1.16; planned for restructuring and redeployment in the near future.

• report extract experiment parameters

• updateversion Updates the Autosubmit version of your experiment with the current version of the module you are using

• dbfix Fixes the database malformed error in the historical database of your experiment

• pklfix Fixes the blank pkl error of your experiment

• updatedescrip Updates the description of your experiment (See: How to update the description of your experiment)

Tutorials (How to)

• Create an Experiment

• Configure Experiments

• How to prepare an experiment to run in two independent job_list. (Priority jobs, Two-step-run) (OLD METHOD)

• How to restart the experiment

TODO add workflow_validation.

• How to monitor job statistics

• How to archive an experiment

• Advanced Configuration