autosubmit.job

class autosubmit.job.job.Job(name=None, job_id=None, status=None, priority=None, loaded_data=None)

Class to handle all the tasks with Jobs at HPC.

A job is created by default with a name, a jobid, a status and a type. It can have children and parents. The inheritance reflects the dependency between jobs. If Job2 must wait until Job1 is completed then Job2 is a child of Job1. Inversely Job1 is a parent of Job2

add_children(children)

Add children for the job. It also adds current job as a parent for all the new children

Parameters:

children (list of Job objects) – job’s children to add

add_edge_info(parent, special_conditions)

Adds edge information to the job

Parameters:

• parent (Job) – parent job

• special_conditions (dict) – special variables

add_parent(*parents)

Add parents for the job. It also adds current job as a child for all the new parents

Parameters:

parents (Job) – job’s parents to add

calendar_chunk(parameters)

Calendar for chunks

Parameters:

parameters

Returns:

calendar_split(as_conf: AutosubmitConfig, parameters: dict, set_attributes: bool) → dict

Calculate the calendar splits for the job.

This method processes the calendar splits based on the provided parameters and the Autosubmit configuration.

Parameters:

• as_conf (AutosubmitConfig) – The Autosubmit configuration object.

• parameters (dict) – The dictionary containing job parameters.

• set_attributes (bool) – Flag indicating whether to set attributes directly.

Returns:

The updated parameters dictionary containing calendar split information.

Return type:

dict

check_completion(default_status=-1)

Check whether a COMPLETED file exists on the platform.

This method sets self.new_status (the proposed status), not self.status (the committed status). The caller must later call update_status() to commit the change.

Parameters:

default_status (Status) – Status to propose when the COMPLETED file is absent. Defaults to Status.FAILED.

check_compressed_local_logs() → bool

Checks if the current local log files are compressed versions (.gz or .xz) and updates the local_logs attribute accordingly.

check_end_time(fail_count=-1) → int

Returns end time from stat file

Returns:

date and time

Return type:

int

check_retrials_end_time()

Returns list of end datetime for retrials from total stats file

Returns:

date and time

Return type:

list[int]

check_retrials_start_time()

Returns list of start datetime for retrials from total stats file

Returns:

date and time

Return type:

list[int]

check_running_after(date_limit)

Checks if the job was running after the given date :param date_limit: reference date :type date_limit: datetime.datetime :return: True if job was running after the given date, false otherwise :rtype: bool

check_script(as_conf: AutosubmitConfig, show_logs='false') → bool

Checks if the script is well-formed.

Parameters:

• as_conf – Autosubmit configuration.

• show_logs – Whether to display logs or not.

Returns:

Returns True if the script is well-formed, otherwise returns False.

check_start_time(fail_count=-1)

Returns job’s start time

Returns:

start time

Return type:

str

check_started_after(date_limit)

Checks if the job started after the given date :param date_limit: reference date :type date_limit: datetime.datetime :return: True if job started after the given date, false otherwise :rtype: bool

property checkpoint

Generates a checkpoint step for this job based on job.type.

property children

Returns a list containing all children of the job

Returns:

child jobs

Return type:

set

property children_names_str

Comma separated list of children’s names

property chunk

Current chunk.

property chunk_size

Chunk size used to compute CPMIP metrics.

property chunk_size_unit

Chunk size unit used to compute CPMIP metrics.

construct_real_additional_file_name(file_name: str) → str

Constructs the real name of the file to be sent to the platform.

Parameters:

file_name (str) – The name of the file to be sent.

Returns:

The full path of the file to be sent.

Return type:

str

property cpmip_thresholds

Thresholds for CPMIP metrics.

create_script(as_conf: AutosubmitConfig) → str

Create the script file to be run for the job.

Parameters:

as_conf (AutosubmitConfig) – Configuration object.

Returns:

Script’s filename.

Return type:

str

property custom_directives

List of custom directives.

property delay

Current delay.

property delay_retrials

TODO

delete_parent(parent)

Remove a parent from the job

Parameters:

parent (Job) – parent to remove

property dependencies

Current job dependencies.

property export

TODO.

property fail_count

Number of failed attempts to run this job.

fix_local_logs_timestamps(current_timestamp: str, new_timestamp: str) → None

Renames local log files to update the timestamp in their names without changing the prefix and extension.

It assumes that self.local_logs contains the new timestamp in their names.

Parameters:

• current_timestamp – The current timestamp in the log file names.

• new_timestamp – The new timestamp to replace the current one.

property frequency

TODO.

get_checkpoint_files()

Check if there is a file on the remote host that contains the checkpoint

get_last_retrials() → list[list[datetime]]

Returns the retrials of a job, including the last COMPLETED run.

The selection stops, and does not include when the previous COMPLETED job is located or the list of registers is exhausted.

Returns:

list of dates of retrial [submit, start, finish] in datetime format

Return type:

list of list

get_metric_folder(as_conf: AutosubmitConfig) → str

Returns the default metric folder for the job.

Returns:

The metric folder path.

Return type:

str

get_new_remotelog_name(count=-1)

Checks if remote log file exists on remote host if it exists, remote_log variable is updated :param

has_children()

Returns true if job has any children, else return false

Returns:

true if job has any children, otherwise return false

Return type:

bool

has_parents()

Returns true if job has any parents, else return false

Returns:

true if job has any parent, otherwise return false

Return type:

bool

property hyperthreading

Detects if hyperthreading is enabled or not.

inc_fail_count()

Increments fail count

static is_a_completed_retrial(fields: list) → bool

Returns true only if there are 4 fields: submit start finish status, and status equals COMPLETED.

is_ancestor(job)

Check if the given job is an ancestor :param job: job to be checked if is an ancestor :return: True if job is an ancestor, false otherwise :rtype bool

is_over_wallclock(effective_wallclock=None) → bool

Check if the job is over the wallclock time, it is an alternative method to avoid platform issues.

is_parent(job)

Check if the given job is a parent :param job: job to be checked if is a parent :return: True if job is a parent, false otherwise :rtype bool

property long_name

Job’s long name. If not set, returns name

Returns:

long name

Return type:

str

property member

Current member.

property memory

Memory requested for the job.

property memory_per_task

Memory requested per task.

property name

Current job full name.

property nodes

Number of nodes that the job will use.

property notify_on

Send mail notification on job status change.

property packed

TODO

property parents

Returns parent jobs list

Returns:

parent jobs

Return type:

set

property partition

Returns the queue to be used by the job. Chooses between serial and parallel platforms

:return HPCPlatform object for the job to use :rtype: HPCPlatform

property platform: Platform

Returns the platform to be used by the job. Chooses between serial and parallel platforms

Returns:

HPCPlatform object for the job to use

Return type:

HPCPlatform

process_scheduler_parameters(job_platform: Platform, chunk: int) → None

Parsers yaml data stored in the dictionary and calculates the components of the heterogeneous job if any.

property processors

Number of processors that the job will use.

property processors_per_node

Number of processors per node that the job can use.

property queue

Returns the queue to be used by the job. Chooses between serial and parallel platforms.

:return HPCPlatform object for the job to use :rtype: HPCPlatform

read_header_tailer_script(script_path: str, as_conf: AutosubmitConfig, is_header: bool)

Opens and reads a script. If it is not a BASH script it will fail :(

Will strip away the line with the hash bang (#!)

Parameters:

• script_path – relative to the experiment directory path to the script

• as_conf – Autosubmit configuration file

• is_header – boolean indicating if it is header extended script

recover_last_ready_date() → None

Recovers the last ready date for this job

reset_logs() → None

Reset job log counters.

property retrials

Max amount of retrials to run this job.

retrieve_logfiles() → RecoveryReport

Retrieves log files from the remote host for all pending attempts.

Returns:

A structured report of what was attempted and what succeeded.

Return type:

RecoveryReport

property scratch_free_space

Percentage of free space required on the scratch.

property script

Allows to launch inline code instead of using the file parameter

property sdate

Current start date.

property section

Type of the job, as given on job configuration file.

send_cpmip_notification(as_conf) → None

Capture CPMIP metrics for job and send them as a notification.

Called before job attributes are cleared upon termination. If capture fails (returns None) the notification is silently skipped. If the notification itself fails the error is logged but not re-raised.

Parameters:

as_conf – experiment_configuration

set_ready_date() → None

Sets the ready start date for the job

property shape

Returns the shape of the job. Chooses between serial and parallel platforms

:return HPCPlatform object for the job to use :rtype: HPCPlatform

property split

Current split.

property splits

Max number of splits.

property status_str

String representation of the current status

property synchronize

TODO.

property tasks

Number of tasks that the job will use.

property threads

Number of threads that the job will use.

property total_processors

Number of processors requested by job. Reduces ‘:’ separated format if necessary.

update_content(as_conf: AutosubmitConfig, parameters: dict) → tuple[str, list[str]]

Create the script content to be run for the job.

Parameters:

• as_conf – Autosubmit configuration.

• parameters – Parameters dictionary.

Returns:

A tuple with the job script template and a list with the additional file names.

update_current_parameters(as_conf: AutosubmitConfig, parameters: dict) → dict

Populate and update CURRENT_XXX parameters and placeholders in the given parameters dictionary.

Parameters:

• as_conf (AutosubmitConfig) – Autosubmit configuration object containing platforms_data, jobs_data and other experiment-level settings.

• parameters (dict) – Parameters dictionary to be updated. This dict is modified

Returns:

The same parameters dictionary updated.

Return type:

dict

update_job_variables_final_values(parameters: dict) → None

Jobs variables final values based on parameters dict instead of as_conf This function is called to handle %CURRENT_% placeholders as they are filled up dynamically for each job

update_local_logs() → None

Updates the local log filenames based on the fail count.

update_parameters(as_conf: AutosubmitConfig, set_attributes: bool = False, reset_logs: bool = False) → dict

Refresh the job’s parameters value.

This method reloads the Autosubmit configuration and updates the job’s parameters based on the configuration and the current state of the job.

Parameters:

• as_conf (AutosubmitConfig) – The Autosubmit configuration object.

• set_attributes (bool) – Flag indicating whether to set attributes, defaults to False.

• reset_logs (bool) – Flag indicating whether to reset logs, defaults to False.

Returns:

None

update_placeholders(as_conf: AutosubmitConfig, parameters: dict, replace_by_empty=False) → dict

Find and substitute dynamic placeholders in parameters using the provided Autosubmit configuration helpers.

Parameters:

• as_conf (AutosubmitConfig) – Autosubmit configuration object.

• parameters (dict) – Parameters dictionary potentially containing placeholders.

• replace_by_empty (bool) – Flag indicating whether to replace dynamic variables with empty strings.

Returns:

Parameters with placeholders substituted.

Return type:

dict

update_start_time(count=-1)

Updates the job’s start time based on the count of retries. :param count: The retry count. :type count: int

update_status(as_conf: AutosubmitConfig) → Status

Updates job status, checking COMPLETED file if needed.

Parameters:

as_conf – Autosubmit configuration.

Returns:

The new status.

update_submit_time_and_job_id(attempt: int) → None

Update the submit time and job ID of the job from the database.

Parameters:

attempt (int) – The retry count used to determine the matching database record.

update_submit_time_on_db() → None

Updates an existing job submission entry in the history database for the current fail count.

Unlike write_submit_time(), this method does not insert a new record but instead updates the existing one identified by the job name and the current fail_count.

property validate_template

Whether to print validate information about the job.

property wallclock

Duration for which nodes used by job will remain allocated.

write_end_time(completed, count=-1)

Writes end timestamp to TOTAL_STATS file and jobs_data.db :param completed: True if the job has been completed, False otherwise :type completed: bool :param count: number of retrials :type count: int

write_start_time(fail_count: int = -1)

Writes start date and time to TOTAL_STATS file and the history database.

Parameters:

fail_count (int) – The fail count to identify the correct database row. Defaults to self.fail_count.

Returns:

True if successful, False otherwise

Return type:

bool

write_stats(attempt: int) → None

Gathers the stat file, writes statistics into the job_data.db, and updates the total_stat file. Considers whether the job is a vertical wrapper and the number of retrials to gather.

Parameters:

attempt (int) – The last retrial count.

write_submit_time() → None

Writes submit date and time to the TOTAL_STATS file.

property x11

Whether to use X11 forwarding

property x11_options

Allows to set salloc parameters for x11

class autosubmit.job.job.RecoveryAttempt(attempt: int, success: bool, local_logs: tuple[str, str], remote_logs: tuple[str, str], error: str | None = None)

Result of recovering logs for a single attempt.

class autosubmit.job.job.RecoveryReport(job_name: str, attempts: list[~autosubmit.job.job.RecoveryAttempt] = <factory>, final_updated_log: int = 0, all_succeeded: bool = False)

Structured report of log recovery across all pending attempts.

class autosubmit.job.job.WrapperJob(name: str, job_id: int, status: str, priority: int, job_list: List[Job], total_wallclock: str, platform: ParamikoPlatform, as_config: AutosubmitConfig, hold: bool = False, sections=None, method=None, wr_type=None, num_processors=None)

Defines a wrapper from a package.

Calls Job constructor.

Parameters:

• name – Name of the Package

• job_id – ID of the first Job of the package

• status – ‘READY’ when coming from submit_ready_jobs()

• priority – 0 when coming from submit_ready_jobs()

• job_list – List of jobs in the package

• total_wallclock – Wallclock of the package

• platform – Platform object defined for the package

• as_config – Autosubmit basic configuration object

• hold – Whether the wrapper job is held on submission.

check_and_update_status(as_conf: AutosubmitConfig) → bool

Check the status of the wrapper job and its inner jobs. :param as_conf: Autosubmit configuration object. :type as_conf: AutosubmitConfig :return: True if the status of the wrapper job has changed, otherwise False. :rtype: bool

class autosubmit.job.job_common.Status

Class to handle the status of a job

class autosubmit.job.job_common.Type

Class to handle the status of a job

autosubmit.job.job_common.increase_wallclock_by_chunk(current, increase, chunk)

Receives the wallclock times an increases it according to a quantity times the number of the current chunk. The result cannot be larger than the platform max_wallclock. If Chunk = 0 then no increment.

Parameters:

• current (str) – WALLCLOCK HH:MM

• increase (str) – WCHUNKINC HH:MM

• chunk (int) – chunk number

Returns:

HH:MM wallclock

Return type:

str

autosubmit.job.job_common.parse_output_number(string_number)

Parses number in format 1.0K 1.0M 1.0G

Parameters:

string_number (str) – String representation of number

Returns:

number in float format

Return type:

float

class autosubmit.job.job_list.JobList(expid, config, parser_factory, job_list_persistence)

Class to manage the list of jobs to be run by autosubmit

add_special_conditions(job: Job, special_conditions: dict, filters_to_apply: dict, parent: Job) → None

Add special conditions to the job edge.

Parameters:

• job – Job

• special_conditions – dict

• filters_to_apply – dict

• parent – parent job

Returns:

assign_unique_fake_id(package: JobPackageThread) → None

Assign a unique negative fake ID to a package’s first job for preview use.

Prevents job_package_map key collisions when jobs have no real platform ID yet (e.g. during inspect -cw or create -cw).

Parameters:

package – The job package whose first job will receive a fake ID.

backup_save()

Persists the job list.

check_checkpoint(job, parent)

Check if a checkpoint step exists for this edge

check_scripts(as_conf) → bool

When we have created the scripts, all parameters should have been substituted. %PARAMETER% handlers not allowed.

Parameters:

as_conf (AutosubmitConfig) – experiment configuration

check_special_status() → List[Job]

Check if all parents of a job have the correct status for checkpointing.

Returns:

jobs_to_check - Jobs that fulfill the special conditions.

clear_preview_wrapper() → None

Clear the wrapper preview table used by the -cw flag.

Drops and recreates the wrapper_job_package table so that stale preview data from a previous inspect -cw or create -cw run does not persist between calls.

property expid

Returns the experiment identifier

Returns:

experiment’s identifier

Return type:

str

find_and_delete_redundant_relations(problematic_jobs: dict) → None

Jobs with intrinsic rules than can’t be safely not added without messing other workflows. The graph will have the least amount of edges added as much as safely possible before this function. Structure: problematic_jobs structure is {section: {child_name: [parent_names]}}

Returns:

generate(as_conf, date_list, member_list, num_chunks, chunk_ini, parameters, date_format, default_retrials, default_job_type, wrapper_jobs={}, new=True, run_only_members=[], show_log=True, monitor=False, force=False, create=False)

Creates all jobs needed for the current workflow.

Parameters:

• create (bool)

• force (bool)

• as_conf (AutosubmitConfig) – AutosubmitConfig object

• date_list (list) – list of dates

• member_list (list) – list of members

• num_chunks (int) – number of chunks

• chunk_ini (int) – initial chunk

• parameters (dict) – parameters

• date_format (str) – date format ( D/M/Y )

• default_retrials (int) – default number of retrials

• default_job_type (str) – default job type

• wrapper_jobs (dict) – wrapper jobs

• new (bool) – new

• run_only_members (list) – run only members

• show_log (bool) – show log

• monitor (bool) – monitor

get_active(platform=None, wrapper=False)

Returns a list of active jobs (In platforms queue + Ready).

Parameters:

• wrapper

• platform (HPCPlatform) – job platform

Returns:

active jobs

Return type:

list

get_all(platform=None, wrapper=False)

Returns a list of all jobs.

Parameters:

• wrapper

• platform (HPCPlatform) – job platform

Returns:

all jobs

Return type:

list

get_chunk_list()

Get inner chunk list.

Returns:

chunk list

Return type:

list

get_completed(platform=None, wrapper=False)

Returns a list of completed jobs

Parameters:

• wrapper

• platform (HPCPlatform) – job platform

Returns:

completed jobs

Return type:

list

get_completed_failed_without_logs() → list[Job]

Returns a list of completed or failed jobs without updated logs.

Returns:

List of completed and failed jobs without updated logs.

Return type:

List[Job]

get_date_list()

Get inner date list.

Returns:

date list

Return type:

list

get_delayed(platform=None)

Returns a list of delayed jobs.

Parameters:

platform (HPCPlatform) – job platform

Returns:

delayed jobs

Return type:

list

get_failed(platform=None, wrapper=False)

Returns a list of failed jobs.

Parameters:

• wrapper

• platform (HPCPlatform) – job platform

Returns:

failed jobs

Return type:

list

get_held_jobs(platform=None)

Returns a list of jobs in the platforms (Held).

Parameters:

platform (HPCPlatform) – job platform

Returns:

jobs in platforms

Return type:

list

get_in_queue(platform=None, wrapper=False)

Returns a list of jobs in the platforms (Submitted, Running, Queuing, Unknown,Held).

Parameters:

• wrapper

• platform (HPCPlatform) – job platform

Returns:

jobs in platforms

Return type:

list

get_in_queue_grouped_id(platform: Platform) → dict[int, list[Job]]

Gets the queued jobs, grouped by their IDs. Same ID, same dictionary key. Each dictionary value is a list.

Parameters:

platform – Optional platform, if None, will fetch all jobs without a platform.

Returns:

A dictionary where job IDs are keys, and the values are lists with the jobs objects.

get_job_by_name(name: str) → Job | None

Returns the job that its name matches parameter name.

Parameters:

name (str) – name to look for

Returns:

found job

Return type:

job

get_job_list()

Get inner job list.

Returns:

job list

Return type:

list

get_job_related(select_jobs_by_name='', select_all_jobs_by_section='', filter_jobs_by_section='', two_step_start=True)

Parameters:

• two_step_start

• select_jobs_by_name – job name

• select_all_jobs_by_section – section name

• filter_jobs_by_section – section, date , member? , chunk?

Returns:

jobs_list names

Return type:

list

get_jobs_by_section(section_list: list, banned_jobs: list | None = None, get_only_non_completed: bool = False) → list

Get jobs by section.

This method filters jobs based on the provided section list and banned jobs list. It can also filter out completed jobs if specified.

Parameters: section_list (list): List of sections to filter jobs by. banned_jobs (list, optional): List of jobs names to exclude from the result. Defaults to an empty list. get_only_non_completed (bool, optional): If True, only non-completed jobs are included. Defaults to False.

Returns: list: List of jobs that match the criteria.

get_member_list()

Get inner member list.

Returns:

member list

Return type:

list

get_ordered_jobs_by_date_member(section)

Get the dictionary of jobs ordered according to wrapper’s expression divided by date and member.

Returns:

jobs ordered divided by date and member

Return type:

dict

get_packages_persistence(reset=False) → JobPackagePersistence

Return (lazily creating if needed) the JobPackagePersistence for this experiment.

get_prepared(platform=None)

Returns a list of prepared jobs.

Parameters:

platform (HPCPlatform) – job platform

Returns:

prepared jobs

Return type:

list

get_queuing(platform=None, wrapper=False)

Returns a list of jobs queuing.

Parameters:

• wrapper

• platform (HPCPlatform) – job platform

Returns:

queuedjobs

Return type:

list

get_ready(platform=None, hold=False, wrapper=False)

Returns a list of ready jobs.

Parameters:

• wrapper

• hold

• platform (HPCPlatform) – job platform

Returns:

ready jobs

Return type:

list

get_running(platform=None, wrapper=False)

Returns a list of jobs running.

Parameters:

• wrapper

• platform (HPCPlatform) – job platform

Returns:

running jobs

Return type:

list

get_submitted(platform=None, hold=False, wrapper=False)

Returns a list of submitted jobs.

Parameters:

• wrapper

• hold

• platform (HPCPlatform) – job platform

Returns:

submitted jobs

Return type:

list

get_uncompleted(platform=None, wrapper=False)

Returns a list of completed jobs.

Parameters:

• wrapper

• platform (HPCPlatform) – job platform

Returns:

completed jobs

Return type:

list

get_unknown(platform=None, wrapper=False)

Returns a list of jobs on unknown state.

Parameters:

• wrapper

• platform (HPCPlatform) – job platform

Returns:

unknown state jobs

Return type:

list

get_unsubmitted(platform=None, wrapper=False)

Returns a list of unsubmitted jobs.

Parameters:

• wrapper

• platform (HPCPlatform) – job platform

Returns:

all jobs

Return type:

list

get_waiting(platform=None, wrapper=False)

Returns a list of jobs waiting.

Parameters:

• wrapper

• platform (HPCPlatform) – job platform

Returns:

waiting jobs

Return type:

list

get_waiting_remote_dependencies(platform_type='slurm')

Returns a list of jobs waiting on slurm scheduler.

Parameters:

platform_type (str) – platform type

Returns:

waiting jobs

Return type:

list

is_wrapper_still_running(job: Job) → bool

Check if the job’s wrapper is still active.

Parameters:

job – Job to check.

Returns:

True if the wrapper is still running, False otherwise.

load(create=False, backup=False)

Recreates a stored job list from the persistence.

Returns:

loaded job list object

Return type:

JobList

load_wrappers(preview: bool = False) → None

Load wrapper jobs and their inner jobs from the database.

Populates job_package_map and packages_dict with the loaded wrapper and inner-job data.

Parameters:

preview – If True, load from the preview tables.

property parameters

List of parameters common to all jobs

Returns:

parameters

Return type:

dict

print_with_status(status_change: dict[Any, Any] | None = None, nocolor=False, existing_list=None) → str

Returns the string representation of the dependency tree of the Job List.

Parameters:

• status_change (dict) – List of changes in the list, supplied in set status

• nocolor (Boolean) – True if the result should not include color codes

• existing_list (List of Job Objects) – External List of Jobs that will be printed, this excludes the inner list of jobs.

Returns:

String representation of the Job List

Return type:

String

recover_all_completed_jobs_from_exp_history(platform: Platform | None = None) → set[str]

Recover all completed jobs from experiment history

Parameters:

platform – Platform to filter by.

Returns:

Set of completed job names.

recover_last_data(finished_jobs: list[Job] | None = None) → None

Recover job IDs and log names for completed, failed, and skipped jobs from experiment history.

Parameters:

finished_jobs – Optional list of finished Job objects to recover data for.

Returns:

None

Return type:

None

recover_logs() → bool

Update jobs’ log recovered status.

Iterate over the current job list and mark jobs whose stdout/stderr logs have been recovered.

remove_rerun_only_jobs() → None

Removes all jobs to be run only in reruns.

rerun(job_list_unparsed, as_conf, monitor=False)

Updates job list to rerun the jobs specified by a job list.

Parameters:

• job_list_unparsed (str) – list of jobs to rerun

• as_conf (AutosubmitConfig) – experiment configuration

• monitor (bool) – if True, the job list will be monitored

static retrieve_packages(BasicConfig, expid, current_jobs=None)

Retrieves dictionaries that map the collection of packages in the experiment

Parameters:

• BasicConfig (Configuration Object) – Basic configuration

• expid (String) – Experiment ID

• current_jobs (list) – list of names of current jobs

Returns:

job to package, package to job, package to package_id, package to symbol

Return type:

Dictionary(Job Object, Package), Dictionary(Package, List of Job Objects), Dictionary(String, String), Dictionary(String, String)

static retrieve_times(status_code, name, tmp_path, make_exception=False, job_times=None, seconds=False, job_data_collection: JobData | None = None) → None | JobRow

Retrieve job timestamps from database.

Parameters:

• status_code (Integer) – Code of the Status of the job

• name (String) – Name of the job

• tmp_path (String) – Path to the tmp folder of the experiment

• make_exception (Boolean) – flag for testing purposes

• job_times (Dictionary Key: job name, Value: 5-tuple (submit time, start time, finish time, status, detail id)) – Detail from as_times.job_times for the experiment

• seconds (bool) – seconds

• job_data_collection (JobData) – Collection of Jobs

Returns:

minutes the job has been queuing, minutes the job has been running, and the text that represents it

Return type:

JobRow

save()

Persists the job list.

save_wrapper_info(wrapper_job)

Save wrapper job info to the database for all wrapper jobs in the job list.

save_wrappers(submitted_scripts: dict, as_conf: AutosubmitConfig, inspect: bool = False) → None

Save wrapper jobs in the job list and the packages dict.

Parameters:

• submitted_scripts – Nested dict {section: {name: package}} of packages produced by the wrapper packager.

• as_conf – Experiment configuration object.

• inspect – If True, persist to the preview tables (-cw flag).

sort_by_id()

Returns a list of jobs sorted by id.

Returns:

jobs sorted by ID

Return type:

list

sort_by_name()

Returns a list of jobs sorted by name.

Returns:

jobs sorted by name

Return type:

list

sort_by_status()

Returns a list of jobs sorted by status.

Returns:

job sorted by status

Return type:

list

sort_by_type()

Returns a list of jobs sorted by type.

Returns:

job sorted by type

Return type:

list

split_by_platform()

Splits the job list by platform name

Returns:

job list per platform

Return type:

dict

update_from_file(store_change=True)

Updates jobs list on the fly from and update file

Parameters:

store_change – if True, renames the update file to avoid reloading it at the next iteration

update_genealogy()

When we have created the job list, every type of job is created. Update genealogy remove jobs that have no templates.

update_list(as_conf: AutosubmitConfig, store_change: bool = True, fromSetStatus: bool = False, submitter: object | None = None, first_time: bool = False) → bool

Updates job list, resetting failed jobs and changing to READY all WAITING jobs with all parents COMPLETED

Parameters:

• first_time

• submitter

• fromSetStatus

• store_change

• as_conf (AutosubmitConfig) – autosubmit config object

Returns:

True if job status were modified, False otherwise

Return type:

bool