autosubmit.platform¶

class autosubmit.platforms.ecplatform.EcPlatform(expid, name, config, scheduler)¶

Bases: autosubmit.platforms.paramiko_platform.ParamikoPlatform

Class to manage queues with ecaccess

Parameters:	expid (str) – experiment’s identifier scheduler (str (pbs, loadleveler)) – scheduler to use

connect()¶

In this case, it does nothing because connection is established for each command

Returns:	True
Return type:	bool

delete_file(filename)¶

Deletes a file from this platform

Parameters:	filename (str) – file name
Returns:	True if successful or file does no exists
Return type:	bool

get_checkjob_cmd(job_id)¶

Returns command to check job status on remote platforms

Parameters:	job_id – id of job to check job_id – int
Returns:	command to check job status
Return type:	str

get_file(filename, must_exist=True, relative_path='', ignore_log=False, wrapper_failed=False)¶

Copies a file from the current platform to experiment’s tmp folder

Parameters:	filename (str) – file name must_exist (bool) – If True, raises an exception if file can not be copied relative_path (str) – path inside the tmp folder
Returns:	True if file is copied successfully, false otherwise
Return type:	bool

get_mkdir_cmd()¶

Gets command to create directories on HPC

Returns:	command to create directories on HPC
Return type:	str

get_ssh_output()¶

Gets output from last command executed

Returns:	output from last command
Return type:	str

get_submit_cmd(job_script, job, hold=False, export='')¶

Get command to add job to scheduler

Parameters:	job_type – job_script – path to job script job_script – str hold – submit a job in a held status hold – boolean export – modules that should’ve downloaded export – string
Returns:	command to submit job to platforms
Return type:	str

get_submitted_job_id(output)¶: Parses submit command output to extract job id :param output: output to parse :type output: str :return: job id :rtype: str

jobs_in_queue()¶

Returns empty list because ecacces does not support this command

Returns:	empty list
Return type:	list

move_file(src, dest, must_exist=False)¶: Moves a file on the platform (includes .err and .out) :param src: source name :type src: str :param dest: destination name :param must_exist: ignore if file exist or not :type dest: str

parse_job_output(output)¶

Parses check job command output so it can be interpreted by autosubmit

Parameters:	output (str) – output to parse
Returns:	job status
Return type:	str

restore_connection()¶

In this case, it does nothing because connection is established for each command

Returns:	True
Return type:	bool

send_command(command, ignore_log=False)¶

Sends given command to HPC

Parameters:	command (str) – command to send
Returns:	True if executed, False if failed
Return type:	bool

send_file(filename, check=True)¶: Sends a local file to the platform :param filename: name of the file to send :type filename: str

test_connection()¶

In this case, it does nothing because connection is established for each command

Returns:	True
Return type:	bool

update_cmds()¶: Updates commands for platforms

class autosubmit.platforms.lsfplatform.LsfPlatform(expid, name, config)¶

Bases: autosubmit.platforms.paramiko_platform.ParamikoPlatform

Class to manage jobs to host using LSF scheduler

Parameters:	expid (str) – experiment’s identifier

get_checkjob_cmd(job_id)¶

Returns command to check job status on remote platforms

Parameters:	job_id – id of job to check job_id – int
Returns:	command to check job status
Return type:	str

get_mkdir_cmd()¶

Gets command to create directories on HPC

Returns:	command to create directories on HPC
Return type:	str

get_submit_cmd(job_script, job, export='')¶

Get command to add job to scheduler

Parameters:	job_type – job_script – path to job script job_script – str hold – submit a job in a held status hold – boolean export – modules that should’ve downloaded export – string
Returns:	command to submit job to platforms
Return type:	str

get_submitted_job_id(output)¶: Parses submit command output to extract job id :param output: output to parse :type output: str :return: job id :rtype: str

parse_job_output(output)¶

Parses check job command output so it can be interpreted by autosubmit

Parameters:	output (str) – output to parse
Returns:	job status
Return type:	str

update_cmds()¶: Updates commands for platforms

class autosubmit.platforms.pbsplatform.PBSPlatform(expid, name, config, version)¶

Bases: autosubmit.platforms.paramiko_platform.ParamikoPlatform

Class to manage jobs to host using PBS scheduler

Parameters:	expid (str) – experiment’s identifier version (str) – scheduler version

get_checkjob_cmd(job_id)¶

Returns command to check job status on remote platforms

Parameters:	job_id – id of job to check job_id – int
Returns:	command to check job status
Return type:	str

get_mkdir_cmd()¶

Gets command to create directories on HPC

Returns:	command to create directories on HPC
Return type:	str

get_submit_cmd(job_script, job, export='')¶

Get command to add job to scheduler

Parameters:	job_type – job_script – path to job script job_script – str hold – submit a job in a held status hold – boolean export – modules that should’ve downloaded export – string
Returns:	command to submit job to platforms
Return type:	str

get_submitted_job_id(output)¶: Parses submit command output to extract job id :param output: output to parse :type output: str :return: job id :rtype: str

parse_job_output(output)¶

Parses check job command output so it can be interpreted by autosubmit

Parameters:	output (str) – output to parse
Returns:	job status
Return type:	str

update_cmds()¶: Updates commands for platforms

class autosubmit.platforms.sgeplatform.SgePlatform(expid, name, config)¶

Bases: autosubmit.platforms.paramiko_platform.ParamikoPlatform

Class to manage jobs to host using SGE scheduler

Parameters:	expid (str) – experiment’s identifier

connect()¶

In this case, it does nothing because connection is established for each command

Returns:	True
Return type:	bool

get_checkjob_cmd(job_id)¶

Returns command to check job status on remote platforms

Parameters:	job_id – id of job to check job_id – int
Returns:	command to check job status
Return type:	str

get_mkdir_cmd()¶

Gets command to create directories on HPC

Returns:	command to create directories on HPC
Return type:	str

get_submit_cmd(job_script, job, export='')¶

Get command to add job to scheduler

Parameters:	job_type – job_script – path to job script job_script – str hold – submit a job in a held status hold – boolean export – modules that should’ve downloaded export – string
Returns:	command to submit job to platforms
Return type:	str

get_submitted_job_id(output)¶: Parses submit command output to extract job id :param output: output to parse :type output: str :return: job id :rtype: str

parse_job_output(output)¶

Parses check job command output so it can be interpreted by autosubmit

Parameters:	output (str) – output to parse
Returns:	job status
Return type:	str

restore_connection()¶

In this case, it does nothing because connection is established for each command

Returns:	True
Return type:	bool

test_connection()¶

In this case, it does nothing because connection is established for each command

Returns:	True
Return type:	bool

update_cmds()¶: Updates commands for platforms

class autosubmit.platforms.slurmplatform.SlurmPlatform(expid, name, config)¶

Bases: autosubmit.platforms.paramiko_platform.ParamikoPlatform

Class to manage jobs to host using SLURM scheduler

Parameters:	expid (str) – experiment’s identifier

get_checkAlljobs_cmd(jobs_id)¶

Returns command to check jobs status on remote platforms

Parameters:	jobs_id – id of jobs to check job_id – str
Returns:	command to check job status
Return type:	str

get_checkjob_cmd(job_id)¶

Returns command to check job status on remote platforms

Parameters:	job_id – id of job to check job_id – int
Returns:	command to check job status
Return type:	str

get_mkdir_cmd()¶

Gets command to create directories on HPC

Returns:	command to create directories on HPC
Return type:	str

get_submit_cmd(job_script, job, hold=False, export='')¶

Get command to add job to scheduler

Parameters:	job_type – job_script – path to job script job_script – str hold – submit a job in a held status hold – boolean export – modules that should’ve downloaded export – string
Returns:	command to submit job to platforms
Return type:	str

get_submitted_job_id(outputlines)¶: Parses submit command output to extract job id :param output: output to parse :type output: str :return: job id :rtype: str

parse_Alljobs_output(output, job_id)¶: Parses check jobs command output so it can be interpreted by autosubmit :param output: output to parse :param job_id: select the job to parse :type output: str :return: job status :rtype: str

parse_job_finish_data(output, packed)¶

Parses the context of the sacct query to SLURM for a single job. Only normal jobs return submit, start, finish, joules, ncpus, nnodes.

When a wrapper has finished, capture finish time.

Parameters:	output (str) – The sacct output job_id (int) – Id in SLURM for the job packed (bool) – true if job belongs to package
Returns:	submit, start, finish, joules, ncpus, nnodes, detailed_data
Return type:	int, int, int, int, int, int, json object (str)

parse_job_output(output)¶

Parses check job command output so it can be interpreted by autosubmit

Parameters:	output (str) – output to parse
Returns:	job status
Return type:	str

submit_Script(hold=False)¶

Sends a Submit file Script, execute it in the platform and retrieves the Jobs_ID of all jobs at once.

Parameters:	job (autosubmit.job.job.Job) – job object
Returns:	job id for submitted jobs
Return type:	list(str)

update_cmds()¶: Updates commands for platforms

class autosubmit.platforms.locplatform.LocalPlatform(expid, name, config)¶

Bases: autosubmit.platforms.paramiko_platform.ParamikoPlatform

Class to manage jobs to localhost

Parameters:	expid (str) – experiment’s identifier

check_file_exists(src, wrapper_failed=False)¶: Moves a file on the platform :param src: source name :type src: str :param dest: destination name :param must_exist: ignore if file exist or not :type dest: str

connect()¶

Creates ssh connection to host

Returns:	True if connection is created, False otherwise
Return type:	bool

delete_file(filename, del_cmd=False)¶

Deletes a file from this platform

Parameters:	filename (str) – file name
Returns:	True if successful or file does no exists
Return type:	bool

get_checkjob_cmd(job_id)¶

Returns command to check job status on remote platforms

Parameters:	job_id – id of job to check job_id – int
Returns:	command to check job status
Return type:	str

get_file(filename, must_exist=True, relative_path='', ignore_log=False, wrapper_failed=False)¶

Copies a file from the current platform to experiment’s tmp folder

Parameters:	filename (str) – file name must_exist (bool) – If True, raises an exception if file can not be copied relative_path (str) – path inside the tmp folder
Returns:	True if file is copied successfully, false otherwise
Return type:	bool

get_logs_files(exp_id, remote_logs)¶

Overriding the parent’s implementation. Do nothing because the log files are already in the local platform (redundancy).

Parameters:	exp_id (str) – experiment id remote_logs ((str, str)) – names of the log files

get_mkdir_cmd()¶

Gets command to create directories on HPC

Returns:	command to create directories on HPC
Return type:	str

get_ssh_output()¶

Gets output from last command executed

Returns:	output from last command
Return type:	str

get_submit_cmd(job_script, job, hold=False, export='')¶

Get command to add job to scheduler

Parameters:	job_type – job_script – path to job script job_script – str hold – submit a job in a held status hold – boolean export – modules that should’ve downloaded export – string
Returns:	command to submit job to platforms
Return type:	str

get_submitted_job_id(output)¶: Parses submit command output to extract job id :param output: output to parse :type output: str :return: job id :rtype: str

move_file(src, dest, must_exist=False)¶: Moves a file on the platform (includes .err and .out) :param src: source name :type src: str :param dest: destination name :param must_exist: ignore if file exist or not :type dest: str

parse_job_output(output)¶

Parses check job command output so it can be interpreted by autosubmit

Parameters:	output (str) – output to parse
Returns:	job status
Return type:	str

send_command(command, ignore_log=False)¶

Sends given command to HPC

Parameters:	command (str) – command to send
Returns:	True if executed, False if failed
Return type:	bool

send_file(filename)¶: Sends a local file to the platform :param filename: name of the file to send :type filename: str

test_connection()¶: Test if the connection is still alive, reconnect if not.

update_cmds()¶: Updates commands for platforms