Job#
The job module provides classes that run analyses on a remote server.
If a JobBuilder is attached to a Public API Dataset, the analyses will be ran through jobs, with writting/submitting of PBS files.
- class osekit.utils.job.Job(script_path: Path, script_args: dict | None = None, config: JobConfig | None = None, name: str = 'osekit_analysis', output_folder: Path | None = None)#
Job that concerns a specific analysis.
Initialize a Job.
Parameters#
- script_path: Path
Path to the script file the job must run.
- script_args: dict | None
Additional arguments to pass to the script file.
- config: JobConfig | None
Optional configuration to pass to the server request.
- name: str
Name of the job.
- output_folder: Path | None
Folder in which the output files (.out and .err) will be written.
- property job_id: str#
Job ID.
- property job_info: dict | None#
Information about the job as returned by a qstat request.
- property mem: str#
Maximum amount of physical memory used by the job.
- property name: str#
Name of the job.
- property nb_nodes: int#
Number of nodes on which the job runs.
- property ncpus: int#
Number of total cores used per node.
- property output_folder: Path | None#
Folder in which the output files (.out and .err) will be written.
- property path: Path | None#
Path of the job file.
- progress() None#
Bring the job to the next state.
- property queue: Literal['omp', 'mpi']#
Queue in which the job will be submitted.
- property script_args: dict#
Additional arguments to pass to the script file.
- property script_path: Path#
Path to the script file the job must run.
- property status: JobStatus#
Status of the job.
UNPREPARED: The job file hasn’t been written yet. PREPARED: The job file has been written but not submitted. QUEUED: The job has been queued. RUNNING: The job is currently running. COMPLETED: The job has been completed.
- submit_pbs() None#
Submit the PBS file of the job to a PBS queueing system.
- update_info() None#
Request info about the job and update it.
- update_status() JobStatus#
Request info about the job and update its status.
Returns#
- JobStatus:
The updated status of the job.
- property venv_activate_script: str#
Bash script used for activating the conda virtual environment.
- property venv_name: str#
Name of the conda virtual environment in which the job is running.
- property walltime: Timedelta#
Maximum amount of real time during which the job can be running.
- property walltime_str: str#
String representation of the walltime.
- class osekit.utils.job.JobBuilder(config: ~osekit.utils.job.JobConfig = <class 'osekit.utils.job.JobConfig'>)#
Class that should be attached to a Public API Dataset for working with jobs.
If a Dataset has a JobBuilder, it will use it to run analyses through jobs.
Initialize a JobBuilder instance.
Parameters#
- config: JobConfig
Config of the jobs built by this job builder.
- create_job(script_path: Path, script_args: dict | None = None, name: str = 'osekit_analysis', output_folder: Path | None = None) None#
Create a new job instance.
Parameters#
- script_path: Path
Path to the script file the job must run.
- script_args: dict | None
Additional arguments to pass to the script file.
- name: str
Name of the job.
- output_folder: Path | None
Folder in which the output files (.out and .err) will be written.
- submit_pbs() None#
Submit all repared jobs to the PBS queueing system.
- class osekit.utils.job.JobConfig(nb_nodes: int = 1, ncpus: int = 2, mem: str = '8gb', walltime: str | Timedelta = '01:00:00', venv_name: str = 'osmose', queue: Literal['omp', 'mpi'] = 'omp')#
Config used for creating a job.
Parameters#
- nb_nodes: int
Number of nodes on which the job runs.
- ncpus: int
Number of total cores used per node.
- mem: str
Maximum amount of physical memory used by the job.
- walltime: str | Timedelta
Maximum amount of real time during which the job can be running.
- venv_name: str
Name (or path) of the conda virtual environment in which the job is running.
- queue: Literal[“omp”, “mpi”]
Queue in which the job will be submitted.