ndcctools.work_queue.RemoteTask Class Reference
Inheritance diagram for ndcctools.work_queue.RemoteTask:
Public Member Functions | |
| def | __init__ (self, fn, coprocess, *args, **kwargs) |
| Create a new remote task specification. More... | |
| def | specify_fn_args (self, args=[], kwargs={}) |
| Specify function arguments. More... | |
| def | specify_exec_method (self, remote_task_exec_method) |
| Specify how the remote task should execute. More... | |
 Public Member Functions inherited from ndcctools.work_queue.Task | |
| def | __init__ (self, command) |
| Create a new task specification. More... | |
| def | clone (self) |
| Return a copy of this task. More... | |
| def | specify_command (self, command) |
| Set the command to be executed by the task. More... | |
| def | specify_coprocess (self, coprocess) |
| Set the coprocess at the worker that should execute the task's command. More... | |
| def | specify_algorithm (self, algorithm) |
| Set the worker selection algorithm for task. More... | |
| def | specify_tag (self, tag) |
| Attach a user defined logical name to the task. More... | |
| def | specify_category (self, name) |
| Label the task with the given category. More... | |
| def | specify_feature (self, name) |
| Label the task with the given user-defined feature. More... | |
| def | specify_file (self, local_name, remote_name=None, type=None, flags=None, cache=None, failure_only=None) |
| Add a file to the task. More... | |
| def | specify_url (self, url, remote_name, type=None, flags=None, cache=None, failure_only=None) |
| Add a url to the task which will be provided as an input file. More... | |
| def | specify_file_command (self, cmd, remote_name, type=None, flags=None, cache=None, failure_only=None) |
| Add an input file produced by a Unix shell command. More... | |
| def | specify_file_piece (self, local_name, remote_name=None, start_byte=0, end_byte=0, type=None, flags=None, cache=None, failure_only=None) |
| Add a file piece to the task. More... | |
| def | specify_input_file (self, local_name, remote_name=None, flags=None, cache=None) |
| Add a input file to the task. More... | |
| def | specify_output_file (self, local_name, remote_name=None, flags=None, cache=None, failure_only=None) |
| Add a output file to the task. More... | |
| def | specify_directory (self, local_name, remote_name=None, type=None, flags=None, recursive=False, cache=None, failure_only=None) |
| Add a directory to the task. More... | |
| def | specify_buffer (self, buffer, remote_name, flags=None, cache=None) |
| Add an input bufer to the task. More... | |
| def | specify_snapshot_file (self, filename) |
| When monitoring, indicates a json-encoded file that instructs the monitor to take a snapshot of the task resources. More... | |
| def | specify_max_retries (self, max_retries) |
| Indicate the number of times the task should be retried. More... | |
| def | specify_cores (self, cores) |
| Indicate the number of cores required by this task. More... | |
| def | specify_memory (self, memory) |
| Indicate the memory (in MB) required by this task. More... | |
| def | specify_disk (self, disk) |
| Indicate the disk space (in MB) required by this task. More... | |
| def | specify_gpus (self, gpus) |
| Indicate the number of GPUs required by this task. More... | |
| def | specify_priority (self, priority) |
| Indicate the the priority of this task (larger means better priority, default is 0). More... | |
| def | specify_environment_variable (self, name, value=None) |
| Set this environment variable before running the task. More... | |
| def | specify_monitor_output (self, directory) |
| Set a name for the resource summary output directory from the monitor. More... | |
| def | tag (self) |
| Get the user-defined logical name for the task. More... | |
| def | category (self) |
| Get the category name for the task. More... | |
| def | command (self) |
| Get the shell command executed by the task. More... | |
| def | priority (self) |
| Get the priority of the task. More... | |
| def | algorithm (self) |
| Get the algorithm for choosing worker to run the task. More... | |
| def | std_output (self) |
| Get the standard output of the task. More... | |
| def | output (self) |
| Get the standard output of the task. More... | |
| def | id (self) |
| Get the task id number. More... | |
| def | return_status (self) |
| Get the exit code of the command executed by the task. More... | |
| def | result (self) |
| Get the result of the task as an integer code, such as successful, missing file, etc. More... | |
| def | result_str (self) |
| Return a string that explains the result of a task. More... | |
| def | total_submissions (self) |
| Get the number of times the task has been resubmitted internally. More... | |
| def | exhausted_attempts (self) |
| Get the number of times the task has been failed given resource exhaustion. More... | |
| def | host (self) |
| Get the address and port of the host on which the task ran. More... | |
| def | hostname (self) |
| Get the name of the host on which the task ran. More... | |
| def | submit_time (self) |
| Get the time at which this task was submitted. More... | |
| def | finish_time (self) |
| Get the time at which this task was finished. More... | |
| def | total_cmd_exhausted_execute_time (self) |
| Get the total time the task executed and failed given resource exhaustion. More... | |
| def | app_delay (self) |
| Get the time spent in upper-level application (outside of work_queue_wait). More... | |
| def | send_input_start (self) |
| Get the time at which the task started to transfer input files. More... | |
| def | send_input_finish (self) |
| Get the time at which the task finished transferring input files. More... | |
| def | execute_cmd_start (self) |
| The time at which the task began. More... | |
| def | execute_cmd_finish (self) |
| Get the time at which the task finished (discovered by the manager). More... | |
| def | receive_output_start (self) |
| Get the time at which the task started to transfer output files. More... | |
| def | receive_output_finish (self) |
| Get the time at which the task finished transferring output files. More... | |
| def | total_bytes_received (self) |
| Get the number of bytes received since task started receiving input data. More... | |
| def | total_bytes_sent (self) |
| Get the number of bytes sent since task started sending input data. More... | |
| def | total_bytes_transferred (self) |
| Get the number of bytes transferred since task started transferring input data. More... | |
| def | total_transfer_time (self) |
| Get the time comsumed in microseconds for transferring total_bytes_transferred. More... | |
| def | cmd_execution_time (self) |
| Time spent in microseconds for executing the command until completion on a single worker. More... | |
| def | total_cmd_execution_time (self) |
| Accumulated time spent in microseconds for executing the command on any worker, regardless of whether the task finished (i.e., this includes time running on workers that disconnected). More... | |
| def | resources_measured (self) |
| Get the resources measured for the task execution if resource monitoring is enabled. More... | |
| def | limits_exceeded (self) |
| Get the resources the task exceeded. More... | |
| def | resources_requested (self) |
| Get the resources the task requested to run. More... | |
| def | resources_allocated (self) |
| Get the resources allocated to the task in its latest attempt. More... | |
Detailed Description
Python RemoteTask object
This class is used to create a task that will execute on a worker running a coprocess
Constructor & Destructor Documentation
◆ __init__()
| def ndcctools.work_queue.RemoteTask.__init__ | ( | self, | |
| fn, | |||
| coprocess, | |||
| * | args, | ||
| ** | kwargs | ||
| ) |
Create a new remote task specification.
- Parameters
-
self Reference to the current remote task object. fn The name of the function to be executed on the coprocess coprocess The name of the coprocess which has the function you wish to execute. The coprocess should have a name() method that returns this args positional arguments used in function to be executed by task. Can be mixed with kwargs kwargs keyword arguments used in function to be executed by task.
Member Function Documentation
◆ specify_fn_args()
| def ndcctools.work_queue.RemoteTask.specify_fn_args | ( | self, | |
args = [], |
|||
kwargs = {} |
|||
| ) |
Specify function arguments.
Accepts arrays and dictionarys. This overrides any arguments passed during task creation
- Parameters
-
self Reference to the current remote task object args An array of positional args to be passed to the function kwargs A dictionary of keyword arguments to be passed to the function
◆ specify_exec_method()
| def ndcctools.work_queue.RemoteTask.specify_exec_method | ( | self, | |
| remote_task_exec_method | |||
| ) |
Specify how the remote task should execute.
- Parameters
-
self Reference to the current remote task object remote_task_exec_method Can be one of "fork", "direct", or "thread". Fork creates a child process to execute the function, direct has the worker directly call the function, and thread spawns a thread to execute the function
The documentation for this class was generated from the following file:
- work_queue.py
