ndcctools.taskvine.dask_executor.PythonTaskDask Class Reference
Inheritance diagram for ndcctools.taskvine.dask_executor.PythonTaskDask:
Public Member Functions | |
| def | __init__ (self, m, dag, key, sexpr, *category=None, environment=None, extra_files=None, env_vars=None, retries=5, worker_transfers=False, wrapper=None) |
| Create a new PythonTaskDask. More... | |
 Public Member Functions inherited from ndcctools.taskvine.task.PythonTask | |
| def | __init__ (self, func, *args, **kwargs) |
| Creates a new python task. More... | |
| def | submit_finalize (self) |
| Finalizes the task definition once the manager that will execute is run. More... | |
| def | enable_temp_output (self) |
| Marks the output of this task to stay at the worker. More... | |
| def | set_output_cache (self, cache=False) |
| Set the cache behavior for the output of the task. More... | |
| def | output_file (self) |
| Returns the ndcctools.taskvine.file.File object that represents the output of this task. More... | |
| def | output (self) |
| returns the result of a python task as a python variable More... | |
| def | disable_output_serialization (self) |
| Disables serialization of results to disk when writing to a file for transmission. More... | |
| Public Member Functions inherited from ndcctools.taskvine.task.Task | |
| def | __init__ (self, command, **task_info) |
| Create a new task specification. More... | |
| def | clone (self) |
| Return a copy of this task. More... | |
| def | set_command (self, command) |
| Set the command to be executed by the task. More... | |
| def | set_library_required (self, library) |
| Set the name of the library at the worker that should execute the task's command. More... | |
| def | get_libray_required (self) |
| Get the name of the library at the worker that should execute the task's command. More... | |
| def | needs_library (self, library) |
| Deprecated, see set_library_required. More... | |
| def | set_library_provided (self, library_name) |
| Set the library name provided by this task. More... | |
| def | get_libray_provided (self) |
| Get the name of the library at the worker that should execute the task's command. More... | |
| def | provides_library (self, library) |
| Deprecated, see set_library_provided. More... | |
| def | set_function_slots (self, nslots) |
| Set the number of concurrent functions a library can run. More... | |
| def | set_scheduler (self, scheduler) |
| Set the worker selection scheduler for task. More... | |
| def | set_tag (self, tag) |
| Attach a user defined logical name to the task. More... | |
| def | set_category (self, name) |
| Label the task with the given category. More... | |
| def | add_feature (self, name) |
| Label the task with the given user-defined feature. More... | |
| def | add_input (self, file, remote_name, strict_input=False, mount_symlink=False) |
| Add any input object to a task. More... | |
| def | add_output (self, file, remote_name, watch=False, failure_only=None, success_only=None) |
| Add any output object to a task. More... | |
| def | set_snapshot_file (self, filename) |
| When monitoring, indicates a json-encoded file that instructs the monitor to take a snapshot of the task resources. More... | |
| def | add_starch_package (self, file) |
| Add a Starch package as an execution context. More... | |
| def | add_poncho_package (self, file) |
| Add a Poncho package as an execution context. More... | |
| def | add_execution_context (self, f) |
| Adds an execution context to the task. More... | |
| def | set_retries (self, max_retries) |
| Indicate the number of times the task should be retried. More... | |
| def | set_max_forsaken (self, max_forsaken) |
| Indicate the number of times the task can be returned to the manager without being executed. More... | |
| def | set_cores (self, cores) |
| Indicate the number of cores required by this task. More... | |
| def | set_memory (self, memory) |
| Indicate the memory (in MB) required by this task. More... | |
| def | set_disk (self, disk) |
| Indicate the disk space (in MB) required by this task. More... | |
| def | set_gpus (self, gpus) |
| Indicate the number of GPUs required by this task. More... | |
| def | set_priority (self, priority) |
| Indicate the the priority of this task (larger means better priority, default is 0). More... | |
| def | set_env_var (self, name, value=None) |
| Set this environment variable before running the task. More... | |
| def | set_monitor_output (self, directory) |
| Set a name for the resource summary output directory from the monitor. More... | |
| def | tag (self) |
| Get the user-defined logical name for the task. More... | |
| def | category (self) |
| Get the category name for the task. More... | |
| def | command (self) |
| Get the shell command executed by the task. More... | |
| def | std_output (self) |
| Get the standard output of the task. More... | |
| def | id (self) |
| Get the task id number. More... | |
| def | exit_code (self) |
| Get the exit code of the command executed by the task. More... | |
| def | result (self) |
| Return a string that explains the result of a task. More... | |
| def | completed (self) |
| Return True if task executed and its command terminated normally. More... | |
| def | successful (self) |
| Return True if task executed successfully, (i.e. More... | |
| def | get_metric (self, name) |
| Return various integer performance metrics about a completed task. More... | |
| def | addrport (self) |
| Get the address and port of the host on which the task ran. More... | |
| def | hostname (self) |
| Get the address and port of the host on which the task ran. More... | |
| def | resources_measured (self) |
| Get the resources measured for the task execution if resource monitoring is enabled. More... | |
| def | limits_exceeded (self) |
| Get the resources the task exceeded. More... | |
| def | resources_requested (self) |
| Get the resources the task requested to run. More... | |
| def | resources_allocated (self) |
| Get the resources allocated to the task in its latest attempt. More... | |
| def | add_nopen (self, manager) |
| Adds inputs for nopen library and rules file and sets LD_PRELOAD. More... | |
Detailed Description
Internal class representing a Dask task to be executed, materialized as a special case of a TaskVine PythonTask.
Constructor & Destructor Documentation
◆ __init__()
| def ndcctools.taskvine.dask_executor.PythonTaskDask.__init__ | ( | self, | |
| m, | |||
| dag, | |||
| key, | |||
| sexpr, | |||
| * | category = None, |
||
environment = None, |
|||
extra_files = None, |
|||
env_vars = None, |
|||
retries = 5, |
|||
worker_transfers = False, |
|||
wrapper = None |
|||
| ) |
Create a new PythonTaskDask.
- Parameters
-
self This task object. m TaskVine manager object. dag Dask graph object. key Key of task in graph. sexpr Positional arguments to function. category TaskVine category name. environment TaskVine execution environment. extra_files Additional files to provide to the task. env_vars A dictionary of environment variables. retries Number of times to retry failed task. worker_transfers If true, do not return outputs to manager until required. wrapper
The documentation for this class was generated from the following file:
- dask_executor.py
