cctools
ndcctools.work_queue.PythonTask Class Reference

Python PythonTask object. More...

Inheritance diagram for ndcctools.work_queue.PythonTask:
ndcctools.work_queue.Task

Public Member Functions

 __init__ (self, func, *args, **kwargs)
 Creates a new python task.
 
 output (self)
 returns the result of a python task as a python variable
 
- Public Member Functions inherited from ndcctools.work_queue.Task
 clone (self)
 Return a copy of this task.
 
 specify_command (self, command)
 Set the command to be executed by the task.
 
 specify_coprocess (self, coprocess)
 Set the coprocess at the worker that should execute the task's command.
 
 specify_algorithm (self, algorithm)
 Set the worker selection algorithm for task.
 
 specify_tag (self, tag)
 Attach a user defined logical name to the task.
 
 specify_category (self, name)
 Label the task with the given category.
 
 specify_feature (self, name)
 Label the task with the given user-defined feature.
 
 specify_file (self, local_name, remote_name=None, type=None, flags=None, cache=None, failure_only=None)
 Add a file to the task.
 
 specify_url (self, url, remote_name, type=None, flags=None, cache=None, failure_only=None)
 Add a url to the task which will be provided as an input file.
 
 specify_file_command (self, cmd, remote_name, type=None, flags=None, cache=None, failure_only=None)
 Add an input file produced by a Unix shell command.
 
 specify_file_piece (self, local_name, remote_name=None, start_byte=0, end_byte=0, type=None, flags=None, cache=None, failure_only=None)
 Add a file piece to the task.
 
 specify_input_file (self, local_name, remote_name=None, flags=None, cache=None)
 Add a input file to the task.
 
 specify_output_file (self, local_name, remote_name=None, flags=None, cache=None, failure_only=None)
 Add a output file to the task.
 
 specify_directory (self, local_name, remote_name=None, type=None, flags=None, recursive=False, cache=None, failure_only=None)
 Add a directory to the task.
 
 specify_buffer (self, buffer, remote_name, flags=None, cache=None)
 Add an input bufer to the task.
 
 specify_snapshot_file (self, filename)
 When monitoring, indicates a json-encoded file that instructs the monitor to take a snapshot of the task resources.
 
 specify_max_retries (self, max_retries)
 Indicate the number of times the task should be retried.
 
 specify_cores (self, cores)
 Indicate the number of cores required by this task.
 
 specify_memory (self, memory)
 Indicate the memory (in MB) required by this task.
 
 specify_disk (self, disk)
 Indicate the disk space (in MB) required by this task.
 
 specify_gpus (self, gpus)
 Indicate the number of GPUs required by this task.
 
 specify_priority (self, priority)
 Indicate the the priority of this task (larger means better priority, default is 0).
 
 specify_environment_variable (self, name, value=None)
 Set this environment variable before running the task.
 
 specify_monitor_output (self, directory)
 Set a name for the resource summary output directory from the monitor.
 
 tag (self)
 Get the user-defined logical name for the task.
 
 category (self)
 Get the category name for the task.
 
 command (self)
 Get the shell command executed by the task.
 
 priority (self)
 Get the priority of the task.
 
 algorithm (self)
 Get the algorithm for choosing worker to run the task.
 
 std_output (self)
 Get the standard output of the task.
 
 id (self)
 Get the task id number.
 
 return_status (self)
 Get the exit code of the command executed by the task.
 
 result (self)
 Get the result of the task as an integer code, such as successful, missing file, etc.
 
 result_str (self)
 Return a string that explains the result of a task.
 
 total_submissions (self)
 Get the number of times the task has been resubmitted internally.
 
 exhausted_attempts (self)
 Get the number of times the task has been failed given resource exhaustion.
 
 host (self)
 Get the address and port of the host on which the task ran.
 
 hostname (self)
 Get the name of the host on which the task ran.
 
 submit_time (self)
 Get the time at which this task was submitted.
 
 finish_time (self)
 Get the time at which this task was finished.
 
 total_cmd_exhausted_execute_time (self)
 Get the total time the task executed and failed given resource exhaustion.
 
 app_delay (self)
 Get the time spent in upper-level application (outside of work_queue_wait).
 
 send_input_start (self)
 Get the time at which the task started to transfer input files.
 
 send_input_finish (self)
 Get the time at which the task finished transferring input files.
 
 execute_cmd_start (self)
 The time at which the task began.
 
 execute_cmd_finish (self)
 Get the time at which the task finished (discovered by the manager).
 
 receive_output_start (self)
 Get the time at which the task started to transfer output files.
 
 receive_output_finish (self)
 Get the time at which the task finished transferring output files.
 
 total_bytes_received (self)
 Get the number of bytes received since task started receiving input data.
 
 total_bytes_sent (self)
 Get the number of bytes sent since task started sending input data.
 
 total_bytes_transferred (self)
 Get the number of bytes transferred since task started transferring input data.
 
 total_transfer_time (self)
 Get the time comsumed in microseconds for transferring total_bytes_transferred.
 
 cmd_execution_time (self)
 Time spent in microseconds for executing the command until completion on a single worker.
 
 total_cmd_execution_time (self)
 Accumulated time spent in microseconds for executing the command on any worker, regardless of whether the task finished (i.e., this includes time running on workers that disconnected).
 
 resources_measured (self)
 Get the resources measured for the task execution if resource monitoring is enabled.
 
 limits_exceeded (self)
 Get the resources the task exceeded.
 
 resources_requested (self)
 Get the resources the task requested to run.
 
 resources_allocated (self)
 Get the resources allocated to the task in its latest attempt.
 

Protected Member Functions

 _create_wrapper (self)
 creates the wrapper script which will execute the function.
 

Detailed Description

Python PythonTask object.

this class is used to create a python task

Constructor & Destructor Documentation

◆ __init__()

ndcctools.work_queue.PythonTask.__init__ (   self,
  func,
args,
**  kwargs 
)

Creates a new python task.

Parameters
selfReference to the current python task object
funcpython function to be executed by task
argsarguments used in function to be executed by task
kwargskeyword arguments used in function to be executed by task

Reimplemented from ndcctools.work_queue.Task.

Member Function Documentation

◆ output()

ndcctools.work_queue.PythonTask.output (   self)

returns the result of a python task as a python variable

Parameters
selfreference to the current python task object

Reimplemented from ndcctools.work_queue.Task.

◆ _create_wrapper()

ndcctools.work_queue.PythonTask._create_wrapper (   self)
protected

creates the wrapper script which will execute the function.

pickles output.


The documentation for this class was generated from the following file: