ndcctools.resource_monitor.Categories Class Reference

Class to encapsule all the categories in a workflow. More...

Public Member Functions
def	__init__ (self, all_categories_name='(all)')
	Create an empty set of categories. More...
def	category_names (self)
	Returns a lists of the category categories. More...
def	first_allocation (self, mode, category)
	Compute and return the first allocations for the given category. More...
def	maximum_seen (self, category)
	Return the maximum resource values so far seen for the given category. More...
def	accumulate_summary (self, summary)
	Add the summary (a dictionary) to the respective category. More...
def	waste (self, category, field, allocation)
	Return the waste (unit x time) that would be produced if the accumulated summaries were run under the given allocation. More...
def	wastepercentage (self, category, field, allocation)
	Return the percentage of wasted resources that would be produced if the accumulated summaries were run under the given allocation. More...
def	throughput (self, category, field, allocation)
	Return the throughput that would be obtained if the accumulated summaries were run under the given allocation. More...
def	retries (self, category, field, allocation)
	Return the number of tasks that would be retried if the accumulated summaries were run under the given allocation. More...
def	count (self, category)
	Return the number of summaries in a particular category. More...

Detailed Description

Class to encapsule all the categories in a workflow.

cs = Categories()
cs.accumulate_summary( { 'category': 'some_category', 'wall_time': 60, 'cores': 1, ... } )
print(cs.first_allocation(mode = 'throughput', category = 'some_category'))

Constructor & Destructor Documentation

__init__()

def ndcctools.resource_monitor.Categories.__init__	(		self,
			all_categories_name = '(all)'
	)

Create an empty set of categories.

Parameters

self	Reference to the current object.
all_categories_name	Name of the general category that holds all of the summaries.

Member Function Documentation

category_names()

def ndcctools.resource_monitor.Categories.category_names

(

self

)

Returns a lists of the category categories.

List sorted lexicographicaly, with the exception of self.all_categories_name, which it is always the last entry.

Parameters

self	Reference to the current object.

first_allocation()

def ndcctools.resource_monitor.Categories.first_allocation	(		self,
			mode,
			category
	)

Compute and return the first allocations for the given category.

Note: wall_time needs to be defined in the resource summaries to be considered in this optimization.

Parameters

self	Reference to the current object.
mode	Optimization mode. One of 'throughput', 'waste', or 'fixed'.
category	Name of the category

cs = Categories()
fa = cs.first_allocation(mode = 'throughput, category = 'some_category')
print(fa['cores'])
print(fa['memory'])
print(fa['disk'])

maximum_seen()

def ndcctools.resource_monitor.Categories.maximum_seen	(		self,
			category
	)

Return the maximum resource values so far seen for the given category.

Parameters

self	Reference to the current object.
category	Name of the category

cs = Categories()
fa = cs.maximum_seen('some_category')
print(fa['cores'])
print(fa['memory'])
print(fa['disk'])

accumulate_summary()

def ndcctools.resource_monitor.Categories.accumulate_summary	(		self,
			summary
	)

Add the summary (a dictionary) to the respective category.

At least both the 'category' and 'wall_time' keys should be defined.

cs = Categories()
cs.accumulate_summary( { 'category': 'some_category', 'wall_time': 50, 'cores': 1, ... } )

waste()

def ndcctools.resource_monitor.Categories.waste	(		self,
			category,
			field,
			allocation
	)

Return the waste (unit x time) that would be produced if the accumulated summaries were run under the given allocation.

Parameters

self	Reference to the current object.
category	Name of the category
field	Name of the resource (e.g., cores, memory, or disk)
allocation	Value of allocation to test.

wastepercentage()

def ndcctools.resource_monitor.Categories.wastepercentage	(		self,
			category,
			field,
			allocation
	)

Return the percentage of wasted resources that would be produced if the accumulated summaries were run under the given allocation.

Parameters

self	Reference to the current object.
category	Name of the category
field	Name of the resource (e.g., cores, memory, or disk)
allocation	Value of allocation to test.

throughput()

def ndcctools.resource_monitor.Categories.throughput	(		self,
			category,
			field,
			allocation
	)

Return the throughput that would be obtained if the accumulated summaries were run under the given allocation.

Parameters

self	Reference to the current object.
category	Name of the category
field	Name of the resource (e.g., cores, memory, or disk)
allocation	Value of allocation to test.

retries()

def ndcctools.resource_monitor.Categories.retries	(		self,
			category,
			field,
			allocation
	)

Return the number of tasks that would be retried if the accumulated summaries were run under the given allocation.

Parameters

self	Reference to the current object.
category	Name of the category
field	Name of the resource (e.g., cores, memory, or disk)
allocation	Value of allocation to test.

count()

def ndcctools.resource_monitor.Categories.count	(		self,
			category
	)

Return the number of summaries in a particular category.

Parameters

self	Reference to the current object.
category	Name of the category

---

The documentation for this class was generated from the following file:

• resource_monitor.py