FastSurferCNN.utils.parallel

class FastSurferCNN.utils.parallel.SerialExecutor

Represent a serial executor.

Methods

map(fn, *iterables[, timeout, chunksize])	The map function.
shutdown([wait, cancel_futures])	Clean-up the resources associated with the Executor.
submit(_SerialExecutor__fn, *args, **kwargs)	A callable function that returns a Future representing the result.

map(fn, *iterables, timeout=None, chunksize=-1)

The map function.

Parameters:

fnCallable[…, _T]

A callable function to be applied to the items in the iterables.

*iterablesIterable[Any]

One or more iterable objects.

timeoutOptional[float]

Maximum number of seconds to wait for a result. Default is None.

chunksizeint

The size of the chunks, default value is -1.

Returns:

Iterator[_T]

An iterator that yields the results of applying ‘fn’ to the items of ‘iterables’.

shutdown(wait=True, *, cancel_futures=False)

Clean-up the resources associated with the Executor.

It is safe to call this method several times. Otherwise, no other methods can be called after this one.

Args:

wait: If True then shutdown will not return until all running

futures have finished executing and the resources used by the executor have been reclaimed.

cancel_futures: If True then shutdown will cancel all pending

futures. Futures that are completed or running will not be cancelled.

submit(_SerialExecutor__fn, *args, **kwargs)

A callable function that returns a Future representing the result.

Parameters:

__fnCallable[…, _T]

A callable function to be executed.

*args

Potential arguments to be passed to the callable function.

**kwargs

Keyword arguments to be passed to the callable function.

Returns:

“Future[_T]”

A Future object representing the execution result of the callable function.

FastSurferCNN.utils.parallel.get_num_threads()

Determine the number of available threads.

Tries to get the process’s CPU affinity for usable thread count; defaults to total CPU count on failure.

Returns:

int

Number of threads available to the process or total CPU count.

FastSurferCNN.utils.parallel.iterate(pool, func, iterable)

Iterate over iterable, yield pairs of elements and func(element).

Parameters:

poolExecutor

The Executor object (dummy object to have a common API with pipeline).

funccallable()

Function to use.

iterableIterable

Iterable to draw objects to process with func from.

Yields:

element_Ti

Elements

_T

Results of func corresponding to element: func(element).

FastSurferCNN.utils.parallel.pipeline(pool, func, iterable, *, pipeline_size=1)

Pipeline a function to be executed in the pool.

Analogous to iterate, but run func in a different thread for the next element while the current element is returned.

Parameters:

poolExecutor

Thread pool executor for parallel execution.

funccallable()

Function to use.

iterableIterable

Iterable containing input elements.

pipeline_sizeint, default=1

Size of the processing pipeline.

Yields:

element_Ti

Elements

_T

Results of func corresponding to element: func(element).

FastSurferCNN.utils.parallel.set_num_threads(threads, wait_finish=False)

Sets the number of threads to use.

Parameters:

threadsint

The number of threads to use, values < 1 reset the thread count to automatically determined.

wait_finishbool, default=False

Wait for the executor to finish current processing (warning, this may have side-effects!).

Returns:

int

The value as int.

FastSurferCNN.utils.parallel.shutdown_executors(wait=True, *, cancel_futures=False)

This is a cleanup function and should only be called at the end of a script!

This first shuts down the process_executor and then the thread_executor.

Notes

The executors are not shut down at the same time, but rather, if wait is True, we first wait for the process_executor to “finish” shutting down.