Pyrseus: Serial Executors and the ExecutorCtx Factory

Pyrseus extends Python’s concurrent.futures asynchronous and concurrent programming package with:

• a collection of non-concurrent executors for light workloads and troubleshooting,

• ExecutorCtx, a factory for easily switching between different executors, and

• a collection of ready-built ExecutorCtx plugins, supporting executors from concurrent.futures, ipyparallel, loky, mpi4py, and itself. Where relevant, optional cloudpickle-enhanced plugins are also provided.

General Information

• Installing Pyrseus

• Plugins for ExecutorCtx

• Writing Your Own Plugin

• Contributing to Pyrseus

API Documentation

pyrseus

The pyrseus package provides the ExecutorCtx factory class, its plugins, and various standalone serial Executor classes.

Usage Examples

Notebooks

• ExecutorCtx vs. Classes
  • Setup
  • Using Explicit Classes
  • Using ExecutorCtx with Explicit Plugins
  • Using ExecutorCtx with Implicit Plugins
• Debugging Tasks with Pyrseus
  • Setup
  • Non-executor Usage
  • Failures with ProcessPoolExecutor
  • Troubleshooting with Serial Executors
  • Testing Cloudpickle Serially
  • Trying a Cloudpickle-enabled Executor
• OnError Modes
  • Setup
  • “Uncaught Exceptions”
  • Wait: Fail-after Uncaught Exceptions
  • Cancel: Fail-faster for Uncaught Exceptions
  • Kill: Fail-fast for Uncaught Exceptions
• ExecutorCtx vs. get_executor
  • Setup
  • A Single Set of Tasks
  • Multiple Sets of Tasks