Source code for pyrseus.executors.pnocatch

"""
Provides a variant of `NoCatchExecutor` that tests the picklability of all
submitted tasks and their return values.
"""

from functools import partial

from pyrseus.core.pickle import call_with_round_trip_pickling
from pyrseus.executors.nocatch import NoCatchExecutor




class PNoCatchExecutor(NoCatchExecutor):


    def __init__(self, **round_trip_kwargs):
        """
        Pickle-testing variant of `.NoCatchExecutor`, using the built-in
        `pickle` module.

        Summary
        -------

        - *Common Use Cases:* for troubleshooting with extra `pickle` testing,
          as a fail-fast variant of
          `~pyrseus.executors.pinline.PInlineExecutor`.

        - *Concurrency:* This is a non-concurrent, serial-only executor. All
          tasks are immediately run in the same process and thread they were
          submitted in.

        - *Exceptions:* This executor has *non-standard* exception-handling
          semantics: no task exceptions are caught and captured in their
          futures. Exceptions are propagated out immediately.

        - *Default max_workers:* Not applicable.

        - *Pickling:* This executor takes extra time to pickle and unpickle all
          tasks and their results. If you aren't troubleshooting such issues and
          prefer lower overhead, consider using
          `~pyrseus.executors.nocatch.NoCatchExecutor` instead.

        Details
        -------

        This variant pickles each submitted task and the task's results, using
        the built-in `pickle` module. This is primarily useful for
        troubleshooting pickling problems occurring in multi-process executors,
        by performing the pickling and unpickling locally.

        Here's an example showing a pickling failure cause by trying to pickle a
        lambda with an executor type that uses `pickle` instead of
        |cloudpickle|_.

        Consider this lambda.

            >>> import pickle
            >>> needs_cloudpickle = lambda: "this only works with cpnocatch, not pnocatch"

        It can't be pickled with `pickle`.

            >>> pickle.dumps(needs_cloudpickle, -1)
            Traceback (most recent call last):
            ...
            _pickle.PicklingError: Can't pickle ...

        Since this class tests pickling with `pickle`, it correctly fails, as it
        would in a multi process executor that uses `pickle`.

            >>> with PNoCatchExecutor() as exe:
            ...     # As with NoCatchExecutor, the exception is propagated immediately
            ...     # at submit() time. We don't have to wait till result() time.
            ...     fut = exe.submit(needs_cloudpickle)
            Traceback (most recent call last):
            ...
            _pickle.PicklingError: Can't pickle ...

        Because the failure happens within the same process and thread as the
        submit call, it is easy to debug by tracing into it with a debugger.

        See :doc:`../executors` for a list of related executors.
        """
        super().__init__()
        self._round_trip_kwargs = round_trip_kwargs




    def submit(self, fcn, /, *args, **kwargs):
        testing_closure = partial(
            call_with_round_trip_pickling,
            fcn,
            args,
            kwargs,
            **self._round_trip_kwargs,
        )
        return super().submit(testing_closure)