Skip to content

feat: add apipe for sequential asynchronous pipelines - #252

Merged
dbrattli merged 3 commits into
dbrattli:mainfrom
edmundmills:async-pipe
Nov 12, 2026
Merged

dbrattli merged 3 commits into
dbrattli:mainfrom
edmundmills:async-pipe

Conversation

@edmundmills

@edmundmills edmundmills commented May 11, 2025 •

Copy link
Copy Markdown
Contributor

Add apipe for pipelines that mix synchronous functions and coroutine-returning functions:

from expression import apipe

value = await apipe(user_id, fetch_user, normalize_user, save_user)

Execution starts when awaited and proceeds sequentially. Native coroutine inputs and stage outputs are awaited once; Option, Result, tasks, futures, and custom awaitables remain values. Use an async wrapper to resolve other awaitables. Existing pipe semantics are unchanged.

Expose apipe from expression and expression.core. Add expression.extra.async_result.bind for asynchronous Result mapping with a preserved error type. Execution is iterative, and exceptions and cancellation propagate without running later stages. A Result bind overload supports composition over the whole Result union while preserving case-specific typing.

Based on Edmund Mills (@edmundmills)'s original proof of concept, retained as commit e7024cf. Follow-up commits merge current main, rename the proposed async_pipe API to apipe, address the review findings, and add documentation and runtime/type regressions.

Validation commands (in the project environment):

  • pytest
  • pre-commit run --all-files

@dbrattli dbrattli left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the proof of concept. Before it can be reconsidered, please rebase onto current main and address these blockers:

  1. extra.async_result.bind accepts a mapper returning Result[_TResult, Any] but promises Result[_TResult, _TError]. That is type-unsound: a mapper may introduce a different error type undetected. Preserve _TError in the mapper return type and use Awaitable rather than the overly specific Coroutine.
  2. async_pipe recursively calls itself once per pipeline stage, so a sufficiently long valid runtime pipeline raises RecursionError. Implement the execution loop iteratively while retaining the deliberate iscoroutine behavior that avoids awaiting Option and Result.

Please add tests for error-type preservation and a pipeline longer than Python’s recursion limit; then CI can validate the rebased implementation.

@dbrattli dbrattli changed the title Proof of concept for async pipe feat: add apipe for sequential asynchronous pipelines Nov 12, 2026
@dbrattli
dbrattli merged commit 65b3967 into dbrattli:main Nov 12, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants