Skip to content

feat(testing): implement entity support in the in-memory backend - #341

Open
YunchuWang wants to merge 2 commits into
mainfrom
feat/in-memory-entity-support
Open

feat(testing): implement entity support in the in-memory backend#341
YunchuWang wants to merge 2 commits into
mainfrom
feat/in-memory-entity-support

Conversation

@YunchuWang

Copy link
Copy Markdown
Member

Problem

Any orchestration that touches an entity is untestable against the in-memory backend today. processAction has no case for SENDENTITYMESSAGE, so it hits the default branch and throws.

Proven on unpatched main — an orchestration whose only statement is ctx.entities.signalEntity(...):

PROBE_RESULT status=3 failure=Unknown orchestrator action type '8' for orchestration '...'.
This likely means the in-memory backend needs to be updated to handle a newly introduced action type.

status=3 is FAILED. Not a dropped message — the orchestration dies. Same probe with this PR:

PROBE_RESULT status=COMPLETED failure=none

Why implement rather than fail fast

I checked all four sibling SDKs for precedent:

SDK In-memory testing backend Entities in it
Python durabletask/testing/in_memory_backend.py (1814 lines) ✅ full
.NET InProcessTestHost/Sidecar/InMemoryOrchestrationService.cs (944 lines) ❌ 0 references
Java none n/a
Go none (sqlite + postgres only) n/a
JS src/testing/in-memory-backend.ts (899 lines) ❌ 0 references

Python is the reference. Its _process_action dispatches sendEntityMessage to _process_send_entity_message_action, which branches on the same four oneof variants this PR handles. This change follows that design.

Worth noting: Python's dispatch handles exactly six actions and has no terminateOrchestration — independently matching the JS in-memory backend's own action set.

What changed

in-memory-backend.ts

  • EntityState, EntityWorkItem, PendingLockRequest; entity map, queue, and in-flight set.
  • processSendEntityMessageAction handles all four variants. Each echoes its confirmation event into orchestration history keyed by the originating action ID — required so replay validation (validateEntityAction) matches on the next execution.
  • getNextEntityWorkItem() drains all queued operations into one batch and marks the entity in-flight so it is not dispatched concurrently.
  • completeEntityTask() persists state, delivers results, applies side effects, and advances the completion token so stale completions are rejected.
  • Result correlation is by index against the dispatched operations, which is sound because TaskEntityShim.executeAsync pushes exactly one result per operation. callEntity responses become EntityOperationCompleted / EntityOperationFailed on the caller; signals are fire-and-forget and produce none.
  • Lock manager: canGrantLock / grantLock / tryGrantLock / tryGrantPendingLocks; unlock releases and re-evaluates the pending queue.
  • Entity side effects: sendSignal and startNewOrchestration.
  • reset() clears entity state; hasPendingWork() includes the entity queue.

test-worker.tsaddEntity / addNamedEntity, entity polling in the processing loop, and batch execution through the existing TaskEntityShim. A call to an unregistered entity now fails that operation with an explanatory error instead of hanging the caller forever.

test-client.tssignalEntity() and getEntity(), matching Python's SignalEntity / GetEntity RPCs and the real TaskHubGrpcClient signatures.

Ordering detail

grantLock pushes EntityLockGranted into parent.pendingEvents while that same instance's actions are being processed. This is safe because completeOrchestration clears pendingEvents before the action loop, not after. The lock test covers it.

Tests

11 new tests in test/in-memory-backend-entities.spec.ts. They assert observable effects, not just absence of a crash — entity state actually mutates, and callEntity return values reach the orchestration:

  • signal from an orchestration mutates entity state
  • multiple signals batch into one execution (1+2+3 → 6)
  • callEntity returns its value to the caller
  • state persists across separate batches
  • a throwing operation surfaces to the caller as a catchable error
  • calling an unregistered entity fails instead of hanging
  • lockEntities grants, then release() unlocks
  • client-initiated signalEntity reaches the entity
  • getEntity returns undefined for a nonexistent entity
  • an entity signalling another entity is delivered
  • reset() clears entity state

Validation

Check Result
npm run build exit 0
npm run lint exit 0
npx jest (durabletask-js) 1187 passed / 1187, 68/68 suites
Revert probe orchestration FAILED without the fix, COMPLETED with it

Baseline is 67 suites / 1176 tests; this adds 1 suite / 11 tests. No regressions.

The azure-functions-durable package reports 8 suites failing on Cannot find module '@azure/functions'. I confirmed this is pre-existing by stashing this branch's changes and re-running that package — identical 8 failed / 2 failed. It is a dependency-install gap in my environment, unrelated to this change.

Scope

Matching Python, an entity is dispatched whenever it has queued operations; operations are not filtered by critical section while a lock is held. Lock state is tracked and lockEntities blocks other lock requests, but a concurrent signal to a locked entity is delivered immediately rather than deferred. Full critical-section message filtering is a larger change and is not attempted here.

Orchestrations that touched an entity could not be tested against the
in-memory backend at all: SENDENTITYMESSAGE was not handled by
processAction, so the default branch threw "Unknown orchestrator action
type '8'" and failed the orchestration outright.

This adds real entity support, mirroring the Python SDK's in-memory
backend (durabletask/testing/in_memory_backend.py), which is the only
sibling SDK that implements entities in a testing backend:

- Entity state, queue, and in-flight tracking on the backend.
- All four SendEntityMessageAction variants: entityOperationSignaled,
  entityOperationCalled, entityLockRequested, entityUnlockSent. Each
  echoes its confirmation event into orchestration history (keyed by the
  originating action ID) so replay validation matches.
- Entity batch dispatch: operations are drained into a single batch and
  executed through the existing TaskEntityShim.
- Result delivery: results are index-aligned with dispatched operations,
  so callEntity responses come back as EntityOperationCompleted /
  EntityOperationFailed to the calling orchestration.
- Entity side effects: sendSignal and startNewOrchestration.
- Lock manager: grant/queue/release with pending-request re-evaluation.
- Client parity with Python's SignalEntity/GetEntity RPCs via
  TestOrchestrationClient.signalEntity() and getEntity().
- reset() clears entity state; hasPendingWork() accounts for the queue.

An unregistered entity now fails its operations with an explanatory
error instead of leaving the caller hanging forever.
Copilot AI review requested due to automatic review settings July 30, 2026 16:53

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds end-to-end Entity support to the DurableTask JS in-memory testing backend, enabling orchestrations that signal/call entities (and use entity locks) to be executed and asserted in unit tests without failing on unsupported action types.

Changes:

  • Added entity dispatch/execution plumbing to the in-memory backend (entity state store, work-item queueing, result delivery, lock management, and entity side effects).
  • Extended the test worker + test client APIs to register entities and to signal/query entities in tests.
  • Introduced a dedicated Jest test suite covering core entity behaviors (signals, calls, batching, failures, locks, reset).

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
packages/durabletask-js/src/testing/in-memory-backend.ts Implements entity work queues/state, SENDENTITYMESSAGE handling, lock granting/unlocking, and entity side-effect routing.
packages/durabletask-js/src/testing/test-worker.ts Adds entity registration APIs and an entity processing loop using TaskEntityShim batch execution.
packages/durabletask-js/src/testing/test-client.ts Adds signalEntity() and getEntity() convenience APIs for in-memory tests.
packages/durabletask-js/test/in-memory-backend-entities.spec.ts New test suite validating observable entity behaviors (signals, calls, batching, locks, reset).
CHANGELOG.md Notes the addition of entity support in the in-memory testing backend.

Comment on lines +415 to +428
const dispatched = entity.dispatchedOperations;

entity.serializedState = result.getEntitystate()?.getValue() ?? undefined;
entity.lastModifiedAt = new Date();
entity.dispatchedOperations = [];
entity.completionToken = this.nextCompletionToken++;
this.entityInFlight.delete(instanceId);

// Deliver results to callers. Results are index-aligned with the dispatched
// operations because the executor pushes exactly one result per operation.
const results = result.getResultsList();
for (let i = 0; i < dispatched.length; i++) {
this.deliverEntityOperationResult(dispatched[i], results[i]);
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants