feat(testing): implement entity support in the in-memory backend - #341
Open
YunchuWang wants to merge 2 commits into
Open
feat(testing): implement entity support in the in-memory backend#341YunchuWang wants to merge 2 commits into
YunchuWang wants to merge 2 commits into
Conversation
Orchestrations that touched an entity could not be tested against the in-memory backend at all: SENDENTITYMESSAGE was not handled by processAction, so the default branch threw "Unknown orchestrator action type '8'" and failed the orchestration outright. This adds real entity support, mirroring the Python SDK's in-memory backend (durabletask/testing/in_memory_backend.py), which is the only sibling SDK that implements entities in a testing backend: - Entity state, queue, and in-flight tracking on the backend. - All four SendEntityMessageAction variants: entityOperationSignaled, entityOperationCalled, entityLockRequested, entityUnlockSent. Each echoes its confirmation event into orchestration history (keyed by the originating action ID) so replay validation matches. - Entity batch dispatch: operations are drained into a single batch and executed through the existing TaskEntityShim. - Result delivery: results are index-aligned with dispatched operations, so callEntity responses come back as EntityOperationCompleted / EntityOperationFailed to the calling orchestration. - Entity side effects: sendSignal and startNewOrchestration. - Lock manager: grant/queue/release with pending-request re-evaluation. - Client parity with Python's SignalEntity/GetEntity RPCs via TestOrchestrationClient.signalEntity() and getEntity(). - reset() clears entity state; hasPendingWork() accounts for the queue. An unregistered entity now fails its operations with an explanatory error instead of leaving the caller hanging forever.
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adds end-to-end Entity support to the DurableTask JS in-memory testing backend, enabling orchestrations that signal/call entities (and use entity locks) to be executed and asserted in unit tests without failing on unsupported action types.
Changes:
- Added entity dispatch/execution plumbing to the in-memory backend (entity state store, work-item queueing, result delivery, lock management, and entity side effects).
- Extended the test worker + test client APIs to register entities and to signal/query entities in tests.
- Introduced a dedicated Jest test suite covering core entity behaviors (signals, calls, batching, failures, locks, reset).
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| packages/durabletask-js/src/testing/in-memory-backend.ts | Implements entity work queues/state, SENDENTITYMESSAGE handling, lock granting/unlocking, and entity side-effect routing. |
| packages/durabletask-js/src/testing/test-worker.ts | Adds entity registration APIs and an entity processing loop using TaskEntityShim batch execution. |
| packages/durabletask-js/src/testing/test-client.ts | Adds signalEntity() and getEntity() convenience APIs for in-memory tests. |
| packages/durabletask-js/test/in-memory-backend-entities.spec.ts | New test suite validating observable entity behaviors (signals, calls, batching, locks, reset). |
| CHANGELOG.md | Notes the addition of entity support in the in-memory testing backend. |
Comment on lines
+415
to
+428
| const dispatched = entity.dispatchedOperations; | ||
|
|
||
| entity.serializedState = result.getEntitystate()?.getValue() ?? undefined; | ||
| entity.lastModifiedAt = new Date(); | ||
| entity.dispatchedOperations = []; | ||
| entity.completionToken = this.nextCompletionToken++; | ||
| this.entityInFlight.delete(instanceId); | ||
|
|
||
| // Deliver results to callers. Results are index-aligned with the dispatched | ||
| // operations because the executor pushes exactly one result per operation. | ||
| const results = result.getResultsList(); | ||
| for (let i = 0; i < dispatched.length; i++) { | ||
| this.deliverEntityOperationResult(dispatched[i], results[i]); | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Any orchestration that touches an entity is untestable against the in-memory backend today.
processActionhas no case forSENDENTITYMESSAGE, so it hits thedefaultbranch and throws.Proven on unpatched
main— an orchestration whose only statement isctx.entities.signalEntity(...):status=3isFAILED. Not a dropped message — the orchestration dies. Same probe with this PR:Why implement rather than fail fast
I checked all four sibling SDKs for precedent:
durabletask/testing/in_memory_backend.py(1814 lines)InProcessTestHost/Sidecar/InMemoryOrchestrationService.cs(944 lines)src/testing/in-memory-backend.ts(899 lines)Python is the reference. Its
_process_actiondispatchessendEntityMessageto_process_send_entity_message_action, which branches on the same four oneof variants this PR handles. This change follows that design.Worth noting: Python's dispatch handles exactly six actions and has no
terminateOrchestration— independently matching the JS in-memory backend's own action set.What changed
in-memory-backend.tsEntityState,EntityWorkItem,PendingLockRequest; entity map, queue, and in-flight set.processSendEntityMessageActionhandles all four variants. Each echoes its confirmation event into orchestration history keyed by the originating action ID — required so replay validation (validateEntityAction) matches on the next execution.getNextEntityWorkItem()drains all queued operations into one batch and marks the entity in-flight so it is not dispatched concurrently.completeEntityTask()persists state, delivers results, applies side effects, and advances the completion token so stale completions are rejected.TaskEntityShim.executeAsyncpushes exactly one result per operation.callEntityresponses becomeEntityOperationCompleted/EntityOperationFailedon the caller; signals are fire-and-forget and produce none.canGrantLock/grantLock/tryGrantLock/tryGrantPendingLocks; unlock releases and re-evaluates the pending queue.sendSignalandstartNewOrchestration.reset()clears entity state;hasPendingWork()includes the entity queue.test-worker.ts—addEntity/addNamedEntity, entity polling in the processing loop, and batch execution through the existingTaskEntityShim. A call to an unregistered entity now fails that operation with an explanatory error instead of hanging the caller forever.test-client.ts—signalEntity()andgetEntity(), matching Python'sSignalEntity/GetEntityRPCs and the realTaskHubGrpcClientsignatures.Ordering detail
grantLockpushesEntityLockGrantedintoparent.pendingEventswhile that same instance's actions are being processed. This is safe becausecompleteOrchestrationclearspendingEventsbefore the action loop, not after. The lock test covers it.Tests
11 new tests in
test/in-memory-backend-entities.spec.ts. They assert observable effects, not just absence of a crash — entity state actually mutates, andcallEntityreturn values reach the orchestration:callEntityreturns its value to the callerlockEntitiesgrants, thenrelease()unlockssignalEntityreaches the entitygetEntityreturnsundefinedfor a nonexistent entityreset()clears entity stateValidation
npm run buildnpm run lintnpx jest(durabletask-js)FAILEDwithout the fix,COMPLETEDwith itBaseline is 67 suites / 1176 tests; this adds 1 suite / 11 tests. No regressions.
The
azure-functions-durablepackage reports 8 suites failing onCannot find module '@azure/functions'. I confirmed this is pre-existing by stashing this branch's changes and re-running that package — identical 8 failed / 2 failed. It is a dependency-install gap in my environment, unrelated to this change.Scope
Matching Python, an entity is dispatched whenever it has queued operations; operations are not filtered by critical section while a lock is held. Lock state is tracked and
lockEntitiesblocks other lock requests, but a concurrent signal to a locked entity is delivered immediately rather than deferred. Full critical-section message filtering is a larger change and is not attempted here.