Optional deferred acknowledgment and a bounded response wait in the RPC call request node - #36
Open
ShvaykaD wants to merge 14 commits into
Open
Conversation
…legacy-preserving defaults
…esult when forceAck is disabled
…d validate the timeout
… migration, fall back to expiration time for the response deadline
…ESCE, parameterized deadline test, clearer nodeDetails order
…t is not overridden
…ad of mocking the scheduler
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
TbSendRPCRequestNodeacknowledges the incoming message right after dispatching the RPC, so the queue offset commits before the outcome is known while the pending state lives only in per-pod memory. A rule engine restart mid-flight therefore loses the RPC silently — it was counted as a success — and a timeout is reported as a new message on the node's Failure output, which queue retries reprocess as an error wrapper without ever re-executingonMsg. There was also no way to bound the rule engine's wait: it is always derived from the RPC expiration time, which for REST-initiated RPC comes from the request body with a 5 s floor and no maximum.Goal
Let an operator make server-side RPC through the rule engine recoverable and bound how long it waits, with existing rule chains behaving exactly as before.
Approach
Two opt-in options, both defaulting to today's behavior:
forceAck(defaulttrue) —falseholds the incoming message and completes it on the outcome (tellSuccess/tellFailureon the original message) instead of acknowledging at dispatch. The original request becomes the completion unit, so it stays uncommitted: a restart mid-flight redelivers and re-executes it, and a timeout becomes a retryable failure of the real request. Downstream payloads are identical in both modes.overrideResponseTimeout(defaultfalse) — makes the node'stimeoutInSecondsbound the wait. The node sends an absoluteruleEngineResponseDeadline(expirationTimewhen off,now + timeoutInSecondswhen on);scheduleTimeoutkeeps its original arithmetic and only reads that deadline, so the default path is unchanged and no sentinel logic lives in the service.init()rejects a negativetimeoutInSecondsunrecoverably, which removes the need for any runtime guard.0stays valid, since the UI has always allowed it.version = 1plusupgrade(), and a single atomicUPDATE rule_nodeappended to the LTS4.3.1.4patch, committed together with the schema version so the new code never sees an un-migrated configuration.No proto / wire / device / transport change, and no change to core-side RPC persistence.
With
forceAck=false, reprocessing means at-least-once delivery — the same command can reach the device twice — a real re-attempt requires a queue whose pack timeout and processing strategy allow it, and the node emits Success/Failure debug entries instead ofACK. A configured cap gets the same+1sgrace as the expiration-derived wait and no longer clamps to the remaining expiration.Tests
TbSendRPCRequestNodeTestcovers both acknowledgment modes (the two pre-existing response tests are the untouched legacy guard), the deadline with the override on and off includingtimeoutInSeconds = 0, theinit()validation, andupgrade()from version0.DefaultTbRuleEngineRpcServiceTestis unchanged — with the deadline computed in the node, the service has no branch left to test.