CAMEL-24598: Fix flaky test JGroupsRaftProducerTest.shouldSendBody - #26193
Draft
gnodet wants to merge 1 commit into
Draft
CAMEL-24598: Fix flaky test JGroupsRaftProducerTest.shouldSendBody#26193gnodet wants to merge 1 commit into
gnodet wants to merge 1 commit into
Conversation
apupier
requested changes
Sep 8, 2026
apupier
left a comment
apupier
left a comment
Contributor
There was a problem hiding this comment.
several unrelated changes
apupier
requested changes
Sep 8, 2026
apupier
left a comment
apupier
left a comment
Contributor
There was a problem hiding this comment.
test errors:
[camel-jgroups-raft] [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 51.07 s <<< FAILURE! -- in org.apache.camel.component.jgroups.raft.cluster.JGroupsRaftMasterTest
[camel-jgroups-raft] [ERROR] org.apache.camel.component.jgroups.raft.cluster.JGroupsRaftMasterTest.test -- Time elapsed: 51.07 s <<< ERROR!
org.apache.camel.FailedToStartRouteException: Failed to start route: route-A because: java.lang.SecurityException: join of A rejected as it would create a view with duplicate members (current view: [B|4] (3) [B, C, A])
at org.apache.camel.impl.engine.InternalRouteStartupManager.doStartOrResumeRouteConsumers(InternalRouteStartupManager.java:431)
at org.apache.camel.impl.engine.InternalRouteStartupManager.doStartRouteConsumers(InternalRouteStartupManager.java:341)
at org.apache.camel.impl.engine.InternalRouteStartupManager.safelyStartRouteServices(InternalRouteStartupManager.java:217)
at org.apache.camel.impl.engine.InternalRouteStartupManager.lambda$doStartOrResumeRoutes$1(InternalRouteStartupManager.java:129)
at org.apache.camel.impl.engine.AbstractCamelContext.lambda$startingRoutes$2(AbstractCamelContext.java:1193)
at org.apache.camel.util.concurrent.ContextValueFactory.where(ContextValueFactory.java:65)
at org.apache.camel.util.concurrent.ContextValue.where(ContextValue.java:177)
at org.apache.camel.impl.engine.AbstractCamelContext.startingRoutes(AbstractCamelContext.java:1191)
at org.apache.camel.impl.engine.InternalRouteStartupManager.doStartOrResumeRoutes(InternalRouteStartupManager.java:117)
at org.apache.camel.impl.engine.AbstractCamelContext.doStartCamel(AbstractCamelContext.java:3239)
at org.apache.camel.impl.engine.AbstractCamelContext.doStartContext(AbstractCamelContext.java:2841)
at org.apache.camel.impl.engine.AbstractCamelContext.doStart(AbstractCamelContext.java:2796)
at org.apache.camel.support.service.BaseService.start(BaseService.java:132)
at org.apache.camel.impl.engine.AbstractCamelContext.start(AbstractCamelContext.java:2375)
at org.apache.camel.impl.DefaultCamelContext.start(DefaultCamelContext.java:214)
at org.apache.camel.component.jgroups.raft.cluster.JGroupsRaftMasterTest.test(JGroupsRaftMasterTest.java:67)
Caused by: java.lang.RuntimeException: java.lang.SecurityException: join of A rejected as it would create a view with duplicate members (current view: [B|4] (3) [B, C, A])
at org.apache.camel.support.cluster.AbstractCamelClusterService$ViewHolder.lambda$new$0(AbstractCamelClusterService.java:254)
at org.apache.camel.util.ReferenceCount.retain(ReferenceCount.java:51)
at org.apache.camel.support.cluster.AbstractCamelClusterService$ViewHolder.retain(AbstractCamelClusterService.java:273)
at org.apache.camel.support.cluster.AbstractCamelClusterService.lambda$getView$3(AbstractCamelClusterService.java:156)
at org.apache.camel.util.concurrent.LockHelper.callWithWriteLock(LockHelper.java:94)
at org.apache.camel.support.cluster.AbstractCamelClusterService.getView(AbstractCamelClusterService.java:141)
at org.apache.camel.component.master.MasterConsumer.doStart(MasterConsumer.java:111)
at org.apache.camel.support.service.BaseService.start(BaseService.java:132)
at org.apache.camel.support.service.ServiceHelper.startService(ServiceHelper.java:129)
at org.apache.camel.impl.engine.AbstractCamelContext.startService(AbstractCamelContext.java:3637)
at org.apache.camel.impl.engine.InternalRouteStartupManager.doStartOrResumeRouteConsumers(InternalRouteStartupManager.java:425)
... 15 more
Caused by: java.lang.SecurityException: join of A rejected as it would create a view with duplicate members (current view: [B|4] (3) [B, C, A])
at org.jgroups.protocols.pbcast.ClientGmsImpl.isJoinResponseValid(ClientGmsImpl.java:188)
at org.jgroups.protocols.pbcast.ClientGmsImpl.installViewIfValidJoinRsp(ClientGmsImpl.java:154)
at org.jgroups.protocols.pbcast.ClientGmsImpl.joinInternal(ClientGmsImpl.java:109)
at org.jgroups.protocols.pbcast.ClientGmsImpl.join(ClientGmsImpl.java:38)
at org.jgroups.protocols.pbcast.GMS.down(GMS.java:841)
at org.jgroups.protocols.FlowControl.down(FlowControl.java:201)
at org.jgroups.protocols.FlowControl.down(FlowControl.java:201)
at org.jgroups.stack.Protocol.down(Protocol.java:308)
at org.jgroups.protocols.FRAG2.down(FRAG2.java:101)
at org.jgroups.protocols.raft.election.BaseElection.down(BaseElection.java:133)
at org.jgroups.protocols.raft.RAFT.down(RAFT.java:620)
at org.jgroups.stack.Protocol.down(Protocol.java:308)
at org.jgroups.stack.ProtocolStack.down(ProtocolStack.java:945)
at org.jgroups.JChannel.down(JChannel.java:546)
at org.jgroups.JChannel._connect(JChannel.java:753)
at org.jgroups.JChannel.connect(JChannel.java:345)
at org.apache.camel.component.jgroups.raft.cluster.JGroupsRaftClusterView.doStart(JGroupsRaftClusterView.java:92)
at org.apache.camel.support.service.BaseService.start(BaseService.java:132)
at org.apache.camel.support.cluster.AbstractCamelClusterService$ViewHolder.startView(AbstractCamelClusterService.java:287)
at org.apache.camel.support.cluster.AbstractCamelClusterService$ViewHolder.lambda$new$0(AbstractCamelClusterService.java:252)
... 25 more
[camel-jgroups-raft] [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 51.07 s <<< FAILURE! -- in org.apache.camel.component.jgroups.raft.cluster.JGroupsRaftClusteredRoutePolicyTest
[camel-jgroups-raft] [ERROR] org.apache.camel.component.jgroups.raft.cluster.JGroupsRaftClusteredRoutePolicyTest.test -- Time elapsed: 51.07 s <<< ERROR!
java.lang.SecurityException: join of A rejected as it would create a view with duplicate members (current view: [B|4] (3) [B, C, A])
at org.jgroups.protocols.pbcast.ClientGmsImpl.isJoinResponseValid(ClientGmsImpl.java:188)
at org.jgroups.protocols.pbcast.ClientGmsImpl.installViewIfValidJoinRsp(ClientGmsImpl.java:154)
at org.jgroups.protocols.pbcast.ClientGmsImpl.joinInternal(ClientGmsImpl.java:109)
at org.jgroups.protocols.pbcast.ClientGmsImpl.join(ClientGmsImpl.java:38)
at org.jgroups.protocols.pbcast.GMS.down(GMS.java:841)
at org.jgroups.protocols.FlowControl.down(FlowControl.java:201)
at org.jgroups.protocols.FlowControl.down(FlowControl.java:201)
at org.jgroups.stack.Protocol.down(Protocol.java:308)
at org.jgroups.protocols.FRAG2.down(FRAG2.java:101)
at org.jgroups.protocols.raft.election.BaseElection.down(BaseElection.java:133)
at org.jgroups.protocols.raft.RAFT.down(RAFT.java:620)
at org.jgroups.stack.Protocol.down(Protocol.java:308)
at org.jgroups.stack.ProtocolStack.down(ProtocolStack.java:945)
at org.jgroups.JChannel.down(JChannel.java:546)
at org.jgroups.JChannel._connect(JChannel.java:753)
at org.jgroups.JChannel.connect(JChannel.java:345)
at org.apache.camel.component.jgroups.raft.cluster.JGroupsRaftClusterView.doStart(JGroupsRaftClusterView.java:92)
at org.apache.camel.support.service.BaseService.start(BaseService.java:132)
at org.apache.camel.support.cluster.AbstractCamelClusterService.lambda$doStart$1(AbstractCamelClusterService.java:123)
at org.apache.camel.util.concurrent.LockHelper.doWithReadLockT(LockHelper.java:54)
at org.apache.camel.support.cluster.AbstractCamelClusterService.doStart(AbstractCamelClusterService.java:119)
at org.apache.camel.support.service.BaseService.start(BaseService.java:132)
at org.apache.camel.support.service.ServiceHelper.startService(ServiceHelper.java:129)
at org.apache.camel.impl.engine.DeferServiceStartupListener.doStart(DeferServiceStartupListener.java:63)
at org.apache.camel.impl.engine.DeferServiceStartupListener.onCamelContextStarted(DeferServiceStartupListener.java:54)
at org.apache.camel.impl.engine.InternalRouteStartupManager.safelyStartRouteServices(InternalRouteStartupManager.java:200)
at org.apache.camel.impl.engine.InternalRouteStartupManager.lambda$doStartOrResumeRoutes$1(InternalRouteStartupManager.java:129)
at org.apache.camel.impl.engine.AbstractCamelContext.lambda$startingRoutes$2(AbstractCamelContext.java:1193)
at org.apache.camel.util.concurrent.ContextValueFactory.where(ContextValueFactory.java:65)
at org.apache.camel.util.concurrent.ContextValue.where(ContextValue.java:177)
at org.apache.camel.impl.engine.AbstractCamelContext.startingRoutes(AbstractCamelContext.java:1191)
at org.apache.camel.impl.engine.InternalRouteStartupManager.doStartOrResumeRoutes(InternalRouteStartupManager.java:117)
at org.apache.camel.impl.engine.AbstractCamelContext.doStartCamel(AbstractCamelContext.java:3239)
at org.apache.camel.impl.engine.AbstractCamelContext.doStartContext(AbstractCamelContext.java:2841)
at org.apache.camel.impl.engine.AbstractCamelContext.doStart(AbstractCamelContext.java:2796)
at org.apache.camel.support.service.BaseService.start(BaseService.java:132)
at org.apache.camel.impl.engine.AbstractCamelContext.start(AbstractCamelContext.java:2375)
at org.apache.camel.impl.DefaultCamelContext.start(DefaultCamelContext.java:214)
at org.apache.camel.component.jgroups.raft.cluster.JGroupsRaftClusteredRoutePolicyTest.test(JGroupsRaftClusteredRoutePolicyTest.java:63)
Contributor
|
🌟 Thank you for your contribution to the Apache Camel project! 🌟 🐫 Apache Camel Committers, please review the following items:
|
Contributor
|
🧪 CI tested the following changed modules:
🔬 Scalpel shadow comparison — Scalpel: 1 tested, 0 compile-only — current: 9 all testedMaveniverse Scalpel detected 1 affected modules (current approach: 9). Modules only in current approach (8)
Skip-tests mode would test 1 modules (1 direct + 0 downstream), skip tests for 0 (generated code, meta-modules) Modules Scalpel would test (1)
All tested modules (9 modules, 1m 40s total)Total reactor time: 1m 40s
Top 20 slowest modules:
|
- waitForLeader: Use Awaitility instead of Thread.sleep, check leader() != null on all connected handles (not just isLeader on one) to ensure followers have discovered the leader before tests proceed. - waitForLeader: Use varargs and skip disconnected handles so the method works correctly after stopping a context. - JGroupsRaftClusteredRoutePolicyTest / JGroupsRaftMasterTest: Explicitly close JChannels after stopping contexts and wait for view to settle before creating new channels with the same member name. This avoids the NO_DUPES rejection from JGroups when the old member is still in the view. - Increase test timeouts from 5s to 30s for more reliable Raft elections. Signed-off-by: Guillaume Nodet <gnodet@gmail.com>
apupier
approved these changes
Sep 9, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
JGroupsRaftProducerTest.shouldSendBodyis flaky due to a Raft leader-election timing race.waitForLeader()only checks that one node considers itself leader (isLeader()), but does NOT wait for follower nodes to learn who the leader is. Whentemplate.request()hits a non-leader node whose REDIRECT protocol has not yet discovered the leader,set()throws aRaftLeaderException. This exception propagates beforepopulateJGroupsRaftHeaders()executes (line 81 inJGroupsRaftProducer.process()), so all headers remain null, causingassertNotNullto fail.Fix
waitForLeader()checkleader() != nullon ALL handles — ensures every node has discovered the leader before the test proceeds withset()callsThread.sleep()with Awaitility — per project conventions (CLAUDE.md), with 500ms poll interval for faster convergenceJGroupsRaftClusterAbstractTestwhich had the identical bugChanges
JGroupsRaftAbstractTest.waitForLeader(): Awaitility +leader() != nullcheck on all handlesJGroupsRaftClusterAbstractTest.waitForLeader(): Same fixJGroupsRaftProducerTest: timeout 5 → 30JGroupsRaftConsumerTest: timeout 5 → 30AI agent (Hermes on behalf of gnodet)