[DX-4492] Speed up `executable` Tests by kalverra · Pull Request #22953 · smartcontractkit/chainlink

kalverra · 2026-06-24T19:50:21Z

Speeds up core/capabilities/remote/executable tests, the 3rd slowest test package in core/.

Results

Before	After	Absolute Delta	% Delta
1m0s	3s	57s	95%

How

t.Parallel()
Ample usage of testing/synctest

Confirmed tests pass and aren't flaky:

❯ make test ARGS="diagnose --iterations 25 -- ./core/..."
❯ make test ARGS="diagnose --iterations 500 --parallel-iterations 5 -- ./core/capabilities/remote/..."
❯ make test ARGS="-race -count=5 ./core/capabilities/remote/..."

Also includes various linting fixes and t.Parallel() calls.

Important Parts to Review

There are a lot of linting fixes and inconsequential changes here, so focus on the more important bits below.

Fixed timer leak in prod: Caught this as we ran more tests in parallel. time.After doesn't get cleaned up until it actually fires, so we were leaking a bunch of these. Using a timer fixes this.
testing/synctest wrapper with and without race: synctest has known issues when running with the -race flag (should be fixed in Go 1.27). Using it in these tests seriously speeds them up and reduces flakes, but breaks the -race runs. So we have a wrapper that turns synctest on and off depending on if -race is present. A bit kludgey, but functional.
Test_RemoteExecutableCapability_ExecutionNotBlockedBySlowCapabilityExecution table test: combined some tests into table tests.

github-actions · 2026-06-24T19:50:51Z

CORA - Pending Reviewers

Codeowners Entry	Overall	Num Files	Owners
`*`	⏳	2	@smartcontractkit/foundations, @smartcontractkit/core
`/core/capabilities/`	⏳	24	@smartcontractkit/keystone, @smartcontractkit/capabilities-team
`/.github/**`	⏳	1	@smartcontractkit/devex-cicd, @smartcontractkit/devex-tooling, @smartcontractkit/core
`/tools/test/`	⏳	2	@smartcontractkit/devex-cicd, @smartcontractkit/devex-tooling, @smartcontractkit/core

For more details, see the full review summary.

github-actions · 2026-06-24T19:51:45Z

I see you updated files related to core. Please run make gocs in the root directory to add a changeset as well as in the text include at least one of the following tags:

#added For any new functionality added.
#breaking_change For any functionality that requires manual action for the node to boot.
#bugfix For bug fixes.
#changed For any change to the existing functionality.
#db_update For any feature that introduces updates to database schema.
#deprecation_notice For any upcoming deprecation functionality.
#internal For changesets that need to be excluded from the final changelog.
#nops For any feature that is NOP facing and needs to be in the official Release Notes for the release.
#removed For any functionality/config that is removed.
#updated For any functionality that is updated.
#wip For any change that is not ready yet and external communication about it should be held off till it is feature complete.

github-actions · 2026-06-24T19:51:51Z

✅ No conflicts with other open PRs targeting develop

trunk-io · 2026-06-24T20:09:28Z

_{View Full Report ↗︎ ⋅ Docs}

…speedUpExecutable

Copilot

Pull request overview

Risk Rating: HIGH

This PR targets a major reduction in runtime for core/capabilities/remote/executable tests by parallelizing tests and using testing/synctest (via a new internal wrapper), plus some small refactors/optimizations in related remote capability code.

Changes:

Parallelized many tests with t.Parallel() and refactored several time-based tests to use synctest.Test/synctest.Wait.
Added core/internal/testutils/synctest wrapper to disable synctest usage under -race (workaround for Go issue #76691).
Minor code cleanups/optimizations (e.g., min/max, timer reuse instead of time.After, reduced allocations).

Areas requiring scrupulous human review:

All production-code concurrency edits (notably the updated wg.Go(...) usage in server/client/dispatcher/executor paths).
synctest wrapper semantics under race vs non-race builds and any assumptions tests may now make about time progression.
Any behavior changes in message batching/hash construction (trigger_publisher.go) and aggregation threshold logic (default_mode.go).

Reviewed changes

Copilot reviewed 28 out of 28 changed files in this pull request and generated 10 comments.

Show a summary per file

File	Description
tools/test/.agents/skills/fix-flaky-tests/SKILL.md	Updates agent skill flow and guidance text.
tools/test/.agents/skills/fix-flaky-tests/references/speed-up-tests.md	New guidance doc for speeding up slow tests.
core/internal/testutils/synctest/synctest.go	Adds non-race synctest wrapper implementation.
core/internal/testutils/synctest/synctest_race.go	Adds race-build no-bubble synctest wrapper implementation.
core/capabilities/remote/utils_test.go	Adds `t.Parallel()` to unit tests.
core/capabilities/remote/trigger_subscriber_test.go	Refactors expiry test using synctest; parallelizes many subtests; adds test helper.
core/capabilities/remote/trigger_registration_load_test.go	Parallelizes load-style tests; uses synctest; switches to `t.Context()`.
core/capabilities/remote/trigger_publisher.go	Small refactors: clearer error vars, `min`, allocation reduction in hash input.
core/capabilities/remote/trigger_publisher_test.go	Parallelizes tests; uses synctest for batching timing; refactors helper signature.
core/capabilities/remote/messagecache/message_cache_test.go	Adds `t.Parallel()` to tests.
core/capabilities/remote/executable/server.go	Concurrency refactor in server start loop goroutine management.
core/capabilities/remote/executable/server_test.go	Heavy test refactors: parallelization, synctest, atomic counters, response collection changes.
core/capabilities/remote/executable/request/server_request_test.go	Adds `t.Parallel()` and adjusts timing assertions.
core/capabilities/remote/executable/request/client_request.go	Replaces `time.After` with a reusable `time.NewTimer`.
core/capabilities/remote/executable/request/client_request_test.go	Parallelizes subtests; small performance tweaks; replaces an Eventually with bounded sleep loop.
core/capabilities/remote/executable/request/client_request_internal_test.go	Parallelizes attestation tests; modernizes loops.
core/capabilities/remote/executable/parallel_executor.go	Concurrency refactor in task execution goroutine management.
core/capabilities/remote/executable/parallel_executor_test.go	Parallelizes tests; switches to `atomic.Int32` patterns and tighter timeouts.
core/capabilities/remote/executable/hasher_test.go	Parallelizes tests; minor slice allocation fixes.
core/capabilities/remote/executable/endtoend_test.go	Consolidates slow-execution tests; uses synctest harness; refactors helper APIs.
core/capabilities/remote/executable/client.go	Concurrency refactor in client background loops goroutine management.
core/capabilities/remote/executable/client_test.go	Parallelizes tests; refactors aggregation-grace coverage and fixes shared-state mutation.
core/capabilities/remote/dispatcher.go	Concurrency refactor in dispatcher receive loop goroutine management; small Send refactor.
core/capabilities/remote/dispatcher_test.go	Parallelizes tests and switches to `t.Context()`.
core/capabilities/remote/combined_client_test.go	Parallelizes tests and switches to `t.Context()`.
core/capabilities/remote/aggregation/signed_report_test.go	Adds `//nolint:paralleltest` notes for shared logger state.
core/capabilities/remote/aggregation/default_mode.go	Simplifies majority threshold computation via `max`.
core/capabilities/remote/aggregation/default_mode_test.go	Parallelizes tests; modernizes loops; adds gosec suppression note.

…speedUpExecutable

Copilot

Pull request overview

Copilot reviewed 29 out of 29 changed files in this pull request and generated 11 comments.

Copilot

Pull request overview

Copilot reviewed 29 out of 29 changed files in this pull request and generated 4 comments.

Comments suppressed due to low confidence (1)

tools/test/.agents/skills/fix-flaky-tests/SKILL.md:80

The loop refers to <diagnose-parallel-iterations> and a “Smoke” profile, but that section/profile no longer exist in this document (the table now starts at “Quick”). This makes the instructions internally inconsistent.

<loop>
1. If user doesn't have recent results, plan a run with `<diagnose-parallel-iterations>` (default: **Smoke** profile) then execute it. On sandbox errors, follow `<possible_execution_issues>`.
2. If no issues, escalate along `<diagnose-iterations>` (e.g. Smoke → Standard → Deep), increasing `--iterations` and keeping parallelism per `<diagnose-parallel-iterations>`. Ask the user before **Deep** if wall time will be large. If still clean and no fix was needed, end with findings; if a fix was applied, require at least **Standard** before FIXED.
3. If issues detected, focus on the ones the user wants to fix.

Copilot

Pull request overview

Copilot reviewed 29 out of 29 changed files in this pull request and generated 4 comments.

…speedUpExecutable

cl-sonarqube-production · 2026-06-26T21:06:53Z

Quality Gate passed

Issues
1 New issue
21 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube

kalverra added 2 commits June 24, 2026 14:56

linting

424d090

synctest!!!

7aa863b

kalverra requested review from a team as code owners June 24, 2026 19:50

product-security-plaid-production Bot requested review from bolekk, chainchad, ecPablo, prashantkumar1982 and vyzaldysanchez June 24, 2026 19:50

Merge branch 'develop' of github.com:smartcontractkit/chainlink into …

7bab8a4

…speedUpExecutable

kalverra changed the title ~~[DX-4492 ]Speed up executable Tests~~ [DX-4492] Speed up executable Tests Jun 24, 2026

kalverra added 2 commits June 24, 2026 17:01

Fix race

2b68aed

synctest without races

5c32a09

kalverra requested a review from a team as a code owner June 25, 2026 19:54

kalverra added 2 commits June 25, 2026 16:02

Merge branch 'develop' of github.com:smartcontractkit/chainlink into …

e2b7c83

…speedUpExecutable

add parallel back

f608e9a

kalverra requested a review from Copilot June 26, 2026 15:47

Copilot started reviewing on behalf of kalverra June 26, 2026 15:47 View session

Copilot AI reviewed Jun 26, 2026

View reviewed changes

kalverra added 3 commits June 26, 2026 11:57

Merge branch 'develop' of github.com:smartcontractkit/chainlink into …

c26c0da

…speedUpExecutable

spelling

b3220c8

PR instructions

ac3ee82

kalverra requested a review from Copilot June 26, 2026 16:11

Copilot started reviewing on behalf of kalverra June 26, 2026 16:11 View session

Copilot AI reviewed Jun 26, 2026

View reviewed changes

prevent races

94e4f2d

kalverra requested a review from Copilot June 26, 2026 17:07

Copilot started reviewing on behalf of kalverra June 26, 2026 17:08 View session

Copilot AI reviewed Jun 26, 2026

View reviewed changes

more race prevention

7685485

kalverra requested a review from Copilot June 26, 2026 17:45

Copilot started reviewing on behalf of kalverra June 26, 2026 17:45 View session

Copilot AI reviewed Jun 26, 2026

View reviewed changes

Comment thread core/capabilities/remote/executable/client_test.go

Comment thread core/capabilities/remote/aggregation/default_mode.go Outdated

Comment thread core/internal/testutils/synctest/synctest.go Outdated

Comment thread core/internal/testutils/synctest/synctest.go Outdated

kalverra added 3 commits June 26, 2026 13:54

comments

cd83ce1

Merge branch 'develop' of github.com:smartcontractkit/chainlink into …

ef9823a

…speedUpExecutable

lint

0c84393

Uh oh!

Conversation

kalverra commented Jun 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Results

How

Important Parts to Review

Uh oh!

github-actions Bot commented Jun 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

CORA - Pending Reviewers

Uh oh!

github-actions Bot commented Jun 24, 2026

Uh oh!

github-actions Bot commented Jun 24, 2026

Uh oh!

trunk-io Bot commented Jun 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cl-sonarqube-production Bot commented Jun 26, 2026

Quality Gate passed

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

kalverra commented Jun 24, 2026 •

edited

Loading

github-actions Bot commented Jun 24, 2026 •

edited

Loading

trunk-io Bot commented Jun 24, 2026 •

edited

Loading