Skip to content

Tunable sentinel poll intervals - #3905

Merged
michaeljguarino merged 9 commits into
masterfrom
more-perf-improvements
Jul 27, 2026
Merged

Tunable sentinel poll intervals#3905
michaeljguarino merged 9 commits into
masterfrom
more-perf-improvements

Conversation

@michaeljguarino

@michaeljguarino michaeljguarino commented Jul 23, 2026

Copy link
Copy Markdown
Member

This apparently never was added to the AgentConfiguration crd, adds it in so we can tune in high cluster deployments

Test Plan

Test environment: https://console.plrldemo.onplural.sh/cd/clusters/a1748282-ce8b-48ab-ae7e-326e74fce04e/services/f3f89a54-d1a7-4bc8-9152-daa07ede918d/components

Checklist

  • I have added a meaningful title and summary to convey the impact of this PR to a user.
  • If required, I have updated the Plural documentation accordingly.
  • I have added tests to cover my changes.
  • I have deployed the agent to a test environment and verified that it works as expected (required only when changing agent code).

Plural Flow: console

@michaeljguarino
michaeljguarino requested a review from a team July 23, 2026 16:07
@michaeljguarino michaeljguarino added the enhancement New feature or request label Jul 23, 2026
@soffi-ai

soffi-ai Bot commented Jul 23, 2026

Copy link
Copy Markdown

Soffi AI Summary

This PR delivers several interconnected enhancements to the AI workbench, sentinel/agent infrastructure, and cluster health tooling:

Tunable Sentinel Poll Intervals (primary motivation): Adds a missing sentinelPollInterval field to the AgentConfiguration CRD (agentconfig_types.go, CRD YAML), enabling operators to tune how frequently sentinels poll in high-cluster deployments — something that was apparently never wired up despite being needed for fleet-scale scenarios.

Verify Agent & PR Followup Workflow: Introduces a new verify agent capability — a new WorkbenchJobActivityType.VERIFY enum value, a WorkbenchJobActivityStatus.REJECTED status, and a workbenchPrFollowup GraphQL mutation — enabling the AI workbench to perform post-PR verification steps and handle PR follow-up interactions.

Sentinel Tooling & Subagent Setup: Adds sentinel tools and subagent configuration to the workbench infrastructure capability (WorkbenchInfrastructure.sentinels field), allowing workbench agents to interact with sentinel resources. Also adds a new VERIFICATION tool category and CODING/OBSERVABILITY categories to WorkbenchToolCategory.

Configurable Health Map Limits: Adds a healthmapClusterCount field to ConsoleConfiguration and wires it through the frontend (ClusterHealthScoresHeatmap, ClustersCharts, Home) so the health map cluster count ceiling is dynamically configurable rather than hardcoded. Fixes a related health map display bug.

Agent Runtime Improvements: Exposes clusterId on AgentRuntimeAttributes (required for user-authenticated upserts), improves the agent runtime permissions modal UI, and adds configurable args to the deployment operator agent.

Cleanup: Removes bound roles from the me GraphQL query (simplifying the login query), removes associated UI code in user management components (PersonaActions, User, GroupsList, misc.tsx), and jitters component SHA expirations to reduce thundering-herd cache invalidation.

Commits

Commit Summary
856b47a Adds a sentinelPollInterval field to the AgentConfiguration CRD and its generated deepcopy/YAML, so sentinel polling frequency can be tuned per-cluster deployment — the core motivation for this PR.
b07733b Removes boundRoles from the me GraphQL query and cleans up associated user management UI components (PersonaActions, User, GroupsList, misc.tsx) that consumed it, reducing query payload size.
325e18f Adds a verify state to the workbench job activity engine, introducing WorkbenchJobActivityType.VERIFY and WorkbenchJobActivityStatus.REJECTED enum values for post-PR verification workflows.
45242b3 Implements the verify agent capability end-to-end, wiring up the new verify activity type into workbench job activity rendering and collapse logic on the frontend.
9834d94 Adds the workbenchPrFollowup GraphQL mutation and its corresponding RootMutationTypeWorkbenchPrFollowupArgs input type, enabling AI workbench jobs to trigger follow-up actions on open PRs.
a4c2804 Makes the health map cluster count limit configurable by adding healthmapClusterCount to ConsoleConfiguration, propagating it through ClusterHealthScoresHeatmap, ClustersCharts, and the home dashboard, and adding a corresponding Helm/secrets entry.
b697cd8 Fixes a health map display bug — likely a rendering or data-binding issue surfaced after the configurable limit work in the prior commit.
a9d1fa7 Adds sentinel tools and subagent setup to the workbench, including new sentinels fields on WorkbenchInfrastructure/WorkbenchInfrastructureAttributes, a VERIFICATION tool category, and CODING/OBSERVABILITY tool categories in the WorkbenchToolCategory enum.
451e55c Jitters component SHA cache expirations to prevent thundering-herd invalidation when many components expire simultaneously, complementing existing jitter logic elsewhere.

Deploy in Soffi


Updated: 2026-07-27 20:51 UTC

@greptile-apps

greptile-apps Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR exposes the sentinel reconciler's poll interval as a tunable setting via the AgentConfiguration CRD, replacing the previously hardcoded 30-second constant in console.go. It also upgrades the sentinel cache to use the cache.Store interface (backed by DynamicCache) so the cache TTL adapts at runtime when the interval changes.

  • Adds sentinelPollInterval to AgentConfigurationSpec, the CRD schema, the CLI flag layer (--sentinel-poll-interval, default 30s), and the ConfigurationManager with full merge/override semantics consistent with all other interval fields.
  • Introduces EffectivePollInterval and ControllerCacheTTLFunc helpers in the sentinel package, plus pollJitterWindow that falls back to 15s when polling is disabled, all covered by a new interval_test.go.
  • Adds deep copy entries for SentinelPollInterval (and incidental StreamingProxy/Memory fields in AgentRuntimeSpec) in the auto-generated file.

Confidence Score: 5/5

Safe to merge — the change is additive, fully backward-compatible (new field is optional with a matching CLI default), and the new dynamic-interval logic is covered by tests.

All changed paths follow the established pattern for configurable intervals used by other reconcilers. The sentinel cache upgrade to DynamicCache mirrors how other caches already work. New test coverage validates disable, jitter window, and cache TTL scenarios. No existing behavior is removed; the hardcoded constant is simply replaced by the same default value flowing through the proper config layer.

No files require special attention.

Important Files Changed

Filename Overview
go/deployment-operator/pkg/controller/sentinel/reconciler.go Upgrades sentinelCache to cache.Store interface, switches to DynamicCache with ControllerCacheTTLFunc, and makes GetPollInterval dynamic via EffectivePollInterval. Adds pollJitterWindow helper with 15s fallback when polling is disabled.
go/deployment-operator/pkg/common/config.go Adds sentinelPollInterval field, its parse/set logic in setValueLocked, override in mergeAgentConfigurationSpec, and GetSentinelPollInterval accessor; consistent with all existing interval fields.
go/deployment-operator/cmd/agent/args/args.go Adds sentinel-poll-interval CLI flag with 30s default, SentinelPollInterval() accessor, and wires it into AgentConfigurationDefaults; pattern is consistent with existing interval flags.
go/deployment-operator/pkg/controller/sentinel/interval_test.go New test file covering disable-by-config, jitter window with custom interval, jitter window fallback when polling is disabled, and dynamic cache TTL calculation.
go/deployment-operator/api/v1alpha1/agentconfig_types.go Adds SentinelPollInterval field to AgentConfigurationSpec with appropriate doc comment and omitempty tag.
go/deployment-operator/api/v1alpha1/zz_generated.deepcopy.go Auto-generated deep copy additions for SentinelPollInterval, StreamingProxy, and Memory fields; standard code-gen pattern.
go/deployment-operator/cmd/agent/console.go Removes the hardcoded 30s sentinelPollInterval constant and replaces it with args.SentinelPollInterval() at the reconciler construction site.
go/deployment-operator/cmd/agent/main_test.go Extends existing AgentConfiguration tests to cover SentinelPollInterval default (30s), override (5m), and reset back to default.
go/deployment-operator/pkg/controller/sentinel/socket_publisher.go Changes sentinelRunCache field type from *cache.Cache to cache.Store interface, consistent with the reconciler change.
go/deployment-operator/config/crd/bases/deployments.plural.sh_agentconfigurations.yaml CRD schema updated to include sentinelPollInterval with accurate description; straightforward generated change.

Reviews (1): Last reviewed commit: "Tunable sentinel poll intervals" | Re-trigger Greptile

@michaeljguarino

Copy link
Copy Markdown
Member Author

plural deploy this

@plural-copilot plural-copilot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@michaeljguarino
michaeljguarino force-pushed the more-perf-improvements branch from b9622e3 to c59c1e9 Compare July 23, 2026 16:31
@michaeljguarino

Copy link
Copy Markdown
Member Author

plural deploy this

@plural-copilot plural-copilot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚀 Deployed for testing via https://github.com/pluralsh/plrl-up-demos/pull/2347

Console image tag bumped to sha-c59c1e9 (commit c59c1e986d77963da855765ac9d8107f60d280be) on the mgmt console service.

@michaeljguarino

Copy link
Copy Markdown
Member Author

plural deploy this

@plural-copilot plural-copilot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚀 Deployed! Console image pinned to sha-c59c1e9 (commit c59c1e9) via GitOps update PR: https://github.com/pluralsh/plrl-up-demos/pull/2348

Once that PR merges and syncs, the test environment will be running this build.

@michaeljguarino

Copy link
Copy Markdown
Member Author

plural deploy this

@plural-copilot plural-copilot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deployed via https://github.com/pluralsh/plrl-up-demos/pull/2353 using image tag sha-4a9865c.

@michaeljguarino
michaeljguarino force-pushed the more-perf-improvements branch from 727035d to 08f96c1 Compare July 24, 2026 13:04
@michaeljguarino

Copy link
Copy Markdown
Member Author

plural deploy this

@plural-copilot plural-copilot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deployed for testing: https://github.com/pluralsh/plrl-up-demos/pull/2368

This updates the console service in the test environment to use image tag sha-08f96c1 (head commit of this PR) so the tunable sentinel poll interval change can be verified.

@michaeljguarino
michaeljguarino force-pushed the more-perf-improvements branch 3 times, most recently from 13e0f7b to 487aac2 Compare July 24, 2026 19:25
@michaeljguarino

Copy link
Copy Markdown
Member Author

plural deploy this

@plural-copilot plural-copilot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@michaeljguarino
michaeljguarino force-pushed the more-perf-improvements branch from a823b4e to 8317b67 Compare July 24, 2026 21:25
@michaeljguarino

Copy link
Copy Markdown
Member Author

plural deploy this

@plural-copilot plural-copilot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@michaeljguarino

Copy link
Copy Markdown
Member Author

plural deploy this

@plural-copilot plural-copilot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deploy PR created: https://github.com/pluralsh/plrl-up-demos/pull/2376
Image tag: sha-1a1ab21

@michaeljguarino
michaeljguarino force-pushed the more-perf-improvements branch 3 times, most recently from 58e1a28 to 02220d1 Compare July 25, 2026 17:10
@michaeljguarino
michaeljguarino force-pushed the more-perf-improvements branch from 02220d1 to 034907f Compare July 26, 2026 03:51
@michaeljguarino

Copy link
Copy Markdown
Member Author

plural deploy this

@plural-copilot plural-copilot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@michaeljguarino

Copy link
Copy Markdown
Member Author

plural deploy this

@plural-copilot plural-copilot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@michaeljguarino
michaeljguarino force-pushed the more-perf-improvements branch 2 times, most recently from 490a67c to cef8b52 Compare July 27, 2026 14:50
@michaeljguarino

Copy link
Copy Markdown
Member Author

plural deploy this

@plural-copilot plural-copilot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@michaeljguarino
michaeljguarino force-pushed the more-perf-improvements branch 2 times, most recently from 78546b1 to 5073478 Compare July 27, 2026 16:05
@michaeljguarino

Copy link
Copy Markdown
Member Author

plural deploy this

@plural-copilot plural-copilot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@michaeljguarino
michaeljguarino force-pushed the more-perf-improvements branch from 5073478 to 1670bbc Compare July 27, 2026 20:37
@michaeljguarino
michaeljguarino force-pushed the more-perf-improvements branch from 1670bbc to 451e55c Compare July 27, 2026 20:51
@michaeljguarino

Copy link
Copy Markdown
Member Author

plural deploy this

@plural-copilot plural-copilot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@michaeljguarino
michaeljguarino merged commit 4f86671 into master Jul 27, 2026
110 checks passed
@michaeljguarino
michaeljguarino deleted the more-perf-improvements branch July 27, 2026 21:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants