Skip to content

feat(agent-runtime): run PostgreSQL schema setup before deployment - #233

Merged
kc-splunk merged 2 commits into
agent-runtime/ai-tierfrom
feat/agent-runtime/AIP-5169
Sep 17, 2026
Merged

kc-splunk merged 2 commits into
agent-runtime/ai-tierfrom
feat/agent-runtime/AIP-5169

Conversation

@kc-splunk

Copy link
Copy Markdown
Collaborator

Summary

Implements AIP-5169 by preparing the Agent Runtime PostgreSQL schema in an AIService-owned Kubernetes Job before creating the AgentRuntime Deployment.

Changes

  • Add the RELATED_IMAGE_AGENT_RUNTIME_SCHEMA_SETUP image contract across the operator manager config, Helm chart, BOM generation, and cluster setup tooling.
  • Create and monitor the schema-setup Job before AgentRuntime deployment.
  • Pass the referenced checkpoint DB Secret through envFrom and preserve existing AgentRuntime feature environment compatibility.
  • Bridge PG_SSLMODE to PGSSLMODE for the schema-setup image.
  • Track schema Job status in AIService status and block deployment until the Job completes.
  • Add Job rerun fingerprinting for deployment inputs and schema inputs, while avoiding secret values in annotations.
  • Reconcile when the referenced checkpoint Secret changes.
  • Update examples and K0S documentation.
  • Fix MinIO installer downloads to use the current AIStor binary endpoints and fail on HTTP errors.

Compatibility

  • Existing SAIA/data-loader behavior is unchanged.
  • Structured PostgreSQL keys may be supplied by the referenced Secret or AgentRuntime feature environment; legacy DATABASE_URL-only SAIA flows are not required to change.

Validation

  • go test ./pkg/ai/features/agentruntime ./internal/controller
  • bash -n tools/cluster_setup/k0s_cluster_with_stack.sh tools/artifacts_download_upload_scripts/install_minio_ec2.sh
  • kubectl kustomize config/manager
  • git diff --check

Testing-only cluster configuration was intentionally excluded from this PR.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 17, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-09-17T09:38:13.220503Z b92a731 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b92a73172e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread tools/cluster_setup/k0s_cluster_with_stack.sh
Comment thread pkg/ai/features/agentruntime/impl.go
Comment thread pkg/ai/features/agentruntime/impl.go Outdated
Comment on lines +430 to +431
if schemaValues["PGSSLMODE"] == "" && schemaValues["PG_SSLMODE"] != "" {
schemaValues["PGSSLMODE"] = schemaValues["PG_SSLMODE"]

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Bridge PG_SSLMODE values supplied through the Secret

When PG_SSLMODE is supplied in the referenced checkpoint Secret rather than feature.env, this map contains no value to bridge, and envFrom exposes only PG_SSLMODE; it cannot rename it to libpq's PGSSLMODE. The schema process therefore ignores the requested SSL mode, potentially failing against databases that require certificate verification or connecting with weaker defaults, despite the documented allowance for settings to come from the Secret.

Useful? React with 👍 / 👎.

Comment on lines +514 to +515
if job.Annotations[schemaJobManagedAnnotation] != schemaJobManagedValue {
return false

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Reject an unmarked schema Job instead of trusting it

If status.schemaJobId names an AIService-owned Job without this annotation—for example, a completed SAIA post-install Job retained when the mutable feature name is changed to agentruntime—returning false makes reconciliation treat that unrelated Job as the PostgreSQL schema run. A completed Job then allows the Agent Runtime Deployment to start without its schema having been installed; an unmarked Job should trigger replacement or an explicit error.

Useful? React with 👍 / 👎.

featureEnv := buildSchemaSetupEnv(ai.Spec.Feature.Env)
job.Spec = batchv1.JobSpec{
BackoffLimit: ptr(schemaJobBackoffLimit),
TTLSecondsAfterFinished: ptr(int32(86400)),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Retain the Job used as the schema completion record

After a successful schema Job reaches this 24-hour TTL, Kubernetes deletes it while status.schemaJobId still contains its name. The owner deletion event, or any subsequent AIService reconciliation, then observes NotFound, clears the status field, and recreates the same schema Job even though neither fingerprint changed, causing every Agent Runtime schema setup to run repeatedly rather than only when its inputs change. Either retain the Job or persist enough completion and fingerprint state outside the expiring object.

Useful? React with 👍 / 👎.

if schemaJobFailed(job) {
return fmt.Errorf("postgres schema setup Job %q failed", job.Name)
}
return fmt.Errorf("postgres schema setup Job %q is still running", job.Name)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Let Job completion events advance reconciliation

While this Job is running, reconciliation returns an error and relies on a later retry to create the Deployment. However, the controller-wide filter in internal/controller/aiservice_controller.go:240-245 rejects a Job update that changes only status, so the local JobChangedPredicate cannot enqueue the AIService when the Complete condition appears. Completion is therefore noticed only on the next exponentially rate-limited error retry, which can leave longer-running schema jobs blocking deployment well after they finish; allow Job status changes through the global filter or use an ordinary timed requeue for this pending state.

Useful? React with 👍 / 👎.

@kc-splunk
kc-splunk merged commit 483a832 into agent-runtime/ai-tier Sep 17, 2026
1 check passed
@kc-splunk
kc-splunk deleted the feat/agent-runtime/AIP-5169 branch September 17, 2026 11:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant