Skip to content

feat(sync): opt-in sync of shared SQL workspaces in pull, diff, push and clone (CLI-25) - #815

Merged
soustruh merged 2 commits into
mainfrom
feat/sync-shared-sql-workspaces
Sep 30, 2026
Merged

soustruh merged 2 commits into
mainfrom
feat/sync-shared-sql-workspaces

Conversation

@soustruh

@soustruh soustruh commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor

This PR is based on #814, which is based on #811, and shows only its own commit. Merge #811, then #814, then this PR, each with a merge commit. The order matters: this PR deletes workspaces only through the --force gate that #811 adds.

What was wrong

sync pull, diff, push and clone always skipped workspaces (keboola.sandboxes in ALWAYS_IGNORED_COMPONENTS). A reference project copied with sync clone had no workspaces in the copy.

What changed

  • A tree turns workspace sync on with the manifest key syncWorkspaces, which sync init --with-workspaces sets. The manifest has the key only while it is on, so other trees do not change. An ignoredComponents entry for keboola.sandboxes still wins, and keboola.mcp-server-tool stays ignored.
  • Scope: SQL workspaces (Snowflake, BigQuery) with runtime.shared: true and no parameters.id. This is the rule of the Keboola UI for a workspace that all users see, and the old Keboola CLI (kbc) uses the same parameters.id test for SQL. A workspace from kbagent workspace create (not shared), a Python or R workspace and a legacy sandbox stay out. A workspace that is already tracked stays tracked when somebody makes it private, so push does not create it again.
  • Pull and diff treat a workspace as a normal configuration. It holds no credentials, and nothing on the server changes it later, so a second pull gives no diff.
  • Push creates and updates only the Storage configuration. It starts no job, creates no SQL editor session and loads no tables. The UI creates the session when a user opens the workspace. Push prints no credentials. It gives a warning when backendSize changes, because the change applies only to a new session.
  • Delete needs sync push --force, as for every configuration after fix(sync): push deletes only with --force and skips remote-deleted configs (#792 G, H) #811. For a workspace, push first deletes the SQL editor sessions of that workspace, also the sessions of other users, and then the configuration, like kbc remote workspace delete. A deleted session also drops its Snowflake or BigQuery workspace, and that cannot be undone. Push keeps the configuration and the manifest entry when the session list fails or is not a list, when a session delete fails, or when the configuration has a parameters.id. When a session delete or the final configuration delete fails, the error names the sessions already deleted. A 404 on a session delete counts as deleted. sync push --dry-run --force lists the sessions that push would delete, and a plain push that holds back a workspace delete says that --force also deletes the sessions.
  • sync push --force has the permission class destructive (FLAG_ESCALATIONS), so --deny-destructive and a policy that denies cli:destructive block it.
  • sync clone creates the workspace configurations and gives a warning for each input table that does not exist in the target, because clone creates buckets but no tables. The link re-pointing from fix(sync): remap shared-code, orchestrator and schedule links, add clone warnings (CLI-24) #814 also covers the shared-code and variables links of a workspace.
  • When the key is turned off, the next pull removes the workspace entries as ignored, but keeps a workspace with local edits and reports it as skipped. Only --theirs removes it.
  • New Editor service client (client/editor.py, host editor.<stack>) with the session list and delete. It never asks for credentials.

Behavior changes

sync push --force is now checked as its own operation, sync.push --force, with the class destructive. This applies to every tree, also without syncWorkspaces. A pipeline that runs sync push --force stops with exit code 6 under these policies:

  • a policy that denies cli:destructive, or --deny-destructive,
  • an allow-list (--mode deny) that names only sync.push,
  • a default-allow policy that denies cli:write and allows sync.push.

To allow a forced push, add --allow "sync.push --force" or a glob such as sync.*. permissions-workflow.md and gotchas.md say this, and a test pins each policy shape. The escalation is static, like auth.logout --remove-projects, so a policy decides before any API call and does not depend on what the push would delete.

Tests

  • tests/test_sync_workspaces.py runs the real pull, diff, push and clone on the filesystem with clients built from MagicMock(spec=KeboolaClient), and checks the full list of client calls. It covers the opt-in on and off, the scope rule, no diff after pull, config-only create and update, the size warning, a plain push that holds back the delete, --force with sessions deleted before the config, listing and delete failures, a 404, the refusal for parameters.id, the dry-run preview, the clone warning, dev branches and the Editor client errors.
  • tests/test_sync_clone_links.py checks that a cloned workspace gets its new shared-code and variables links.
  • make check passes.

Not checked against a live project yet. Before the merge, please run this once on a dev stack: create a shared SQL workspace and open it once, sync pull, delete its directory, sync push --dry-run --force, sync push --force, then check that the sessions are gone and that workspace list --orphaned is empty. Also check that the UI shows a workspace created by push with the correct type and opens it.

Docs: gotchas.md (since vNEXT), sync-workflow.md, commands-reference.md, permissions-workflow.md, keboola-expert.md, context.py, CLAUDE.md, docs/TUTORIAL.md. No version bump, no changelog entry.

Fixes CLI-25

@linear-code

linear-code Bot commented Sep 30, 2026

Copy link
Copy Markdown

CLI-25

@keboola-pr-reviewer-bot keboola-pr-reviewer-bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Reviewer error — this is not a verdict. The PR reviewer could not complete this review, so no approval is implied. The failure has been logged for the operator.

reviewer could not complete: opencode exited 1: (no stderr — check the reviewer logs for the trace summary)

Retry with @keboola-pr-reviewer review once the underlying issue clears.

@soustruh

Copy link
Copy Markdown
Contributor Author

@keboola-pr-reviewer review

@keboola-pr-reviewer-bot keboola-pr-reviewer-bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Reviewer error — this is not a verdict. The PR reviewer could not complete this review, so no approval is implied. The failure has been logged for the operator.

reviewer could not complete: opencode exited 1: (no stderr — check the reviewer logs for the trace summary)

Retry with @keboola-pr-reviewer review once the underlying issue clears.

@soustruh
soustruh requested a review from zajca September 30, 2026 06:31

@zajca zajca left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable findings from the automated review.

Comment thread src/keboola_agent_cli/permissions.py
@soustruh
soustruh requested a review from zajca September 30, 2026 07:55

@zajca zajca left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No actionable findings were found by the automated review.

Base automatically changed from fix/sync-clone-reference-links to main September 30, 2026 08:27
@soustruh
soustruh merged commit 717bde8 into main Sep 30, 2026
@soustruh
soustruh deleted the feat/sync-shared-sql-workspaces branch September 30, 2026 08:28
@soustruh soustruh mentioned this pull request Sep 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants