Skip to content

fix(sync): remap shared-code, orchestrator and schedule links, add clone warnings (CLI-24) - #814

Merged
soustruh merged 2 commits into
mainfrom
fix/sync-clone-reference-links
Sep 30, 2026
Merged

soustruh merged 2 commits into
mainfrom
fix/sync-clone-reference-links

Conversation

@soustruh

Copy link
Copy Markdown
Contributor

This PR is based on #811 and shows only its own commit. Merge #811 first, with a merge commit. GitHub then changes the base of this PR to main.

What was wrong

sync clone copies a reference tree into a new project and runs sync push. Push set only two kinds of links to the new IDs: the configId of keboola.flow job tasks, and the variables links of transformations. These links kept the IDs of the reference project:

  • shared code (shared_code_id, shared_code_row_ids and the {{<row id>}} placeholders in the script),
  • the schedule target (target.configurationId),
  • the tasks of legacy orchestrations (keboola.orchestrator),
  • task configRowIds.

Clone also copied values encrypted for the source project, and it did not say that data apps need a deploy. The result was status: cloned with errors: [] and no warning.

What changed

  • Push sets these links to the new IDs, in the remote configuration and in the local files. This applies to every push that creates configs, not only to clone.
  • When push cannot set a link (a row that was not created, a failed PUT), it adds an error with LINK_UNRESOLVED or the API error. After a failed PUT, the local files already have the new IDs and the manifest hash stays old, so the next push sends the link again.
  • Clone returns warnings[] and prints them, also with --dry-run:
    • a task that runs a config that is not in the tree,
    • values encrypted for the source project, with keys only and advice per kind (plaintext, kbagent encrypt values, or a new OAuth authorization),
    • each created data app, with the data-app deploy command,
    • each cloned schedule, which clone does not activate. The warning has the flow schedule command when only one schedule runs the flow. Clone does not activate schedules on purpose: right after a clone the target has no tables and cannot decrypt the copied values, so scheduled runs would fail.
  • Clone reports the warnings only in the run that creates the configs.
  • New result key link_remaps with a count per link kind. flow_task_remaps still counts only flow tasks.
  • The clone output code moved to commands/_sync_clone_render.py, so commands/sync.py gets smaller.

Tests

tests/test_sync_clone_links.py builds a reference tree with the real init_sync and pull from fake Storage and Data Science clients, and runs the real clone_project into an empty fake target. It checks:

  • each link on the remote and in the local files, and a second clone that returns no_changes,
  • a failed PUT for each link kind, which the next push repairs,
  • rows whose create failed,
  • a --branch clone, a disabled schedule and two schedules on one flow,
  • each warning kind, and that the dry run gives the same warnings.

make check passes.

Docs: gotchas.md (since vNEXT), sync-workflow.md, commands-reference.md, keboola-expert.md, context.py, CLAUDE.md, docs/error-codes.md. No version bump, no changelog entry.

Fixes CLI-24

@linear-code

linear-code Bot commented Sep 30, 2026

Copy link
Copy Markdown

CLI-24

@keboola-pr-reviewer-bot keboola-pr-reviewer-bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Reviewer error — this is not a verdict. The PR reviewer could not complete this review, so no approval is implied. The failure has been logged for the operator.

reviewer could not complete: opencode exited 1: (no stderr — check the reviewer logs for the trace summary)

Retry with @keboola-pr-reviewer review once the underlying issue clears.

@soustruh

Copy link
Copy Markdown
Contributor Author

@keboola-pr-reviewer review

@keboola-pr-reviewer-bot keboola-pr-reviewer-bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Reviewer error — this is not a verdict. The PR reviewer could not complete this review, so no approval is implied. The failure has been logged for the operator.

reviewer could not complete: opencode exited 1: (no stderr — check the reviewer logs for the trace summary)

Retry with @keboola-pr-reviewer review once the underlying issue clears.

@soustruh soustruh closed this Sep 30, 2026
@soustruh soustruh reopened this Sep 30, 2026
@soustruh
soustruh requested a review from zajca September 30, 2026 06:31

@zajca zajca left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable findings from the automated review.

Comment thread src/keboola_agent_cli/services/_sync_clone_warnings.py Outdated
Comment thread src/keboola_agent_cli/services/_sync_clone_warnings.py Outdated
@soustruh
soustruh requested a review from zajca September 30, 2026 07:55

@zajca zajca left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No actionable findings were found by the automated review.

@soustruh
soustruh merged commit c5d443b into main Sep 30, 2026
4 checks passed
@soustruh
soustruh deleted the fix/sync-clone-reference-links branch September 30, 2026 08:27
@soustruh soustruh mentioned this pull request Sep 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants