diff --git a/docs/developer-guide/01-mental-model.md b/docs/developer-guide/01-mental-model.md new file mode 100644 index 0000000..169e5d6 --- /dev/null +++ b/docs/developer-guide/01-mental-model.md @@ -0,0 +1,39 @@ +# Mental model + +The lifecycle the library runs for you, and the operations you implement to plug into it. + +## The managed-resource lifecycle + +You hand the controller a resource type to watch and an implementation of a few operations. The watch enqueues events; a worker fetches the live object, runs the framework-level checks, then asks your code to **observe** it and — based on what you report — **create**, **update**, **delete**, or do nothing, persisting conditions and managing the finalizer along the way. + +```mermaid +stateDiagram-v2 + [*] --> Fetch + Fetch --> Paused: paused + Fetch --> Observe: active + Observe --> Create: missing + Observe --> Update: drifted + Observe --> Delete: being deleted + Observe --> UpToDate: matches + Create --> [*] + Update --> [*] + Delete --> [*] + UpToDate --> [*] + Paused --> [*] +``` + +You implement the boxes; the library implements everything else — the watch and queue, fetching the live object, the pause check, finalizer add/remove, the safety steps around creation, persisting conditions and status, and requeue/retry. The full flow is in [`02-architecture.md`](./02-architecture.md). + +## The operations you implement + +- **Observe** — look at the external world and report whether the resource **exists** and whether it is **up to date**. +- **Create / Update / Delete** — make the external world match the desired state. + +The contract that keeps this safe: + +- **Idempotent and non-blocking.** The framework re-enqueues and retries, so create must tolerate an already-existing resource and delete a missing one. +- **The object arrives untyped.** You read the desired state out of the object's spec directly; there is no generated type. +- **The framework persists conditions and status, not your code.** It writes the `Ready`/`Synced` conditions onto the object's status; you report observations and errors. + +> ### Headline difference from provider-runtime: there is no Connect +> `provider-runtime` has a **Connect** step that builds a client for each resource. This library has none — you register a single implementation for the whole controller, and build whatever clients you need inside the four operations. (The CDC, for instance, constructs its dynamic and Helm clients per reconcile.) Everything else about the lifecycle is the same; see [`04-equivalence.md`](./04-equivalence.md). diff --git a/docs/developer-guide/02-architecture.md b/docs/developer-guide/02-architecture.md new file mode 100644 index 0000000..42b62c4 --- /dev/null +++ b/docs/developer-guide/02-architecture.md @@ -0,0 +1,59 @@ +# Architecture + +What the library provides, how a reconcile flows, and the truth about worker scaling. + +## What's in the box + +- **The controller** — the watch over a resource type, a priority work queue, a pool of workers, and the dispatch that routes each event to one of your operations. +- **Event handling** — the mapping of "what changed" to "which operation runs" is configurable (the CDC, for example, treats an update as an observe). +- **The priority queue** — de-duplicating, priority-aware, and rate-limited, so bursts are fair and retries back off. +- **Pluggable-logic helpers** — for reading desired state out of an untyped object and for writing conditions and status back. +- **Runtime type resolution** — turning a kind into the resource type to act on, since nothing is known at compile time. +- **Cross-cutting** — logging, optional telemetry, and an optional metrics server. + +## How a reconcile flows + +```mermaid +flowchart TB + W[watch over the resource type] -->|enqueue| Q[priority queue] + Q --> WK[worker] + WK -->|fetch the live object| K8s[(Kubernetes API)] + WK --> CH{framework checks: pause, finalizer, create-safety} + CH --> D{dispatch} + D -->|observe| OBS[Observe] + D -->|create / update / delete| CUD[Create / Update / Delete] + OBS -.->|self-enqueue create or update if needed| Q +``` + +A worker pulls an event, fetches the live object, runs the framework checks (skip if paused; drop the finalizer if it's an orphan being deleted; refuse if a create is unconfirmed; add the finalizer for live objects), then dispatches to one of your operations. `Observe` doesn't create or update directly — it reports its findings, and the framework enqueues the appropriate follow-up. + +## Worker scaling: the reality + +The number of workers is a **fixed value passed at startup — there is no autoscaling.** The repository includes a design note on the subject, but it is **advisory**, not an implemented feature: its conclusion is that adding workers in a single instance mostly causes contention, and that the right way to scale is to **shard** — run several controller instances, each handling a slice of the resources via a label selector — and to **rate-limit** against slow external APIs. The priority queue is the fairness mechanism within an instance. So the levers are: the worker count, the priority queue, the rate limiter, and operationally, sharding. Don't go looking for an autoscaler — there isn't one. + +## Two things to keep in mind + +- **Status goes to the status subresource.** Conditions and status are written there, so the watched resource must define a status subresource. +- **Keep "up to date" stable.** The controller writes status, finalizers, and annotations as part of normal operation; if `Observe` reports "out of date" on inputs that didn't really change, you'll get needless churn. (Updates that only change the object's version, not its spec, are already ignored to avoid self-trigger loops.) + +## Management and deletion policies + +Two annotations on the watched object let an operator narrow what the controller is allowed to do. They're set by whoever applies the object, but they directly decide **which of your operations the controller actually calls** — so it matters when you're building a controller (don't assume your `Create` runs just because the object is missing). + +**Management policy** — the annotation `krateo.io/management-policy` — gates the allowed actions: + +| Value | Observe | Create | Update | Delete | +| --- | :---: | :---: | :---: | :---: | +| `default` (when the annotation is absent) | ✓ | ✓ | ✓ | ✓ | +| `observe-create-update` | ✓ | ✓ | ✓ | — | +| `observe-delete` | ✓ | — | — | ✓ | +| `observe` | ✓ | — | — | — | + +`default` is full management. `observe` is the read-only case — the object is owned by something else and the controller only observes it. Under any non-`default` value, the disallowed operations are simply never invoked, even if `Observe` reports the object as missing or drifted. + +**Deletion policy** — the annotation `krateo.io/deletion-policy` — decides what happens to the *external* resource when the watched object is deleted: + +- `delete` (the default when absent) — the external resource is deleted too. +- `orphan` — the external resource is left in place. + +**How they combine on delete.** When the object is being deleted, the controller deletes the external resource only when the management policy permits it **and** the deletion policy asks for it — concretely, when management is `default` and deletion is `delete` (the default), or when management is `observe-delete`. In every other case the external resource is orphaned. One subtlety to keep in mind: `observe-delete` deletes regardless of the deletion policy. diff --git a/docs/developer-guide/03-building-a-controller.md b/docs/developer-guide/03-building-a-controller.md new file mode 100644 index 0000000..1d3c281 --- /dev/null +++ b/docs/developer-guide/03-building-a-controller.md @@ -0,0 +1,21 @@ +# Building a controller + +The steps to stand up a dynamic controller on this library, using the **composition-dynamic-controller (CDC)** and the runnable example as references. + +## 1. Build the controller + +Give the controller the resource type to watch and the options it needs: a resync interval, a rate limiter, optional label/field selectors (to scope or shard it), and — if you want — a remapping of which change triggers which operation (the CDC remaps update to observe). Client-side throttling is off by default, leaving load to the API server's fairness controls. + +## 2. Implement the operations over untyped objects + +Implement **Observe**, **Create**, **Update**, **Delete**, following the contract in [`01-mental-model.md`](./01-mental-model.md). Read desired state out of the object's spec, and write conditions and status back through the provided helpers (status lands on the status subresource). There is **no Connect step** — build whatever clients you need inside these operations. + +## 3. Register the client and run + +Register your implementation with the controller and run it with a chosen number of workers. The CDC does exactly this, and the repository's runnable integration example shows the whole flow end to end against a real cluster with a no-op implementation. + +## Testing + +Unit tests can drive the controller against a fake dynamic client — no cluster required. The integration example covers the real path on a throwaway cluster. + +> One thing to get right: the resource type you watch must line up with how the object's kind resolves to a resource. For kinds with irregular plurals, supply a custom resolver so the controller reads and writes the right resource. diff --git a/docs/developer-guide/04-equivalence.md b/docs/developer-guide/04-equivalence.md new file mode 100644 index 0000000..ff65766 --- /dev/null +++ b/docs/developer-guide/04-equivalence.md @@ -0,0 +1,43 @@ +# Equivalence: provider-runtime ⟷ unstructured-runtime + +> These two libraries deliberately implement the **same managed-resource lifecycle**. `provider-runtime` drives it for **typed** custom resources; `unstructured-runtime` (this library) drives it for **untyped** objects at a resource type chosen at runtime. This appendix lines them up so a change to one can be mirrored in the other. **If you change lifecycle semantics in one, mirror it in the other — divergence is a bug, not a feature.** + +This same appendix appears in the **provider-runtime** developer guide. + +## Concept map + +| Concept | provider-runtime (typed) | unstructured-runtime (dynamic) | +| --- | --- | --- | +| What you manage | a typed custom resource, registered in a scheme | untyped objects at a resource type chosen at runtime | +| Operations you implement | Observe / Create / Update / Delete | the same four operations | +| Per-resource setup | a **Connect** step builds a client for each resource | **none** — you register one client for the whole controller | +| What Observe reports | exists, up-to-date, plus "defaults filled in" and a drift description | exists, up-to-date (minimal) | +| How it's wired | a reconciler into a controller-runtime manager | a controller built directly on lower-level primitives — no manager | +| Work queue | the manager's rate-limited queue | a local priority queue (de-duplicating, priority-aware) | +| Concurrency | a max-concurrent-reconciles setting | a fixed number of workers — **no autoscaling** | +| Finalizer | a configurable finalizer | a fixed finalizer name | +| Conditions / status | standard conditions on the typed resource's status | the same conditions written onto the untyped object's status | +| Standard conditions | `Ready` and `Synced`, with the same reasons | the same | +| Pause | a paused annotation short-circuits to a paused condition | the same | +| Create safety | pending / succeeded / failed create-tracking, plus a grace period | the same tracking | +| Lifecycle policies | the loop may skip operations or orphan on delete | the same | +| Type resolution | scheme / RESTMapper (compile-time types) | runtime pluralization | +| Event recorder, logger | shared Krateo helpers | the same | +| Origin | trimmed fork of crossplane-runtime's managed reconciler | the dynamic analog of the same lifecycle | + +## Invariants that must stay equivalent + +- **Branching from Observe** — missing ⇒ create; exists but drifted ⇒ update; otherwise mark success and requeue. +- **Finalizer discipline** — add the finalizer before creating the external resource; remove it only after a confirmed delete. +- **Create safety** — mark the create pending before doing it, record success or failure after, and refuse to proceed while a create is unconfirmed. +- **Pause** — a paused resource short-circuits to a paused condition without touching the external resource. +- **Conditions** — maintain `Ready` and `Synced` with the same reasons; the framework persists them, not your code. +- **Idempotency** — operations must be idempotent and non-blocking, and a conflict is a requeue, not an error. + +## Where they legitimately differ (and why) + +- **The Connect step** exists only in provider-runtime — typed providers often build a per-resource client; the dynamic controller registers a single client instead. +- **What Observe reports is richer** in provider-runtime (it also carries "defaults filled in" and a drift description); the dynamic side keeps the minimal two-signal form. +- **The plumbing differs** — provider-runtime rides a controller-runtime manager; unstructured-runtime wires the lower-level primitives itself and brings its own priority queue. +- **Concurrency** — a max-concurrent setting versus a fixed worker count (the dynamic side explicitly favors sharding over autoscaling). +- **Type handling** — compile-time types versus runtime resolution of untyped objects. diff --git a/docs/developer-guide/README.md b/docs/developer-guide/README.md new file mode 100644 index 0000000..6861f4d --- /dev/null +++ b/docs/developer-guide/README.md @@ -0,0 +1,28 @@ +# unstructured-runtime — Developer Guide + +A contributor-facing guide to the library that gives Krateo's **dynamic** controllers a managed-resource loop over untyped objects: implement a small set of operations, and the framework reconciles any resource type. + +> Audience: engineers **building a controller on top of this library, or maintaining the library itself**. This guide explains *ideas and flows*, not line-by-line code. For product concepts, see [docs.krateo.io](https://docs.krateo.io). + +## What it is + +`unstructured-runtime` is a controller framework for reconciling Kubernetes resources **dynamically** — using untyped objects rather than generated, compile-time types. A controller author targets a resource type chosen at runtime and implements a handful of operations; the framework owns everything else: the watch over that resource type, a de-duplicating priority work queue, a worker pool, finalizers, status and conditions, event recording, retry and rate-limiting, and metrics. + +It is **"unstructured"** because the consuming controller doesn't know the resource's type at compile time. The primary consumer, the **composition-dynamic-controller (CDC)**, is told its resource type at startup. Unlike `provider-runtime`, this library does not build on a controller-runtime manager; it wires the lower-level pieces directly. See [`01-mental-model.md`](./01-mental-model.md). + +> **Sibling library.** `provider-runtime` is the **typed analog** of this library — the same lifecycle applied to a compile-time-typed resource. The two are meant to stay functionally equivalent; the mapping is in [`04-equivalence.md`](./04-equivalence.md). + +## Documents in this folder + +| Document | What it covers | +| --- | --- | +| [`01-mental-model.md`](./01-mental-model.md) | The managed-resource lifecycle and the operations you implement (and the headline difference from provider-runtime: no Connect step). | +| [`02-architecture.md`](./02-architecture.md) | What the library provides, how a reconcile flows, and the truth about worker scaling. | +| [`03-building-a-controller.md`](./03-building-a-controller.md) | The steps to stand up a controller, using the CDC and the runnable example as references. | +| [`04-equivalence.md`](./04-equivalence.md) | How `provider-runtime` and `unstructured-runtime` line up, concept by concept. | + +## See also + +- **Ecosystem overview (canonical)** — how Krateo Composable Operations fits together lives in the **core-provider** repo: `core-provider/docs/developer-guide/00-ecosystem-overview.md`. +- **The exemplar consumer** — the **composition-dynamic-controller** is the reference controller built on this library. +- **Runnable example** — the integration example in the repo is the canonical getting-started: build the controller, register a client, run it against a real cluster.