docs: add agent-oriented knowledge base (AGENTS.md + .kb/)

.kb/adding-a-cloud.md

@@ -0,0 +1,29 @@
+# Preface
+
+End-to-end checklist for adding a new cloud backend to pycloudlib. Read before starting a new backend; cross-reference `.kb/cloud-abstraction.md` for the contract.
+
+Read the top-level `.kb/agents.md` file before continuing below.
+
+
+# Overview
+
+A new backend is a subpackage under `pycloudlib/<cloud>/` providing a concrete `BaseCloud` subclass and a concrete `BaseInstance` subclass, wired into the public API, build, config template, docs, examples, and tests. Each step below mirrors how existing backends are structured.
+
+
+# Important
+
+1. **Subpackage** `pycloudlib/<cloud>/` with `__init__.py`, `cloud.py` (`<Cloud>(BaseCloud)`), `instance.py` (`<Cloud>Instance(BaseInstance)`). Add `errors.py`/`util.py` only if needed; cloud-specific exceptions MUST inherit from `PycloudlibException` (root in `pycloudlib/errors.py`).
+2. **Implement every abstract method** of `BaseCloud` and `BaseInstance` (read `cloud.py`/`instance.py` for the list). If a method is genuinely unsupported, raise `PycloudlibError` (see `Openstack.released_image`).
+3. **Register the class** in `pycloudlib/__init__.py` (import + `__all__` entry).
+4. **Add SDK deps** to `pyproject.toml` `[project.dependencies]` and, if the SDK lacks type stubs, add it to the `[[tool.mypy.overrides]] ignore_missing_imports` list. Prefer strict typing where possible.
+5. **Config template**: add a `[<cloud>]` section to `pycloudlib.toml.template` mirroring constructor kwargs; uncommented keys = required. Update `.kb/configuration.md` if precedence behavior changes.
+6. **Docs**: add `docs/clouds/<cloud>.md` (user-facing) and `docs/source/pycloudlib.<cloud>.*.rst` per module (Sphinx autodoc). Add the cloud to the `docs/clouds` toctree glob if not already covered by `clouds/*`.
+7. **Example**: add `examples/<cloud>.py` demonstrating launch/snapshot/cleanup. See `examples/.kb/examples.md`.
+8. **Tests**: add `tests/unit_tests/<cloud>/` with mocked SDK tests (CI runs these). Add `tests/integration_tests/<cloud>/` for live tests, gated by pytest markers `ci`/`main_check` as appropriate (see `tests/.kb/testing.md`).
+9. **Knowledge base**: create `pycloudlib/<cloud>/.kb/<cloud>.md` capturing config keys, SDK auth flow, cloud-specific methods, and gotchas; link it from `pycloudlib/AGENTS.md` _Documents_.
+10. **Verify**: `tox -e ruff`, `tox -e mypy`, `tox -e py38` (or `make test`), and `tox -e docs` (Sphinx with `-W` treats warnings as errors, so missing autodoc targets will fail the build).
+
+
+# Architecture
+
+For the abstract contract each backend must satisfy, read `cloud.py`/`instance.py` directly and see `.kb/cloud-abstraction.md` for the architectural intent. The steps above are the wiring checklist around that contract; this article intentionally does not restate the API surface.

.kb/agents.md

@@ -0,0 +1,45 @@
+# Preface
+
+This repository follows strict conventions for organizing agent-oriented knowledge documents, and this document is required reading for any agent that wants to read or write these. The agent-oriented knowledge base is distinct from the user-facing Sphinx documentation under `docs/`; the latter targets library users, while `.kb/*.md` targets agents working on the codebase.
+
+Read the top-level `.kb/agents.md` file before continuing below.
+
+# Overview
+
+Every directory in this repository, including the root, may have its own `AGENTS.md` file and `.kb/` subdirectory. The `AGENTS.md` file gives a general view of the directory, while more specific knowledge lives in `.kb/*.md` files with dashed lowercase names (e.g. `.kb/cloud-abstraction.md`).
+
+The design of this structure has the following key goals:
+
+- **Mechanical** - Agents are the main actors reading and writing the knowledge base.
+- **Generic** - Benefits any agentic workflow, no matter the editor or platform.
+- **Distilled** - Avoids verbose task or plan logs that pollute the context window.
+- **Hierarchical** - Avoids excessive information in a single place that also pollutes the context window.
+- **Human** - Information is readily available and reviewable in a friendly format.
+
+
+# Important
+
+- Read local `AGENTS.md` files upon navigating into a directory.
+- Keep the `.kb/*.md` files updated whenever there is something relevant to be documented or updated; remove content that becomes stale.
+- Follow the header conventions outlined below. Only the _Preface_ header is required; omit other headers if empty or trivial.
+- Do NOT duplicate content that lives in source docstrings or the Sphinx `docs/` tree. Reference it instead.
+- File names use dashed-lowercase (`some-topic.md`), matching pycloudlib's existing doc style.
+
+
+# Headers
+
+The following are the ONLY top-level headers allowed across the `.kb/*.md` files in this repository, to maintain semantic standardization across projects. Sub-headers are okay.
+
+- _Preface_ - A brief introduction outlining the scope and relevance of a specific `*.md` file. This section MUST be at the top of every `.kb/*.md` file so agents can easily grep for it, and the last line of this section MUST be "Read the top-level `.kb/agents.md` file before continuing below." so rules are followed.
+- _Overview_ - High-level summary of the directory, subsystem or knowledge base layout at large. Do NOT use this to list files or directories.
+- _Important_ - Essential directives for the agent outlining critical constraints, behaviors, or rules.
+- _Headers_ - Global registry of header definitions, uniquely hosted at the root `.kb/agents.md` (this file). Do NOT use this header in any other document.
+- _Architecture_ - Structural design details or boundary explanations for a given component. Only use this for software architecture concepts, NOT for directory outlining. Also avoid using this as a code reference (keep that in the code itself).
+- _Directory_ - Only in `AGENTS.md` files, it briefly outlines the contents and structure of the directory the `AGENTS.md` file is in, and potentially nested small directories that do not justify their own `AGENTS.md` file. For `.kb/*.md` files, use _Documents_ instead.
+- _Documents_ - Only in `AGENTS.md` files, it outlines the content of `.kb/*.md` files and also immediately nested `AGENTS.md` child files, to aid agent navigation. It MUST be the last section in the file. It's okay to also mention such filenames inline when they are relevant elsewhere in the text.
+
+For the _Directory_ and _Documents_ listings, format items as a dashed list starting with the file or directory name surrounded by backticks, a dash, and then a brief description:
+
+```
+- `filename` - Terse summary.
+```

.kb/cloud-abstraction.md

@@ -0,0 +1,27 @@
+# Preface
+
+Architectural intent behind the `BaseCloud` / `BaseInstance` abstractions in `pycloudlib/cloud.py` and `pycloudlib/instance.py`. Read before auditing cross-cloud behavior or deciding where new functionality belongs; for the concrete method list and signatures, read the base classes directly.
+
+Read the top-level `.kb/agents.md` file before continuing below.
+
+
+# Overview
+
+`BaseCloud` and `BaseInstance` are ABCs defining a uniform surface every backend implements. The point of the abstraction is that cloud-agnostic callers work unchanged across backends; backends subclass and add cloud-specific helpers without exposing backend-specific shapes to generic code.
+
+
+# Important
+
+- Read `cloud.py` and `instance.py` for the authoritative abstract method list and `__init__` signatures — they are the source of truth and this note intentionally does not restate them.
+- Both base classes are context managers whose `__exit__` calls cleanup and raises `CleanupError` on failure. `CleanupError` usually means leaked resources — do not silently swallow it.
+- Backends track created instances/images in `created_instances`/`created_images` and extend `clean()` (calling `super().clean()` first) for cloud-specific resources (e.g. EC2 VPCs/keys, LXD profiles/snapshots). Generic `launch`/`snapshot` callers rely on this tracking for automatic cleanup.
+- `ImageType` (in `cloud.py`) dispatches image flavors on clouds that support them (Azure, EC2, GCE, LXD); other clouds accept and ignore it via `**kwargs`.
+- Not every abstract method is meaningfully supported by every cloud: some raise `PycloudlibError` (e.g. `Openstack.released_image`/`daily_image`) or fall back to a sibling (e.g. `IBMClassic.daily_image` → `released_image`). Cloud-agnostic code must tolerate this.
+- The `_type` class attribute identifies the backend for logging and helpers.
+
+
+# Architecture
+
+- `BaseInstance` exec is paramiko-based for most clouds, but LXD/QEMU/VMWare translate a non-SSH transport (the `lxc` CLI, a QMP socket, `govc`) into the same `execute`/`run`/`Result` surface. See `.kb/ssh-and-instances.md` for the `Result`/paramiko model.
+- Tag validation lives in `BaseCloud._validate_tag` with per-cloud override rules; invalid tags raise `InvalidTagNameError(tag, rules_failed)`.
+- Configuration is resolved in `BaseCloud.__init__` and shared with instances via the `key_pair`; see `.kb/configuration.md` for precedence.

.kb/configuration.md

@@ -0,0 +1,23 @@
+# Preface
+
+How pycloudlib resolves per-cloud configuration. Read before changing precedence or debugging "must be defined in pycloudlib.toml" errors; for per-cloud keys read `pycloudlib.toml.template` and each cloud's `__init__` docstring.
+
+Read the top-level `.kb/agents.md` file before continuing below.
+
+
+# Overview
+
+Configuration is layered: explicit constructor kwargs > `pycloudlib.toml` values > cloud SDK defaults / env vars. `config.py` parses the TOML; `pycloudlib.toml.template` is the authoritative per-cloud key reference.
+
+
+# Important
+
+- In `pycloudlib.toml.template`, **uncommented keys are required**, commented keys are optional with shown defaults. Do not check a filled-in `pycloudlib.toml` into version control; secrets belong in a secret manager.
+- `Config` (in `config.py`) subclasses `dict` only to raise a clearer `KeyError` on missing-key `__getitem__` access; code using `.get(key)` gets `None` and must handle it. Don't rely on this difference leaking into call sites.
+- When adding a cloud, add a `[<cloud>]` section to `pycloudlib.toml.template` mirroring its constructor kwargs.
+
+
+# Architecture
+
+- `parse_config` (in `config.py`) tries, in order: the `config_file` constructor arg > `$PYCLOUDLIB_CONFIG` > `~/.config/pycloudlib.toml` > `/etc/pycloudlib.toml`. First existing, parseable file wins; later paths are not merged. Read `config.py` for the exact `CONFIG_PATHS` list and `ConfigFile` type.
+- Each cloud's `__init__` resolves each value as `kwarg or self.config.get("key") or default` before constructing its SDK client; `BaseCloud.required_values` only validates that at least one supplied value is non-None. See each `pycloudlib/<cloud>/.kb/<cloud>.md` for that cloud's auth flow.

.kb/development.md

@@ -0,0 +1,25 @@
+# Preface
+
+Dev environment, lint, typecheck, and test commands for pycloudlib. Read before running checks; for env/flag specifics read `tox.ini`, `Makefile`, and `pyproject.toml` directly.
+
+Read the top-level `.kb/agents.md` file before continuing below.
+
+
+# Overview
+
+pycloudlib uses `uv` (wrapped by a thin `Makefile`) and `tox` for the multi-check workflow. Supported Python range is >=3.8; CI covers 3.8/3.10/3.12. ruff handles lint+format, mypy handles typing, pytest runs unit tests with `--doctest-modules`.
+
+
+# Important
+
+- After editing any Python in `pycloudlib/` or `examples/`, run ruff and mypy. Prefer `make test` (full tox) or the individual tox envs defined in `tox.ini`.
+- Default `tox` envlist only **checks** (does not reformat): `ruff`, `mypy`, `py38`. Use `tox -e format` to apply formatting.
+- Unit tests live in `tests/unit_tests/` and run in CI; integration tests in `tests/integration_tests/` need live cloud credentials and are marker-gated (see `tests/.kb/testing.md`).
+- New code: prefer Markdown for new docs (per `docs/contributing.md`), follow ruff's pep257 docstring convention, and keep PRs to a single issue.
+
+
+# Architecture
+
+- `pyproject.toml` is the source of truth for dependencies, ruff/mypy/pytest config, and build (hatchling). mypy has `ignore_missing_imports` overrides for many cloud SDKs and `check_untyped_defs = false` for a known TODO module list — read the `[tool.mypy.overrides]` sections rather than assuming; prefer fixing typing over widening the relaxed list.
+- `tox.ini` defines all envs (`pytest`, `py38`/`py310`/`py312`, `mypy`, `ruff`, `format`, `docs`, `integration-tests*`); read it for exact commands/flags.
+- `Makefile` wraps `uv` (`build`/`install`/`test`/`venv`/`clean`/`publish`); `uv.lock` pins the dependency set; `VERSION` is read by hatchling.

.kb/ssh-and-instances.md

@@ -0,0 +1,23 @@
+# Preface
+
+SSH key handling, paramiko usage in `BaseInstance`, and the `Result` exec model. Read when debugging instance connectivity or key wiring; for signatures and defaults read `instance.py`/`key.py`/`result.py` directly.
+
+Read the top-level `.kb/agents.md` file before continuing below.
+
+
+# Overview
+
+`BaseInstance` reaches instances over SSH via paramiko using a `KeyPair`; commands return a `Result`. SSH clients are lazily opened and reused for the instance lifetime.
+
+
+# Important
+
+- Do not log or echo key material. `KeyPair.__str__` includes paths but not key contents; preserve that boundary when handling keys.
+- `Result` is a `str` subclass whose string value is stdout, so `bool(result)` reflects success and `if not result:` detects failure. See `result.py` for the full attribute surface; do not assume attributes beyond what's defined there.
+- Backends that do not use paramiko for transport (LXD: `lxc` CLI via `subp`; QEMU: QMP socket; VMWare: `govc`) still expose the same `execute`/`run`/`Result` surface for cloud-agnostic callers, translating their transport into a `Result`. See each `pycloudlib/<cloud>/.kb/<cloud>.md`.
+
+
+# Architecture
+
+- `BaseInstance` holds cached, lazily-initialized `_ssh_client`/`_sftp_client` reused across `execute`/`run`/file transfer; the connect logic (with its paramiko exception handling and `boot_timeout`/`ready_timeout` retry) lives in `instance.py`.
+- `BaseCloud` builds `self.key_pair` in `__init__` from config (`public_key_path`/`private_key_path`/`key_name`); `KeyPair` path/`public_key_content`/`UnsetSSHKeyError` behavior is defined in `key.py`. A cloud may expose `use_key(...)` to swap keys at runtime — consult the cloud's `cloud.py`.

AGENTS.md

@@ -0,0 +1,44 @@
+# Preface
+
+pycloudlib is a Python library (Python >=3.8) that launches, interacts with, and snapshots Ubuntu cloud instances across many public and private clouds through a uniform `BaseCloud` / `BaseInstance` abstraction. This file is relevant to any task touching the source tree, build, tests, docs, or examples.
+
+Read the top-level `.kb/agents.md` file before continuing below.
+
+
+# Overview
+
+The package supports Azure, EC2, GCE, IBM VPC, IBM Classic, LXD (containers and VMs), OCI, Openstack, QEMU, and VMWare. Each cloud lives in its own subpackage under `pycloudlib/<cloud>/` and implements the abstract API defined in `pycloudlib/cloud.py` and `pycloudlib/instance.py`. The library is consumed both as a library (via `pycloudlib` top-level exports) and via the example scripts under `examples/`.
+
+
+# Architecture
+
+- `BaseCloud` (`pycloudlib/cloud.py`) and `BaseInstance` (`pycloudlib/instance.py`) define the abstract contract every backend satisfies; read them for the method list, and see `.kb/cloud-abstraction.md` for architectural intent.
+- Each `<cloud>` subpackage provides a concrete cloud class and instance class plus optional helpers (`vpc.py`, `util.py`, `errors.py`, `_util.py`).
+- Cross-cutting modules live at the package root: `config.py` (TOML config), `key.py` (`KeyPair`), `result.py` (`Result`), `errors.py` (root exception hierarchy), `util.py` (release maps, shell helpers), `constants.py`.
+- Configuration is layered: constructor kwargs > `pycloudlib.toml` (see `pycloudlib.toml.template`) > cloud SDK defaults / env vars; see `.kb/configuration.md`.
+
+See `.kb/cloud-abstraction.md` for the abstract API surface agents should rely on, and each `pycloudlib/<cloud>/.kb/<cloud>.md` for backend-specific behavior and gotchas.
+
+
+# Directory
+
+- `pycloudlib/` - The Python package; see `pycloudlib/AGENTS.md` for module-level detail.
+- `tests/` - Unit tests (`unit_tests/`, run in CI) and integration tests (`integration_tests/`, require live cloud credentials). See `tests/AGENTS.md`.
+- `examples/` - Runnable scripts demonstrating each cloud's API. See `examples/.kb/examples.md`.
+- `docs/` - Sphinx user-facing documentation (MyST Markdown + reStructuredText). See `docs/.kb/documentation.md`.
+- `pyproject.toml` - Build (hatchling), dependencies, ruff/mypy/pytest config.
+- `tox.ini` - Tox environments: `ruff`, `mypy`, `py38`/`py310`/`py312` (pytest), `format`, `docs`, `integration-tests*`.
+- `Makefile` - Thin wrappers around `uv` (`build`, `install`, `test`, `venv`, `clean`).
+- `pycloudlib.toml.template` - Reference template for the per-cloud config file users copy to `~/.config/pycloudlib.toml` or `/etc/pycloudlib.toml`.
+- `VERSION` - Single-line version read by hatchling.
+- `uv.lock` - Locked dependency set for the `uv` workflow.
+
+
+# Documents
+
+- `.kb/agents.md` - General rules for the knowledge base reading and writing.
+- `.kb/development.md` - Dev environment, lint/typecheck/test commands, and Python version coverage.
+- `.kb/configuration.md` - `pycloudlib.toml` resolution precedence and per-cloud required keys.
+- `.kb/cloud-abstraction.md` - The `BaseCloud` / `BaseInstance` contract agents should rely on across backends.
+- `.kb/ssh-and-instances.md` - SSH key handling, paramiko usage, and the `Result` exec model.
+- `.kb/adding-a-cloud.md` - Steps to add a new cloud backend end-to-end.

VERSION

@@ -1 +1 @@
-1!11.1.3
+1!11.1.4

docs/.kb/documentation.md

@@ -0,0 +1,24 @@
+# Preface
+
+The `docs/` tree is the user-facing Sphinx documentation (distinct from the agent-oriented `.kb/` knowledge base). Read before editing docs or the Sphinx build; for config read `docs/conf.py` and `docs/index.rst`.
+
+Read the top-level `.kb/agents.md` file before continuing below.
+
+
+# Overview
+
+Sphinx builds MyST Markdown (`.md`) and reStructuredText (`.rst`) into the published docs at pycloudlib.readthedocs.io. Per-cloud guides live in `docs/clouds/*.md`; API autodoc stubs in `docs/source/pycloudlib.<module>.rst`; the index/config in `docs/index.rst`/`docs/conf.py`.
+
+
+# Important
+
+- Build with `tox -e docs` (separate env because it `cd`s into `docs/`); it runs `sphinx-build ... -W`, so **warnings are errors** — a missing autodoc target or broken cross-reference fails the build.
+- Prefer **Markdown** for new prose docs (per `docs/contributing.md`); use `.rst` only where Sphinx autodoc tooling expects it.
+- The `.kb/` knowledge base and `AGENTS.md` files are NOT part of the Sphinx toctree (they live outside `docs/` and are agent-oriented only); do not link them from `docs/index.rst`.
+- `docs/_build/` is generated output — never edit it (cleaned by `make -C docs clean` / root `make clean`).
+
+
+# Architecture
+
+- `index.rst` defines the toctree (read it for the exact groups); `docs/clouds/*` and `docs/examples/*` are globs that pick up new files automatically, so adding a cloud's `docs/clouds/<cloud>.md` and `docs/source/pycloudlib.<cloud>.*.rst` is sufficient for autodoc.
+- `docs/conf.py` inserts `..` into `sys.path` so autodoc can import `pycloudlib`; do not break that path setup. `docs/_static/`/`docs/_templates/` hold theme assets.