From e92dd464a619a4a89e7b8fdcbcc5256a22f9951f Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Th=C3=A9o=20LAGACHE?= Date: Fri, 11 Sep 2026 09:52:56 +0200 Subject: [PATCH 01/24] docs: add CLI refactor design spec Design validated in brainstorming: flat layout with services/engines/ui layers, OOP commands, Jinja2 remote templates with manifest, compose rendered from .env + databases.json (no state file), non-interactive mode, error hierarchy, OTel-ready telemetry, CI hardening and release workflow replacing ./release. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_017ACLkgaiP48pBMyi7kZZ4a --- .../specs/2026-09-11-cli-refactor-design.md | 546 ++++++++++++++++++ 1 file changed, 546 insertions(+) create mode 100644 docs/superpowers/specs/2026-09-11-cli-refactor-design.md diff --git a/docs/superpowers/specs/2026-09-11-cli-refactor-design.md b/docs/superpowers/specs/2026-09-11-cli-refactor-design.md new file mode 100644 index 0000000..067d703 --- /dev/null +++ b/docs/superpowers/specs/2026-09-11-cli-refactor-design.md @@ -0,0 +1,546 @@ +# Refonte du CLI Portabase — Design + +Date : 2026-09-11 +Statut : validé en brainstorming, en attente de relecture avant plan d'implémentation. + +## 1. Objectifs et périmètre + +Refonte structurelle du CLI sans changement de dépendances majeures (Typer, Rich, questionary, requests, PyYAML conservés ; Jinja2 ajouté). + +Objectifs : + +- Supprimer la duplication entre `commands/agent.py` et `commands/db.py` (~500 lignes du wizard "ajouter une base" copiées). +- Remplacer la génération de `docker-compose.yml` par chirurgie texte (`.replace`, regex, ancres) par un rendu Jinja2 complet et déterministe. +- POO sur toute l'application : commandes, services, moteurs, composants UI en classes. Les utilitaires purs (`slugify`, `generate_password`, `validate_edge_key`) restent des fonctions. +- Chaque commande configurable intégralement par paramètres, sans mode interactif. +- Bibliothèque de composants `ui/` (approche shadcn : tokens, composants stateless, façade unique). +- Catcher d'erreurs unique avec hiérarchie d'exceptions et codes de sortie stables. +- Couche télémétrie prête pour OpenTelemetry, opt-in, sans dépendance immédiate. +- Plus d'auto-update : notification seulement, mise à jour manuelle. +- CI de PR (lint, Gitleaks, Plumber, validation des templates, build smoke), suppression du script `./release`. + +Hors périmètre de cette spec : + +- Tests automatisés (spec suivante ; la structure en tient compte : injection de dépendances, services sans I/O terminal, job `test` vide en CI). +- Sortie machine `--json`. +- Exporter OTel réel (le contrat est posé, l'implémentation viendra avec un endpoint). +- Fichier de spec déclaratif `build -f spec.yml`. +- Support Windows (inchangé : code présent, hors matrice de build). + +## 2. Décisions structurantes + +| Sujet | Décision | +|---|---| +| Layout | Plat, conservé (`main.py` racine, `--paths=.`). Nouveaux dossiers `services/`, `engines/`, `ui/`. | +| État d'une install | Aucun fichier d'état ajouté. Source de vérité = `.env` (variables runtime des conteneurs uniquement) + `databases.json` (contrat agent, inchangé) + lecture structurelle du compose existant pour le seul fait non dérivable (`host_gateway`). | +| Compose | Artefact dérivé, propriété du CLI, re-rendu intégralement à chaque commande mutante. Personnalisations utilisateur via `docker-compose.override.yml` (mécanisme Compose natif). | +| Templates | 100 % remote (S3), versionnés par version CLI exacte, manifest avec sha256, cache disque. Suppression du fallback `latest`. Source dans `templates/` à la racine du dépôt. | +| Création multi-DB | `portabase agent` crée un agent sans base ; les bases s'ajoutent par `portabase db add` (un appel par base). Pas de DSL `--db engine:opts`. | +| Moteurs DB | Classes Python (`engines/`), registre à imports explicites. Pas de manifeste data-driven. | +| Input UI | questionary uniquement. `rich.prompt` et `typer.prompt` bannis (ruff). | +| Non-interactif | Flag `--non-interactive`, env `PORTABASE_NON_INTERACTIVE`, ou `stdin` non-TTY. Géré par `ui.Form`, pas par les commandes. | +| Erreurs | `PortabaseError` + sous-classes, codes de sortie distincts, un seul `try` dans `main.py`. `except:` nus interdits. | +| Télémétrie | Interface `Telemetry`, `NoopTelemetry` par défaut, opt-in via config globale, jamais de prompt. | +| Updater | Notification après la commande (cache 24 h, silencieux si hors ligne), `portabase update` manuel avec vérification de checksum. | +| Release | `bump.yml` (`workflow_dispatch`) remplace `./release`. Workflows de release sur tag inchangés. | +| Mot de passe | `generate_password` retire `$` et `` ` `` des symboles (cassent `--requirepass "${PASSWORD}"` via shell). Ne s'applique qu'aux nouvelles bases. | + +## 3. Structure des fichiers + +``` +cli/ +├── main.py # build_app(), catcher d'erreurs, codes de sortie +├── pyproject.toml # + jinja2 ; pyinstaller/ruff/pytest en groupe dev +│ +├── commands/ +│ ├── base.py # Command ABC, CommandGroup +│ ├── agent.py # AgentCommand +│ ├── dashboard.py # DashboardCommand +│ ├── build.py # BuildCommand +│ ├── lifecycle.py # Start/Stop/Restart/Logs/Uninstall (ex-common.py) +│ ├── db.py # DbCommands : add / remove / list +│ ├── config.py # ConfigCommands : get / set +│ └── update.py # UpdateCommand +│ +├── services/ +│ ├── project.py # AgentProject, DashboardProject, DatabaseSpec, detect_kind() +│ ├── envfile.py # EnvFile +│ ├── compose_facts.py # ComposeFacts (lecture structurelle, jamais d'écriture) +│ ├── renderer.py # ComposeRenderer, RenderResult +│ ├── templates.py # TemplateRepository, Manifest +│ ├── docker.py # DockerRunner +│ ├── ports.py # PortAllocator +│ ├── http.py # HttpClient +│ ├── updater.py # UpdateChecker, Updater +│ └── telemetry.py # Telemetry ABC, NoopTelemetry, ConsoleTelemetry, TelemetryFactory +│ +├── engines/ +│ ├── __init__.py # registry = EngineRegistry([...]) — imports explicites +│ ├── base.py # DbEngine ABC, Field +│ ├── registry.py # EngineRegistry +│ ├── sql.py # StandardSqlEngine + Postgres/PostgresCluster/MySQL/MariaDB/MSSQL/Firebird +│ ├── keyvalue.py # KeyValueEngine + Redis/Valkey +│ ├── mongo.py # MongoEngine +│ ├── sqlite.py # SqliteEngine +│ └── docker_volume.py # DockerVolumeEngine +│ +├── ui/ +│ ├── __init__.py # façade UI +│ ├── theme.py # PALETTE → RICH_THEME + QUESTIONARY_STYLE +│ ├── form.py # Form (flag → prompt → défaut → erreur) +│ └── components/ +│ ├── base.py # Component(console) +│ ├── banner.py message.py section.py summary.py table.py +│ ├── status.py hints.py diff.py prompt.py +│ +├── core/ +│ ├── errors.py # PortabaseError + sous-classes +│ ├── config.py # GlobalConfig (~/.portabase/config.json) +│ ├── version.py # current_version() +│ └── utils.py # slugify, generate_password, validate_edge_key — fonctions pures +│ +├── templates/ # source des templates remote (assets, pas un package Python) +│ ├── agent.yml.j2 +│ ├── dashboard.yml.j2 +│ ├── engines.map.json # clé moteur → template (pour engines-check et manifest) +│ └── engines/ +│ ├── postgresql.yml.j2 mysql.yml.j2 mariadb.yml.j2 mssql.yml.j2 +│ ├── firebird.yml.j2 mongodb.yml.j2 redis.yml.j2 valkey.yml.j2 +│ +├── scripts/ +│ └── render_check.py # rend tous les templates avec fixtures, valide YAML + compose config +│ +├── .github/workflows/ +│ ├── ci.yml # PR : lint, render-check, engines-check, gitleaks, plumber, build-smoke, test +│ ├── bump.yml # workflow_dispatch : bump version + tag +│ ├── templates-hotfix.yml # workflow_dispatch : re-upload templates vers une version existante +│ ├── release.yml, release-candidate.yml, python.yml, github.yml # inchangés (hors durcissement) +│ └── templates-upload.yml # + génération manifest.json, source templates/ +│ +├── .gitleaks.toml +└── supprimés : release, templates/compose.py, templates/__init__.py, commands/common.py, + core/network.py, core/docker.py, .github/assets/templates/ +``` + +Règle de dépendance, descendante uniquement : + +- `commands` → `services`, `engines`, `ui`, `core` +- `services` → `engines`, `core` (jamais `ui` : un service lève, n'affiche rien) +- `engines` → `core` +- `ui` → `core` + +## 4. Commandes + +### 4.1 `Command` + +```python +class Command(ABC): + name: str + help: str + panel: str = "General" + + def __init__(self, ui: UI, telemetry: Telemetry): ... + def register(self, app: typer.Typer) -> None: + app.command(self.name, help=self.help, rich_help_panel=self.panel)(self.run) + + @abstractmethod + def run(self, *args, **kwargs) -> None: ... +``` + +`base.py` wrappe `run` dans `telemetry.span(f"command.{name}")`. Les dépendances (`DockerRunner`, `TemplateRepository`, `EngineRegistry`, `PortAllocator`) sont injectées par constructeur dans `main.build_app()`. + +Signatures Typer en `Annotated[...]`. Chaque option qui correspond à une question du wizard a une valeur par défaut `None` : présente → utilisée, absente → prompt (interactif) ou défaut/erreur (non-interactif). Une seule méthode `_collect()` par commande, aucun `if non_interactive` dans la logique métier. + +### 4.2 Inventaire + +| Commande | Options notables | Effet | +|---|---|---| +| `agent NAME` | `--key`, `--tz`, `--polling`, `--host-gateway/--no-host-gateway`, `--start`, `--force`, `--non-interactive` | crée le dossier, `.env`, `databases.json` vide, rend le compose. Ne crée aucune base. | +| `dashboard NAME` | `--port`, `--db-mode external\|internal\|custom`, `--db-host/--db-port/--db-name/--db-user/--db-password-stdin`, `--start`, `--force` | crée `.env`, rend le compose. | +| `db add NAME` | `--engine`, `--mode new\|existing`, `--auth/--no-auth`, `--name`, `--host`, `--port`, `--database`, `--user`, `--password`, `--password-stdin`, `--path`, `--volume`, `--container`, `--label`, `--keep-ownership`, `--clean-mode` | collecte selon le moteur, mute `.env` + `databases.json`, re-rend. Flag fourni mais non pertinent pour le moteur/mode → `ValidationError`. | +| `db remove NAME` | `--id` ou `--name`, `--purge-volume` | retire l'entrée, retire les variables `.env` du service, re-rend. Le volume Docker n'est supprimé que sur `--purge-volume`. | +| `db list NAME` | — | lecture seule. | +| `build PATH` | `--diff`, `--stdout`, `--inline-env`, `--output DIR` | re-rend depuis l'état. Sans option : écrit en place (= migration legacy). `--inline-env` substitue les valeurs au lieu de `${VAR}` avec avertissement secrets en clair. | +| `start/stop/restart/logs/uninstall PATH` | inchangées (`uninstall --force`) | n'utilisent pas le renderer, fonctionnent sur toute install. | +| `config get/set` | inchangées + clés `telemetry`, `telemetry_endpoint`, `channel` | config globale. | +| `update` | — | mise à jour manuelle avec vérification checksum. | + +Options globales : `--verbose`, `--debug`, `--no-color`, `--non-interactive`. Détection `kind` d'un dossier : `databases.json` présent → agent ; `PROJECT_SECRET` dans `.env` → dashboard. + +Le choix `back` dans les selects disparaît : interactif = Ctrl-C (`UserAbort`) ou entrée "cancel" en fin de liste. + +## 5. État, templates, rendu + +### 5.1 Modèle de données (`services/project.py`) + +Objets en mémoire construits depuis le disque, jamais persistés tels quels. + +```python +@dataclass(frozen=True) +class DatabaseSpec: + id: str; engine: str; name: str; managed: bool + host: str | None; port: int | None; database: str | None + username: str | None; password: str | None + path: str | None # sqlite + volume: str | None; container: str | None # docker-volume + options: dict + + @classmethod + def from_json(cls, raw: dict, env: EnvFile) -> "DatabaseSpec": ... + def to_json(self) -> dict: ... # format databases.json actuel, inchangé + @property + def env_prefix(self) -> str: ... # "db-pg-a1f2" → "DB_PG_A1F2" + + +@dataclass +class AgentProject: + path: Path; env: EnvFile; databases: list[DatabaseSpec]; host_gateway: bool + + @property + def needs_docker_socket(self) -> bool # une entrée docker-volume + @property + def managed(self) -> list[DatabaseSpec] + @property + def sqlite_mounts(self) -> list[tuple[str, str]] # database commence par /config/ → ./x:/config/x + + +@dataclass +class DashboardProject: + path: Path; env: EnvFile + @property + def db_mode(self) -> Literal["external", "custom", "internal"] + # POSTGRES_HOST absent → internal ; == "db" → external ; sinon custom +``` + +Détection `managed` : `.env` contient `{PREFIX}_PORT` pour ce `host` (toutes les bases `new` l'écrivent, aucune `existing`). Si l'agent tolère les clés inconnues dans `databases.json`, une clé explicite `managed: true` sera ajoutée et la détection deviendra le fallback — à vérifier côté agent. + +Cas limites : + +- `host` managé sans `{PREFIX}_PORT` dans `.env` → `ui.warning`, la base est traitée comme externe. +- Deux entrées avec le même `host` → `ConfigError` avant tout rendu. +- `.env` ou `databases.json` absent → `ConfigError("Not a Portabase agent folder")`. + +### 5.2 `EnvFile` (`services/envfile.py`) + +Remplace `write_env_file`. Parse `KEY="v"`, `KEY='v'`, `KEY=v`, `export KEY=`, commentaires, lignes vides. Conserve l'ordre et les commentaires (liste de lignes typées). `merge()` met à jour en place et ajoute en fin ; `remove(prefix)` retire les `PREFIX_*`. Écriture toujours quotée `"…"`, `"` et `\` échappés. Sauvegarde atomique (tmp + `os.replace`). + +`.env` ne contient que des variables consommées par les conteneurs. Aucune métadonnée CLI. + +### 5.3 `ComposeFacts` (`services/compose_facts.py`) + +`yaml.safe_load` du compose existant, lecture seule, jamais réécrit. Expose `host_gateway` (présence de `extra_hosts` sur `services.agent`, forme liste ou dict tolérée). Compose absent ou invalide → valeurs par défaut + `ui.warning`, jamais d'erreur. + +### 5.4 `TemplateRepository` (`services/templates.py`) + +- URL : `{TEMPLATE_BASE_URL}/{version}/manifest.json` puis fichiers listés. +- Résolution de version : `current_version()` ; sinon `PORTABASE_TEMPLATES_VERSION` ; sinon `PORTABASE_TEMPLATES_DIR` (court-circuite S3) ; sinon `TemplateError`. En dev non-frozen, `./templates` à côté de `main.py` est utilisé automatiquement s'il existe. +- Cache `~/.portabase/cache/templates//`. Séquence `ensure()` : GET manifest (10 s) → pour chaque fichier, sha256 identique en cache → skip, sinon GET + vérification sha256 et taille → écriture. Fichiers en cache absents du manifest supprimés. Manifest injoignable avec cache complet → warning et cache ; sans cache → `TemplateError` avec hint. +- Jinja2 : `Environment(undefined=StrictUndefined, keep_trailing_newline=True, autoescape=False)`. `{{ }}` ne collisionne pas avec `${}` Compose. + +Manifest : + +```json +{ + "schema": 1, + "version": "26.09.0", + "generated_at": "2026-09-11T14:02:17Z", + "commit": "858d4926…", + "files": { + "agent.yml.j2": { "sha256": "…", "size": 612 }, + "engines/postgresql.yml.j2": { "sha256": "…", "size": 498 } + }, + "engines": { + "postgresql": "engines/postgresql.yml.j2", + "postgresql-cluster": "engines/postgresql.yml.j2" + } +} +``` + +`schema` inconnu → `TemplateError`. `version` ≠ version demandée → `TemplateError`. `engines` sert à `engines-check` en CI et à `get_engine(key)`. + +### 5.5 `ComposeRenderer` (`services/renderer.py`) + +```python +class ComposeRenderer: + def __init__(self, templates: TemplateRepository, engines: EngineRegistry): ... + def render_agent(self, project: AgentProject, inline: bool = False) -> RenderResult: ... + def render_dashboard(self, project: DashboardProject, inline: bool = False) -> RenderResult: ... +``` + +Contexte `agent.yml.j2` : `host_gateway`, `docker_socket`, `mounts` (sqlite), `services` (liste de `{name, volume, body}` où `body` est le rendu du template moteur). Le renderer passe aux templates moteurs des variables **déjà formées** (`port_var = "${DB_PG_A1F2_PORT}"` ou valeur littérale si `inline`) : la logique de nommage reste en Python, les templates restent lisibles. + +`RenderResult` : `compose: str`, `databases: list[dict]`. `write(path)` valide d'abord (`yaml.safe_load` du compose → sinon `TemplateError`, un template remote cassé ne corrompt jamais une install), puis écrit `docker-compose.yml` et `databases.json` atomiquement. Le compose porte un en-tête `# Generated by Portabase CLI . Do not edit — use docker-compose.override.yml.` + +Ordre dans une commande mutante : collecte → `templates.ensure()` → mutation `.env`/`databases.json` en mémoire → rendu → validation → écriture. Le manifest est vérifié avant toute mutation. + +### 5.6 Templates + +`agent.yml.j2` : + +```jinja +services: + agent: + restart: unless-stopped + image: portabase/agent:latest + volumes: + - ./databases.json:/config/config.json +{%- for m in mounts %} + - {{ m.host }}:{{ m.container }} +{%- endfor %} +{%- if docker_socket %} + - /var/run/docker.sock:/var/run/docker.sock +{%- endif %} +{%- if host_gateway %} + extra_hosts: + - "localhost:host-gateway" +{%- endif %} + environment: + TZ: "${TZ}" + EDGE_KEY: "${EDGE_KEY}" + LOG_LEVEL: "${LOG_LEVEL}" + POLLING: "${POLLING}" + networks: + - portabase +{% for s in services %} +{{ s.body }} +{%- endfor %} +{% if services %} +volumes: +{%- for s in services %} + {{ s.volume }}: +{%- endfor %} +{% endif %} +networks: + portabase: + name: portabase_network + external: true +``` + +Templates moteurs : un par moteur, variante auth par `{% if auth %}` (10 snippets actuels → 8 templates ; `postgresql-cluster` réutilise `postgresql.yml.j2`). `dashboard.yml.j2` : `{% if db_mode == "external" %}` autour du service `db`, de `depends_on` et du volume — remplace les trois `re.sub` de `dashboard.py`. + +### 5.7 Installs legacy + +Aucun marqueur de version nécessaire. `AgentProject.load()` fonctionne sur toute install (`.env` + `databases.json` existent déjà). Au premier `RenderResult.write()` sur un compose sans l'en-tête `# Generated by Portabase CLI`, le fichier est copié en `docker-compose.legacy.yml` et un avertissement est affiché. `portabase build PATH --diff` permet de voir le diff avant. Les commandes `start/stop/logs` ne déclenchent rien. + +Différences attendues au premier rendu d'une install ancienne : `restart: unless-stopped` ajouté sur redis/valkey (absent des snippets actuels) ; à mentionner dans le changelog rc. + +## 6. Moteurs DB (`engines/`) + +```python +@dataclass(frozen=True) +class Field: + name: str; prompt: str + kind: Literal["text", "int", "secret", "bool", "choice"] + default: Any = None; choices: tuple[str, ...] = () + validator: Callable[[Any], Any] | None = None + + +class DbEngine(ABC): + key: str; display: str; default_port: int + template: str | None # None = aucun service Compose (sqlite, docker-volume, existing) + auth_variants: bool = False + warning: str | None = None + + def fields_existing(self) -> list[Field]: ... # défaut : host, port, database, username, password + def fields_new(self) -> list[Field]: ... # défaut : [] (tout généré) + def option_fields(self) -> list[Field]: ... # défaut : [] + def generate(self, service: str, auth: bool, ports: PortAllocator) -> DatabaseSpec: ... + def env_vars(self, spec: DatabaseSpec) -> dict[str, str]: ... + def template_ctx(self, spec: DatabaseSpec, inline: bool) -> dict: ... + def agent_entry(self, spec: DatabaseSpec) -> dict: ... # projection databases.json +``` + +Hiérarchie : `StandardSqlEngine` (postgresql, postgresql-cluster, mysql, mariadb, mssql, firebird), `KeyValueEngine` (redis, valkey), `MongoEngine`, `SqliteEngine`, `DockerVolumeEngine`. Les sous-classes ne surchargent que leurs particularités : + +| Moteur | Particularité | +|---|---| +| postgresql | `option_fields` : `keep_ownership` (bool), `clean_mode` (clean/none/drop_schemas/drop_database) | +| postgresql-cluster | `warning` superuser ; pas d'options | +| firebird | `agent_entry.name = "mirror.fdb"` ; var `_ROOT_PASS` | +| mssql | `agent_entry.username = "sa"` | +| redis, valkey | `agent_entry.database = "0"` ; `auth_variants = True` ; no-auth → `_PORT` seul | +| mongodb | `auth_variants = True` | +| sqlite | `fields_new` : nom de fichier ; `fields_existing` : chemin ; pas de template ; mount si chemin relatif | +| docker-volume | `fields` : volume, container (optionnel), label ; `warning` socket ; pas de template | + +`EngineRegistry` : dict `key → instance`, imports explicites (compatible PyInstaller). `get(key)` inconnu → `ValidationError` avec la liste des clés. + +## 7. `ui/` + +### 7.1 Principes + +- Tokens uniques (`ui/theme.py`) : `PALETTE` → `RICH_THEME` et `QUESTIONARY_STYLE`. +- Composants stateless, un par fichier, `Component(console)`. +- Façade `UI` : seule chose importée par `commands/`. Rich et questionary ne sont jamais importés hors de `ui/`. +- Le markup Rich est autorisé dans les arguments texte (`ui.success("Added [bold]x[/bold]")`). +- `NO_COLOR` / `--no-color` → `Console(no_color=True)`, style questionary vide. +- Jamais de prompt à l'intérieur d'un `ui.status()` (structurellement garanti : les services ne promptent pas). + +### 7.2 Composants + +| Composant | Remplace | +|---|---| +| `Banner` | `print_banner` | +| `Message` (`success/info/warning/error`) | ~60 `console.print("[success]✔ …")` | +| `Section` | `Panel("[bold]Database Setup[/bold]")` | +| `Summary` (masque auto des clés `password/secret/key`) | `Table(show_header=False)` de dashboard | +| `DataTable` | `Table` de `db list` | +| `Status` (context manager, hint injecté) | `console.status(msg + hint)` | +| `Hint` | `get_random_hint` | +| `Diff` | nouveau, pour `build --diff` | +| `Prompt` (`text/integer/secret/confirm/select/path`) | `rich.prompt.*`, `questionary.*` épars | + +Règle anti-dérive : un composant n'existe que s'il a un appelant. L'inventaire ci-dessus est un plafond. + +### 7.3 `Form` + +```python +class Form: + def ask(self, field: Field, value: Any | None) -> Any: + # 1. valeur du flag → validée + # 2. non-interactif : défaut, sinon ValidationError("Missing --") + # 3. interactif : prompt selon field.kind (dispatch dict), None (Ctrl-C) → UserAbort + # validation en boucle jusqu'à valeur acceptée + def collect(self, fields: list[Field], values: dict) -> dict: ... + def text(...), integer(...), confirm(...), choice(...) # raccourcis +``` + +`non_interactive` résolu une fois dans `main.py`. `ui.confirm()` en non-interactif renvoie le défaut ; les confirmations destructives ont `default=False` et un flag `--force`. + +## 8. Erreurs, télémétrie, updater + +### 8.1 Hiérarchie (`core/errors.py`) + +| Classe | `code` | exit | +|---|---|---| +| `PortabaseError` | `E_GENERIC` | 1 | +| `UserAbort` | `E_ABORT` | 130 | +| `ValidationError` | `E_VALIDATION` | 2 | +| `ConfigError` | `E_CONFIG` | 3 | +| `DockerError` | `E_DOCKER` | 4 | +| `TemplateError` | `E_TEMPLATE` | 5 | +| `NetworkError` | `E_NETWORK` | 6 | +| `UpdateError` | `E_UPDATE` | 7 | + +Constructeur : `(message, *, hint=None, cause=None)`. Les exceptions tierces (`requests`, `subprocess`, `yaml`, `jinja2`) sont wrappées à la frontière du service. `typer.Exit` n'est plus levé hors de `main.py`. Ruff : `E722`, `BLE001`, `S110`, `TID251`. + +### 8.2 Catcher (`main.py`) + +`app(standalone_mode=False)` dans un seul `try` : `UserAbort` → "Cancelled." exit 130 ; `PortabaseError` → `ui.error(e)` (message, hint, code ; `--verbose` ajoute cause et traceback), `telemetry.error(e)`, exit `e.exit_code` ; `click.UsageError` → mappé en `ValidationError` ; `KeyboardInterrupt` → exit 130 ; `Exception` → "Unexpected error", télémétrie `unexpected=True`, exit 1. `finally: telemetry.flush()`. + +### 8.3 Télémétrie (`services/telemetry.py`) + +```python +class Telemetry(ABC): + def session(self, **attrs) -> ContextManager # span racine par invocation + def span(self, name: str, **attrs) -> ContextManager + def event(self, name: str, **attrs) -> None + def error(self, exc: Exception, unexpected: bool = False) -> None + def flush(self) -> None +``` + +Implémentations : `NoopTelemetry` (défaut), `ConsoleTelemetry` (`--debug`, stderr), `OtelTelemetry` (futur, import lazy, construit seulement si `telemetry=true` et `telemetry_endpoint` défini). Spans : `Command.run`, `TemplateRepository.ensure`, `ComposeRenderer.render`, `DockerRunner.compose`. Attributs : commande, moteur, mode, durée, code de sortie, `error.code`, version CLI, OS. Jamais : nom d'agent, chemin, clé, credentials, contenu de fichier. + +Opt-in : `portabase config set telemetry true` ou `PORTABASE_TELEMETRY=1`. Une ligne d'information à la première exécution, aucun prompt. + +### 8.4 Updater (`services/updater.py`) + +`UpdateChecker.notify(ui)` appelé après la commande, cache 24 h (`~/.portabase/cache/release.json`), silencieux si hors ligne, `--stdout` ou non-interactif. `Updater.apply()` vérifie le sha256 via `checksums.txt` de la release avant remplacement du binaire. Canal `beta` conservé. + +## 9. CI, sécurité, release + +### 9.1 `ci.yml` (`pull_request`, `push: main`) + +| Job | Contenu | +|---|---| +| `lint` | `ruff check`, `ruff format --check` | +| `render-check` | `scripts/render_check.py` : rend `agent.yml.j2` (0 base, chaque moteur auth/no-auth, socket, host_gateway, mounts sqlite) et `dashboard.yml.j2` × 3 modes via le vrai `ComposeRenderer` (`PORTABASE_TEMPLATES_DIR=./templates`), puis `yaml.safe_load` et `docker compose config` avec `.env` fixture | +| `engines-check` | chaque `DbEngine.template` existe dans `templates/`, chaque template a un moteur, `engines.map.json` cohérent | +| `gitleaks` | action pinnée ; `.gitleaks.toml` allowlist `templates/**` et `HINTS` | +| `plumber` | action drop-in, `verify-attestation: true` | +| `build-smoke` | PyInstaller linux/amd64, `./dist/portabase --version`, `agent smoke --key --non-interactive` avec templates locaux | +| `test` | `pytest` — vide, réservé à la spec tests | + +### 9.2 Durcissement + +- Toutes les actions pinnées par SHA avec commentaire de version ; Dependabot `github-actions` et `uv` hebdomadaires. +- `permissions: {}` au top de chaque workflow, permissions explicites par job. `packages: write` retiré (inutilisé). +- `templates-upload.yml` : plus de `~/.s3cfg` par heredoc ; credentials par variables d'environnement. +- `actions/attest-build-provenance` sur les binaires. + +### 9.3 Release + +`bump.yml` (`workflow_dispatch`, inputs `version`, `channel: stable|rc`) : validation regex, `stable` uniquement depuis `main`, `sed` `pyproject.toml` + `CITATION.cff`, commit `chore(release): X`, tag, push. Les workflows sur tag restent inchangés. `./release` supprimé. Pas de release-please (historique non conventional). Si `main` exige une PR, le workflow ouvre une PR au lieu de pousser — à régler selon la protection de branche. + +`templates-hotfix.yml` (`workflow_dispatch`, input `version`) : re-sync `templates/` vers `templates//` et régénère le manifest. Réservé aux corrections compatibles avec le code de cette version. + +`templates-upload.yml` : source `templates/`, génération de `manifest.json` (sha256, taille, version, commit, date, mapping moteurs depuis `engines.map.json`) avant `s3cmd sync`. + +### 9.4 `pyproject.toml` + +```toml +dependencies = ["typer", "rich", "questionary", "requests", "pyyaml", "jinja2"] +[dependency-groups] +dev = ["pyinstaller", "ruff", "pytest"] +``` + +## 10. Ordre des chantiers + +Graphe de dépendances, pas un calendrier. + +``` +[A] Hygiène CI ─────────────────────────────────────────────┐ indépendant + ci.yml, pin SHA, permissions, bump.yml, pyproject │ + │ +[B] Fondations │ + core/errors, ui/, services/{envfile,docker,http,ports}, │ + main.py catcher, Command ABC │ + │ │ + ├──► [C] Lifecycle en POO (start/stop/…/config/update) + │ (ancien code agent/db/dashboard via LegacyCommand) + │ + └──► [D] Templates .j2 + TemplateRepository + engines/ + render-check + │ + ▼ + [E] Rendu : project, compose_facts, renderer, build + │ + ▼ + [F] agent / dashboard / db réécrits, ancien code supprimé + │ + ▼ + [G] OTel réel, --json, spec tests +``` + +Contraintes : + +- B avant tout code métier. +- D avant E (le renderer se construit contre des templates réels). +- E avant F. +- C et F ne touchent pas les mêmes fichiers ; C peut aller avant ou après D/E. +- A avant F de préférence : `render-check` et `build-smoke` sont le seul filet avant la spec tests. + +Points de livraison : + +| Après | État | Canal | +|---|---|---| +| A | fonctionnellement identique, CI verte | stable | +| B + C | lifecycle en POO, ui/ et erreurs neuves ; `agent`/`db`/`dashboard` = ancien code via `LegacyCommand` | stable | +| D | templates `.j2` uploadés sous la nouvelle version ; l'ancien code lit `agent.yml`, coexistence sur S3 | rc | +| E + F | bascule complète | rc obligatoire, puis stable | + +Risques et parades : + +| Étape | Risque | Parade | +|---|---|---| +| A | mauvais SHA casse un workflow | tag rc jetable | +| B | sur-conception de `ui/` | un composant = un appelant | +| D | template `.j2` diverge d'un snippet actuel | diff manuel des rendus contre l'ancien CLI, une fois | +| E | `ComposeFacts` lit mal un vieux compose | tolérance, warning, jamais de crash | +| F | install legacy cassée après `db add` | `.legacy.yml`, `build --diff`, changelog rc | +| F | mots de passe existants avec `$` | ne pas régénérer ; correction pour les nouvelles bases seulement | + +## 11. Questions ouvertes + +- L'agent tolère-t-il des clés inconnues dans `databases.json` ? Si oui : clé `managed: true` explicite. +- Protection de la branche `main` : `bump.yml` pousse directement ou ouvre une PR ? +- Garder la clé `engines` dans le manifest (double source de vérité avec le code) ou s'en tenir au registre Python ? From c23839e3afee184a7f18dfa54c4dc3930a43e713 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Th=C3=A9o=20LAGACHE?= Date: Fri, 11 Sep 2026 10:05:59 +0200 Subject: [PATCH 02/24] docs(spec): shared AddDatabaseFlow, generic engine options, split redis/valkey Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_017ACLkgaiP48pBMyi7kZZ4a --- .../specs/2026-09-11-cli-refactor-design.md | 62 ++++++++++++++++--- 1 file changed, 53 insertions(+), 9 deletions(-) diff --git a/docs/superpowers/specs/2026-09-11-cli-refactor-design.md b/docs/superpowers/specs/2026-09-11-cli-refactor-design.md index 067d703..3655b72 100644 --- a/docs/superpowers/specs/2026-09-11-cli-refactor-design.md +++ b/docs/superpowers/specs/2026-09-11-cli-refactor-design.md @@ -35,7 +35,8 @@ Hors périmètre de cette spec : | État d'une install | Aucun fichier d'état ajouté. Source de vérité = `.env` (variables runtime des conteneurs uniquement) + `databases.json` (contrat agent, inchangé) + lecture structurelle du compose existant pour le seul fait non dérivable (`host_gateway`). | | Compose | Artefact dérivé, propriété du CLI, re-rendu intégralement à chaque commande mutante. Personnalisations utilisateur via `docker-compose.override.yml` (mécanisme Compose natif). | | Templates | 100 % remote (S3), versionnés par version CLI exacte, manifest avec sha256, cache disque. Suppression du fallback `latest`. Source dans `templates/` à la racine du dépôt. | -| Création multi-DB | `portabase agent` crée un agent sans base ; les bases s'ajoutent par `portabase db add` (un appel par base). Pas de DSL `--db engine:opts`. | +| Création multi-DB | En non-interactif, `portabase agent` crée un agent sans base ; les bases s'ajoutent par `portabase db add` (un appel par base). En interactif, `agent` enchaîne sur une boucle « Add a database? » qui réutilise le même flux que `db add`. Pas de DSL `--db engine:opts`. | +| Options moteur | Flag générique répétable `-o/--option KEY=VALUE`, validé contre `DbEngine.option_fields()`. Pas de flag Typer par option. | | Moteurs DB | Classes Python (`engines/`), registre à imports explicites. Pas de manifeste data-driven. | | Input UI | questionary uniquement. `rich.prompt` et `typer.prompt` bannis (ruff). | | Non-interactif | Flag `--non-interactive`, env `PORTABASE_NON_INTERACTIVE`, ou `stdin` non-TTY. Géré par `ui.Form`, pas par les commandes. | @@ -60,7 +61,9 @@ cli/ │ ├── lifecycle.py # Start/Stop/Restart/Logs/Uninstall (ex-common.py) │ ├── db.py # DbCommands : add / remove / list │ ├── config.py # ConfigCommands : get / set -│ └── update.py # UpdateCommand +│ ├── update.py # UpdateCommand +│ └── flows/ +│ └── add_database.py # AddDatabaseFlow : collecte + application, partagé par agent et db add │ ├── services/ │ ├── project.py # AgentProject, DashboardProject, DatabaseSpec, detect_kind() @@ -79,7 +82,8 @@ cli/ │ ├── base.py # DbEngine ABC, Field │ ├── registry.py # EngineRegistry │ ├── sql.py # StandardSqlEngine + Postgres/PostgresCluster/MySQL/MariaDB/MSSQL/Firebird -│ ├── keyvalue.py # KeyValueEngine + Redis/Valkey +│ ├── redis.py # RedisEngine +│ ├── valkey.py # ValkeyEngine │ ├── mongo.py # MongoEngine │ ├── sqlite.py # SqliteEngine │ └── docker_volume.py # DockerVolumeEngine @@ -155,9 +159,9 @@ Signatures Typer en `Annotated[...]`. Chaque option qui correspond à une questi | Commande | Options notables | Effet | |---|---|---| -| `agent NAME` | `--key`, `--tz`, `--polling`, `--host-gateway/--no-host-gateway`, `--start`, `--force`, `--non-interactive` | crée le dossier, `.env`, `databases.json` vide, rend le compose. Ne crée aucune base. | +| `agent NAME` | `--key`, `--tz`, `--polling`, `--host-gateway/--no-host-gateway`, `--start`, `--force`, `--non-interactive` | crée le dossier, `.env`, `databases.json` vide, rend le compose. En interactif, enchaîne sur une boucle « Add a database? » (`AddDatabaseFlow`, rendu après chaque ajout). En non-interactif, ne crée aucune base. | | `dashboard NAME` | `--port`, `--db-mode external\|internal\|custom`, `--db-host/--db-port/--db-name/--db-user/--db-password-stdin`, `--start`, `--force` | crée `.env`, rend le compose. | -| `db add NAME` | `--engine`, `--mode new\|existing`, `--auth/--no-auth`, `--name`, `--host`, `--port`, `--database`, `--user`, `--password`, `--password-stdin`, `--path`, `--volume`, `--container`, `--label`, `--keep-ownership`, `--clean-mode` | collecte selon le moteur, mute `.env` + `databases.json`, re-rend. Flag fourni mais non pertinent pour le moteur/mode → `ValidationError`. | +| `db add NAME` | `--engine`, `--mode new\|existing`, `--auth/--no-auth`, `--name`, `--host`, `--port`, `--database`, `--user`, `--password`, `--password-stdin`, `--path`, `--volume`, `--container`, `--label`, `-o/--option KEY=VALUE` (répétable) | collecte via `AddDatabaseFlow` selon le moteur, mute `.env` + `databases.json`, re-rend. Flag ou option fourni mais non pertinent pour le moteur/mode → `ValidationError`. | | `db remove NAME` | `--id` ou `--name`, `--purge-volume` | retire l'entrée, retire les variables `.env` du service, re-rend. Le volume Docker n'est supprimé que sur `--purge-volume`. | | `db list NAME` | — | lecture seule. | | `build PATH` | `--diff`, `--stdout`, `--inline-env`, `--output DIR` | re-rend depuis l'état. Sans option : écrit en place (= migration legacy). `--inline-env` substitue les valeurs au lieu de `${VAR}` avec avertissement secrets en clair. | @@ -169,6 +173,31 @@ Options globales : `--verbose`, `--debug`, `--no-color`, `--non-interactive`. D Le choix `back` dans les selects disparaît : interactif = Ctrl-C (`UserAbort`) ou entrée "cancel" en fin de liste. +### 4.3 `AddDatabaseFlow` (`commands/flows/add_database.py`) + +Le wizard d'ajout de base est un objet réutilisable, pas une commande. C'est la duplication actuelle entre `agent.py` et `db.py` qui disparaît. + +```python +class AddDatabaseFlow: + def __init__(self, ui: UI, engines: EngineRegistry, ports: PortAllocator): ... + + def collect(self, values: dict) -> DatabaseSpec: + """values = flags parsés (engine, mode, auth, host…, options). + Champ manquant → prompt (interactif) / défaut / ValidationError (non-interactif).""" + + def apply(self, project: AgentProject, spec: DatabaseSpec) -> None: + """Mute project.env (variables du service si managed) et project.databases, en mémoire.""" +``` + +Séquence de `collect` : moteur (`--engine` ou select) → affiche `engine.warning` s'il existe → mode (`--mode` ou select ; sqlite et docker-volume n'ont pas de mode `existing`/`new` au sens service : sqlite distingue fichier créé vs chemin existant, docker-volume n'a qu'un mode) → variante auth si `engine.auth_variants` → `Form.collect(engine.fields_new() | fields_existing(), values)` → `Form.collect(engine.option_fields(), values["options"])` → `engine.generate(...)` ou construction depuis les réponses. + +Utilisation : + +- `DbAddCommand.run` : `AgentProject.load` → `templates.ensure()` → `flow.collect(flags)` → `flow.apply` → `renderer.render_agent` → `write`. +- `AgentCommand.run` (interactif seulement) : après le premier rendu, `while ui.confirm("Add a database?", default=True)` : `flow.collect({})` → `flow.apply` → rendu + écriture. Rendu après chaque ajout : un Ctrl-C au milieu laisse un état cohérent sur disque. + +`flows/` vit dans `commands/` parce qu'il prompte via `ui` ; il ne fait aucune I/O fichier (c'est `RenderResult.write` qui écrit). + ## 5. État, templates, rendu ### 5.1 Modèle de données (`services/project.py`) @@ -331,7 +360,7 @@ Différences attendues au premier rendu d'une install ancienne : `restart: unles class Field: name: str; prompt: str kind: Literal["text", "int", "secret", "bool", "choice"] - default: Any = None; choices: tuple[str, ...] = () + default: Any = None; choices: tuple[str, ...] = (); help: str | None = None validator: Callable[[Any], Any] | None = None @@ -348,23 +377,38 @@ class DbEngine(ABC): def env_vars(self, spec: DatabaseSpec) -> dict[str, str]: ... def template_ctx(self, spec: DatabaseSpec, inline: bool) -> dict: ... def agent_entry(self, spec: DatabaseSpec) -> dict: ... # projection databases.json + def agent_database(self, spec: DatabaseSpec) -> str: ... # défaut : spec.database ; hook pour "0", chemin… ``` -Hiérarchie : `StandardSqlEngine` (postgresql, postgresql-cluster, mysql, mariadb, mssql, firebird), `KeyValueEngine` (redis, valkey), `MongoEngine`, `SqliteEngine`, `DockerVolumeEngine`. Les sous-classes ne surchargent que leurs particularités : +Hiérarchie : `StandardSqlEngine` (postgresql, postgresql-cluster, mysql, mariadb, mssql, firebird), `RedisEngine`, `ValkeyEngine`, `MongoEngine`, `SqliteEngine`, `DockerVolumeEngine`. Redis et Valkey sont deux classes indépendantes dans deux fichiers, sans base commune (images, commandes et healthchecks divergent ; ce qu'elles partagent — `agent_database = "0"`, `auth_variants` — passe par les hooks de `DbEngine`). Les sous-classes ne surchargent que leurs particularités : | Moteur | Particularité | |---|---| -| postgresql | `option_fields` : `keep_ownership` (bool), `clean_mode` (clean/none/drop_schemas/drop_database) | +| postgresql | `option_fields` : `keep_ownership` (bool, défaut False), `clean_mode` (choice clean/none/drop_schemas/drop_database, défaut clean) | | postgresql-cluster | `warning` superuser ; pas d'options | | firebird | `agent_entry.name = "mirror.fdb"` ; var `_ROOT_PASS` | | mssql | `agent_entry.username = "sa"` | -| redis, valkey | `agent_entry.database = "0"` ; `auth_variants = True` ; no-auth → `_PORT` seul | +| redis | `agent_database = "0"` ; `auth_variants = True` ; no-auth → `_PORT` seul | +| valkey | idem redis, classe et template distincts | | mongodb | `auth_variants = True` | | sqlite | `fields_new` : nom de fichier ; `fields_existing` : chemin ; pas de template ; mount si chemin relatif | | docker-volume | `fields` : volume, container (optionnel), label ; `warning` socket ; pas de template | `EngineRegistry` : dict `key → instance`, imports explicites (compatible PyInstaller). `get(key)` inconnu → `ValidationError` avec la liste des clés. +### 6.1 Options moteur + +Certains moteurs exposent des options que l'agent lit dans `databases.json` (`options` : aujourd'hui `keep_ownership` et `clean_mode` pour PostgreSQL). Le système doit accepter de nouvelles options sans toucher à la signature Typer. + +- Déclaration : `DbEngine.option_fields() -> list[Field]`. Un `Field` comme les autres : nom, prompt, type, défaut, choix, validateur. +- Saisie non-interactive : flag générique répétable `-o KEY=VALUE` / `--option KEY=VALUE` sur `db add`. Parsé en `dict[str, str]`, converti selon `Field.kind` (`bool` : `true/false/1/0/yes/no`, `int`, `choice` validé contre `choices`). Clé inconnue pour ce moteur → `ValidationError` listant les options valides. +- Saisie interactive : `Form.collect(engine.option_fields(), values["options"])`, un prompt par option non fournie, avec le texte d'aide actuel (par exemple l'explication de `--no-owner` / `pg_restore --clean`) porté par `Field.help`. +- Stockage : `DatabaseSpec.options: dict` (valeurs typées). +- Projection : `agent_entry()` n'écrit dans `options` que les valeurs différentes du défaut. Comportement actuel conservé : `keep_ownership` absent si False, `clean_mode` absent si `clean`. Aucune clé `options` si vide. +- Affichage : `db list` montre les options non-défaut ; `Summary` les inclut lors de l'ajout. + +Ajouter une option = une ligne dans `option_fields()` du moteur concerné. + ## 7. `ui/` ### 7.1 Principes From 8acecd93b1fb332f40f68f1275e385969fb47846 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Th=C3=A9o=20LAGACHE?= Date: Fri, 11 Sep 2026 10:13:03 +0200 Subject: [PATCH 03/24] docs: add plan 1 (CI hygiene, pinned actions, bump workflow) Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_017ACLkgaiP48pBMyi7kZZ4a --- .../plans/2026-09-11-plan-1-ci-hygiene.md | 954 ++++++++++++++++++ 1 file changed, 954 insertions(+) create mode 100644 docs/superpowers/plans/2026-09-11-plan-1-ci-hygiene.md diff --git a/docs/superpowers/plans/2026-09-11-plan-1-ci-hygiene.md b/docs/superpowers/plans/2026-09-11-plan-1-ci-hygiene.md new file mode 100644 index 0000000..9f0648a --- /dev/null +++ b/docs/superpowers/plans/2026-09-11-plan-1-ci-hygiene.md @@ -0,0 +1,954 @@ +# Plan 1 — CI, hygiène et release (chantier A) + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** CI de PR bloquante (lint, secrets, sécurité pipeline, build smoke), workflows durcis et pinnés, `./release` remplacé par `bump.yml` — sans changer une ligne de comportement du CLI. + +**Architecture:** Un workflow `ci.yml` sur PR/push main avec des jobs indépendants. Les workflows de release existants restent structurellement identiques, seulement pinnés par SHA et restreints en permissions. La configuration ruff vit dans `pyproject.toml` avec des exclusions explicites pour le code legacy qui sera supprimé aux plans 2–4. + +**Tech Stack:** GitHub Actions, uv 0.9, ruff 0.16, pytest, PyInstaller 6.17, gitleaks-action v2, getplumber/plumber, Dependabot. + +**Spec:** `docs/superpowers/specs/2026-09-11-cli-refactor-design.md` — sections 9 (CI, sécurité, release) et 10 (étape A). + +## Global Constraints + +- Python `>=3.12` (pyproject actuel). Ne pas changer. +- Aucune modification de comportement du CLI dans ce plan. Seuls `pyproject.toml`, `.gitignore`, `.github/**`, `.gitleaks.toml` et le formatage (`ruff format`) bougent. +- Toutes les `uses:` pinnées par SHA complet + commentaire `# vX.Y.Z`. +- `permissions: {}` au top de chaque workflow ; permissions explicites par job. +- Le job `test` existe mais ne collecte aucun test (réservé à la spec tests). +- Aucun test unitaire dans ce plan (consigne utilisateur). Chaque tâche a des étapes de vérification exécutables. +- Commits en Conventional Commits (`chore`, `ci`, `build`, `style`). +- `gh` n'est pas authentifié sur ce poste : les appels `gh api` sur dépôts publics fonctionnent, `gh` sur `Portabase/cli` (protection de branche, secrets) ne fonctionne pas. Vérifier ces points dans l'interface GitHub. + +SHAs résolus le 2026-09-11 (à réutiliser tels quels) : + +| Action | Tag | SHA | +|---|---|---| +| actions/checkout | v4 | `11d5960a326750d5838078e36cf38b85af677262` | +| actions/upload-artifact | v4 | `ea165f8d65b6e75b540449e92b4886f43607fa02` | +| actions/download-artifact | v4 | `d3f86a106a0bac45b974a628896c90dbdf5c8093` | +| actions/attest-build-provenance | v2 | `e8998f949152b193b063cb0ec769d69d929409be` | +| astral-sh/setup-uv | v3 | `caf0cab7a618c569241d31dcd442f54681755d39` | +| astral-sh/ruff-action | v3 | `4919ec5cf1f49eff0871dbcea0da843445b837e6` | +| softprops/action-gh-release | v2 | `3bb12739c298aeb8a4eeaf626c5b8d85266b0e65` | +| mikepenz/release-changelog-builder-action | v5 | `c9dc8369bccbc41e0ac887f8fd674f5925d315f7` | +| gitleaks/gitleaks-action | v2 | `ff98106e4c7b2bc287b24eaf42907196329070c7` | +| getplumber/plumber | (doc officielle) | `3feac69e925e9771f8a495f4177af754d568c1ad` | + +Pour re-résoudre un SHA : `gh api repos///git/ref/tags/ --jq .object.sha` (si `.object.type == "tag"`, résoudre encore via `repos///git/tags/ --jq .object.sha`). + +--- + +## File Structure + +| Fichier | Action | Responsabilité | +|---|---|---| +| `pyproject.toml` | modifier | deps runtime/dev, config ruff, config pytest | +| `.gitignore` | modifier | retirer `uv.lock` (tracké, requis par `--frozen`) | +| `commands/*.py`, `core/*.py`, `main.py` | reformater seulement | `ruff format` mécanique, aucun changement sémantique | +| `.gitleaks.toml` | créer | allowlist des faux positifs | +| `.github/workflows/ci.yml` | créer | lint, test, gitleaks, plumber, build-smoke | +| `.github/workflows/python.yml` | modifier | pin SHA, permissions, `--frozen`, attestation | +| `.github/workflows/github.yml` | modifier | pin SHA, permissions par job | +| `.github/workflows/templates-upload.yml` | modifier | pin SHA, permissions, s3cmd sans `~/.s3cfg` | +| `.github/workflows/release.yml`, `release-candidate.yml` | modifier | `permissions: {}` top-level, retirer `packages: write` | +| `.github/dependabot.yml` | créer | github-actions + uv hebdo | +| `.github/workflows/bump.yml` | créer | remplace `./release` | +| `release` | supprimer | — | +| `.github/CONTRIBUTING.md` | modifier | procédure de release | + +--- + +### Task 1 : `pyproject.toml` — dépendances, ruff, pytest + +**Files:** +- Modify: `pyproject.toml` +- Modify: `.gitignore` + +**Interfaces:** +- Produces: commandes `uv run ruff check .`, `uv run ruff format --check .`, `uv run pytest` utilisables localement et en CI ; groupe `dev` avec `pyinstaller`, `ruff`, `pytest`. + +- [ ] **Step 1: Réécrire `pyproject.toml`** + +Remplacer le contenu intégral par : + +```toml +[project] +name = "portabase-cli" +version = "26.07.6" +description = "The official command line interface (CLI) for managing and deploying Portabase instances with ease." +readme = "README.md" +requires-python = ">=3.12" +dependencies = [ + "typer>=0.20.0", + "rich>=14.2.0", + "questionary>=2.1.0", + "requests>=2.32.5", + "pyyaml>=6.0.3", +] + +[dependency-groups] +dev = [ + "pyinstaller>=6.17.0", + "ruff>=0.16.0", + "pytest>=8.3", +] + +[tool.ruff] +target-version = "py312" +line-length = 88 +extend-exclude = [".venv", "build", "dist"] + +[tool.ruff.lint] +select = [ + "E", "F", "W", # pycodestyle / pyflakes + "I", # isort + "UP", # pyupgrade + "B", # bugbear + "BLE", # blind except + "S110", # try-except-pass + "E722", # bare except + "TID251", # banned imports (activé au plan 2 : rich.prompt, typer.prompt) + "SIM", + "TRY201", + "PLW1510", # subprocess.run sans check= +] +ignore = [ + "B008", # typer.Argument(...) / typer.Option(...) en défaut : idiome Typer + "E501", # line length géré par ruff format +] + +# Code legacy supprimé aux plans 2-4. Ne pas étendre cette liste : tout nouveau +# fichier doit passer sans exception. +[tool.ruff.lint.per-file-ignores] +"commands/agent.py" = ["BLE001", "E722", "S110", "SIM102"] +"commands/db.py" = ["BLE001", "E722", "S110"] +"commands/dashboard.py" = ["BLE001"] +"commands/common.py" = ["BLE001", "PLW1510"] +"core/config.py" = ["BLE001", "E722", "S110"] +"core/utils.py" = ["BLE001", "E722", "S110", "PLR1730"] +"core/updater.py" = ["BLE001", "TRY201"] +"core/network.py" = ["BLE001"] + +[tool.ruff.lint.isort] +known-first-party = ["commands", "core", "templates"] + +[tool.pytest.ini_options] +testpaths = ["tests"] +addopts = "-q" +``` + +- [ ] **Step 2: Retirer `uv.lock` de `.gitignore`** + +`.gitignore` devient : + +``` +dist/ +build/ +.venv/ +__pycache__/ +*.spec +``` + +- [ ] **Step 3: Régénérer le lock et synchroniser** + +Run: `uv lock && uv sync --all-groups` +Expected: `uv.lock` mis à jour (pyinstaller passe en groupe dev, ruff et pytest ajoutés), `.venv` contient `ruff` et `pytest`. + +- [ ] **Step 4: Vérifier que le CLI démarre toujours** + +Run: `uv run python main.py --version` +Expected: `Portabase CLI version: 26.07.6` (plus éventuel message de mise à jour). + +- [ ] **Step 5: Vérifier le lint** + +Run: `uv run ruff check .` +Expected: `All checks passed!`. Si des erreurs subsistent, ajuster **uniquement** `per-file-ignores` pour les fichiers legacy listés ; ne pas modifier le code Python. + +- [ ] **Step 6: Commit** + +```bash +git add pyproject.toml uv.lock .gitignore +git commit -m "build: move pyinstaller to dev group, add ruff and pytest config + +Legacy files get per-file-ignores for bare/blind excepts; those files are +rewritten in later plans and the ignores are removed with them." +``` + +--- + +### Task 2 : Formatage mécanique + +**Files:** +- Modify: tous les fichiers signalés par `ruff format --check` (5 fichiers au 2026-09-11) + +**Interfaces:** +- Produces: `uv run ruff format --check .` passe. + +- [ ] **Step 1: Lister les fichiers à reformater** + +Run: `uv run ruff format --check .` +Expected: `5 files would be reformatted, 17 files already formatted` (nombres indicatifs). + +- [ ] **Step 2: Appliquer** + +Run: `uv run ruff format .` + +- [ ] **Step 3: Vérifier que rien de sémantique n'a changé** + +Run: `git diff --stat && uv run python main.py --help` +Expected: diff uniquement sur espaces/quotes/retours à la ligne ; `--help` affiche les commandes `agent`, `dashboard`, `start`, `stop`, `restart`, `logs`, `uninstall`, `db`, `config`, `update`. + +- [ ] **Step 4: Vérifier lint + format ensemble** + +Run: `uv run ruff check . && uv run ruff format --check .` +Expected: les deux passent. + +- [ ] **Step 5: Commit** + +```bash +git add -A commands core main.py +git commit -m "style: apply ruff format" +``` + +--- + +### Task 3 : `.gitleaks.toml` + +**Files:** +- Create: `.gitleaks.toml` + +**Interfaces:** +- Produces: config lue par `gitleaks/gitleaks-action` (Task 4) et par `gitleaks detect` en local. + +- [ ] **Step 1: Créer le fichier** + +```toml +# Gitleaks configuration for Portabase CLI. +# Extends the default ruleset; only adds allowlists for known false positives. + +title = "portabase-cli" + +[extend] +useDefault = true + +[allowlist] +description = "Known false positives" +paths = [ + # Compose templates contain PASSWORD=${...} placeholders, never real secrets. + '''templates/.*''', + '''\.github/assets/templates/.*''', + # Lock file: hashes only. + '''uv\.lock''', +] +regexes = [ + # Compose interpolation placeholders. + '''\$\{[A-Z0-9_]+\}''', + # Test/fixture edge keys are base64 JSON with these field names, not credentials. + '''"masterKeyB64"''', +] +``` + +- [ ] **Step 2: Scanner l'historique en local** + +Run: `uvx --from gitleaks gitleaks detect --source . --config .gitleaks.toml --redact --no-banner || docker run --rm -v "$PWD:/repo" -w /repo ghcr.io/gitleaks/gitleaks:v8 detect --source . --config .gitleaks.toml --redact --no-banner` + +(Le binaire gitleaks n'est pas distribué via PyPI ; la première commande échouera, la seconde via Docker fonctionne. Si aucun des deux n'est disponible, passer : la CI fera le scan à la Task 4.) + +Expected: `no leaks found`. Si des fuites réelles sont trouvées dans l'historique : **s'arrêter et le signaler** — ne pas allowlister, ne pas réécrire l'historique sans décision explicite. + +- [ ] **Step 3: Commit** + +```bash +git add .gitleaks.toml +git commit -m "ci: add gitleaks config with template placeholder allowlist" +``` + +--- + +### Task 4 : `ci.yml` — lint, test, gitleaks, plumber, build-smoke + +**Files:** +- Create: `.github/workflows/ci.yml` + +**Interfaces:** +- Consumes: config ruff/pytest de Task 1, `.gitleaks.toml` de Task 3. +- Produces: check requis `CI / lint`, `CI / test`, `CI / gitleaks`, `CI / plumber`, `CI / build-smoke` sur chaque PR. Le job `build-smoke` sera enrichi au Plan 4 (invocation `agent --non-interactive`). + +- [ ] **Step 1: Créer le workflow** + +```yaml +name: CI + +on: + pull_request: + push: + branches: [main] + +permissions: {} + +concurrency: + group: ci-${{ github.ref }} + cancel-in-progress: true + +jobs: + lint: + name: lint + runs-on: ubuntu-latest + permissions: + contents: read + steps: + - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4 + - uses: astral-sh/setup-uv@caf0cab7a618c569241d31dcd442f54681755d39 # v3 + - name: Install + run: uv sync --frozen --all-groups + - name: Ruff check + run: uv run ruff check . --output-format=github + - name: Ruff format + run: uv run ruff format --check . + + test: + name: test + runs-on: ubuntu-latest + permissions: + contents: read + steps: + - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4 + - uses: astral-sh/setup-uv@caf0cab7a618c569241d31dcd442f54681755d39 # v3 + - name: Install + run: uv sync --frozen --all-groups + - name: Pytest + # Exit code 5 = no tests collected. Accepted until the test suite exists. + run: | + set +e + uv run pytest + code=$? + set -e + if [ "$code" -ne 0 ] && [ "$code" -ne 5 ]; then exit "$code"; fi + + gitleaks: + name: gitleaks + runs-on: ubuntu-latest + permissions: + contents: read + steps: + - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4 + with: + fetch-depth: 0 + - uses: gitleaks/gitleaks-action@ff98106e4c7b2bc287b24eaf42907196329070c7 # v2 + env: + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + GITLEAKS_CONFIG: .gitleaks.toml + + plumber: + name: plumber + runs-on: ubuntu-24.04 + permissions: + contents: read + security-events: write + steps: + - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4 + - uses: getplumber/plumber@3feac69e925e9771f8a495f4177af754d568c1ad + with: + score-push: false + upload-sarif: true + # First run: observe only. Tighten to min-score once the baseline is known. + soft-fail: true + + build-smoke: + name: build-smoke + runs-on: ubuntu-latest + permissions: + contents: read + steps: + - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4 + - uses: astral-sh/setup-uv@caf0cab7a618c569241d31dcd442f54681755d39 # v3 + - name: Install + run: uv sync --frozen --all-groups + - name: Build binary + run: | + rm -rf build dist *.spec + uv run pyinstaller \ + --onefile \ + --name portabase_smoke \ + --paths=. \ + --collect-all rich \ + --collect-all requests \ + --collect-data certifi \ + --add-data "pyproject.toml:." \ + main.py + - name: Smoke + run: | + ./dist/portabase_smoke --version + ./dist/portabase_smoke --help +``` + +- [ ] **Step 2: Valider la syntaxe YAML localement** + +Run: `uv run python -c "import yaml,sys; yaml.safe_load(open('.github/workflows/ci.yml')); print('ok')"` +Expected: `ok`. + +- [ ] **Step 3: Vérifier localement ce que fera le job lint** + +Run: `uv sync --frozen --all-groups && uv run ruff check . --output-format=github && uv run ruff format --check .` +Expected: aucune sortie d'erreur. + +- [ ] **Step 4: Vérifier localement ce que fera le job test** + +Run: `uv run pytest; echo "exit=$?"` +Expected: `exit=5` (aucun test collecté). + +- [ ] **Step 5: Vérifier localement ce que fera build-smoke** + +Run: `rm -rf build dist *.spec && uv run pyinstaller --onefile --name portabase_smoke --paths=. --collect-all rich --collect-all requests --collect-data certifi --add-data "pyproject.toml:." main.py && ./dist/portabase_smoke --version` +Expected: `Portabase CLI version: 26.07.6`. Puis `rm -rf build dist *.spec`. + +- [ ] **Step 6: Commit** + +```bash +git add .github/workflows/ci.yml +git commit -m "ci: add PR workflow (lint, test, gitleaks, plumber, build smoke)" +``` + +--- + +### Task 5 : Durcir `python.yml` (build binaires) + +**Files:** +- Modify: `.github/workflows/python.yml` + +**Interfaces:** +- Consumes: appelé par `release.yml` / `release-candidate.yml` via `workflow_call`. +- Produces: artefacts `portabase__` inchangés + attestation de provenance. + +- [ ] **Step 1: Réécrire le workflow** + +```yaml +name: Build Python Binaries + +on: + workflow_call: + +permissions: {} + +jobs: + build: + name: Build for ${{ matrix.os }} (${{ matrix.arch }}) + runs-on: ${{ matrix.runner }} + permissions: + contents: read + id-token: write + attestations: write + strategy: + matrix: + include: + - os: linux + arch: amd64 + runner: ubuntu-latest + - os: linux + arch: arm64 + runner: ubuntu-24.04-arm + - os: macos + arch: arm64 + runner: macos-latest + - os: macos + arch: amd64 + runner: macos-15-intel + + steps: + - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4 + + - uses: astral-sh/setup-uv@caf0cab7a618c569241d31dcd442f54681755d39 # v3 + + - name: Install + run: uv sync --frozen --all-groups + + - name: Build binary + run: | + rm -rf build dist *.spec + uv run pyinstaller \ + --onefile \ + --name portabase_${{ matrix.os }}_${{ matrix.arch }} \ + --paths=. \ + --collect-all rich \ + --collect-all requests \ + --collect-data certifi \ + --add-data "pyproject.toml:." \ + main.py + + - name: Smoke + run: ./dist/portabase_${{ matrix.os }}_${{ matrix.arch }} --version + + - name: Attest provenance + uses: actions/attest-build-provenance@e8998f949152b193b063cb0ec769d69d929409be # v2 + with: + subject-path: dist/portabase_${{ matrix.os }}_${{ matrix.arch }} + + - name: Upload artifacts + uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4 + with: + name: portabase_${{ matrix.os }}_${{ matrix.arch }} + path: dist/portabase_${{ matrix.os }}_${{ matrix.arch }} +``` + +Changements par rapport à l'actuel : `uv python install` remplacé par `uv sync --frozen` (respecte `.python-version` et le lock) ; étape `Smoke` ; attestation ; permissions explicites. + +- [ ] **Step 2: Valider YAML** + +Run: `uv run python -c "import yaml; yaml.safe_load(open('.github/workflows/python.yml')); print('ok')"` +Expected: `ok`. + +- [ ] **Step 3: Commit** + +```bash +git add .github/workflows/python.yml +git commit -m "ci: pin actions, scope permissions, attest binaries in build workflow" +``` + +--- + +### Task 6 : Durcir `github.yml` (release GitHub + Discord) + +**Files:** +- Modify: `.github/workflows/github.yml` + +**Interfaces:** +- Consumes: artefacts de Task 5. +- Produces: release GitHub identique à aujourd'hui. + +- [ ] **Step 1: Modifier uniquement l'en-tête et les `uses:`** + +Remplacer le bloc `jobs:` d'en-tête et les trois `uses:` ; le reste (changelog config, script Discord) reste identique. + +En-tête (après le bloc `on:` existant, avant `jobs:`) — ajouter : + +```yaml +permissions: {} +``` + +Job : + +```yaml +jobs: + create-release: + runs-on: ubuntu-latest + permissions: + contents: write + steps: + - name: Check out the repo + uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4 + with: + fetch-depth: 0 + + - name: Download artifacts + if: inputs.artifact_name != '' + uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4 + with: + pattern: ${{ inputs.artifact_name }} + path: dist + merge-multiple: true +``` + +Et plus bas : + +```yaml + - name: Build Changelog + id: build_changelog + uses: mikepenz/release-changelog-builder-action@c9dc8369bccbc41e0ac887f8fd674f5925d315f7 # v5 +``` + +```yaml + - name: Create GitHub Release + uses: softprops/action-gh-release@3bb12739c298aeb8a4eeaf626c5b8d85266b0e65 # v2 +``` + +- [ ] **Step 2: Vérifier qu'aucun `@vN` non pinné ne reste** + +Run: `grep -nE 'uses: .*@v[0-9]' .github/workflows/github.yml` +Expected: aucune sortie. + +- [ ] **Step 3: Valider YAML** + +Run: `uv run python -c "import yaml; yaml.safe_load(open('.github/workflows/github.yml')); print('ok')"` +Expected: `ok`. + +- [ ] **Step 4: Commit** + +```bash +git add .github/workflows/github.yml +git commit -m "ci: pin actions and scope permissions in release workflow" +``` + +--- + +### Task 7 : Durcir `templates-upload.yml` (S3 sans fichier de credentials) + +**Files:** +- Modify: `.github/workflows/templates-upload.yml` + +**Interfaces:** +- Produces: même arborescence S3 qu'aujourd'hui (`cli/public/templates//` et `latest/`). La source reste `.github/assets/templates/` jusqu'au Plan 3 qui la déplace vers `templates/` et ajoute le manifest. + +- [ ] **Step 1: Réécrire le workflow** + +```yaml +name: Upload Templates to S3 + +on: + workflow_call: + inputs: + version: + required: true + type: string + is_prerelease: + required: true + type: boolean + secrets: + S3_ENDPOINT: + required: true + S3_ACCESS_KEY: + required: true + S3_SECRET_KEY: + required: true + S3_BUCKET: + required: true + +permissions: {} + +jobs: + upload: + runs-on: ubuntu-latest + permissions: + contents: read + env: + # s3cmd reads these flags; no config file is written to disk. + S3CMD_ARGS: >- + --access_key=${{ secrets.S3_ACCESS_KEY }} + --secret_key=${{ secrets.S3_SECRET_KEY }} + --host=${{ secrets.S3_ENDPOINT }} + --host-bucket=%(bucket)s.${{ secrets.S3_ENDPOINT }} + --ssl + steps: + - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4 + + - name: Install s3cmd + run: sudo apt-get update && sudo apt-get install -y s3cmd + + - name: Upload versioned templates + run: | + CLEAN_VERSION="${{ inputs.version }}" + CLEAN_VERSION="${CLEAN_VERSION#v}" + s3cmd $S3CMD_ARGS sync .github/assets/templates/ \ + "s3://${{ secrets.S3_BUCKET }}/cli/public/templates/${CLEAN_VERSION}/" --acl-public + + - name: Upload latest templates (stable only) + if: ${{ !inputs.is_prerelease }} + run: | + s3cmd $S3CMD_ARGS sync .github/assets/templates/ \ + "s3://${{ secrets.S3_BUCKET }}/cli/public/templates/latest/" --acl-public +``` + +Note : les secrets passés en arguments de ligne de commande sont masqués dans les logs par GitHub (`***`). C'est le compromis retenu ; l'alternative (`~/.s3cfg`) laisse les secrets en clair sur le disque du runner. + +- [ ] **Step 2: Valider YAML** + +Run: `uv run python -c "import yaml; yaml.safe_load(open('.github/workflows/templates-upload.yml')); print('ok')"` +Expected: `ok`. + +- [ ] **Step 3: Commit** + +```bash +git add .github/workflows/templates-upload.yml +git commit -m "ci: pass S3 credentials to s3cmd as flags instead of writing ~/.s3cfg" +``` + +--- + +### Task 8 : Permissions top-level sur `release.yml` et `release-candidate.yml` + +**Files:** +- Modify: `.github/workflows/release.yml:10-13` +- Modify: `.github/workflows/release-candidate.yml:12-15` + +**Interfaces:** +- Produces: workflows appelants avec permissions minimales ; les jobs `uses:` héritent des permissions déclarées dans les workflows appelés (Tasks 5–7). + +- [ ] **Step 1: Dans les deux fichiers, remplacer** + +```yaml +permissions: + contents: write + packages: write +``` + +par + +```yaml +permissions: + contents: write + id-token: write + attestations: write + security-events: write +``` + +Un workflow appelant doit déclarer au moins les permissions que les workflows appelés demandent (`contents: write` pour la release, `id-token`/`attestations` pour l'attestation). `packages: write` n'était utilisé par aucun job. + +- [ ] **Step 2: Vérifier** + +Run: `grep -n "packages" .github/workflows/*.yml` +Expected: aucune sortie. + +- [ ] **Step 3: Commit** + +```bash +git add .github/workflows/release.yml .github/workflows/release-candidate.yml +git commit -m "ci: drop unused packages permission, declare attestation permissions" +``` + +--- + +### Task 9 : Dependabot + +**Files:** +- Create: `.github/dependabot.yml` + +**Interfaces:** +- Produces: PRs hebdomadaires pour les SHAs d'actions et les dépendances uv. + +- [ ] **Step 1: Créer le fichier** + +```yaml +version: 2 +updates: + - package-ecosystem: github-actions + directory: / + schedule: + interval: weekly + groups: + actions: + patterns: ["*"] + commit-message: + prefix: "ci" + + - package-ecosystem: uv + directory: / + schedule: + interval: weekly + groups: + python: + patterns: ["*"] + commit-message: + prefix: "build" +``` + +- [ ] **Step 2: Valider YAML** + +Run: `uv run python -c "import yaml; yaml.safe_load(open('.github/dependabot.yml')); print('ok')"` +Expected: `ok`. + +- [ ] **Step 3: Commit** + +```bash +git add .github/dependabot.yml +git commit -m "ci: enable dependabot for actions and uv" +``` + +--- + +### Task 10 : `bump.yml` remplace `./release` + +**Files:** +- Create: `.github/workflows/bump.yml` +- Delete: `release` +- Modify: `.github/CONTRIBUTING.md` + +**Interfaces:** +- Produces: déclenchement manuel qui commit `chore(release): X`, tague `X` et pousse. Le push du tag déclenche `release.yml` ou `release-candidate.yml` selon le motif, exactement comme le script. + +- [ ] **Step 1: Créer le workflow** + +```yaml +name: Bump version + +on: + workflow_dispatch: + inputs: + version: + description: "Version (e.g. 26.09.0 or 26.09.0rc1). No leading v." + required: true + type: string + channel: + description: "stable: only from main. rc: any branch." + required: true + type: choice + options: [stable, rc] + default: rc + +permissions: {} + +jobs: + bump: + runs-on: ubuntu-latest + permissions: + contents: write + steps: + - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4 + with: + fetch-depth: 0 + # Use a PAT if branch protection blocks GITHUB_TOKEN pushes to main. + token: ${{ secrets.GITHUB_TOKEN }} + + - name: Validate version against channel + env: + VERSION: ${{ inputs.version }} + CHANNEL: ${{ inputs.channel }} + REF: ${{ github.ref_name }} + run: | + set -euo pipefail + if [[ "$VERSION" == v* ]]; then + echo "::error::Version must not start with 'v'"; exit 1 + fi + if [[ "$CHANNEL" == "stable" ]]; then + if [[ "$REF" != "main" ]]; then + echo "::error::stable releases are only allowed from main (got $REF)"; exit 1 + fi + if [[ ! "$VERSION" =~ ^[0-9]+\.[0-9]+\.[0-9]+$ ]]; then + echo "::error::stable version must match X.Y.Z"; exit 1 + fi + else + if [[ ! "$VERSION" =~ ^[0-9]+\.[0-9]+\.[0-9]+([-.]?(rc|alpha|beta|a|b)[0-9]*(\.[0-9]+)?)$ ]]; then + echo "::error::rc version must match X.Y.Z(rc|a|b|alpha|beta)N"; exit 1 + fi + fi + if git rev-parse "$VERSION" >/dev/null 2>&1; then + echo "::error::Tag $VERSION already exists"; exit 1 + fi + + - name: Update version files + env: + VERSION: ${{ inputs.version }} + run: | + set -euo pipefail + DATE=$(date -u +%F) + sed -i "s/^version = \".*\"/version = \"$VERSION\"/" pyproject.toml + if [ -f CITATION.cff ]; then + sed -i "s/^version: .*/version: $VERSION/" CITATION.cff + sed -i "s/^date-released: .*/date-released: \"$DATE\"/" CITATION.cff + fi + git diff --stat + + - name: Commit, tag, push + env: + VERSION: ${{ inputs.version }} + run: | + set -euo pipefail + git config user.name "github-actions[bot]" + git config user.email "41898282+github-actions[bot]@users.noreply.github.com" + git add pyproject.toml CITATION.cff + if git diff --cached --quiet; then + echo "No version change to commit" + else + git commit -m "chore(release): $VERSION" + fi + git tag -a "$VERSION" -m "Release $VERSION" + git push origin HEAD + git push origin "$VERSION" +``` + +Différences avec le script : pas de `package.json` / `Cargo.toml` (absents du dépôt) ; `git add .` remplacé par un `add` ciblé ; identité bot. + +Point à vérifier dans l'interface GitHub (Settings → Branches) : si `main` exige une PR, `git push origin HEAD` sera refusé pour `GITHUB_TOKEN`. Deux solutions : (a) autoriser `github-actions[bot]` à contourner la règle ; (b) créer un PAT fine-grained (Contents: write) stocké en secret `RELEASE_TOKEN` et remplacer `token: ${{ secrets.GITHUB_TOKEN }}` par `token: ${{ secrets.RELEASE_TOKEN }}`. Note : un push effectué avec `GITHUB_TOKEN` ne déclenche **pas** d'autres workflows par design GitHub — **le push du tag ne déclenchera donc pas `release.yml`**. Avec un PAT (`RELEASE_TOKEN`), il le déclenche. → **Utiliser un PAT est obligatoire** pour que le tag lance la release. Créer le secret avant le premier usage. + +- [ ] **Step 2: Remplacer le token par le PAT** + +Dans le workflow ci-dessus, `token: ${{ secrets.GITHUB_TOKEN }}` → `token: ${{ secrets.RELEASE_TOKEN }}` et supprimer le commentaire au-dessus. Le secret `RELEASE_TOKEN` (fine-grained PAT, dépôt `Portabase/cli`, permissions Contents: Read and write, Metadata: Read) doit être créé par un mainteneur dans Settings → Secrets → Actions. + +- [ ] **Step 3: Supprimer le script** + +Run: `git rm release` + +- [ ] **Step 4: Documenter dans CONTRIBUTING.md** + +Ajouter une section à la fin de `.github/CONTRIBUTING.md` : + +```markdown +## Releasing + +Releases are cut from GitHub Actions, never from a local machine. + +1. Open **Actions → Bump version → Run workflow**. +2. Pick the branch (`main` for stable, any branch for a release candidate). +3. Enter the version without a leading `v` (`26.09.0` for stable, `26.09.0rc1` for a candidate) and the matching channel. +4. The workflow commits `chore(release): `, creates the tag and pushes. The tag triggers the build, the GitHub release, the Discord notification and the template upload. + +Stable versions must match `X.Y.Z` and can only be cut from `main`. +``` + +- [ ] **Step 5: Valider YAML** + +Run: `uv run python -c "import yaml; yaml.safe_load(open('.github/workflows/bump.yml')); print('ok')"` +Expected: `ok`. + +- [ ] **Step 6: Commit** + +```bash +git add .github/workflows/bump.yml .github/CONTRIBUTING.md +git commit -m "ci: replace ./release script with bump workflow" +``` + +--- + +### Task 11 : Vérification de bout en bout sur GitHub + +**Files:** aucun. + +**Interfaces:** +- Consumes: tout ce qui précède. + +- [ ] **Step 1: Pousser une branche et ouvrir une PR** + +```bash +git checkout -b ci/hygiene +git push -u origin ci/hygiene +gh pr create --fill --title "ci: PR workflow, pinned actions, bump workflow" --body "Implements plan 1 (chantier A) of docs/superpowers/specs/2026-09-11-cli-refactor-design.md. No CLI behaviour change." +``` + +(`gh` non authentifié ici : créer la PR depuis l'interface si la commande échoue.) + +- [ ] **Step 2: Vérifier les checks** + +Expected dans l'onglet Checks : `lint`, `test`, `gitleaks`, `plumber`, `build-smoke` tous verts. `plumber` publie un rapport SARIF dans Security → Code scanning ; noter le score obtenu. + +- [ ] **Step 3: Si `plumber` remonte des findings sur les workflows** + +Les traiter dans la même PR si triviaux (permission manquante, action non pinnée oubliée). Sinon ouvrir une issue avec la liste et laisser `soft-fail: true`. + +- [ ] **Step 4: Créer le secret `RELEASE_TOKEN`** + +Settings → Secrets and variables → Actions → New repository secret. PAT fine-grained, dépôt `Portabase/cli`, Contents: Read and write, Metadata: Read. + +- [ ] **Step 5: Merger, puis tester `bump.yml` avec un rc jetable** + +Actions → Bump version → branche `main`, version `26.07.7rc1`, channel `rc`. Expected : commit `chore(release): 26.07.7rc1` sur `main`, tag créé, `release-candidate.yml` déclenché, binaires attestés publiés en pre-release, templates uploadés sous `templates/26.07.7rc1/`. + +- [ ] **Step 6: Rendre les checks requis** + +Settings → Branches → `main` → Require status checks : `lint`, `test`, `gitleaks`, `build-smoke`. Laisser `plumber` non requis tant que `soft-fail: true`. + +--- + +## Self-review + +**Spec coverage (§9, §10 A) :** +- 9.1 `ci.yml` : lint ✔ (T4), test vide ✔ (T4), gitleaks ✔ (T3, T4), plumber ✔ (T4), build-smoke `--version` ✔ (T4 ; l'invocation `agent --non-interactive` arrive au Plan 4), `render-check` et `engines-check` → Plan 3 (dépendent des templates `.j2` et du registre). +- 9.2 pin SHA ✔ (T4–T8), Dependabot ✔ (T9), `permissions: {}` ✔, `packages: write` retiré ✔ (T8), `~/.s3cfg` supprimé ✔ (T7), attestation ✔ (T5). +- 9.3 `bump.yml` ✔ (T10), `./release` supprimé ✔, pas de release-please ✔, question branch protection → T10/T11. `templates-hotfix.yml` et manifest → Plan 3. +- 9.4 `pyproject.toml` ✔ (T1) ; `jinja2` ajouté au Plan 3 quand il est utilisé. +- 10 A : shippable stable ✔ (T11 step 5 le prouve avec un rc). + +**Placeholder scan :** aucun TBD/TODO. Toutes les étapes ont leur contenu ou leur commande. + +**Type consistency :** noms de jobs identiques entre T4 et T11 (`lint`, `test`, `gitleaks`, `plumber`, `build-smoke`) ; secret `RELEASE_TOKEN` cohérent T10/T11 ; SHAs identiques entre tâches. + +**Écart connu :** T1 `per-file-ignores` liste des fichiers/règles déduits du run ruff du 2026-09-11 ; si ruff remonte une règle non listée sur un fichier legacy, l'ajouter à la liste de ce fichier (pas de correction de code). From 43a97499b64e60fbd72f5c9df16c370cb6feb879 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Th=C3=A9o=20LAGACHE?= Date: Fri, 11 Sep 2026 10:21:37 +0200 Subject: [PATCH 04/24] docs: add plan 2 (foundations: errors, ui, services, Command; lifecycle rewrite) Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_017ACLkgaiP48pBMyi7kZZ4a --- ...2026-09-11-plan-2-foundations-lifecycle.md | 2381 +++++++++++++++++ 1 file changed, 2381 insertions(+) create mode 100644 docs/superpowers/plans/2026-09-11-plan-2-foundations-lifecycle.md diff --git a/docs/superpowers/plans/2026-09-11-plan-2-foundations-lifecycle.md b/docs/superpowers/plans/2026-09-11-plan-2-foundations-lifecycle.md new file mode 100644 index 0000000..7ec1a50 --- /dev/null +++ b/docs/superpowers/plans/2026-09-11-plan-2-foundations-lifecycle.md @@ -0,0 +1,2381 @@ +# Plan 2 — Fondations et lifecycle (chantiers B + C) + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Poser les fondations POO (erreurs, `ui/`, services d'infrastructure, `Command`, catcher, télémétrie no-op, updater sans auto-update) et réécrire les commandes de lifecycle/config/update dessus, tout en gardant `agent`, `dashboard` et `db` sur l'ancien code via un adaptateur — le CLI reste shippable en stable à la fin. + +**Architecture:** `main.py` construit les dépendances (`UI`, `Telemetry`, `GlobalConfig`, `HttpClient`, `DockerRunner`) et les injecte dans des classes `Command` enregistrées sur Typer. Un seul `try` dans `main()` traduit `PortabaseError` en message + code de sortie. Les commandes legacy sont enregistrées telles quelles par `LegacyCommand` ; elles continuent d'importer `core.utils.console` jusqu'au Plan 4. + +**Tech Stack:** Python 3.12, Typer 0.25 / Click 8.4, Rich 15, questionary 2.1, requests. + +**Spec:** `docs/superpowers/specs/2026-09-11-cli-refactor-design.md` — sections 3, 4.1, 7, 8, 10 (B, C). + +## Global Constraints + +- Prérequis : Plan 1 exécuté (ruff configuré, CI en place). +- Règle de dépendance descendante : `commands → services, engines, ui, core` ; `services → engines, core` (jamais `ui`) ; `ui → core` ; `core → rien`. Vérifiée par ruff `TID251` (Task 12). +- `rich.prompt`, `typer.prompt`, `typer.confirm`, `print` interdits hors `ui/` (ruff `TID251`, activé Task 12 avec exceptions legacy). +- `typer.Exit` n'est levé nulle part hors des fichiers legacy ; le nouveau code lève `PortabaseError`. +- Pas de tests unitaires (consigne). Chaque tâche a des vérifications exécutables ; les commandes Docker sont vérifiées avec un dossier agent réel si Docker est disponible, sinon sur leurs chemins d'erreur. +- Ne pas toucher `commands/agent.py`, `commands/db.py`, `commands/dashboard.py`, `core/network.py`, `core/docker.py`, `templates/compose.py` (supprimés au Plan 4). `core/utils.py` : seulement retirer `current_version` (Task 2). +- Déviation spec assumée : `Field` vit dans `core/fields.py` (partagé par `ui.Form` et `engines`), pas dans `engines/base.py`. +- Nom de la clé de config existante conservé : `update_channel` (valeurs `stable` / `beta`). +- Commits Conventional Commits, un par tâche minimum. + +--- + +## File Structure + +| Fichier | Action | Responsabilité | +|---|---|---| +| `core/errors.py` | créer | hiérarchie `PortabaseError` | +| `core/version.py` | créer | `current_version()`, `parse_version()`, `is_prerelease()` | +| `core/utils.py` | modifier | retirer `current_version` (re-export pour legacy) | +| `core/config.py` | modifier | ajouter classe `GlobalConfig` ; fonctions legacy conservées | +| `core/fields.py` | créer | `Field` | +| `ui/theme.py` | créer | `PALETTE`, `RICH_THEME`, `QUESTIONARY_STYLE` | +| `ui/components/base.py` | créer | `Component` | +| `ui/components/hints.py` | créer | `HINTS`, `Hint` | +| `ui/components/message.py` | créer | `Message` | +| `ui/components/banner.py` | créer | `Banner` | +| `ui/components/section.py` | créer | `Section` | +| `ui/components/status.py` | créer | `Status` | +| `ui/components/progress.py` | créer | `Progress` (téléchargement) | +| `ui/components/prompt.py` | créer | `Prompt` (questionary) | +| `ui/form.py` | créer | `Form` | +| `ui/__init__.py` | créer | façade `UI` | +| `services/http.py` | créer | `HttpClient` | +| `services/docker.py` | créer | `DockerRunner` | +| `services/telemetry.py` | créer | `Telemetry`, `NoopTelemetry`, `ConsoleTelemetry`, `TelemetryHub`, `TelemetryFactory` | +| `services/updater.py` | créer | `Release`, `UpdateChecker`, `Updater` | +| `commands/base.py` | créer | `Command`, `CommandGroup`, `LegacyCommand` | +| `commands/lifecycle.py` | créer | `Start/Stop/Restart/Logs/Uninstall` | +| `commands/config.py` | réécrire | `ConfigCommands` | +| `commands/update.py` | créer | `UpdateCommand` | +| `main.py` | réécrire | `Settings`, `build_app`, `main` | +| `commands/common.py`, `core/updater.py` | supprimer | — | +| `pyproject.toml` | modifier | `TID251`, per-file-ignores mis à jour | + +--- + +### Task 1 : `core/errors.py` + +**Files:** +- Create: `core/errors.py` + +**Interfaces:** +- Produces: `PortabaseError(message, *, hint=None, cause=None)` avec attributs `message`, `hint`, `cause`, classe-attributs `code: str`, `exit_code: int` ; sous-classes `UserAbort`, `ValidationError`, `ConfigError`, `DockerError`, `TemplateError`, `NetworkError`, `UpdateError`. + +- [ ] **Step 1: Écrire le module** + +```python +"""Exception hierarchy. Every error the CLI reports to a user is one of these.""" + +from __future__ import annotations + + +class PortabaseError(Exception): + code: str = "E_GENERIC" + exit_code: int = 1 + + def __init__( + self, + message: str, + *, + hint: str | None = None, + cause: BaseException | None = None, + ) -> None: + super().__init__(message) + self.message = message + self.hint = hint + self.cause = cause + if cause is not None: + self.__cause__ = cause + + def __str__(self) -> str: + return self.message + + +class UserAbort(PortabaseError): + code = "E_ABORT" + exit_code = 130 + + def __init__(self, message: str = "Cancelled.", **kwargs) -> None: + super().__init__(message, **kwargs) + + +class ValidationError(PortabaseError): + code = "E_VALIDATION" + exit_code = 2 + + +class ConfigError(PortabaseError): + code = "E_CONFIG" + exit_code = 3 + + +class DockerError(PortabaseError): + code = "E_DOCKER" + exit_code = 4 + + +class TemplateError(PortabaseError): + code = "E_TEMPLATE" + exit_code = 5 + + +class NetworkError(PortabaseError): + code = "E_NETWORK" + exit_code = 6 + + +class UpdateError(PortabaseError): + code = "E_UPDATE" + exit_code = 7 +``` + +- [ ] **Step 2: Vérifier** + +Run: `uv run python -c "from core.errors import *; e = DockerError('daemon down', hint='start it'); print(e.code, e.exit_code, e, e.hint); assert isinstance(e, PortabaseError)"` +Expected: `E_DOCKER 4 daemon down start it`. + +- [ ] **Step 3: Commit** + +```bash +git add core/errors.py +git commit -m "feat(core): add PortabaseError hierarchy with stable codes and exit codes" +``` + +--- + +### Task 2 : `core/version.py` et `core/fields.py` + +**Files:** +- Create: `core/version.py` +- Create: `core/fields.py` +- Modify: `core/utils.py:197-214` (fonction `current_version`) + +**Interfaces:** +- Produces: `current_version() -> str` ; `parse_version(v: str) -> tuple` ; `is_prerelease(v: str) -> bool` ; `Field` dataclass. + +- [ ] **Step 1: Écrire `core/version.py`** + +```python +"""CLI version helpers. Version is read from the bundled pyproject.toml.""" + +from __future__ import annotations + +import re +import sys +import tomllib +from functools import lru_cache +from pathlib import Path + +UNKNOWN = "unknown" +_PRE = re.compile(r"^(\d+)\.(\d+)\.(\d+)(?:[-.]?(rc|alpha|beta|a|b)(\d*))?$", re.I) + + +@lru_cache(maxsize=1) +def current_version() -> str: + try: + base = Path(sys._MEIPASS) if getattr(sys, "frozen", False) else Path(__file__).parent.parent + with open(base / "pyproject.toml", "rb") as f: + return tomllib.load(f)["project"]["version"] + except (FileNotFoundError, KeyError, tomllib.TOMLDecodeError, AttributeError): + return UNKNOWN + + +def is_prerelease(version: str) -> bool: + m = _PRE.match(version.strip().lstrip("v")) + return bool(m and m.group(4)) + + +def parse_version(version: str) -> tuple[int, int, int, int, int]: + """Sortable tuple. Pre-releases sort before the final release of the same number. + + (major, minor, patch, pre_rank, pre_number) — pre_rank: 0 alpha/a, 1 beta/b, 2 rc, 3 final. + """ + m = _PRE.match(version.strip().lstrip("v")) + if not m: + return (0, 0, 0, 0, 0) + major, minor, patch = (int(m.group(i)) for i in (1, 2, 3)) + tag = (m.group(4) or "").lower() + rank = {"alpha": 0, "a": 0, "beta": 1, "b": 1, "rc": 2, "": 3}[tag] + num = int(m.group(5)) if m.group(5) else 0 + return (major, minor, patch, rank, num) +``` + +- [ ] **Step 2: Écrire `core/fields.py`** + +```python +"""Declarative input field. Used by ui.Form to prompt or validate a value.""" + +from __future__ import annotations + +from collections.abc import Callable +from dataclasses import dataclass +from typing import Any, Literal + +FieldKind = Literal["text", "int", "secret", "bool", "choice", "path"] + + +@dataclass(frozen=True) +class Field: + name: str + prompt: str + kind: FieldKind = "text" + default: Any = None + choices: tuple[str, ...] = () + help: str | None = None + validator: Callable[[Any], Any] | None = None + + @property + def flag(self) -> str: + return "--" + self.name.replace("_", "-") +``` + +- [ ] **Step 3: Retirer `current_version` de `core/utils.py`** + +Supprimer la fonction `current_version` (lignes ~197–214) et ajouter en tête des imports : + +```python +from core.version import current_version # noqa: F401 — re-export for legacy modules +``` + +`core/network.py` et `core/updater.py` importent `current_version` depuis `core.utils` ; le re-export les garde fonctionnels jusqu'à leur suppression. + +- [ ] **Step 4: Vérifier** + +Run: `uv run python -c "from core.version import *; print(current_version(), is_prerelease('26.09.0rc1'), parse_version('26.09.0rc1') < parse_version('26.09.0'), parse_version('26.10.0') > parse_version('26.9.9'))" && uv run python main.py --version` +Expected: `26.07.6 True True True` puis `Portabase CLI version: 26.07.6`. + +- [ ] **Step 5: Commit** + +```bash +git add core/version.py core/fields.py core/utils.py +git commit -m "feat(core): add version helpers and Field descriptor" +``` + +--- + +### Task 3 : `GlobalConfig` + +**Files:** +- Modify: `core/config.py` + +**Interfaces:** +- Produces: `GlobalConfig(path: Path = GLOBAL_CONFIG_FILE)` avec `get(key, default=None)`, `set(key, value)`, `all() -> dict`, `cache_dir: Path` (`~/.portabase/cache`) ; propriétés typées `update_channel: str | None`, `telemetry: bool`, `telemetry_endpoint: str | None`. +- Les fonctions module-level existantes restent (legacy). + +- [ ] **Step 1: Ajouter la classe en fin de `core/config.py`** + +```python +class GlobalConfig: + """~/.portabase/config.json. Unknown keys are preserved.""" + + KNOWN_KEYS = ("update_channel", "telemetry", "telemetry_endpoint") + + def __init__(self, path: Path = GLOBAL_CONFIG_FILE) -> None: + self.path = path + self.cache_dir = path.parent / "cache" + + def all(self) -> dict: + if not self.path.exists(): + return {} + try: + with open(self.path, encoding="utf-8") as f: + data = json.load(f) + except (OSError, json.JSONDecodeError): + return {} + return data if isinstance(data, dict) else {} + + def get(self, key: str, default=None): + return self.all().get(key, default) + + def set(self, key: str, value) -> None: + data = self.all() + data[key] = value + self.path.parent.mkdir(parents=True, exist_ok=True) + tmp = self.path.with_suffix(".json.tmp") + with open(tmp, "w", encoding="utf-8") as f: + json.dump(data, f, indent=2) + os.replace(tmp, self.path) + + @property + def update_channel(self) -> str | None: + return self.get("update_channel") + + @property + def telemetry(self) -> bool: + return str(self.get("telemetry", "false")).lower() in ("1", "true", "yes") + + @property + def telemetry_endpoint(self) -> str | None: + return self.get("telemetry_endpoint") +``` + +Ajouter au-dessus des fonctions legacy le commentaire : + +```python +# --- Legacy helpers below: used by commands/agent.py, db.py, dashboard.py, core/updater.py. +# --- Removed in plan 4. New code uses GlobalConfig. +``` + +- [ ] **Step 2: Vérifier** + +Run: `uv run python -c " +from pathlib import Path; import tempfile +from core.config import GlobalConfig +c = GlobalConfig(Path(tempfile.mkdtemp())/'config.json') +print(c.all(), c.telemetry); c.set('update_channel','beta'); c.set('telemetry', True) +print(c.update_channel, c.telemetry, c.all())"` +Expected: `{} False` puis `beta True {'update_channel': 'beta', 'telemetry': True}`. + +- [ ] **Step 3: Commit** + +```bash +git add core/config.py +git commit -m "feat(core): add GlobalConfig class over ~/.portabase/config.json" +``` + +--- + +### Task 4 : `ui/theme.py` et composants d'affichage + +**Files:** +- Create: `ui/__init__.py` (vide pour l'instant, rempli Task 6) +- Create: `ui/theme.py` +- Create: `ui/components/__init__.py` (vide) +- Create: `ui/components/base.py` +- Create: `ui/components/hints.py` +- Create: `ui/components/message.py` +- Create: `ui/components/banner.py` +- Create: `ui/components/section.py` +- Create: `ui/components/status.py` +- Create: `ui/components/progress.py` + +**Interfaces:** +- Produces: `Component(console)` ; `Hint(console).random() -> str` ; `Message(console).success/info/warning(text)`, `.error(exc: PortabaseError, *, verbose: bool, unexpected: bool)` ; `Banner(console)()` ; `Section(console)(title)` ; `Status(console)(text) -> ContextManager` ; `Progress(console).download(description, total) -> ContextManager[Callable[[int], None]]`. + +- [ ] **Step 1: `ui/theme.py`** + +```python +"""Single source of visual tokens. Rich theme and questionary style derive from PALETTE.""" + +from __future__ import annotations + +from questionary import Style +from rich.theme import Theme + +PALETTE = { + "brand": "#ff6600", + "accent": "#5f00d7", + "info": "cyan", + "warning": "magenta", + "danger": "red", + "success": "green", + "muted": "grey50", +} + +RICH_THEME = Theme( + { + "info": f"dim {PALETTE['info']}", + "warning": PALETTE["warning"], + "danger": f"bold {PALETTE['danger']}", + "success": f"bold {PALETTE['success']}", + "title": f"bold white on {PALETTE['accent']}", + "key": f"bold {PALETTE['brand']}", + "value": "white", + "hint": f"italic {PALETTE['muted']}", + "brand": f"bold {PALETTE['brand']}", + } +) + +QUESTIONARY_STYLE = Style( + [ + ("qmark", f"fg:{PALETTE['brand']} bold"), + ("question", "bold"), + ("pointer", f"fg:{PALETTE['brand']} bold"), + ("highlighted", f"fg:black bg:{PALETTE['brand']} bold"), + ("selected", f"fg:{PALETTE['brand']} bold"), + ("answer", f"fg:{PALETTE['brand']}"), + ] +) + +QUESTIONARY_STYLE_PLAIN = Style([]) +``` + +- [ ] **Step 2: `ui/components/base.py`** + +```python +from __future__ import annotations + +from rich.console import Console + + +class Component: + """Stateless renderable bound to a console. Instantiate per call.""" + + def __init__(self, console: Console) -> None: + self.console = console +``` + +- [ ] **Step 3: `ui/components/hints.py`** + +Reprendre la liste `HINTS` de `core/utils.py:42-66` telle quelle. + +```python +from __future__ import annotations + +import random + +from ui.components.base import Component + +HINTS = [ + "The Edge Key contains the connection details for dashboard and agent communication.", + "Portabase uses Docker Compose to isolate your databases.", + "You can list all configured databases using 'portabase db list '.", + "Running 'portabase stop' will gracefully shut down your containers.", + "The agent polls the github for configuration updates.", + "Logs can be viewed in real-time with 'portabase logs '.", + "Custom environment variables can be added to the generated .env file.", + "Need to update? Use 'portabase update' to get the latest version.", + "You can add multiple databases to a single agent during setup.", + "Portabase Dashboard provides a web interface to manage your infrastructure.", + "Is Docker not running? The CLI will offer to start it for you!", + "All configurations are stored locally in the component's folder.", + "The 'portabase restart' command is useful after manual .env modifications.", + "Portabase is open-source! Check our GitHub to contribute.", + "Using the --start flag with 'agent' or 'dashboard' skips the final prompt.", + "Internal databases are automatically backed up when using volumes.", + "The dashboard requires a PostgreSQL database to store its own data.", + "You can change the update channel to 'beta' in the config for early features.", + "Portabase network ensures secure communication between your containers.", + "Lost your Edge Key? You can find it in the dashboard.", + "The 'portabase uninstall' command safely removes containers and their data.", + "Use 'portabase --version' to check your current installation details.", + "The 'databases.json' file keeps track of all managed database instances.", +] + + +class Hint(Component): + def random(self) -> str: + return f"[hint]{random.choice(HINTS)}[/hint]" + + def __call__(self, text: str | None = None) -> None: + self.console.print(f"[hint]{text}[/hint]" if text else self.random()) +``` + +- [ ] **Step 4: `ui/components/message.py`** + +```python +from __future__ import annotations + +import traceback + +from core.errors import PortabaseError +from ui.components.base import Component + + +class Message(Component): + def success(self, text: str) -> None: + self.console.print(f"[success]✔ {text}[/success]") + + def info(self, text: str) -> None: + self.console.print(f"[info]ℹ {text}[/info]") + + def warning(self, text: str) -> None: + self.console.print(f"[warning]⚠ {text}[/warning]") + + def error(self, exc: PortabaseError, *, verbose: bool = False, unexpected: bool = False) -> None: + label = "Unexpected error" if unexpected else "Error" + self.console.print(f"[danger]✖ {label}:[/danger] {exc.message}") + if exc.hint: + self.console.print(f" [hint]↳ {exc.hint}[/hint]") + if verbose or unexpected: + self.console.print(f" [hint]code: {exc.code}[/hint]") + if verbose and exc.cause is not None: + self.console.print(f" [hint]cause: {type(exc.cause).__name__}: {exc.cause}[/hint]") + if verbose: + self.console.print("".join(traceback.format_exception(exc)), highlight=False, markup=False) +``` + +- [ ] **Step 5: `ui/components/banner.py`** + +```python +from __future__ import annotations + +from rich.align import Align + +from ui.components.base import Component +from ui.components.hints import Hint + +BANNER = """ +[brand]█▀█ █▀█ █▀█ ▀█▀ ▄▀█ █▄▄ ▄▀█ █▀ █▀▀[/brand] +[brand]█▀▀ █▄█ █▀▄ █ █▀█ █▄█ █▀█ ▄█ ██▄[/brand] +[hint]Deploy your infrastructure anywhere.[/hint] +""" + + +class Banner(Component): + def __call__(self) -> None: + self.console.print(Align.center(BANNER)) + self.console.print(Align.center(Hint(self.console).random() + "\n")) +``` + +- [ ] **Step 6: `ui/components/section.py`** + +```python +from __future__ import annotations + +from rich.panel import Panel + +from ui.components.base import Component + + +class Section(Component): + def __call__(self, title: str) -> None: + self.console.print("") + self.console.print(Panel(f"[bold]{title}[/bold]", style="cyan", expand=False)) +``` + +- [ ] **Step 7: `ui/components/status.py`** + +```python +from __future__ import annotations + +from contextlib import AbstractContextManager + +from ui.components.base import Component +from ui.components.hints import Hint + + +class Status(Component): + def __call__(self, text: str, *, spinner: str = "dots") -> AbstractContextManager: + message = f"[bold magenta]{text}[/bold magenta]\n{Hint(self.console).random()}" + return self.console.status(message, spinner=spinner) +``` + +- [ ] **Step 8: `ui/components/progress.py`** + +```python +from __future__ import annotations + +from collections.abc import Callable, Iterator +from contextlib import contextmanager + +from rich.progress import ( + BarColumn, + DownloadColumn, + Progress as RichProgress, + SpinnerColumn, + TextColumn, + TransferSpeedColumn, +) + +from ui.components.base import Component +from ui.components.hints import Hint + + +class Progress(Component): + @contextmanager + def download(self, description: str, total: int) -> Iterator[Callable[[int], None]]: + """Yields an advance(n_bytes) callable.""" + with RichProgress( + SpinnerColumn(), + TextColumn("[progress.description]{task.description}\n" + Hint(self.console).random()), + BarColumn(), + DownloadColumn(), + TransferSpeedColumn(), + console=self.console, + ) as progress: + task = progress.add_task(description, total=total or None) + yield lambda n: progress.update(task, advance=n) +``` + +- [ ] **Step 9: Vérifier le rendu** + +Run: `uv run python -c " +from rich.console import Console +from ui.theme import RICH_THEME +from ui.components.message import Message +from ui.components.banner import Banner +from ui.components.section import Section +from core.errors import DockerError +c = Console(theme=RICH_THEME) +Banner(c)(); Section(c)('Database Setup') +m = Message(c); m.success('ok'); m.info('note'); m.warning('careful') +m.error(DockerError('daemon down', hint='run: sudo systemctl start docker')) +m.error(DockerError('daemon down', hint='x', cause=RuntimeError('boom')), verbose=True)"` +Expected: bannière orange, panneau cyan, quatre messages avec icônes ✔ ℹ ⚠ ✖, hint indenté, puis le bloc verbose avec `code: E_DOCKER`, `cause: RuntimeError: boom` et une traceback. + +- [ ] **Step 10: Commit** + +```bash +git add ui/ +git commit -m "feat(ui): add theme tokens and display components" +``` + +--- + +### Task 5 : `ui/components/prompt.py` et `ui/form.py` + +**Files:** +- Create: `ui/components/prompt.py` +- Create: `ui/form.py` + +**Interfaces:** +- Consumes: `Field` (Task 2), `UserAbort`/`ValidationError` (Task 1), `QUESTIONARY_STYLE` (Task 4). +- Produces: `Prompt(console, style)` avec `text/integer/secret/confirm/select/path` renvoyant `None` sur Ctrl-C ; `Form(prompt, non_interactive)` avec `ask(field, value)`, `collect(fields, values) -> dict`, raccourcis `text/integer/secret/confirm/choice`. + +- [ ] **Step 1: `ui/components/prompt.py`** + +```python +from __future__ import annotations + +from collections.abc import Sequence + +import questionary +from questionary import Style +from rich.console import Console + +from ui.components.base import Component + + +class Prompt(Component): + """Thin wrapper over questionary. Every method returns None when the user aborts (Ctrl-C).""" + + def __init__(self, console: Console, style: Style) -> None: + super().__init__(console) + self.style = style + + def text(self, message: str, *, default: str | None = None) -> str | None: + return questionary.text(message, default=default or "", style=self.style).ask() + + def integer(self, message: str, *, default: int | None = None) -> int | None: + answer = questionary.text( + message, + default="" if default is None else str(default), + validate=lambda v: v.strip().lstrip("-").isdigit() or "Enter a whole number", + style=self.style, + ).ask() + return None if answer is None else int(answer) + + def secret(self, message: str) -> str | None: + return questionary.password(message, style=self.style).ask() + + def confirm(self, message: str, *, default: bool = False) -> bool | None: + return questionary.confirm(message, default=default, style=self.style).ask() + + def select(self, message: str, choices: Sequence[str], *, default: str | None = None) -> str | None: + return questionary.select(message, choices=list(choices), default=default, style=self.style).ask() + + def path(self, message: str, *, default: str | None = None) -> str | None: + return questionary.path(message, default=default or "", style=self.style).ask() +``` + +- [ ] **Step 2: `ui/form.py`** + +```python +"""Flag → prompt → default → error. The only place that knows about non-interactive mode.""" + +from __future__ import annotations + +from collections.abc import Callable, Sequence +from typing import Any + +from core.errors import UserAbort, ValidationError +from core.fields import Field +from ui.components.prompt import Prompt + +_TRUE = {"1", "true", "yes", "y", "on"} +_FALSE = {"0", "false", "no", "n", "off"} + + +class Form: + def __init__(self, prompt: Prompt, non_interactive: bool) -> None: + self.prompt = prompt + self.non_interactive = non_interactive + self._askers: dict[str, Callable[[Field], Any]] = { + "text": lambda f: self.prompt.text(f.prompt, default=f.default), + "int": lambda f: self.prompt.integer(f.prompt, default=f.default), + "secret": lambda f: self.prompt.secret(f.prompt), + "bool": lambda f: self.prompt.confirm(f.prompt, default=bool(f.default)), + "choice": lambda f: self.prompt.select(f.prompt, f.choices, default=f.default), + "path": lambda f: self.prompt.path(f.prompt, default=f.default), + } + + # ---- core ------------------------------------------------------------- + + def ask(self, field: Field, value: Any | None = None) -> Any: + if value is not None: + return self._coerce_and_validate(field, value) + if self.non_interactive: + if field.default is not None: + return self._coerce_and_validate(field, field.default) + raise ValidationError( + f"Missing {field.flag}", + hint=f"Required in non-interactive mode: {field.prompt}", + ) + return self._ask_until_valid(field) + + def collect(self, fields: Sequence[Field], values: dict[str, Any]) -> dict[str, Any]: + return {f.name: self.ask(f, values.get(f.name)) for f in fields} + + # ---- shortcuts -------------------------------------------------------- + + def text(self, prompt: str, *, value=None, default=None, validator=None, name="value") -> str: + return self.ask(Field(name, prompt, "text", default=default, validator=validator), value) + + def integer(self, prompt: str, *, value=None, default=None, validator=None, name="value") -> int: + return self.ask(Field(name, prompt, "int", default=default, validator=validator), value) + + def secret(self, prompt: str, *, value=None, validator=None, name="value") -> str: + return self.ask(Field(name, prompt, "secret", validator=validator), value) + + def confirm(self, prompt: str, *, value=None, default: bool = False, name="value") -> bool: + return self.ask(Field(name, prompt, "bool", default=default), value) + + def choice(self, prompt: str, choices: Sequence[str], *, value=None, default=None, name="value") -> str: + return self.ask(Field(name, prompt, "choice", default=default, choices=tuple(choices)), value) + + # ---- internals -------------------------------------------------------- + + def _ask_until_valid(self, field: Field) -> Any: + if field.help: + self.prompt.console.print(f"[info]ℹ {field.help}[/info]") + while True: + answer = self._askers[field.kind](field) + if answer is None: + raise UserAbort() + try: + return self._coerce_and_validate(field, answer) + except ValidationError as e: + self.prompt.console.print(f"[danger]✖ {e.message}[/danger]") + + def _coerce_and_validate(self, field: Field, value: Any) -> Any: + value = self._coerce(field, value) + if field.kind == "choice" and value not in field.choices: + raise ValidationError( + f"Invalid value for {field.flag}: {value!r}", + hint="Choices: " + ", ".join(field.choices), + ) + if field.validator is not None: + value = field.validator(value) # raises ValidationError + return value + + @staticmethod + def _coerce(field: Field, value: Any) -> Any: + if field.kind == "int" and not isinstance(value, int): + try: + return int(str(value).strip()) + except ValueError as e: + raise ValidationError(f"{field.flag} must be a whole number, got {value!r}") from e + if field.kind == "bool" and not isinstance(value, bool): + s = str(value).strip().lower() + if s in _TRUE: + return True + if s in _FALSE: + return False + raise ValidationError(f"{field.flag} must be true or false, got {value!r}") + if field.kind in ("text", "secret", "path", "choice"): + return str(value) + return value +``` + +- [ ] **Step 3: Vérifier le mode non-interactif (sans terminal)** + +Run: `uv run python -c " +from rich.console import Console +from ui.theme import QUESTIONARY_STYLE +from ui.components.prompt import Prompt +from ui.form import Form +from core.fields import Field +from core.errors import ValidationError +f = Form(Prompt(Console(), QUESTIONARY_STYLE), non_interactive=True) +print(f.text('Timezone', value=None, default='UTC'), f.integer('Polling', value='7'), f.confirm('Gateway?', value='yes')) +print(f.collect([Field('engine','Engine','choice',choices=('a','b')), Field('port','Port','int',default=5432)], {'engine':'a'})) +try: f.text('Edge key') +except ValidationError as e: print('OK:', e.message, '|', e.hint) +try: f.choice('Mode', ['new','existing'], value='bogus') +except ValidationError as e: print('OK:', e.message, '|', e.hint)"` +Expected : +``` +UTC 7 True +{'engine': 'a', 'port': 5432} +OK: Missing --value | Required in non-interactive mode: Edge key +OK: Invalid value for --value: 'bogus' | Choices: new, existing +``` + +- [ ] **Step 4: Vérifier le mode interactif (terminal requis)** + +Run: `uv run python -c " +from rich.console import Console +from ui.theme import QUESTIONARY_STYLE +from ui.components.prompt import Prompt +from ui.form import Form +f = Form(Prompt(Console(), QUESTIONARY_STYLE), non_interactive=False) +print(f.choice('Mode', ['new','existing'], default='new')) +print(f.integer('Port', default=5432))"` +Répondre aux deux prompts. Puis relancer et faire Ctrl-C au premier prompt. +Expected: valeurs saisies affichées ; sur Ctrl-C, traceback se terminant par `core.errors.UserAbort: Cancelled.` (le catcher n'est pas encore branché — attendu). + +- [ ] **Step 5: Commit** + +```bash +git add ui/components/prompt.py ui/form.py +git commit -m "feat(ui): add questionary Prompt and Form with non-interactive resolution" +``` + +--- + +### Task 6 : Façade `UI` + +**Files:** +- Modify: `ui/__init__.py` + +**Interfaces:** +- Produces: `UI(console=None, *, non_interactive=False, verbose=False, no_color=False)` ; `configure(**kwargs)` ; `banner()`, `success/info/warning(text)`, `error(exc, unexpected=False)`, `hint(text=None)`, `section(title)`, `status(text)`, `progress()`, `confirm(q, default=False, value=None) -> bool`, `form() -> Form`, `print(renderable)`. Attribut `console`. + +- [ ] **Step 1: Écrire la façade** + +```python +"""Facade: the only thing `commands/` imports from ui. Rich and questionary stay inside ui/.""" + +from __future__ import annotations + +from rich.console import Console + +from core.errors import PortabaseError +from ui.components.banner import Banner +from ui.components.hints import Hint +from ui.components.message import Message +from ui.components.progress import Progress +from ui.components.prompt import Prompt +from ui.components.section import Section +from ui.components.status import Status +from ui.form import Form +from ui.theme import QUESTIONARY_STYLE, QUESTIONARY_STYLE_PLAIN, RICH_THEME + + +class UI: + def __init__( + self, + console: Console | None = None, + *, + non_interactive: bool = False, + verbose: bool = False, + no_color: bool = False, + ) -> None: + self.non_interactive = non_interactive + self.verbose = verbose + self.no_color = no_color + self.console = console or self._make_console() + + def configure(self, *, non_interactive: bool | None = None, verbose: bool | None = None, no_color: bool | None = None) -> None: + if non_interactive is not None: + self.non_interactive = non_interactive + if verbose is not None: + self.verbose = verbose + if no_color is not None and no_color != self.no_color: + self.no_color = no_color + self.console = self._make_console() + + def _make_console(self) -> Console: + return Console(theme=RICH_THEME, no_color=self.no_color) + + # ---- output ----------------------------------------------------------- + + def print(self, renderable) -> None: + self.console.print(renderable) + + def banner(self) -> None: + Banner(self.console)() + + def success(self, text: str) -> None: + Message(self.console).success(text) + + def info(self, text: str) -> None: + Message(self.console).info(text) + + def warning(self, text: str) -> None: + Message(self.console).warning(text) + + def error(self, exc: PortabaseError, *, unexpected: bool = False) -> None: + Message(self.console).error(exc, verbose=self.verbose, unexpected=unexpected) + + def hint(self, text: str | None = None) -> None: + Hint(self.console)(text) + + def section(self, title: str) -> None: + Section(self.console)(title) + + def status(self, text: str): + return Status(self.console)(text) + + def progress(self) -> Progress: + return Progress(self.console) + + # ---- input ------------------------------------------------------------ + + def form(self) -> Form: + style = QUESTIONARY_STYLE_PLAIN if self.no_color else QUESTIONARY_STYLE + return Form(Prompt(self.console, style), self.non_interactive) + + def confirm(self, question: str, *, default: bool = False, value: bool | None = None) -> bool: + return self.form().confirm(question, value=value, default=default) +``` + +- [ ] **Step 2: Vérifier** + +Run: `uv run python -c " +from ui import UI +ui = UI(non_interactive=True) +ui.banner(); ui.section('Test'); ui.success('a'); ui.warning('b'); ui.hint() +print('confirm default:', ui.confirm('Really?', default=False)) +with ui.status('Working...'): import time; time.sleep(0.5) +ui.configure(no_color=True); ui.success('no color')"` +Expected: rendu, `confirm default: False` sans prompt, spinner 0,5 s, dernière ligne sans couleur. + +- [ ] **Step 3: Commit** + +```bash +git add ui/__init__.py +git commit -m "feat(ui): add UI facade" +``` + +--- + +### Task 7 : `services/http.py` et `services/docker.py` + +**Files:** +- Create: `services/__init__.py` (vide) +- Create: `services/http.py` +- Create: `services/docker.py` + +**Interfaces:** +- Produces: + - `HttpClient(timeout=10.0)` : `get_json(url) -> Any`, `get_text(url) -> str`, `download(url, dest: Path, on_progress: Callable[[int], None] | None = None, *, timeout=30.0) -> int` (octets), `head_content_length(url) -> int | None`. Lèvent `NetworkError`. + - `DockerRunner(docker_bin: str | None = None)` : `available() -> bool`, `daemon_running() -> bool`, `start_daemon() -> bool`, `ensure_network(name)`, `compose(cwd, args, *, check=True, capture=False) -> subprocess.CompletedProcess`, `project_name(cwd) -> str`. Lèvent `DockerError`. + +- [ ] **Step 1: `services/http.py`** + +```python +"""requests wrapper. Every failure becomes NetworkError; nothing else leaks out.""" + +from __future__ import annotations + +from collections.abc import Callable +from pathlib import Path +from typing import Any + +import requests + +from core.errors import NetworkError + +_HINT = "Check your internet connection or proxy settings." + + +class HttpClient: + def __init__(self, timeout: float = 10.0, user_agent: str = "portabase-cli") -> None: + self.timeout = timeout + self.session = requests.Session() + self.session.headers["User-Agent"] = user_agent + + def get_json(self, url: str) -> Any: + try: + r = self.session.get(url, timeout=self.timeout) + r.raise_for_status() + return r.json() + except requests.RequestException as e: + raise NetworkError(f"GET {url} failed: {e}", hint=_HINT, cause=e) from e + except ValueError as e: + raise NetworkError(f"GET {url}: response is not JSON", cause=e) from e + + def get_text(self, url: str) -> str: + try: + r = self.session.get(url, timeout=self.timeout) + r.raise_for_status() + return r.text + except requests.RequestException as e: + raise NetworkError(f"GET {url} failed: {e}", hint=_HINT, cause=e) from e + + def status(self, url: str) -> int: + """HTTP status without raising on 4xx/5xx. Network failure still raises.""" + try: + return self.session.get(url, timeout=self.timeout, stream=True).status_code + except requests.RequestException as e: + raise NetworkError(f"GET {url} failed: {e}", hint=_HINT, cause=e) from e + + def download( + self, + url: str, + dest: Path, + on_progress: Callable[[int], None] | None = None, + *, + timeout: float = 30.0, + ) -> int: + written = 0 + try: + with self.session.get(url, stream=True, timeout=timeout) as r: + r.raise_for_status() + with open(dest, "wb") as f: + for chunk in r.iter_content(chunk_size=64 * 1024): + if not chunk: + continue + f.write(chunk) + written += len(chunk) + if on_progress: + on_progress(len(chunk)) + except requests.RequestException as e: + dest.unlink(missing_ok=True) + raise NetworkError(f"Download of {url} failed: {e}", hint=_HINT, cause=e) from e + return written + + def content_length(self, url: str) -> int | None: + try: + r = self.session.head(url, timeout=self.timeout, allow_redirects=True) + value = r.headers.get("content-length") + return int(value) if value else None + except (requests.RequestException, ValueError): + return None +``` + +- [ ] **Step 2: `services/docker.py`** + +Reprend `core/docker.py` + `check_system`/`start_docker` de `core/utils.py`, sans aucune sortie terminal. + +```python +"""Docker CLI runner. No terminal output; callers decide what to show.""" + +from __future__ import annotations + +import platform +import shutil +import subprocess +import time +from pathlib import Path + +from core.errors import DockerError +from core.utils import slugify_project_name + +_START_COMMANDS = { + "Linux": ["sudo", "systemctl", "start", "docker"], + "Darwin": ["open", "--background", "-a", "Docker"], + "Windows": ["cmd", "/c", "start", "docker"], +} + + +class DockerRunner: + def __init__(self, docker_bin: str | None = None) -> None: + self._bin = docker_bin + + @property + def binary(self) -> str: + if self._bin is None: + found = shutil.which("docker") + if found is None: + raise DockerError( + "Docker not found (binary missing).", + hint="Install Docker: https://docs.docker.com/get-docker/", + ) + self._bin = found + return self._bin + + def available(self) -> bool: + return shutil.which("docker") is not None + + def daemon_running(self) -> bool: + try: + subprocess.run( + [self.binary, "info"], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, check=True + ) + return True + except (subprocess.CalledProcessError, OSError): + return False + + def start_daemon(self, *, wait_seconds: int = 20) -> bool: + cmd = _START_COMMANDS.get(platform.system()) + if cmd is None: + return False + try: + subprocess.run(cmd, check=True) + except (subprocess.CalledProcessError, OSError) as e: + raise DockerError(f"Failed to start Docker: {e}", cause=e) from e + deadline = time.monotonic() + wait_seconds + while time.monotonic() < deadline: + if self.daemon_running(): + return True + time.sleep(2) + return False + + def ensure_network(self, name: str) -> None: + inspect = subprocess.run( + [self.binary, "network", "inspect", name], + stdout=subprocess.DEVNULL, + stderr=subprocess.DEVNULL, + check=False, + ) + if inspect.returncode == 0: + return + try: + subprocess.run([self.binary, "network", "create", name], stdout=subprocess.DEVNULL, check=True) + except subprocess.CalledProcessError as e: + raise DockerError(f"Could not create Docker network '{name}'.", cause=e) from e + + @staticmethod + def project_name(cwd: Path) -> str: + return slugify_project_name(cwd.resolve().name) + + def compose( + self, + cwd: Path, + args: list[str], + *, + check: bool = True, + capture: bool = False, + ) -> subprocess.CompletedProcess: + cmd = [self.binary, "compose", "-p", self.project_name(cwd), *args] + try: + return subprocess.run( + cmd, + cwd=cwd, + check=check, + capture_output=capture, + text=capture, + ) + except subprocess.CalledProcessError as e: + raise DockerError( + f"docker compose {' '.join(args)} failed (exit {e.returncode}).", + hint=f"Run it manually in {cwd} to see the full output.", + cause=e, + ) from e +``` + +- [ ] **Step 3: Vérifier** + +Run: `uv run python -c " +from services.http import HttpClient +from services.docker import DockerRunner +from core.errors import NetworkError, DockerError +h = HttpClient(timeout=5) +print(type(h.get_json('https://api.github.com/repos/Portabase/cli')).__name__) +try: h.get_json('https://127.0.0.1:1/nope') +except NetworkError as e: print('NetworkError OK:', e.code) +d = DockerRunner(); print('docker available:', d.available(), '| daemon:', d.available() and d.daemon_running()) +try: DockerRunner(docker_bin='/nonexistent').compose(__import__('pathlib').Path('.'), ['version']) +except (DockerError, OSError) as e: print('error path OK:', type(e).__name__)"` +Expected: `dict`, `NetworkError OK: E_NETWORK`, état Docker local, `error path OK: FileNotFoundError` ou `DockerError` (les deux acceptables ici ; `OSError` est traité au niveau commande, Task 9). + +- [ ] **Step 4: Commit** + +```bash +git add services/ +git commit -m "feat(services): add HttpClient and DockerRunner" +``` + +--- + +### Task 8 : `services/telemetry.py` + +**Files:** +- Create: `services/telemetry.py` + +**Interfaces:** +- Produces: `Telemetry` ABC (`session(**attrs)`, `span(name, **attrs)`, `event(name, **attrs)`, `error(exc, unexpected=False)`, `flush()`) ; `NoopTelemetry` ; `ConsoleTelemetry(stream=sys.stderr)` ; `TelemetryHub(inner)` avec `.set(inner)` ; `TelemetryFactory.build(config: GlobalConfig, *, debug: bool) -> TelemetryHub`. + +- [ ] **Step 1: Écrire le module** + +```python +"""Telemetry contract. Noop by default; OTel exporter can be plugged later without touching callers. + +Never record: agent names, paths, keys, credentials, file contents. +""" + +from __future__ import annotations + +import sys +import time +from abc import ABC, abstractmethod +from collections.abc import Iterator +from contextlib import contextmanager +from typing import Any, TextIO + +from core.config import GlobalConfig + + +class Telemetry(ABC): + @abstractmethod + def session(self, **attrs: Any): + """Context manager: root span for one CLI invocation.""" + + @abstractmethod + def span(self, name: str, **attrs: Any): + """Context manager: child span.""" + + @abstractmethod + def event(self, name: str, **attrs: Any) -> None: ... + + @abstractmethod + def error(self, exc: BaseException, *, unexpected: bool = False) -> None: ... + + def flush(self) -> None: + return None + + +class NoopTelemetry(Telemetry): + @contextmanager + def session(self, **attrs: Any) -> Iterator[None]: + yield + + @contextmanager + def span(self, name: str, **attrs: Any) -> Iterator[None]: + yield + + def event(self, name: str, **attrs: Any) -> None: + return None + + def error(self, exc: BaseException, *, unexpected: bool = False) -> None: + return None + + +class ConsoleTelemetry(Telemetry): + """--debug: prints spans and events to stderr. Development aid, not an exporter.""" + + def __init__(self, stream: TextIO = sys.stderr) -> None: + self.stream = stream + self._depth = 0 + + def _log(self, line: str) -> None: + self.stream.write(" " * self._depth + f"[telemetry] {line}\n") + self.stream.flush() + + @contextmanager + def session(self, **attrs: Any) -> Iterator[None]: + with self.span("session", **attrs): + yield + + @contextmanager + def span(self, name: str, **attrs: Any) -> Iterator[None]: + self._log(f"▶ {name} {attrs}") + self._depth += 1 + start = time.perf_counter() + try: + yield + finally: + self._depth -= 1 + self._log(f"◀ {name} {time.perf_counter() - start:.3f}s") + + def event(self, name: str, **attrs: Any) -> None: + self._log(f"• {name} {attrs}") + + def error(self, exc: BaseException, *, unexpected: bool = False) -> None: + code = getattr(exc, "code", type(exc).__name__) + self._log(f"✖ error code={code} unexpected={unexpected}") + + +class TelemetryHub(Telemetry): + """Delegates to a swappable implementation. Commands hold the hub; main swaps the inner.""" + + def __init__(self, inner: Telemetry | None = None) -> None: + self.inner: Telemetry = inner or NoopTelemetry() + + def set(self, inner: Telemetry) -> None: + self.inner = inner + + def session(self, **attrs: Any): + return self.inner.session(**attrs) + + def span(self, name: str, **attrs: Any): + return self.inner.span(name, **attrs) + + def event(self, name: str, **attrs: Any) -> None: + self.inner.event(name, **attrs) + + def error(self, exc: BaseException, *, unexpected: bool = False) -> None: + self.inner.error(exc, unexpected=unexpected) + + def flush(self) -> None: + self.inner.flush() + + +class TelemetryFactory: + @staticmethod + def build(config: GlobalConfig, *, debug: bool = False) -> TelemetryHub: + if debug: + return TelemetryHub(ConsoleTelemetry()) + # Opt-in and endpoint present → OTel exporter (future plan). Until then: noop. + return TelemetryHub(NoopTelemetry()) +``` + +- [ ] **Step 2: Vérifier** + +Run: `uv run python -c " +from services.telemetry import * +from core.config import GlobalConfig +hub = TelemetryFactory.build(GlobalConfig(), debug=True) +with hub.session(cli_version='x'): + with hub.span('command.start', command='start'): + hub.event('compose', args='up') + hub.error(RuntimeError('boom'), unexpected=True) +hub.set(NoopTelemetry()) +with hub.span('silent'): pass +print('ok')"` +Expected: lignes `[telemetry]` imbriquées sur stderr pour session/command/event/error, rien pour `silent`, puis `ok`. + +- [ ] **Step 3: Commit** + +```bash +git add services/telemetry.py +git commit -m "feat(services): add Telemetry contract with noop, console and hub implementations" +``` + +--- + +### Task 9 : `commands/base.py` — `Command`, `CommandGroup`, `LegacyCommand` + +**Files:** +- Create: `commands/base.py` + +**Interfaces:** +- Consumes: `UI`, `Telemetry`, `DockerRunner`, erreurs. +- Produces: + - `Command(ui, telemetry)` : attributs de classe `name`, `help`, `panel`, `no_args_is_help=False` ; `register(app)` ; `run(...)` abstraite ; helpers `require_docker(docker)`, `require_project_dir(path) -> Path`. + - `CommandGroup(ui, telemetry)` : `name`, `help`, `commands: list[Command]`, `typer() -> typer.Typer`, `register(app)`. + - `LegacyCommand(ui, telemetry, fn, *, name, help, panel, no_args_is_help=True)`. + +- [ ] **Step 1: Écrire le module** + +```python +"""Command base classes. Typer registers bound `run` methods; dependencies come via constructors.""" + +from __future__ import annotations + +import functools +from abc import ABC, abstractmethod +from collections.abc import Callable +from pathlib import Path + +import typer + +from core.errors import ConfigError, DockerError, UserAbort +from services.docker import DockerRunner +from services.telemetry import Telemetry +from ui import UI + + +class Command(ABC): + name: str + help: str + panel: str = "General" + no_args_is_help: bool = False + + def __init__(self, ui: UI, telemetry: Telemetry) -> None: + self.ui = ui + self.telemetry = telemetry + + # ---- registration ----------------------------------------------------- + + def register(self, app: typer.Typer) -> None: + app.command( + self.name, + help=self.help, + rich_help_panel=self.panel, + no_args_is_help=self.no_args_is_help, + )(self._traced(self.run)) + + def _traced(self, fn: Callable) -> Callable: + @functools.wraps(fn) + def wrapper(*args, **kwargs): + with self.telemetry.span(f"command.{self.name}"): + return fn(*args, **kwargs) + + return wrapper + + @abstractmethod + def run(self, *args, **kwargs) -> None: ... + + # ---- shared helpers --------------------------------------------------- + + def require_docker(self, docker: DockerRunner) -> None: + """Binary present and daemon up, offering to start it when interactive.""" + if not docker.available(): + raise DockerError( + "Docker not found (binary missing).", + hint="Install Docker: https://docs.docker.com/get-docker/", + ) + if docker.daemon_running(): + return + self.ui.warning("Docker is installed but the daemon is not running.") + if self.ui.confirm("Do you want to try starting Docker?", default=False): + with self.ui.status("Waiting for Docker to start..."): + started = docker.start_daemon() + if started: + self.ui.success("Docker started successfully.") + return + raise DockerError("Docker is required to continue.", hint="Start the Docker daemon and retry.") + + @staticmethod + def require_project_dir(path: Path) -> Path: + path = path.resolve() + if not (path / "docker-compose.yml").exists(): + raise ConfigError( + f"No Portabase configuration found in: {path}", + hint="Expected a docker-compose.yml created by 'portabase agent' or 'portabase dashboard'.", + ) + return path + + def confirm_or_abort(self, question: str, *, default: bool = False, value: bool | None = None) -> None: + if not self.ui.confirm(question, default=default, value=value): + raise UserAbort() + + +class CommandGroup: + name: str + help: str + panel: str = "General" + + def __init__(self, ui: UI, telemetry: Telemetry) -> None: + self.ui = ui + self.telemetry = telemetry + + @property + @abstractmethod + def commands(self) -> list[Command]: ... + + def typer(self) -> typer.Typer: + sub = typer.Typer(help=self.help, no_args_is_help=True) + for cmd in self.commands: + cmd.register(sub) + return sub + + def register(self, app: typer.Typer) -> None: + app.add_typer(self.typer(), name=self.name, rich_help_panel=self.panel) + + +class LegacyCommand(Command): + """Adapter for the pre-refactor function-style commands. Removed in plan 4.""" + + def __init__( + self, + ui: UI, + telemetry: Telemetry, + fn: Callable, + *, + name: str, + help: str, + panel: str, + no_args_is_help: bool = True, + ) -> None: + super().__init__(ui, telemetry) + self.name, self.help, self.panel, self.no_args_is_help = name, help, panel, no_args_is_help + self._fn = fn + + def register(self, app: typer.Typer) -> None: + app.command( + self.name, + help=self.help, + rich_help_panel=self.panel, + no_args_is_help=self.no_args_is_help, + )(self._traced(self._fn)) + + def run(self, *args, **kwargs) -> None: + return self._fn(*args, **kwargs) +``` + +Note : `_traced` utilise `functools.wraps`, donc Typer voit la signature de la fonction d'origine (`__wrapped__`) — c'est ce qui permet d'envelopper sans casser l'introspection des paramètres. + +- [ ] **Step 2: Vérifier l'introspection Typer à travers `_traced`** + +Run: `uv run python -c " +from typing import Annotated +import typer +from commands.base import Command +from ui import UI +from services.telemetry import NoopTelemetry +class Hello(Command): + name, help, panel = 'hello', 'Say hello', 'Test' + def run(self, name: Annotated[str, typer.Argument()], loud: Annotated[bool, typer.Option('--loud')] = False): + print('hello', name.upper() if loud else name) +app = typer.Typer(add_completion=False) +@app.callback() +def root(): pass +Hello(UI(), NoopTelemetry()).register(app) +app(['hello', 'bob', '--loud'], standalone_mode=False)"` +Expected: `hello BOB`. + +- [ ] **Step 3: Commit** + +```bash +git add commands/base.py +git commit -m "feat(commands): add Command, CommandGroup and LegacyCommand base classes" +``` + +--- + +### Task 10 : `services/updater.py` + +**Files:** +- Create: `services/updater.py` + +**Interfaces:** +- Consumes: `HttpClient`, `GlobalConfig`, `core.version`. +- Produces: `Release(tag, assets: dict[str, str], prerelease: bool)` ; `UpdateChecker(http, config, current: str)` : `include_prerelease -> bool`, `latest(force=False) -> Release | None` (cache 24 h, `None` si réseau KO), `available() -> str | None` (tag plus récent ou `None`) ; `Updater(http, current: str)` : `asset_name() -> str`, `target_path() -> Path`, `download(release, on_progress) -> Path` (vérifie sha256 via `checksums.txt`), `install(tmp: Path, target: Path) -> None`. + +- [ ] **Step 1: Écrire le module** + +```python +"""Update check (notify only) and manual update with checksum verification.""" + +from __future__ import annotations + +import hashlib +import json +import os +import platform +import shutil +import subprocess +import sys +import tempfile +import time +from collections.abc import Callable +from dataclasses import dataclass +from pathlib import Path + +from core.config import GlobalConfig +from core.errors import NetworkError, UpdateError +from core.version import UNKNOWN, is_prerelease, parse_version +from services.http import HttpClient + +GITHUB_REPO = "Portabase/cli" +RELEASES_URL = f"https://api.github.com/repos/{GITHUB_REPO}/releases" +CACHE_TTL = 24 * 3600 + + +@dataclass(frozen=True) +class Release: + tag: str + assets: dict[str, str] # name -> browser_download_url + prerelease: bool + + @classmethod + def from_api(cls, data: dict) -> Release: + return cls( + tag=str(data.get("tag_name", "")).lstrip("v"), + assets={a["name"]: a["browser_download_url"] for a in data.get("assets", [])}, + prerelease=bool(data.get("prerelease", False)), + ) + + +def platform_asset_name() -> str: + system = platform.system().lower() + system = "macos" if system == "darwin" else system + machine = platform.machine().lower() + arch = "arm64" if machine in ("arm64", "aarch64") else "amd64" + name = f"portabase_{system}_{arch}" + return name + ".exe" if system == "windows" else name + + +def is_frozen() -> bool: + return bool(getattr(sys, "frozen", False)) + + +class UpdateChecker: + def __init__(self, http: HttpClient, config: GlobalConfig, current: str) -> None: + self.http = http + self.config = config + self.current = current + self.cache_file = config.cache_dir / "release.json" + + @property + def include_prerelease(self) -> bool: + channel = self.config.update_channel + if channel: + return channel == "beta" + return is_prerelease(self.current) + + def fetch_latest(self) -> Release | None: + """Network call. Returns None when nothing is published.""" + if self.include_prerelease: + releases = self.http.get_json(RELEASES_URL) + return Release.from_api(releases[0]) if releases else None + return Release.from_api(self.http.get_json(f"{RELEASES_URL}/latest")) + + def latest(self, *, force: bool = False) -> Release | None: + """Cached 24h. Returns None on any network failure — never raises.""" + if not force: + cached = self._read_cache() + if cached is not None: + return cached + try: + release = self.fetch_latest() + except NetworkError: + return None + if release is not None: + self._write_cache(release) + return release + + def available(self, *, force: bool = False) -> str | None: + if self.current == UNKNOWN: + return None + release = self.latest(force=force) + if release is None: + return None + if parse_version(release.tag) > parse_version(self.current): + return release.tag + return None + + def _read_cache(self) -> Release | None: + try: + with open(self.cache_file, encoding="utf-8") as f: + data = json.load(f) + if time.time() - float(data.get("checked_at", 0)) > CACHE_TTL: + return None + if data.get("channel_pre") != self.include_prerelease: + return None + return Release(tag=data["tag"], assets=data.get("assets", {}), prerelease=bool(data.get("prerelease"))) + except (OSError, ValueError, KeyError): + return None + + def _write_cache(self, release: Release) -> None: + try: + self.cache_file.parent.mkdir(parents=True, exist_ok=True) + with open(self.cache_file, "w", encoding="utf-8") as f: + json.dump( + { + "checked_at": time.time(), + "channel_pre": self.include_prerelease, + "tag": release.tag, + "assets": release.assets, + "prerelease": release.prerelease, + }, + f, + ) + except OSError: + pass + + +class Updater: + CHECKSUMS_ASSET = "checksums.txt" + + def __init__(self, http: HttpClient, current: str) -> None: + self.http = http + self.current = current + + def target_path(self) -> Path: + if is_frozen(): + return Path(sys.executable).resolve() + if platform.system().lower() == "windows": + return Path(os.environ.get("APPDATA", "")) / "Portabase" / "portabase.exe" + default = Path("/usr/local/bin/portabase") + return default if default.exists() else Path.home() / ".local" / "bin" / "portabase" + + def download(self, release: Release, on_progress: Callable[[int], None] | None = None) -> Path: + name = platform_asset_name() + url = release.assets.get(name) + if url is None: + raise UpdateError( + f"No binary for this platform ({name}) in release {release.tag}.", + hint="Available: " + ", ".join(sorted(release.assets)) if release.assets else None, + ) + fd, tmp = tempfile.mkstemp(prefix="portabase_update_") + os.close(fd) + tmp_path = Path(tmp) + try: + self.http.download(url, tmp_path, on_progress, timeout=60) + self._verify(release, name, tmp_path) + except Exception: + tmp_path.unlink(missing_ok=True) + raise + return tmp_path + + def expected_size(self, release: Release) -> int | None: + url = release.assets.get(platform_asset_name()) + return self.http.content_length(url) if url else None + + def _verify(self, release: Release, name: str, path: Path) -> None: + url = release.assets.get(self.CHECKSUMS_ASSET) + if url is None: + raise UpdateError(f"Release {release.tag} has no {self.CHECKSUMS_ASSET}; refusing to install.") + expected = None + for line in self.http.get_text(url).splitlines(): + parts = line.split() + if len(parts) == 2 and parts[1].lstrip("*") == name: + expected = parts[0].lower() + if expected is None: + raise UpdateError(f"{name} not listed in {self.CHECKSUMS_ASSET}; refusing to install.") + digest = hashlib.sha256() + with open(path, "rb") as f: + for chunk in iter(lambda: f.read(1 << 20), b""): + digest.update(chunk) + if digest.hexdigest() != expected: + raise UpdateError("Checksum mismatch for downloaded binary; refusing to install.") + + def install(self, tmp: Path, target: Path) -> None: + system = platform.system().lower() + if system != "windows": + tmp.chmod(0o755) + target.parent.mkdir(parents=True, exist_ok=True) + backup = target.with_name(target.name + ".old") + writable = os.access(target.parent, os.W_OK) and (not target.exists() or os.access(target, os.W_OK)) + try: + if writable or system == "windows": + if target.exists(): + backup.unlink(missing_ok=True) + target.rename(backup) + shutil.move(str(tmp), str(target)) + else: + if target.exists(): + subprocess.run(["sudo", "mv", str(target), str(backup)], check=True) + subprocess.run(["sudo", "mv", str(tmp), str(target)], check=True) + subprocess.run(["sudo", "chmod", "+x", str(target)], check=True) + except (OSError, subprocess.CalledProcessError) as e: + raise UpdateError(f"Could not install to {target}: {e}", cause=e) from e +``` + +- [ ] **Step 2: Vérifier le checker (réseau requis)** + +Run: `uv run python -c " +from pathlib import Path; import tempfile +from services.http import HttpClient +from services.updater import UpdateChecker, platform_asset_name +from core.config import GlobalConfig +cfg = GlobalConfig(Path(tempfile.mkdtemp())/'config.json') +c = UpdateChecker(HttpClient(), cfg, '0.0.1') +r = c.latest(force=True); print('latest:', r.tag, 'pre:', r.prerelease, 'assets:', len(r.assets)) +print('cached:', c.latest().tag == r.tag, '| available from 0.0.1:', c.available()) +print('asset for this machine:', platform_asset_name(), platform_asset_name() in r.assets)"` +Expected: tag de la dernière release stable (ex. `26.07.6`), `cached: True`, `available from 0.0.1: `, asset présent `True` sur linux/macos. + +- [ ] **Step 3: Commit** + +```bash +git add services/updater.py +git commit -m "feat(services): add UpdateChecker (notify, cached) and Updater with checksum verification" +``` + +--- + +### Task 11 : `commands/lifecycle.py`, `commands/config.py`, `commands/update.py` + +**Files:** +- Create: `commands/lifecycle.py` +- Modify: `commands/config.py` (réécriture complète) +- Create: `commands/update.py` +- Delete: `commands/common.py` (Task 12, après bascule de `main.py`) + +**Interfaces:** +- Consumes: `Command`, `CommandGroup`, `DockerRunner`, `UpdateChecker`, `Updater`, `GlobalConfig`. +- Produces: classes `StartCommand`, `StopCommand`, `RestartCommand`, `LogsCommand`, `UninstallCommand` (constructeur `(ui, telemetry, docker)`) ; `ConfigCommands(ui, telemetry, config)` groupe `config` avec `show`, `get`, `set`, `channel` ; `UpdateCommand(ui, telemetry, checker, updater)`. + +- [ ] **Step 1: `commands/lifecycle.py`** + +```python +"""start / stop / restart / logs / uninstall. No rendering: work on any folder with a compose file.""" + +from __future__ import annotations + +import shutil +from pathlib import Path +from typing import Annotated + +import typer + +from commands.base import Command +from services.docker import DockerRunner +from services.telemetry import Telemetry +from ui import UI + +PathArg = Annotated[Path, typer.Argument(help="Path to the component folder")] + + +class _ComposeCommand(Command): + panel = "Lifecycle" + no_args_is_help = True + verb: str + compose_args: list[str] + done: str + + def __init__(self, ui: UI, telemetry: Telemetry, docker: DockerRunner) -> None: + super().__init__(ui, telemetry) + self.docker = docker + + def run(self, path: PathArg) -> None: + path = self.require_project_dir(path) + self.require_docker(self.docker) + with self.ui.status(f"{self.verb} {path.name}..."): + self.docker.compose(path, self.compose_args) + self.ui.success(self.done) + + +class StartCommand(_ComposeCommand): + name, help = "start", "Start a Portabase component." + verb, compose_args, done = "Starting", ["up", "-d"], "Started" + + +class StopCommand(_ComposeCommand): + name, help = "stop", "Stop a Portabase component." + verb, compose_args, done = "Stopping", ["stop"], "Stopped" + + +class RestartCommand(_ComposeCommand): + name, help = "restart", "Restart a Portabase component." + verb, compose_args, done = "Restarting", ["restart"], "Restarted" + + +class LogsCommand(Command): + name, help, panel = "logs", "View logs of a Portabase component.", "Lifecycle" + no_args_is_help = True + + def __init__(self, ui: UI, telemetry: Telemetry, docker: DockerRunner) -> None: + super().__init__(ui, telemetry) + self.docker = docker + + def run( + self, + path: PathArg, + follow: Annotated[bool, typer.Option("--follow/--no-follow", "-f", help="Follow log output")] = True, + ) -> None: + path = self.require_project_dir(path) + self.require_docker(self.docker) + args = ["logs", "-f"] if follow else ["logs"] + try: + self.docker.compose(path, args, check=False) + except KeyboardInterrupt: + pass + + +class UninstallCommand(Command): + name, help, panel = "uninstall", "Uninstall and delete a Portabase component.", "Lifecycle" + no_args_is_help = True + + def __init__(self, ui: UI, telemetry: Telemetry, docker: DockerRunner) -> None: + super().__init__(ui, telemetry) + self.docker = docker + + def run( + self, + path: PathArg, + force: Annotated[bool, typer.Option("--force", "-f", help="Skip confirmation")] = False, + ) -> None: + path = self.require_project_dir(path) + self.require_docker(self.docker) + if not force: + self.ui.warning(f"This will delete containers, volumes and all data in {path}.") + self.confirm_or_abort("Are you sure?", default=False) + with self.ui.status("Uninstalling..."): + self.docker.compose(path, ["down", "-v"]) + try: + shutil.rmtree(path) + except OSError as e: + self.ui.warning(f"Could not remove directory: {e}") + self.ui.success("Uninstalled") +``` + +- [ ] **Step 2: `commands/config.py` (réécriture)** + +```python +"""Global configuration (~/.portabase/config.json).""" + +from __future__ import annotations + +from typing import Annotated + +import typer + +from commands.base import Command, CommandGroup +from core.config import GlobalConfig +from core.errors import ValidationError +from services.telemetry import Telemetry +from ui import UI + +CHANNELS = ("stable", "beta") +BOOL_KEYS = ("telemetry",) + + +class _ConfigCommand(Command): + panel = "Configuration" + + def __init__(self, ui: UI, telemetry: Telemetry, config: GlobalConfig) -> None: + super().__init__(ui, telemetry) + self.config = config + + +class ConfigShow(_ConfigCommand): + name, help = "show", "Show the current configuration." + + def run(self) -> None: + data = self.config.all() + self.ui.info(f"Configuration file: {self.config.path}") + for key in GlobalConfig.KNOWN_KEYS: + value = data.get(key, "[hint]unset[/hint]") + self.ui.print(f" [key]{key}[/key]: {value}") + for key in sorted(set(data) - set(GlobalConfig.KNOWN_KEYS)): + self.ui.print(f" [key]{key}[/key]: {data[key]} [hint](unknown key)[/hint]") + + +class ConfigGet(_ConfigCommand): + name, help = "get", "Print one configuration value." + no_args_is_help = True + + def run(self, key: Annotated[str, typer.Argument(help="Configuration key")]) -> None: + value = self.config.get(key) + if value is None: + raise ValidationError(f"'{key}' is not set.", hint="Known keys: " + ", ".join(GlobalConfig.KNOWN_KEYS)) + self.ui.print(str(value)) + + +class ConfigSet(_ConfigCommand): + name, help = "set", "Set a configuration value." + no_args_is_help = True + + def run( + self, + key: Annotated[str, typer.Argument(help="Configuration key")], + value: Annotated[str, typer.Argument(help="Value")], + ) -> None: + if key == "update_channel" and value not in CHANNELS: + raise ValidationError(f"Invalid channel '{value}'.", hint="Choose 'stable' or 'beta'.") + stored: object = value + if key in BOOL_KEYS: + lowered = value.lower() + if lowered not in ("true", "false", "1", "0", "yes", "no"): + raise ValidationError(f"'{key}' expects true or false.") + stored = lowered in ("true", "1", "yes") + self.config.set(key, stored) + self.ui.success(f"{key} = {stored}") + + +class ConfigChannel(_ConfigCommand): + """Kept for compatibility with the previous `config channel ` command.""" + + name, help = "channel", "Set the update channel (stable or beta)." + no_args_is_help = True + + def run(self, name: Annotated[str, typer.Argument(help="stable or beta")]) -> None: + ConfigSet(self.ui, self.telemetry, self.config).run("update_channel", name.lower()) + + +class ConfigCommands(CommandGroup): + name, help, panel = "config", "Manage global CLI configuration.", "Configuration" + + def __init__(self, ui: UI, telemetry: Telemetry, config: GlobalConfig) -> None: + super().__init__(ui, telemetry) + self.config = config + + @property + def commands(self) -> list[Command]: + deps = (self.ui, self.telemetry, self.config) + return [ConfigShow(*deps), ConfigGet(*deps), ConfigSet(*deps), ConfigChannel(*deps)] +``` + +- [ ] **Step 3: `commands/update.py`** + +```python +"""Manual update. Auto-update is gone; main.py only prints a notice after commands.""" + +from __future__ import annotations + +from commands.base import Command +from core.errors import UpdateError +from core.version import UNKNOWN, parse_version +from services.telemetry import Telemetry +from services.updater import Release, UpdateChecker, Updater, is_frozen +from ui import UI + + +class UpdateCommand(Command): + name, help, panel = "update", "Update the CLI to the latest version.", "System" + + def __init__(self, ui: UI, telemetry: Telemetry, checker: UpdateChecker, updater: Updater) -> None: + super().__init__(ui, telemetry) + self.checker = checker + self.updater = updater + + def run(self) -> None: + if not is_frozen(): + self.ui.warning("The update command is only available for the binary version of Portabase CLI.") + self.ui.info("If you installed from source, use [bold]git pull[/bold] to update.") + return + + current = self.checker.current + release = self._latest() + if release.tag == current: + self.ui.success(f"Portabase CLI is already up to date ({current}).") + return + if current != UNKNOWN and parse_version(release.tag) < parse_version(current): + self.ui.warning(f"Current version ({current}) is newer than the latest remote version ({release.tag}).") + self.confirm_or_abort("Continue with the downgrade?", default=False) + + target = self.updater.target_path() + self.ui.info(f"Updating Portabase CLI from {current} to {release.tag}") + self.ui.info(f"Target installation path: {target}") + + total = self.updater.expected_size(release) or 0 + with self.ui.progress().download(f"Downloading {release.tag}...", total) as advance: + tmp = self.updater.download(release, advance) + self.updater.install(tmp, target) + self.ui.success(f"Successfully updated to {release.tag}!") + + def _latest(self) -> Release: + try: + release = self.checker.fetch_latest() + except Exception as e: # NetworkError + raise UpdateError("Could not fetch latest release data from GitHub.", cause=e) from e + if release is None: + raise UpdateError("No release found for this channel.") + return release +``` + +- [ ] **Step 4: Vérifier le lint des nouveaux fichiers** + +Run: `uv run ruff check commands/lifecycle.py commands/config.py commands/update.py commands/base.py services ui core` +Expected: `All checks passed!`. (`except Exception` dans `_latest` : remplacer par `except NetworkError` en important `NetworkError` depuis `core.errors` si ruff `BLE001` se plaint — c'est de toute façon plus précis.) + +- [ ] **Step 5: Commit** + +```bash +git add commands/lifecycle.py commands/config.py commands/update.py +git commit -m "feat(commands): rewrite lifecycle, config and update commands as classes" +``` + +--- + +### Task 12 : `main.py` — câblage, catcher, bascule + +**Files:** +- Modify: `main.py` (réécriture complète) +- Delete: `commands/common.py`, `core/updater.py` +- Modify: `pyproject.toml` (`TID251`, per-file-ignores) + +**Interfaces:** +- Consumes: tout ce qui précède + fonctions legacy `commands.agent.agent`, `commands.dashboard.dashboard`, `commands.db.app`. +- Produces: `Settings`, `build_app(ui, telemetry, config, settings) -> tuple[typer.Typer, UpdateChecker]`, `main() -> None`. + +- [ ] **Step 1: Réécrire `main.py`** + +```python +"""Entry point. Builds dependencies, registers commands, owns the single error boundary.""" + +from __future__ import annotations + +import os +import platform +import sys +from dataclasses import dataclass +from typing import Annotated + +import click +import typer + +from commands import agent as legacy_agent +from commands import dashboard as legacy_dashboard +from commands import db as legacy_db +from commands.base import LegacyCommand +from commands.config import ConfigCommands +from commands.lifecycle import LogsCommand, RestartCommand, StartCommand, StopCommand, UninstallCommand +from commands.update import UpdateCommand +from core.config import GlobalConfig +from core.errors import PortabaseError, UserAbort, ValidationError +from core.version import current_version +from services.docker import DockerRunner +from services.http import HttpClient +from services.telemetry import ConsoleTelemetry, TelemetryFactory, TelemetryHub +from services.updater import UpdateChecker, Updater, is_frozen +from ui import UI + + +@dataclass +class Settings: + non_interactive: bool = False + verbose: bool = False + debug: bool = False + no_color: bool = False + + @classmethod + def from_env(cls) -> Settings: + return cls( + non_interactive=os.environ.get("PORTABASE_NON_INTERACTIVE", "").lower() in ("1", "true", "yes") + or not sys.stdin.isatty(), + no_color=bool(os.environ.get("NO_COLOR")), + ) + + +def build_app( + ui: UI, telemetry: TelemetryHub, config: GlobalConfig, settings: Settings +) -> tuple[typer.Typer, UpdateChecker]: + app = typer.Typer(no_args_is_help=True, add_completion=False, rich_markup_mode="rich") + http = HttpClient() + docker = DockerRunner() + version = current_version() + checker = UpdateChecker(http, config, version) + updater = Updater(http, version) + + def version_callback(value: bool) -> None: + if value: + ui.print(f"Portabase CLI version: {version}") + latest = checker.available(force=True) + if latest: + ui.warning(f"A new version is available: [bold]{latest}[/bold]") + raise typer.Exit() + + @app.callback() + def root( + _version: Annotated[ + bool | None, + typer.Option("--version", help="Show the version and exit.", callback=version_callback, is_eager=True), + ] = None, + verbose: Annotated[bool, typer.Option("--verbose", help="Show error causes and tracebacks.")] = False, + debug: Annotated[bool, typer.Option("--debug", help="Verbose plus telemetry trace on stderr.")] = False, + no_color: Annotated[bool, typer.Option("--no-color", help="Disable colours.")] = False, + non_interactive: Annotated[ + bool, + typer.Option("--non-interactive", envvar="PORTABASE_NON_INTERACTIVE", help="Never prompt; fail on missing input."), + ] = False, + ) -> None: + """Portabase CLI to manage agents, dashboards and databases.""" + settings.verbose = verbose or debug + settings.debug = debug + settings.no_color = settings.no_color or no_color + settings.non_interactive = settings.non_interactive or non_interactive + ui.configure(verbose=settings.verbose, no_color=settings.no_color, non_interactive=settings.non_interactive) + if debug: + telemetry.set(ConsoleTelemetry()) + + commands = [ + LegacyCommand(ui, telemetry, legacy_agent.agent, name="agent", help="Create a new Portabase Agent instance.", panel="Creation"), + LegacyCommand(ui, telemetry, legacy_dashboard.dashboard, name="dashboard", help="Create a new Portabase Dashboard instance.", panel="Creation"), + StartCommand(ui, telemetry, docker), + StopCommand(ui, telemetry, docker), + RestartCommand(ui, telemetry, docker), + LogsCommand(ui, telemetry, docker), + UninstallCommand(ui, telemetry, docker), + UpdateCommand(ui, telemetry, checker, updater), + ] + for cmd in commands: + cmd.register(app) + + app.add_typer(legacy_db.app, name="db", rich_help_panel="Configuration") # legacy, replaced in plan 4 + ConfigCommands(ui, telemetry, config).register(app) + + return app, checker + + +def _notify_update(ui: UI, checker: UpdateChecker, settings: Settings, invoked: str | None) -> None: + if not is_frozen() or settings.non_interactive or invoked in ("update", None): + return + latest = checker.available() + if latest: + ui.print("") + ui.warning(f"A new version of Portabase CLI is available: [bold]{latest}[/bold] (current: {checker.current})") + ui.info("Run [bold]portabase update[/bold] to update.") + + +def main() -> None: + settings = Settings.from_env() + config = GlobalConfig() + ui = UI(non_interactive=settings.non_interactive, no_color=settings.no_color) + telemetry = TelemetryFactory.build(config, debug=False) + app, checker = build_app(ui, telemetry, config, settings) + invoked = next((a for a in sys.argv[1:] if not a.startswith("-")), None) + exit_code = 0 + + try: + with telemetry.session(cli_version=current_version(), os=platform.system()): + app(standalone_mode=False) + except UserAbort as e: + ui.warning(e.message) + telemetry.event("abort") + exit_code = e.exit_code + except PortabaseError as e: + ui.error(e) + telemetry.error(e) + exit_code = e.exit_code + except click.exceptions.NoArgsIsHelpError: + exit_code = 0 # help already printed by Typer + except click.exceptions.Exit as e: # typer.Exit from legacy code or --help + exit_code = e.exit_code + except click.exceptions.Abort: # typer.Abort from legacy code + ui.warning("Cancelled.") + exit_code = 130 + except click.UsageError as e: + err = ValidationError(e.format_message(), hint="Run 'portabase --help' for usage.") + ui.error(err) + telemetry.error(err) + exit_code = err.exit_code + except KeyboardInterrupt: + ui.console.print("") + ui.warning("Cancelled.") + exit_code = 130 + except Exception as e: # noqa: BLE001 — last resort: a bug, not an expected error + wrapped = PortabaseError("Unexpected error: " + str(e), cause=e) + ui.error(wrapped, unexpected=True) + telemetry.error(e, unexpected=True) + exit_code = 1 + finally: + telemetry.flush() + + if exit_code == 0: + _notify_update(ui, checker, settings, invoked) + raise SystemExit(exit_code) + + +if __name__ == "__main__": + main() +``` + +- [ ] **Step 2: Supprimer les modules remplacés** + +Run: `git rm commands/common.py core/updater.py` + +Puis vérifier qu'aucun import ne subsiste : +Run: `grep -rn "commands.common\|core.updater\|check_for_updates\|update_cli" --include=*.py . | grep -v ".venv"` +Expected: aucune sortie. + +- [ ] **Step 3: Mettre à jour `pyproject.toml`** + +Remplacer le bloc `[tool.ruff.lint.per-file-ignores]` par : + +```toml +# Code legacy supprimé au plan 4. Ne pas étendre cette liste. +[tool.ruff.lint.per-file-ignores] +"commands/agent.py" = ["BLE001", "E722", "S110", "SIM102", "TID251"] +"commands/db.py" = ["BLE001", "E722", "S110", "TID251"] +"commands/dashboard.py" = ["BLE001", "TID251"] +"core/config.py" = ["BLE001", "E722", "S110"] +"core/utils.py" = ["BLE001", "E722", "S110", "PLR1730", "TID251"] +"core/network.py" = ["BLE001", "TID251"] +"main.py" = ["TID251"] + +[tool.ruff.lint.flake8-tidy-imports.banned-api] +"rich.prompt".msg = "Use ui.form() / ui.confirm() instead." +"rich.console".msg = "Only ui/ may build a Console. Use the UI facade." +"typer.prompt".msg = "Use ui.form() instead." +"typer.confirm".msg = "Use ui.confirm() instead." +``` + +Et ajouter `"TID251"` dans `select` s'il n'y est pas déjà (il y est depuis Plan 1). `main.py` importe `click` et `typer.Exit`, pas de prompt : `TID251` sur `main.py` est là uniquement pour `ui.console.print` ? Non — `rich.console` n'y est pas importé. Retirer `"main.py" = ["TID251"]` si `ruff check` passe sans. + +Ajouter `known-first-party = ["commands", "core", "services", "ui", "templates"]` dans `[tool.ruff.lint.isort]`. + +- [ ] **Step 4: Lint complet** + +Run: `uv run ruff check . && uv run ruff format --check .` +Expected: passe. Sinon `uv run ruff format .` puis corriger les erreurs signalées **dans les nouveaux fichiers uniquement**. + +- [ ] **Step 5: Vérifier l'aide et les erreurs de saisie** + +Run: `uv run python main.py; echo "exit=$?"` +Expected: aide affichée, `exit=0`. + +Run: `uv run python main.py --help | head -30` +Expected: panneaux `Creation` (agent, dashboard), `Lifecycle` (start, stop, restart, logs, uninstall), `Configuration` (db, config), `System` (update) ; options `--version`, `--verbose`, `--debug`, `--no-color`, `--non-interactive`. + +Run: `uv run python main.py start; echo "exit=$?"` +Expected: aide de `start` (no_args_is_help), `exit=0`. + +Run: `uv run python main.py bogus; echo "exit=$?"` +Expected: `✖ Error: No such command 'bogus'.` + hint, `exit=2`. + +Run: `uv run python main.py start /tmp/does-not-exist; echo "exit=$?"` +Expected: `✖ Error: No Portabase configuration found in: /tmp/does-not-exist` + hint, `exit=3`. + +Run: `uv run python main.py --verbose start /tmp/does-not-exist 2>&1 | grep -c "code: E_CONFIG"` +Expected: `1`. + +- [ ] **Step 6: Vérifier config** + +Run: `uv run python main.py config show && uv run python main.py config set update_channel beta && uv run python main.py config get update_channel && uv run python main.py config channel stable && uv run python main.py config set update_channel nope; echo "exit=$?"` +Expected: affichage, `✔ update_channel = beta`, `beta`, `✔ update_channel = stable`, puis `✖ Error: Invalid channel 'nope'.` `exit=2`. + +- [ ] **Step 7: Vérifier update et --version (non-frozen)** + +Run: `uv run python main.py update; echo "exit=$?"; uv run python main.py --version; echo "exit=$?"` +Expected: avertissement "only available for the binary version", `exit=0` ; version puis éventuellement "A new version is available", `exit=0`. + +- [ ] **Step 8: Vérifier le mode non-interactif et Ctrl-C** + +Run: `uv run python main.py --non-interactive uninstall /tmp/does-not-exist; echo "exit=$?"` +Expected: `E_CONFIG`, `exit=3` (l'erreur dossier précède la confirmation). + +Créer un faux projet : `mkdir -p /tmp/pb-fake && touch /tmp/pb-fake/docker-compose.yml`. +Run: `uv run python main.py --non-interactive uninstall /tmp/pb-fake; echo "exit=$?"` +Expected (Docker présent) : confirm par défaut `False` → `⚠ Cancelled.` `exit=130`, dossier intact. (Docker absent : `E_DOCKER`, `exit=4`.) + +Run: `uv run python main.py uninstall /tmp/pb-fake` puis Ctrl-C au prompt. +Expected: `⚠ Cancelled.`, `exit=130`, pas de traceback. + +- [ ] **Step 9: Vérifier les commandes legacy à travers le catcher** + +Run: `uv run python main.py agent; echo "exit=$?"` puis `uv run python main.py db list /tmp/does-not-exist; echo "exit=$?"` +Expected: aide de `agent` `exit=0` ; message legacy `No Portabase configuration found` (ancien style) et `exit=1` (via `typer.Exit(1)` → `click.exceptions.Exit`). + +- [ ] **Step 10: Vérifier le lifecycle réel (si Docker disponible)** + +```bash +cd /tmp && rm -rf pb-smoke && uv --directory /home/soluce/Documents/PROJETS/Portabase/cli run python /home/soluce/Documents/PROJETS/Portabase/cli/main.py dashboard pb-smoke --port 8899 +``` +Répondre `internal` au choix DB, `N` à "Start dashboard now?". Puis : + +```bash +M=/home/soluce/Documents/PROJETS/Portabase/cli/main.py +uv --directory /home/soluce/Documents/PROJETS/Portabase/cli run python $M start /tmp/pb-smoke +uv --directory /home/soluce/Documents/PROJETS/Portabase/cli run python $M logs /tmp/pb-smoke --no-follow | tail -3 +uv --directory /home/soluce/Documents/PROJETS/Portabase/cli run python $M restart /tmp/pb-smoke +uv --directory /home/soluce/Documents/PROJETS/Portabase/cli run python $M stop /tmp/pb-smoke +uv --directory /home/soluce/Documents/PROJETS/Portabase/cli run python $M uninstall /tmp/pb-smoke --force +ls /tmp/pb-smoke 2>&1 +``` +Expected: `✔ Started`, quelques lignes de logs, `✔ Restarted`, `✔ Stopped`, `✔ Uninstalled`, `No such file or directory`. + +- [ ] **Step 11: Commit** + +```bash +git add main.py pyproject.toml +git commit -m "refactor: wire commands through DI container and single error boundary + +Lifecycle, config and update run on the new Command classes; agent, +dashboard and db stay on legacy code behind LegacyCommand until plan 4. +Auto-update is replaced by a post-command notice." +``` + +--- + +### Task 13 : Build smoke, PR + +**Files:** aucun nouveau. + +- [ ] **Step 1: Binaire local** + +Run: `rm -rf build dist *.spec && uv run pyinstaller --onefile --name portabase_smoke --paths=. --collect-all rich --collect-all requests --collect-data certifi --add-data "pyproject.toml:." main.py && ./dist/portabase_smoke --version && ./dist/portabase_smoke config show && ./dist/portabase_smoke start /tmp/nope; echo "exit=$?"; rm -rf build dist *.spec` +Expected: version, config, `E_CONFIG` `exit=3`. Aucun `ModuleNotFoundError` (questionary, services, ui embarqués via `--paths=.`). + +- [ ] **Step 2: PR** + +```bash +git checkout -b refactor/foundations +git push -u origin refactor/foundations +``` +Ouvrir la PR « refactor: foundations (errors, ui, services, Command) + lifecycle rewrite ». Checks Plan 1 verts attendus. + +- [ ] **Step 3: Release candidate (optionnel mais recommandé)** + +Après merge : Actions → Bump version → `26.08.0rc1`, channel `rc`. Installer le binaire rc sur une machine avec une install existante et dérouler `start/stop/logs/restart` + `--version` (la notification de mise à jour après commande s'affiche seulement en binaire). + +--- + +## Self-review + +**Spec coverage :** +- §3 structure : `core/errors`, `core/version`, `core/config` (GlobalConfig), `ui/*`, `services/{http,docker,telemetry,updater}`, `commands/{base,lifecycle,config,update}`, `main.py` ✔. `services/{envfile,ports,templates,renderer,project,compose_facts}`, `engines/`, `commands/{agent,dashboard,build,db,flows}` → Plans 3–4. `core/fields.py` : déviation documentée. +- §4.1 `Command`, `register`, `_traced`, injection ✔ (T9). `Annotated` ✔. +- §7 ui : tokens ✔, composants Banner/Message/Section/Status/Hint/Prompt ✔ + Progress (appelant : update). `Summary`, `DataTable`, `Diff` → Plan 4 (appelants). `Form` ✔ avec flag→prompt→défaut→erreur, `UserAbort` sur `None` ✔. `NO_COLOR` ✔. Pas de prompt sous status : respecté dans lifecycle (confirm avant status). +- §8.1 hiérarchie et codes ✔. §8.2 catcher, `standalone_mode=False`, mapping click ✔ (T12). §8.3 télémétrie contrat + noop + console + hub ✔ ; opt-in config lu par `TelemetryFactory` (endpoint ignoré tant qu'aucun exporter — documenté). §8.4 updater : notif après commande, cache 24 h, silencieux offline, checksum ✔. +- §10 B+C : shippable, legacy via `LegacyCommand` ✔. + +**Placeholders :** aucun. + +**Cohérence des types :** `UI.confirm(question, *, default, value)` utilisé par `Command.confirm_or_abort` et `require_docker` ✔ ; `Telemetry.span` context manager utilisé par `_traced` ✔ ; `UpdateChecker.available(force=)` utilisé par `version_callback` et `_notify_update` ✔ ; `Updater.expected_size/download/install` utilisés par `UpdateCommand` ✔ ; `HttpClient.content_length` utilisé par `Updater.expected_size` ✔ (`head_content_length` cité dans l'interface T7 = `content_length` ; nom retenu : `content_length`). + +**Écarts connus :** +- `UpdateCommand._latest` : utiliser `except NetworkError` (T11 step 4). From a7cf34ba1fc81f3b66e42b451a185b4162f30c36 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Th=C3=A9o=20LAGACHE?= Date: Fri, 11 Sep 2026 10:27:54 +0200 Subject: [PATCH 05/24] docs: add plan 3 (Jinja2 templates, engine registry, template repository) Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_017ACLkgaiP48pBMyi7kZZ4a --- .../2026-09-11-plan-3-templates-engines.md | 1944 +++++++++++++++++ 1 file changed, 1944 insertions(+) create mode 100644 docs/superpowers/plans/2026-09-11-plan-3-templates-engines.md diff --git a/docs/superpowers/plans/2026-09-11-plan-3-templates-engines.md b/docs/superpowers/plans/2026-09-11-plan-3-templates-engines.md new file mode 100644 index 0000000..3b5be5d --- /dev/null +++ b/docs/superpowers/plans/2026-09-11-plan-3-templates-engines.md @@ -0,0 +1,1944 @@ +# Plan 3 — Templates Jinja2 et moteurs DB (chantier D) + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Introduire les templates Jinja2 versionnés (source `templates/` à la racine, manifest, cache, `TemplateRepository`), le registre de moteurs DB en classes, et les garde-fous CI (`render-check`, `engines-check`, manifest à l'upload, hotfix) — sans encore brancher le rendu sur les commandes (Plan 4). Le CLI reste fonctionnel : les commandes legacy continuent de lire `agent.yml` / `dashboard.yml` (conservés dans `templates/` jusqu'au Plan 4). + +**Architecture:** `TemplateRepository` résout une version → dossier local (`./templates` en dev, cache `~/.portabase/cache/templates//` en binaire), vérifie un `manifest.json` (sha256) et expose des `jinja2.Template`. Chaque `DbEngine` déclare ses champs, génère un `DatabaseSpec`, produit ses variables `.env`, son contexte de template et sa projection `databases.json`. `render_check.py` rend chaque template avec des fixtures et valide le YAML puis `docker compose config`. + +**Tech Stack:** Jinja2 3.1, PyYAML, Python 3.12, GitHub Actions, s3cmd, jq. + +**Spec:** `docs/superpowers/specs/2026-09-11-cli-refactor-design.md` — sections 5.4, 5.6, 6, 6.1, 9.1 (`render-check`, `engines-check`), 9.3 (hotfix, manifest), 10 (D). + +## Global Constraints + +- Prérequis : Plans 1 et 2 exécutés. +- Règle de dépendance : `engines → core` uniquement. `services → engines, core`. Vérifié par revue ; ruff ne le détecte pas. +- Déviations spec assumées : + - `DatabaseSpec` vit dans `core/specs.py` (produit par `engines`, consommé par `services`), pas dans `services/project.py`. + - Pas de `mysql.yml.j2` : le moteur `mysql` utilise `engines/mariadb.yml.j2`, comme le code legacy (image `mariadb:latest`). Changer d'image casserait les volumes des installs existantes. +- Les fichiers legacy `agent.yml` et `dashboard.yml` sont déplacés tels quels dans `templates/` et restent uploadés (le code legacy les fetch sous `/`). Supprimés au Plan 4. +- Conventions de nommage legacy conservées à l'identique (service `db-pg-`, `db-mongo-auth-`, db `pg_`, user `admin`, firebird user `alice` / `mirror.fdb`, mssql `sa` / `master` / name `MSSQL`, redis/valkey `database: "0"`), pour que les nouvelles installs ressemblent aux anciennes. +- `generate_password` retire `$` et `` ` `` des symboles (Task 1). +- Pas de tests unitaires. `render_check.py` est la vérification exécutable de ce plan et devient un job CI. +- Aucune commande utilisateur ne change dans ce plan. + +--- + +## File Structure + +| Fichier | Action | Responsabilité | +|---|---|---| +| `core/utils.py` | modifier | `generate_password` sans `$`/`` ` `` | +| `core/specs.py` | créer | `DatabaseSpec` | +| `services/ports.py` | créer | `PortAllocator` | +| `engines/__init__.py` | créer | `registry` | +| `engines/base.py` | créer | `DbEngine` | +| `engines/registry.py` | créer | `EngineRegistry` | +| `engines/sql.py` | créer | `StandardSqlEngine`, `PostgresEngine`, `PostgresClusterEngine`, `MySqlEngine`, `MariaDbEngine`, `MssqlEngine`, `FirebirdEngine` | +| `engines/redis.py` | créer | `RedisEngine` | +| `engines/valkey.py` | créer | `ValkeyEngine` | +| `engines/mongo.py` | créer | `MongoEngine` | +| `engines/sqlite.py` | créer | `SqliteEngine` | +| `engines/docker_volume.py` | créer | `DockerVolumeEngine` | +| `templates/agent.yml.j2`, `dashboard.yml.j2`, `engines/*.yml.j2` | créer | templates Jinja2 | +| `templates/engines.map.json` | créer | clé moteur → template | +| `templates/agent.yml`, `dashboard.yml` | déplacer depuis `.github/assets/templates/` | legacy | +| `services/templates.py` | créer | `Manifest`, `TemplateRepository` | +| `scripts/render_check.py` | créer | validation des templates | +| `.github/workflows/ci.yml` | modifier | jobs `render-check`, `engines-check` | +| `.github/workflows/templates-upload.yml` | modifier | source `templates/`, manifest | +| `.github/workflows/templates-hotfix.yml` | créer | re-upload d'une version | +| `pyproject.toml` | modifier | `jinja2` | +| `.gitleaks.toml` | modifier | chemin `templates/` déjà allowlisté ; retirer `.github/assets/templates` | + +--- + +### Task 1 : `core/specs.py`, `services/ports.py`, mot de passe + +**Files:** +- Create: `core/specs.py` +- Create: `services/ports.py` +- Modify: `core/utils.py:70-92` (`generate_password`) + +**Interfaces:** +- Produces: `DatabaseSpec` (frozen dataclass) avec `env_prefix`, `is_service`, `with_options()` ; `PortAllocator().free() -> int` ; `generate_password(length=16)` sans `$` ni `` ` ``. + +- [ ] **Step 1: `core/specs.py`** + +```python +"""Typed view of one databases.json entry plus what the CLI needs to render it.""" + +from __future__ import annotations + +from dataclasses import dataclass, field, replace +from typing import Any + + +@dataclass(frozen=True) +class DatabaseSpec: + id: str + engine: str + name: str + managed: bool = False # True: a Compose service is rendered for it + host: str | None = None # service name when managed, remote host otherwise + port: int | None = None # container/remote port (what the agent connects to) + host_port: int | None = None # published port on the Docker host (managed only) + database: str | None = None + username: str | None = None + password: str | None = None + root_password: str | None = None # firebird + path: str | None = None # sqlite + volume: str | None = None # docker-volume + container: str | None = None # docker-volume + options: dict[str, Any] = field(default_factory=dict) + + @property + def env_prefix(self) -> str: + if not self.host: + raise ValueError("env_prefix requires a host/service name") + return self.host.upper().replace("-", "_") + + @property + def auth(self) -> bool: + return bool(self.password) + + def with_options(self, options: dict[str, Any]) -> DatabaseSpec: + return replace(self, options=dict(options)) +``` + +- [ ] **Step 2: `services/ports.py`** + +```python +"""Free TCP port allocation. Remembers ports handed out during the process to avoid duplicates.""" + +from __future__ import annotations + +import socket + + +class PortAllocator: + def __init__(self) -> None: + self._given: set[int] = set() + + def free(self) -> int: + for _ in range(50): + with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s: + s.bind(("", 0)) + port = s.getsockname()[1] + if port not in self._given: + self._given.add(port) + return port + raise RuntimeError("Could not allocate a free port") + + +class FixedPortAllocator(PortAllocator): + """Deterministic ports for render checks and fixtures.""" + + def __init__(self, start: int = 40000) -> None: + super().__init__() + self._next = start + + def free(self) -> int: + port = self._next + self._next += 1 + return port +``` + +- [ ] **Step 3: Corriger `generate_password` dans `core/utils.py`** + +Remplacer la ligne `symbols = "!@#$%^&*()-_=+[]{}|;:,.<>?"` par : + +```python + # No '$' (Compose interpolation / shell), no '`' or quotes (shell command args in templates). + symbols = "!@#%^&*()-_=+[]{}|;:,.<>?" +``` + +- [ ] **Step 4: Vérifier** + +Run: `uv run python -c " +from core.specs import DatabaseSpec +from services.ports import PortAllocator, FixedPortAllocator +from core.utils import generate_password +s = DatabaseSpec(id='1', engine='postgresql', name='x', managed=True, host='db-pg-a1f2', password='p') +print(s.env_prefix, s.auth, s.with_options({'a':1}).options) +p = PortAllocator(); a, b = p.free(), p.free(); print(a != b, FixedPortAllocator().free()) +pw = generate_password(); print(len(pw), '\$' not in pw and '\`' not in pw)"` +Expected: `DB_PG_A1F2 True {'a': 1}`, `True 40000`, `16 True`. + +- [ ] **Step 5: Commit** + +```bash +git add core/specs.py services/ports.py core/utils.py +git commit -m "feat: add DatabaseSpec, PortAllocator; drop shell-unsafe symbols from generated passwords" +``` + +--- + +### Task 2 : `engines/base.py` et `engines/registry.py` + +**Files:** +- Create: `engines/__init__.py` (rempli Task 4) +- Create: `engines/base.py` +- Create: `engines/registry.py` + +**Interfaces:** +- Produces: `DbEngine` ABC : + - classe-attributs `key`, `display`, `default_port: int | None`, `template: str | None`, `auth_variants=False`, `warning=None`, `has_modes=True` + - `fields_existing() -> list[Field]`, `fields_new() -> list[Field]`, `option_fields() -> list[Field]` + - `generate(*, auth: bool, ports: PortAllocator, answers: dict) -> DatabaseSpec` + - `from_existing(answers: dict) -> DatabaseSpec` + - `env_vars(spec) -> dict[str, str]` + - `template_ctx(spec, *, inline: bool = False) -> dict` + - `agent_entry(spec) -> dict` + - `describe(spec) -> str` (pour `db list` : « host:port », « Local File », « volume: x ») + - helpers `new_id()`, `service_name(slug, auth)`, `var(spec, suffix, value, inline)` +- `EngineRegistry(engines)` : `get(key)`, `keys()`, `choices()`, `__iter__`. + +- [ ] **Step 1: `engines/base.py`** + +```python +"""DbEngine: everything the CLI needs to know about one database engine.""" + +from __future__ import annotations + +import secrets +import uuid +from abc import ABC, abstractmethod +from typing import Any + +from core.fields import Field +from core.specs import DatabaseSpec +from services.ports import PortAllocator + +STANDARD_EXISTING_FIELDS = ( + Field("host", "Host", "text", default="localhost"), + Field("port", "Port", "int"), # default filled per engine + Field("database", "Database Name", "text"), + Field("username", "Username", "text"), + Field("password", "Password", "secret"), +) + + +class DbEngine(ABC): + key: str + display: str + default_port: int | None = None + template: str | None = None # e.g. "engines/postgresql.yml.j2"; None: no Compose service + auth_variants: bool = False # offer with-auth / no-auth when creating a container + warning: str | None = None # shown before collecting answers + has_modes: bool = True # new/existing choice applies + + # ---- declarations ----------------------------------------------------- + + def fields_existing(self) -> list[Field]: + return [ + Field("port", "Port", "int", default=self.default_port) if f.name == "port" else f + for f in STANDARD_EXISTING_FIELDS + ] + + def fields_new(self) -> list[Field]: + return [] + + def option_fields(self) -> list[Field]: + return [] + + # ---- construction ----------------------------------------------------- + + @abstractmethod + def generate(self, *, auth: bool, ports: PortAllocator, answers: dict[str, Any]) -> DatabaseSpec: ... + + def from_existing(self, answers: dict[str, Any]) -> DatabaseSpec: + return DatabaseSpec( + id=self.new_id(), + engine=self.key, + name=answers.get("label") or "External DB", + managed=False, + host=answers["host"], + port=int(answers["port"]), + database=answers["database"], + username=answers["username"], + password=answers["password"], + ) + + # ---- rendering inputs ------------------------------------------------- + + def env_vars(self, spec: DatabaseSpec) -> dict[str, str]: + """Variables written to .env for a managed service. Default: PORT, DB, USER, PASS.""" + p = spec.env_prefix + return { + f"{p}_PORT": str(spec.host_port), + f"{p}_DB": spec.database or "", + f"{p}_USER": spec.username or "", + f"{p}_PASS": spec.password or "", + } + + def template_ctx(self, spec: DatabaseSpec, *, inline: bool = False) -> dict[str, Any]: + return { + "name": spec.host, + "volume": f"{spec.host}-data", + "auth": spec.auth, + "port_var": self.var(spec, "PORT", spec.host_port, inline), + "db_var": self.var(spec, "DB", spec.database, inline), + "user_var": self.var(spec, "USER", spec.username, inline), + "password_var": self.var(spec, "PASS", spec.password, inline), + } + + def agent_entry(self, spec: DatabaseSpec) -> dict[str, Any]: + """Projection to databases.json. Same shape as the legacy CLI.""" + entry: dict[str, Any] = { + "name": spec.name, + "database": self.agent_database(spec), + "type": self.key, + "username": spec.username or "", + "password": spec.password or "", + "port": spec.port, + "host": spec.host, + "generated_id": spec.id, + } + options = self.non_default_options(spec) + if options: + entry["options"] = options + return entry + + def agent_database(self, spec: DatabaseSpec) -> str: + return spec.database or "" + + def describe(self, spec: DatabaseSpec) -> str: + return f"{spec.host}:{spec.port}" + + # ---- helpers ---------------------------------------------------------- + + def non_default_options(self, spec: DatabaseSpec) -> dict[str, Any]: + defaults = {f.name: f.default for f in self.option_fields()} + return {k: v for k, v in spec.options.items() if k in defaults and v != defaults[k]} + + @staticmethod + def new_id() -> str: + return str(uuid.uuid4()) + + @staticmethod + def service_name(slug: str, auth: bool = False) -> str: + suffix = "auth-" if auth else "" + return f"db-{slug}-{suffix}{secrets.token_hex(2)}" + + @staticmethod + def var(spec: DatabaseSpec, suffix: str, value: Any, inline: bool) -> str: + return str(value if value is not None else "") if inline else f"${{{spec.env_prefix}_{suffix}}}" +``` + +- [ ] **Step 2: `engines/registry.py`** + +```python +from __future__ import annotations + +from collections.abc import Iterable, Iterator + +from core.errors import ValidationError +from engines.base import DbEngine + + +class EngineRegistry: + def __init__(self, engines: Iterable[DbEngine]) -> None: + self._by_key: dict[str, DbEngine] = {} + for engine in engines: + if engine.key in self._by_key: + raise ValueError(f"Duplicate engine key: {engine.key}") + self._by_key[engine.key] = engine + + def get(self, key: str) -> DbEngine: + try: + return self._by_key[key] + except KeyError: + raise ValidationError( + f"Unknown engine '{key}'.", + hint="Available: " + ", ".join(self.keys()), + ) from None + + def keys(self) -> list[str]: + return list(self._by_key) + + def choices(self) -> list[str]: + return self.keys() + + def __iter__(self) -> Iterator[DbEngine]: + return iter(self._by_key.values()) + + def __contains__(self, key: str) -> bool: + return key in self._by_key +``` + +- [ ] **Step 3: Vérifier** + +Run: `uv run python -c " +from engines.base import DbEngine +from engines.registry import EngineRegistry +from core.errors import ValidationError +print([f.name for f in DbEngine.fields_existing(type('E',(DbEngine,),{'key':'x','display':'X','default_port':1,'generate':lambda *a,**k: None})())]) +try: EngineRegistry([]).get('nope') +except ValidationError as e: print(e.message, '|', e.hint)"` +Expected: `['host', 'port', 'database', 'username', 'password']` puis `Unknown engine 'nope'. | Available: `. + +- [ ] **Step 4: Commit** + +```bash +git add engines/ +git commit -m "feat(engines): add DbEngine base class and EngineRegistry" +``` + +--- + +### Task 3 : Moteurs SQL (`engines/sql.py`) + +**Files:** +- Create: `engines/sql.py` + +**Interfaces:** +- Produces: `StandardSqlEngine` et sous-classes `PostgresEngine` (`postgresql`), `PostgresClusterEngine` (`postgresql-cluster`), `MySqlEngine` (`mysql`), `MariaDbEngine` (`mariadb`), `MssqlEngine` (`mssql`), `FirebirdEngine` (`firebird`). + +- [ ] **Step 1: Écrire le module** + +```python +"""SQL engines rendered as Compose services. Naming mirrors the legacy CLI.""" + +from __future__ import annotations + +import secrets +from typing import Any + +from core.fields import Field +from core.specs import DatabaseSpec +from core.utils import generate_password +from engines.base import DbEngine +from services.ports import PortAllocator + + +class StandardSqlEngine(DbEngine): + slug: str # service name fragment: db--xxxx + db_prefix: str # generated database name: _xxxxxxxx + default_user = "admin" + + def generate(self, *, auth: bool, ports: PortAllocator, answers: dict[str, Any]) -> DatabaseSpec: + db_name = f"{self.db_prefix}_{secrets.token_hex(4)}" + return DatabaseSpec( + id=self.new_id(), + engine=self.key, + name=db_name, + managed=True, + host=self.service_name(self.slug), + port=self.default_port, + host_port=ports.free(), + database=db_name, + username=self.default_user, + password=generate_password(16), + options=dict(answers.get("options", {})), + ) + + +class PostgresEngine(StandardSqlEngine): + key, display, default_port = "postgresql", "PostgreSQL", 5432 + template, slug, db_prefix = "engines/postgresql.yml.j2", "pg", "pg" + + def option_fields(self) -> list[Field]: + return [ + Field( + "keep_ownership", + "Keep ownership?", + "bool", + default=False, + help=( + "When enabled, omits --no-owner and --no-privileges from the dump. Ownership and role " + "assignments are preserved. By default these flags are applied to keep restores portable " + "across users and environments." + ), + ), + Field( + "clean_mode", + "Clean mode", + "choice", + default="clean", + choices=("clean", "none", "drop_schemas", "drop_database"), + help=( + "How the target database is cleaned before a restore. clean: pg_restore --clean --if-exists. " + "none: no pre-clean. drop_schemas: drop every non-system schema CASCADE (works on managed " + "Postgres). drop_database: DROP DATABASE + CREATE DATABASE — requires CREATEDB or superuser; " + "most managed providers do not allow it." + ), + ), + ] + + +class PostgresClusterEngine(StandardSqlEngine): + key, display, default_port = "postgresql-cluster", "PostgreSQL Cluster", 5432 + template, slug, db_prefix = "engines/postgresql.yml.j2", "pg", "pg" + warning = ( + "Postgres Cluster requires a superuser. Cluster backup/restore uses pg_dumpall, which dumps all " + "databases and global objects (roles, tablespaces). The provided user must be a Postgres superuser." + ) + + +class MariaDbEngine(StandardSqlEngine): + key, display, default_port = "mariadb", "MariaDB", 3306 + template, slug, db_prefix = "engines/mariadb.yml.j2", "mariadb", "mysql" + + +class MySqlEngine(MariaDbEngine): + """Legacy behaviour: a 'mysql' container is a MariaDB image. Kept for volume compatibility.""" + + key, display = "mysql", "MySQL" + + +class MssqlEngine(StandardSqlEngine): + key, display, default_port = "mssql", "Microsoft SQL Server", 1433 + template, slug, db_prefix = "engines/mssql.yml.j2", "mssql", "master" + + def generate(self, *, auth: bool, ports: PortAllocator, answers: dict[str, Any]) -> DatabaseSpec: + return DatabaseSpec( + id=self.new_id(), + engine=self.key, + name="MSSQL", + managed=True, + host=self.service_name(self.slug), + port=self.default_port, + host_port=ports.free(), + database="master", + username="sa", + password=generate_password(16), + ) + + def env_vars(self, spec: DatabaseSpec) -> dict[str, str]: + p = spec.env_prefix + return {f"{p}_PORT": str(spec.host_port), f"{p}_PASS": spec.password or ""} + + +class FirebirdEngine(StandardSqlEngine): + key, display, default_port = "firebird", "Firebird", 3050 + template, slug, db_prefix = "engines/firebird.yml.j2", "firebird", "fb" + DATA_DIR = "/var/lib/firebird/data" + + def generate(self, *, auth: bool, ports: PortAllocator, answers: dict[str, Any]) -> DatabaseSpec: + db_file = "mirror.fdb" + return DatabaseSpec( + id=self.new_id(), + engine=self.key, + name=db_file, + managed=True, + host=self.service_name(self.slug), + port=self.default_port, + host_port=ports.free(), + database=f"{self.DATA_DIR}/{db_file}", + username="alice", + password=generate_password(16), + root_password=generate_password(16), + ) + + def env_vars(self, spec: DatabaseSpec) -> dict[str, str]: + base = super().env_vars(spec) + # Compose template expects the bare file name; databases.json carries the container path. + base[f"{spec.env_prefix}_DB"] = (spec.database or "").rsplit("/", 1)[-1] + base[f"{spec.env_prefix}_ROOT_PASS"] = spec.root_password or "" + return base + + def template_ctx(self, spec: DatabaseSpec, *, inline: bool = False) -> dict[str, Any]: + ctx = super().template_ctx(spec, inline=inline) + ctx["db_var"] = self.var(spec, "DB", (spec.database or "").rsplit("/", 1)[-1], inline) + ctx["root_password_var"] = self.var(spec, "ROOT_PASS", spec.root_password, inline) + return ctx +``` + +- [ ] **Step 2: Vérifier** + +Run: `uv run python -c " +from engines.sql import * +from services.ports import FixedPortAllocator +p = FixedPortAllocator() +for E in (PostgresEngine, MySqlEngine, MssqlEngine, FirebirdEngine): + e = E(); s = e.generate(auth=True, ports=p, answers={'options': {'clean_mode': 'none'}}) + print(E.key, s.host[:9], sorted(e.env_vars(s)), e.agent_entry(s).get('options'), e.template_ctx(s)['port_var']) +print(PostgresEngine().agent_entry(PostgresEngine().generate(auth=True, ports=p, answers={})).get('options'))"` +Expected (hex variable) : +``` +postgresql db-pg-xxx ['DB_PG_XXXX_DB', 'DB_PG_XXXX_PASS', 'DB_PG_XXXX_PORT', 'DB_PG_XXXX_USER'] {'clean_mode': 'none'} ${DB_PG_XXXX_PORT} +mysql db-mariad [... 4 vars] None ... +mssql db-mssql- [..._PASS, ..._PORT] None ... +firebird db-fireb [..._DB, ..._PASS, ..._PORT, ..._ROOT_PASS, ..._USER] None ... +None +``` +La dernière ligne : options par défaut → pas de clé `options`. + +- [ ] **Step 3: Commit** + +```bash +git add engines/sql.py +git commit -m "feat(engines): add SQL engines (postgresql, cluster, mysql, mariadb, mssql, firebird)" +``` + +--- + +### Task 4 : Redis, Valkey, Mongo, SQLite, Docker volume, registre + +**Files:** +- Create: `engines/redis.py`, `engines/valkey.py`, `engines/mongo.py`, `engines/sqlite.py`, `engines/docker_volume.py` +- Modify: `engines/__init__.py` + +**Interfaces:** +- Produces: `RedisEngine`, `ValkeyEngine`, `MongoEngine`, `SqliteEngine`, `DockerVolumeEngine` ; `engines.registry: EngineRegistry` (instance module-level) ; `engines.ALL: tuple[DbEngine, ...]`. + +- [ ] **Step 1: `engines/redis.py`** + +```python +from __future__ import annotations + +import secrets +from typing import Any + +from core.fields import Field +from core.specs import DatabaseSpec +from core.utils import generate_password +from engines.base import DbEngine +from services.ports import PortAllocator + + +class RedisEngine(DbEngine): + key, display, default_port = "redis", "Redis", 6379 + template = "engines/redis.yml.j2" + auth_variants = True + + def fields_existing(self) -> list[Field]: + return [ + Field("host", "Host", "text", default="localhost"), + Field("port", "Port", "int", default=self.default_port), + Field("database", "Database index", "text", default="0"), + Field("username", "Username (empty if none)", "text", default=""), + Field("password", "Password (empty if none)", "text", default=""), + ] + + def generate(self, *, auth: bool, ports: PortAllocator, answers: dict[str, Any]) -> DatabaseSpec: + return DatabaseSpec( + id=self.new_id(), + engine=self.key, + name=f"redis_{secrets.token_hex(4)}", + managed=True, + host=self.service_name("redis", auth), + port=self.default_port, + host_port=ports.free(), + database="0", + username="", + password=generate_password(16) if auth else None, + ) + + def env_vars(self, spec: DatabaseSpec) -> dict[str, str]: + p = spec.env_prefix + out = {f"{p}_PORT": str(spec.host_port)} + if spec.auth: + out[f"{p}_PASS"] = spec.password or "" + return out + + def agent_database(self, spec: DatabaseSpec) -> str: + return spec.database or "0" +``` + +- [ ] **Step 2: `engines/valkey.py`** + +Identique à Redis sauf identité et template : + +```python +from __future__ import annotations + +import secrets +from typing import Any + +from core.fields import Field +from core.specs import DatabaseSpec +from core.utils import generate_password +from engines.base import DbEngine +from services.ports import PortAllocator + + +class ValkeyEngine(DbEngine): + key, display, default_port = "valkey", "Valkey", 6379 + template = "engines/valkey.yml.j2" + auth_variants = True + + def fields_existing(self) -> list[Field]: + return [ + Field("host", "Host", "text", default="localhost"), + Field("port", "Port", "int", default=self.default_port), + Field("database", "Database index", "text", default="0"), + Field("username", "Username (empty if none)", "text", default=""), + Field("password", "Password (empty if none)", "text", default=""), + ] + + def generate(self, *, auth: bool, ports: PortAllocator, answers: dict[str, Any]) -> DatabaseSpec: + return DatabaseSpec( + id=self.new_id(), + engine=self.key, + name=f"valkey_{secrets.token_hex(4)}", + managed=True, + host=self.service_name("valkey", auth), + port=self.default_port, + host_port=ports.free(), + database="0", + username="", + password=generate_password(16) if auth else None, + ) + + def env_vars(self, spec: DatabaseSpec) -> dict[str, str]: + p = spec.env_prefix + out = {f"{p}_PORT": str(spec.host_port)} + if spec.auth: + out[f"{p}_PASS"] = spec.password or "" + return out + + def agent_database(self, spec: DatabaseSpec) -> str: + return spec.database or "0" +``` + +- [ ] **Step 3: `engines/mongo.py`** + +```python +from __future__ import annotations + +import secrets +from typing import Any + +from core.specs import DatabaseSpec +from core.utils import generate_password +from engines.base import DbEngine +from services.ports import PortAllocator + + +class MongoEngine(DbEngine): + key, display, default_port = "mongodb", "MongoDB", 27017 + template = "engines/mongodb.yml.j2" + auth_variants = True + + def generate(self, *, auth: bool, ports: PortAllocator, answers: dict[str, Any]) -> DatabaseSpec: + db_name = f"mongo_{secrets.token_hex(4)}" + return DatabaseSpec( + id=self.new_id(), + engine=self.key, + name=db_name, + managed=True, + host=self.service_name("mongo", auth), + port=self.default_port, + host_port=ports.free(), + database=db_name, + username="admin" if auth else "", + password=generate_password(16) if auth else None, + ) + + def env_vars(self, spec: DatabaseSpec) -> dict[str, str]: + p = spec.env_prefix + out = {f"{p}_PORT": str(spec.host_port), f"{p}_DB": spec.database or ""} + if spec.auth: + out[f"{p}_USER"] = spec.username or "" + out[f"{p}_PASS"] = spec.password or "" + return out +``` + +- [ ] **Step 4: `engines/sqlite.py`** + +```python +"""SQLite: a file mounted into the agent. No Compose service.""" + +from __future__ import annotations + +from typing import Any + +from core.fields import Field +from core.specs import DatabaseSpec +from engines.base import DbEngine +from services.ports import PortAllocator + +CONFIG_DIR = "/config" + + +class SqliteEngine(DbEngine): + key, display = "sqlite", "SQLite" + template = None + auth_variants = False + + def fields_existing(self) -> list[Field]: + return [Field("path", "Database Path (relative or absolute)", "text")] + + def fields_new(self) -> list[Field]: + return [Field("name", "Database Name", "text", default="local")] + + def generate(self, *, auth: bool, ports: PortAllocator, answers: dict[str, Any]) -> DatabaseSpec: + name = str(answers.get("name") or "local") + if not name.endswith(".sqlite"): + name += ".sqlite" + return DatabaseSpec( + id=self.new_id(), + engine=self.key, + name=name, + managed=False, + path=name, # relative: ./name mounted to /config/name + database=f"{CONFIG_DIR}/{name}", + ) + + def from_existing(self, answers: dict[str, Any]) -> DatabaseSpec: + raw = str(answers["path"]) + absolute = raw.startswith("/") + return DatabaseSpec( + id=self.new_id(), + engine=self.key, + name=answers.get("label") or "External DB", + managed=False, + path=raw, + database=raw if absolute else f"{CONFIG_DIR}/{raw}", + ) + + @staticmethod + def mount_for(spec: DatabaseSpec) -> tuple[str, str] | None: + """(host_path, container_path) if the file must be bind-mounted into the agent.""" + if spec.database and spec.database.startswith(f"{CONFIG_DIR}/"): + rel = spec.database[len(CONFIG_DIR) + 1 :] + return (f"./{rel}", spec.database) + return None + + def env_vars(self, spec: DatabaseSpec) -> dict[str, str]: + return {} + + def agent_entry(self, spec: DatabaseSpec) -> dict[str, Any]: + return {"name": spec.name, "database": spec.database, "type": self.key, "generated_id": spec.id} + + def describe(self, spec: DatabaseSpec) -> str: + return "Local File" +``` + +- [ ] **Step 5: `engines/docker_volume.py`** + +```python +"""Docker volume backup target. Requires the Docker socket on the agent. No Compose service.""" + +from __future__ import annotations + +from typing import Any + +from core.fields import Field +from core.specs import DatabaseSpec +from engines.base import DbEngine +from services.ports import PortAllocator + + +class DockerVolumeEngine(DbEngine): + key, display = "docker-volume", "Docker Volume" + template = None + has_modes = False + warning = "Requires the Docker socket. It will be mounted on the agent (/var/run/docker.sock)." + + def fields_existing(self) -> list[Field]: + return [ + Field("volume", "Volume Name (e.g. databases_sqlite-data)", "text"), + Field("container", "Container Name (optional, enables auto-restart after restore)", "text", default=""), + ] + + def fields_new(self) -> list[Field]: + return self.fields_existing() + + def generate(self, *, auth: bool, ports: PortAllocator, answers: dict[str, Any]) -> DatabaseSpec: + return self.from_existing(answers) + + def from_existing(self, answers: dict[str, Any]) -> DatabaseSpec: + return DatabaseSpec( + id=self.new_id(), + engine=self.key, + name=answers.get("label") or "Docker Volume", + managed=False, + volume=str(answers["volume"]).strip(), + container=(str(answers.get("container") or "").strip() or None), + ) + + def env_vars(self, spec: DatabaseSpec) -> dict[str, str]: + return {} + + def agent_entry(self, spec: DatabaseSpec) -> dict[str, Any]: + entry = {"name": spec.name, "type": self.key, "volume_name": spec.volume, "generated_id": spec.id} + if spec.container: + entry["container_name"] = spec.container + return entry + + def describe(self, spec: DatabaseSpec) -> str: + return f"volume: {spec.volume}" +``` + +- [ ] **Step 6: `engines/__init__.py`** + +```python +"""Engine registry. Explicit imports keep PyInstaller happy (no dynamic discovery).""" + +from __future__ import annotations + +from engines.docker_volume import DockerVolumeEngine +from engines.mongo import MongoEngine +from engines.redis import RedisEngine +from engines.registry import EngineRegistry +from engines.sql import ( + FirebirdEngine, + MariaDbEngine, + MssqlEngine, + MySqlEngine, + PostgresClusterEngine, + PostgresEngine, +) +from engines.sqlite import SqliteEngine +from engines.valkey import ValkeyEngine + +ALL = ( + PostgresEngine(), + PostgresClusterEngine(), + MySqlEngine(), + MariaDbEngine(), + SqliteEngine(), + FirebirdEngine(), + MongoEngine(), + RedisEngine(), + ValkeyEngine(), + MssqlEngine(), + DockerVolumeEngine(), +) + +registry = EngineRegistry(ALL) + +__all__ = ["ALL", "EngineRegistry", "registry"] +``` + +L'ordre = ordre d'affichage dans le select (identique au legacy). + +- [ ] **Step 7: Vérifier** + +Run: `uv run python -c " +from engines import registry +from services.ports import FixedPortAllocator +p = FixedPortAllocator() +print(registry.keys()) +for e in registry: + if e.template is None: continue + for auth in ((True, False) if e.auth_variants else (True,)): + s = e.generate(auth=auth, ports=p, answers={}) + ctx = e.template_ctx(s); assert ctx['name'] == s.host and set(e.env_vars(s)) >= {s.env_prefix + '_PORT'} + print(f'{e.key:20} auth={auth!s:5} {s.host:24} env={len(e.env_vars(s))} entry.db={e.agent_entry(s)[\"database\"]!r}') +sq = registry.get('sqlite'); s = sq.generate(auth=False, ports=p, answers={'name':'x'}); print(sq.agent_entry(s), sq.mount_for(s)) +dv = registry.get('docker-volume'); print(dv.agent_entry(dv.from_existing({'volume':'v','container':''})))"` +Expected: 11 clés dans l'ordre legacy ; une ligne par moteur/variante avec `entry.db` = `'0'` pour redis/valkey, `'master'` mssql, `/var/lib/firebird/data/mirror.fdb` firebird ; sqlite `{'name': 'x.sqlite', 'database': '/config/x.sqlite', 'type': 'sqlite', 'generated_id': ...} ('./x.sqlite', '/config/x.sqlite')` ; docker-volume sans `container_name`. + +- [ ] **Step 8: Commit** + +```bash +git add engines/ +git commit -m "feat(engines): add redis, valkey, mongodb, sqlite, docker-volume engines and registry" +``` + +--- + +### Task 5 : Templates Jinja2 + +**Files:** +- Create: `templates/agent.yml.j2`, `templates/dashboard.yml.j2` +- Create: `templates/engines/postgresql.yml.j2`, `mariadb.yml.j2`, `mssql.yml.j2`, `firebird.yml.j2`, `mongodb.yml.j2`, `redis.yml.j2`, `valkey.yml.j2` +- Create: `templates/engines.map.json` +- Move: `.github/assets/templates/agent.yml` → `templates/agent.yml`, `dashboard.yml` → `templates/dashboard.yml` +- Modify: `pyproject.toml` (`jinja2`), `.gitleaks.toml` + +**Interfaces:** +- Produces: contrat de contexte. + - `agent.yml.j2` : `host_gateway: bool`, `docker_socket: bool`, `mounts: list[{host, container}]`, `services: list[{name, volume, body}]`, `tz_var, edge_key_var, log_level_var, polling_var: str`. + - `dashboard.yml.j2` : `db_mode: "external"|"internal"|"custom"`, `project_name_var, host_port_var, tz_var, log_level_var, project_secret_var, project_url_var, pg_port_var, postgres_db_var, postgres_user_var, postgres_password_var: str`. + - `engines/*.yml.j2` : `name, volume, auth, port_var, db_var, user_var, password_var` (+ `root_password_var` firebird). + +- [ ] **Step 1: Ajouter Jinja2** + +Run: `uv add "jinja2>=3.1"` +Expected: `pyproject.toml` et `uv.lock` mis à jour. + +- [ ] **Step 2: Déplacer les templates legacy** + +Run: `mkdir -p templates/engines && git mv .github/assets/templates/agent.yml templates/agent.yml && git mv .github/assets/templates/dashboard.yml templates/dashboard.yml && rmdir .github/assets/templates 2>/dev/null; ls templates` + +- [ ] **Step 3: `templates/agent.yml.j2`** + +```jinja +services: + agent: + restart: unless-stopped + image: portabase/agent:latest + volumes: + - ./databases.json:/config/config.json +{%- for m in mounts %} + - {{ m.host }}:{{ m.container }} +{%- endfor %} +{%- if docker_socket %} + - /var/run/docker.sock:/var/run/docker.sock +{%- endif %} +{%- if host_gateway %} + extra_hosts: + - "localhost:host-gateway" +{%- endif %} + environment: + TZ: "{{ tz_var }}" + EDGE_KEY: "{{ edge_key_var }}" + LOG_LEVEL: "{{ log_level_var }}" + POLLING: "{{ polling_var }}" + networks: + - portabase +{% for s in services %} +{{ s.body }} +{%- endfor %} +{% if services %} +volumes: +{%- for s in services %} + {{ s.volume }}: +{%- endfor %} +{% endif %} +networks: + portabase: + name: portabase_network + external: true +``` + +- [ ] **Step 4: `templates/dashboard.yml.j2`** + +```jinja +name: {{ project_name_var }} +services: + portabase: + container_name: {{ project_name_var }}-app + image: portabase/portabase:latest + restart: unless-stopped + env_file: + - .env + ports: + - "{{ host_port_var }}:80" + environment: + - TZ={{ tz_var }} + - LOG_LEVEL={{ log_level_var }} + - PROJECT_SECRET={{ project_secret_var }} + - PROJECT_URL={{ project_url_var }} + volumes: + - portabase-data:/data +{%- if db_mode == "external" %} + depends_on: + db: + condition: service_healthy +{%- endif %} + healthcheck: + test: ["CMD-SHELL", "curl -f http://localhost/api/health"] + interval: 30s + timeout: 5s + retries: 3 + start_period: 60s +{%- if db_mode == "external" %} + db: + container_name: {{ project_name_var }}-pg + image: postgres:17-alpine + restart: unless-stopped + ports: + - "{{ pg_port_var }}:5432" + volumes: + - postgres-data:/var/lib/postgresql/data + environment: + - POSTGRES_DB={{ postgres_db_var }} + - POSTGRES_USER={{ postgres_user_var }} + - POSTGRES_PASSWORD={{ postgres_password_var }} + healthcheck: + test: ["CMD-SHELL", "pg_isready -U {{ postgres_user_var }} -d {{ postgres_db_var }}"] + interval: 10s + timeout: 5s + retries: 5 +{%- endif %} +volumes: +{%- if db_mode == "external" %} + postgres-data: +{%- endif %} + portabase-data: +``` + +- [ ] **Step 5: Templates moteurs** + +`templates/engines/postgresql.yml.j2` : + +```jinja + {{ name }}: + image: postgres:17-alpine + restart: unless-stopped + networks: + - portabase + ports: + - "{{ port_var }}:5432" + volumes: + - {{ volume }}:/var/lib/postgresql/data + environment: + - POSTGRES_DB={{ db_var }} + - POSTGRES_USER={{ user_var }} + - POSTGRES_PASSWORD={{ password_var }} + healthcheck: + test: ["CMD-SHELL", "pg_isready -U {{ user_var }} -d {{ db_var }}"] + interval: 10s + timeout: 5s + retries: 5 +``` + +`templates/engines/mariadb.yml.j2` : + +```jinja + {{ name }}: + image: mariadb:latest + restart: unless-stopped + networks: + - portabase + ports: + - "{{ port_var }}:3306" + environment: + - MYSQL_DATABASE={{ db_var }} + - MYSQL_USER={{ user_var }} + - MYSQL_PASSWORD={{ password_var }} + - MYSQL_RANDOM_ROOT_PASSWORD=yes + volumes: + - {{ volume }}:/var/lib/mysql + healthcheck: + test: ["CMD-SHELL", "mariadb-admin ping -h localhost -u {{ user_var }} -p{{ password_var }}"] + interval: 10s + timeout: 5s + retries: 5 +``` + +`templates/engines/mssql.yml.j2` : + +```jinja + {{ name }}: + image: mcr.microsoft.com/azure-sql-edge:latest + restart: unless-stopped + networks: + - portabase + ports: + - "{{ port_var }}:1433" + environment: + - ACCEPT_EULA=Y + - MSSQL_SA_PASSWORD={{ password_var }} + volumes: + - {{ volume }}:/var/opt/mssql + healthcheck: + test: ["CMD-SHELL", "cat /proc/net/tcp6 | grep -q '059901' || exit 1"] + interval: 10s + timeout: 5s + retries: 20 +``` + +`templates/engines/firebird.yml.j2` : + +```jinja + {{ name }}: + image: firebirdsql/firebird + restart: unless-stopped + networks: + - portabase + ports: + - "{{ port_var }}:3050" + volumes: + - {{ volume }}:/var/lib/firebird/data + environment: + - FIREBIRD_DATABASE={{ db_var }} + - FIREBIRD_USER={{ user_var }} + - FIREBIRD_PASSWORD={{ password_var }} + - FIREBIRD_ROOT_PASSWORD={{ root_password_var }} + - FIREBIRD_DATABASE_DEFAULT_CHARSET=UTF8 + healthcheck: + test: ["CMD-SHELL", "nc -z localhost 3050"] + interval: 10s + timeout: 5s + retries: 5 +``` + +`templates/engines/mongodb.yml.j2` : + +```jinja + {{ name }}: + image: mongo:latest + restart: unless-stopped + networks: + - portabase + ports: + - "{{ port_var }}:27017" + environment: +{%- if auth %} + - MONGO_INITDB_ROOT_USERNAME={{ user_var }} + - MONGO_INITDB_ROOT_PASSWORD={{ password_var }} +{%- endif %} + - MONGO_INITDB_DATABASE={{ db_var }} +{%- if auth %} + command: mongod --auth +{%- endif %} + volumes: + - {{ volume }}:/data/db + healthcheck: + test: ["CMD-SHELL", "mongosh --eval 'db.runCommand({ping:1})' --quiet"] + interval: 10s + timeout: 5s + retries: 5 +``` + +`templates/engines/redis.yml.j2` : + +```jinja + {{ name }}: + image: redis:latest + restart: unless-stopped + ports: + - "{{ port_var }}:6379" + volumes: + - {{ volume }}:/data +{%- if auth %} + environment: + - REDIS_PASSWORD={{ password_var }} + command: ["redis-server", "--requirepass", "{{ password_var }}", "--appendonly", "yes"] +{%- else %} + command: ["redis-server", "--appendonly", "yes"] +{%- endif %} + networks: + - portabase + - default + healthcheck: + test: ["CMD-SHELL", "redis-cli {% if auth %}-a {{ password_var }} {% endif %}ping | grep PONG"] + interval: 10s + timeout: 5s + retries: 5 +``` + +`templates/engines/valkey.yml.j2` : + +```jinja + {{ name }}: + image: valkey/valkey:latest + restart: unless-stopped +{%- if auth %} + command: --requirepass "{{ password_var }}" +{%- else %} + environment: + - ALLOW_EMPTY_PASSWORD=yes +{%- endif %} + ports: + - "{{ port_var }}:6379" + volumes: + - {{ volume }}:/data + networks: + - portabase + - default + healthcheck: + test: ["CMD-SHELL", "valkey-cli {% if auth %}-a {{ password_var }} {% endif %}ping | grep PONG"] + interval: 10s + timeout: 5s + retries: 5 +``` + +Différence assumée vs snippets legacy : `restart: unless-stopped` ajouté sur redis et valkey (spec §5.7). + +- [ ] **Step 6: `templates/engines.map.json`** + +```json +{ + "postgresql": "engines/postgresql.yml.j2", + "postgresql-cluster": "engines/postgresql.yml.j2", + "mysql": "engines/mariadb.yml.j2", + "mariadb": "engines/mariadb.yml.j2", + "mssql": "engines/mssql.yml.j2", + "firebird": "engines/firebird.yml.j2", + "mongodb": "engines/mongodb.yml.j2", + "redis": "engines/redis.yml.j2", + "valkey": "engines/valkey.yml.j2" +} +``` + +- [ ] **Step 7: `.gitleaks.toml`** + +Retirer la ligne `'''\.github/assets/templates/.*''',` de `paths`. + +- [ ] **Step 8: Rendu manuel de contrôle** + +Run: `uv run python -c " +import jinja2, yaml +env = jinja2.Environment(loader=jinja2.FileSystemLoader('templates'), undefined=jinja2.StrictUndefined, keep_trailing_newline=True, autoescape=False) +body = env.get_template('engines/redis.yml.j2').render(name='db-redis-auth-ab12', volume='db-redis-auth-ab12-data', auth=True, port_var='\${DB_REDIS_AUTH_AB12_PORT}', db_var='', user_var='', password_var='\${DB_REDIS_AUTH_AB12_PASS}') +out = env.get_template('agent.yml.j2').render(host_gateway=True, docker_socket=True, mounts=[{'host':'./x.sqlite','container':'/config/x.sqlite'}], services=[{'name':'db-redis-auth-ab12','volume':'db-redis-auth-ab12-data','body':body}], tz_var='\${TZ}', edge_key_var='\${EDGE_KEY}', log_level_var='\${LOG_LEVEL}', polling_var='\${POLLING}') +print(out); d = yaml.safe_load(out); print(sorted(d['services']), d['volumes'], d['services']['agent']['extra_hosts']) +for mode in ('external','internal','custom'): + o = env.get_template('dashboard.yml.j2').render(db_mode=mode, project_name_var='pb', host_port_var='8887', tz_var='\${TZ}', log_level_var='\${LOG_LEVEL}', project_secret_var='\${PROJECT_SECRET}', project_url_var='\${PROJECT_URL}', pg_port_var='\${PG_PORT}', postgres_db_var='\${POSTGRES_DB}', postgres_user_var='\${POSTGRES_USER}', postgres_password_var='\${POSTGRES_PASSWORD}') + print(mode, sorted(yaml.safe_load(o)['services']), sorted(yaml.safe_load(o)['volumes']))"` +Expected: compose agent imprimé avec socket, extra_hosts, mount sqlite, service redis ; `['agent', 'db-redis-auth-ab12'] {'db-redis-auth-ab12-data': None} ['localhost:host-gateway']` ; dashboard `external ['db', 'portabase'] ['portabase-data', 'postgres-data']`, `internal ['portabase'] ['portabase-data']`, `custom ['portabase'] ['portabase-data']`. + +- [ ] **Step 9: Commit** + +```bash +git add templates/ pyproject.toml uv.lock .gitleaks.toml +git commit -m "feat(templates): add Jinja2 compose templates at repo root, move legacy templates" +``` + +--- + +### Task 6 : `services/templates.py` — `Manifest`, `TemplateRepository` + +**Files:** +- Create: `services/templates.py` + +**Interfaces:** +- Consumes: `HttpClient`, `GlobalConfig.cache_dir`, `core.version`, `TemplateError`. +- Produces: + - `Manifest(schema, version, files: dict[str, FileEntry], engines: dict[str, str], generated_at, commit)` avec `from_json(data)`, `from_directory(dir, version)`. + - `TemplateRepository(http, cache_dir, version, base_url=TEMPLATE_BASE_URL, local_dir: Path | None = None)` : `resolve() -> Path` (dossier prêt, fetch si besoin), `get(name) -> jinja2.Template`, `engine_template(key) -> jinja2.Template`, `manifest -> Manifest`, propriété `source: str` (`local` / `cache` / `remote`). + - `TemplateRepository.from_environment(http, config) -> TemplateRepository` : lit `PORTABASE_TEMPLATES_DIR`, `PORTABASE_TEMPLATES_VERSION`, détection dev (`./templates` à côté de `main.py` si non frozen). + - `TEMPLATE_BASE_URL` importée depuis `core/config.py` (inchangée). + +- [ ] **Step 1: Écrire le module** + +```python +"""Versioned remote templates with manifest verification and a local cache. + +Resolution order: explicit local dir (dev) → cache hit → remote fetch. No 'latest' fallback: +a CLI version only ever renders with the templates published for that exact version. +""" + +from __future__ import annotations + +import hashlib +import json +import os +import sys +from dataclasses import dataclass +from pathlib import Path + +import jinja2 + +from core.config import TEMPLATE_BASE_URL, GlobalConfig +from core.errors import NetworkError, TemplateError +from core.version import UNKNOWN, current_version +from services.http import HttpClient + +MANIFEST_NAME = "manifest.json" +SUPPORTED_SCHEMA = 1 + + +@dataclass(frozen=True) +class FileEntry: + sha256: str + size: int + + +@dataclass(frozen=True) +class Manifest: + schema: int + version: str + files: dict[str, FileEntry] + engines: dict[str, str] + generated_at: str = "" + commit: str = "" + + @classmethod + def from_json(cls, data: dict) -> Manifest: + try: + schema = int(data["schema"]) + if schema != SUPPORTED_SCHEMA: + raise TemplateError( + f"Unsupported template manifest schema {schema} (this CLI supports {SUPPORTED_SCHEMA}).", + hint="Update the CLI: portabase update", + ) + files = { + name: FileEntry(sha256=str(e["sha256"]).lower(), size=int(e["size"])) + for name, e in data["files"].items() + } + return cls( + schema=schema, + version=str(data["version"]), + files=files, + engines=dict(data.get("engines", {})), + generated_at=str(data.get("generated_at", "")), + commit=str(data.get("commit", "")), + ) + except (KeyError, TypeError, ValueError) as e: + raise TemplateError("Template manifest is malformed.", cause=e) from e + + @classmethod + def from_directory(cls, directory: Path, version: str) -> Manifest: + """Manifest computed from a local directory (dev mode / render checks).""" + files = {} + for path in sorted(directory.rglob("*.j2")): + rel = path.relative_to(directory).as_posix() + files[rel] = FileEntry(sha256=_sha256(path), size=path.stat().st_size) + engines_map = directory / "engines.map.json" + engines = json.loads(engines_map.read_text(encoding="utf-8")) if engines_map.exists() else {} + return cls(schema=SUPPORTED_SCHEMA, version=version, files=files, engines=engines) + + +def _sha256(path: Path) -> str: + h = hashlib.sha256() + with open(path, "rb") as f: + for chunk in iter(lambda: f.read(1 << 20), b""): + h.update(chunk) + return h.hexdigest() + + +class TemplateRepository: + def __init__( + self, + http: HttpClient, + cache_dir: Path, + version: str, + *, + base_url: str = TEMPLATE_BASE_URL, + local_dir: Path | None = None, + ) -> None: + self.http = http + self.version = version + self.base_url = base_url.rstrip("/") + self.local_dir = local_dir + self.cache_dir = cache_dir / "templates" / version + self._manifest: Manifest | None = None + self._env: jinja2.Environment | None = None + self.source = "unresolved" + + # ---- construction ----------------------------------------------------- + + @classmethod + def from_environment(cls, http: HttpClient, config: GlobalConfig) -> TemplateRepository: + version = os.environ.get("PORTABASE_TEMPLATES_VERSION") or current_version() + local = os.environ.get("PORTABASE_TEMPLATES_DIR") + local_dir = Path(local) if local else None + if local_dir is None and not getattr(sys, "frozen", False): + candidate = Path(__file__).resolve().parent.parent / "templates" + if (candidate / "agent.yml.j2").exists(): + local_dir = candidate + return cls(http, config.cache_dir, version, local_dir=local_dir) + + # ---- resolution ------------------------------------------------------- + + @property + def manifest(self) -> Manifest: + if self._manifest is None: + self.resolve() + assert self._manifest is not None + return self._manifest + + def resolve(self) -> Path: + """Ensure a verified template directory exists locally and return it.""" + if self.local_dir is not None: + if not (self.local_dir / "agent.yml.j2").exists(): + raise TemplateError(f"Template directory {self.local_dir} has no agent.yml.j2.") + self._manifest = Manifest.from_directory(self.local_dir, self.version) + self.source = "local" + return self.local_dir + + if self.version == UNKNOWN: + raise TemplateError( + "Cannot resolve template version (CLI version unknown).", + hint="Set PORTABASE_TEMPLATES_DIR to a local templates folder or PORTABASE_TEMPLATES_VERSION.", + ) + + remote_manifest = self._fetch_manifest() + if remote_manifest is None: + cached = self._cached_manifest() + if cached is None: + raise TemplateError( + f"Templates for version {self.version} are unavailable and not cached.", + hint="Check your internet connection, or set PORTABASE_TEMPLATES_DIR.", + ) + self._manifest = cached + self.source = "cache" + self._verify_cache_complete(cached) + return self.cache_dir + + if remote_manifest.version != self.version: + raise TemplateError( + f"Template manifest is for version {remote_manifest.version}, expected {self.version}." + ) + self._sync(remote_manifest) + self._manifest = remote_manifest + self.source = "remote" + return self.cache_dir + + # ---- access ----------------------------------------------------------- + + def get(self, name: str) -> jinja2.Template: + directory = self.resolve() + if name not in self.manifest.files: + raise TemplateError(f"Template '{name}' is not part of version {self.version}.") + if self._env is None: + self._env = jinja2.Environment( + loader=jinja2.FileSystemLoader(str(directory)), + undefined=jinja2.StrictUndefined, + keep_trailing_newline=True, + autoescape=False, + ) + try: + return self._env.get_template(name) + except jinja2.TemplateError as e: + raise TemplateError(f"Template '{name}' failed to load: {e}", cause=e) from e + + def engine_template(self, engine_key: str) -> jinja2.Template: + name = self.manifest.engines.get(engine_key) + if name is None: + raise TemplateError(f"No template mapped for engine '{engine_key}' in version {self.version}.") + return self.get(name) + + # ---- internals -------------------------------------------------------- + + def _url(self, name: str) -> str: + return f"{self.base_url}/{self.version}/{name}" + + def _fetch_manifest(self) -> Manifest | None: + try: + return Manifest.from_json(self.http.get_json(self._url(MANIFEST_NAME))) + except NetworkError: + return None + + def _cached_manifest(self) -> Manifest | None: + path = self.cache_dir / MANIFEST_NAME + if not path.exists(): + return None + try: + return Manifest.from_json(json.loads(path.read_text(encoding="utf-8"))) + except (OSError, ValueError, TemplateError): + return None + + def _verify_cache_complete(self, manifest: Manifest) -> None: + for name, entry in manifest.files.items(): + path = self.cache_dir / name + if not path.exists() or _sha256(path) != entry.sha256: + raise TemplateError( + f"Cached template '{name}' is missing or corrupt and the network is unavailable.", + hint="Reconnect and retry; the cache will be refreshed.", + ) + + def _sync(self, manifest: Manifest) -> None: + self.cache_dir.mkdir(parents=True, exist_ok=True) + for name, entry in manifest.files.items(): + path = self.cache_dir / name + if path.exists() and path.stat().st_size == entry.size and _sha256(path) == entry.sha256: + continue + path.parent.mkdir(parents=True, exist_ok=True) + tmp = path.with_suffix(path.suffix + ".tmp") + try: + self.http.download(self._url(name), tmp) + except NetworkError as e: + raise TemplateError(f"Could not download template '{name}'.", cause=e) from e + if tmp.stat().st_size != entry.size or _sha256(tmp) != entry.sha256: + tmp.unlink(missing_ok=True) + raise TemplateError(f"Template '{name}' failed integrity check (sha256 mismatch).") + os.replace(tmp, path) + for stale in self.cache_dir.rglob("*.j2"): + if stale.relative_to(self.cache_dir).as_posix() not in manifest.files: + stale.unlink(missing_ok=True) + (self.cache_dir / MANIFEST_NAME).write_text( + json.dumps( + { + "schema": manifest.schema, + "version": manifest.version, + "generated_at": manifest.generated_at, + "commit": manifest.commit, + "files": {n: {"sha256": e.sha256, "size": e.size} for n, e in manifest.files.items()}, + "engines": manifest.engines, + }, + indent=2, + ), + encoding="utf-8", + ) +``` + +`TEMPLATE_BASE_URL` reste définie dans `core/config.py` (le legacy `core/network.py` l'importe de là) ; `services/templates.py` l'importe depuis `core.config`. Pas de circularité : `core` n'importe jamais `services`. + +- [ ] **Step 2: Vérifier en mode local (dev)** + +Run: `uv run python -c " +from pathlib import Path +from services.http import HttpClient +from services.templates import TemplateRepository +from core.config import GlobalConfig +r = TemplateRepository.from_environment(HttpClient(), GlobalConfig()) +print(r.source, r.resolve(), r.source, len(r.manifest.files), r.manifest.engines['mysql']) +print(r.engine_template('redis').render(name='n', volume='v', auth=False, port_var='1', db_var='', user_var='', password_var='')[:40].strip())"` +Expected: `unresolved /templates local 9 engines/mariadb.yml.j2` puis `n:` (début du service rendu). + +- [ ] **Step 3: Vérifier le mode remote contre un serveur local** + +```bash +# Terminal 1 — publie templates/ comme S3 sous la version 99.0.0 avec un manifest +mkdir -p /tmp/pb-s3/99.0.0 && cp -r templates/. /tmp/pb-s3/99.0.0/ && cd /tmp/pb-s3/99.0.0 && \ +uv --directory /home/soluce/Documents/PROJETS/Portabase/cli run python -c " +import json, hashlib, pathlib +files = {p.as_posix(): {'sha256': hashlib.sha256(p.read_bytes()).hexdigest(), 'size': p.stat().st_size} for p in sorted(pathlib.Path('.').rglob('*.j2'))} +json.dump({'schema':1,'version':'99.0.0','generated_at':'now','commit':'x','files':files,'engines':json.load(open('engines.map.json'))}, open('manifest.json','w'), indent=2)" && \ +cd /tmp/pb-s3 && python3 -m http.server 8765 +``` + +Terminal 2 : +```bash +uv run python -c " +import tempfile; from pathlib import Path +from services.http import HttpClient +from services.templates import TemplateRepository +cache = Path(tempfile.mkdtemp()) +r = TemplateRepository(HttpClient(), cache, '99.0.0', base_url='http://127.0.0.1:8765') +print(r.resolve(), r.source, sorted(p.name for p in (cache/'templates'/'99.0.0').iterdir())) +r2 = TemplateRepository(HttpClient(), cache, '99.0.0', base_url='http://127.0.0.1:8765'); r2.resolve(); print('second:', r2.source) +r3 = TemplateRepository(HttpClient(), cache, '99.0.0', base_url='http://127.0.0.1:1'); r3.resolve(); print('offline:', r3.source) +try: TemplateRepository(HttpClient(), Path(tempfile.mkdtemp()), '99.0.0', base_url='http://127.0.0.1:1').resolve() +except Exception as e: print('offline no cache:', type(e).__name__, e.code) +try: TemplateRepository(HttpClient(), cache, '98.0.0', base_url='http://127.0.0.1:8765').resolve() +except Exception as e: print('missing version:', type(e).__name__, e.code)" +``` +Expected: `... remote ['agent.yml.j2', 'dashboard.yml.j2', 'engines', 'manifest.json']`, `second: remote` (manifest re-fetché, fichiers en cache non re-téléchargés), `offline: cache`, `offline no cache: TemplateError E_TEMPLATE`, `missing version: TemplateError E_TEMPLATE`. Arrêter le serveur. + +- [ ] **Step 4: Commit** + +```bash +git add services/templates.py +git commit -m "feat(services): add TemplateRepository with manifest verification and cache" +``` + +--- + +### Task 7 : `scripts/render_check.py` + +**Files:** +- Create: `scripts/render_check.py` + +**Interfaces:** +- Consumes: `TemplateRepository` (mode local), `engines.registry`, `FixedPortAllocator`. +- Produces: script exécutable, exit 0 si tous les rendus sont du YAML valide (et `docker compose config` valide si Docker disponible), exit 1 sinon. Réutilisé par la CI (Task 8) et remplacé par un appel à `ComposeRenderer` au Plan 4. + +- [ ] **Step 1: Écrire le script** + +```python +#!/usr/bin/env python3 +"""Render every template with fixture contexts and validate the output. + +Usage: uv run python scripts/render_check.py [--templates DIR] [--no-compose] +Exit 0 on success. Prints one line per rendered case. +""" + +from __future__ import annotations + +import argparse +import os +import shutil +import subprocess +import sys +import tempfile +from pathlib import Path + +import yaml + +sys.path.insert(0, str(Path(__file__).resolve().parent.parent)) + +from core.config import GlobalConfig # noqa: E402 +from engines import registry # noqa: E402 +from services.http import HttpClient # noqa: E402 +from services.ports import FixedPortAllocator # noqa: E402 +from services.templates import TemplateRepository # noqa: E402 + +AGENT_GLOBALS = { + "tz_var": "${TZ}", + "edge_key_var": "${EDGE_KEY}", + "log_level_var": "${LOG_LEVEL}", + "polling_var": "${POLLING}", +} +AGENT_ENV = 'TZ="UTC"\nEDGE_KEY="x"\nLOG_LEVEL="info"\nPOLLING="5"\n' +DASHBOARD_VARS = { + "project_name_var": "pb", + "host_port_var": "${HOST_PORT}", + "tz_var": "${TZ}", + "log_level_var": "${LOG_LEVEL}", + "project_secret_var": "${PROJECT_SECRET}", + "project_url_var": "${PROJECT_URL}", + "pg_port_var": "${PG_PORT}", + "postgres_db_var": "${POSTGRES_DB}", + "postgres_user_var": "${POSTGRES_USER}", + "postgres_password_var": "${POSTGRES_PASSWORD}", +} +DASHBOARD_ENV = ( + 'HOST_PORT="8887"\nTZ="UTC"\nLOG_LEVEL="info"\nPROJECT_SECRET="s"\nPROJECT_URL="http://localhost"\n' + 'PG_PORT="5433"\nPOSTGRES_DB="pb"\nPOSTGRES_USER="pb"\nPOSTGRES_PASSWORD="p"\n' +) + + +class Failure(Exception): + pass + + +def validate(label: str, compose: str, env_text: str, use_compose: bool) -> None: + try: + doc = yaml.safe_load(compose) + except yaml.YAMLError as e: + raise Failure(f"{label}: invalid YAML: {e}\n{compose}") from e + if not isinstance(doc, dict) or "services" not in doc: + raise Failure(f"{label}: no services key\n{compose}") + if use_compose: + with tempfile.TemporaryDirectory() as tmp: + Path(tmp, "docker-compose.yml").write_text(compose, encoding="utf-8") + Path(tmp, ".env").write_text(env_text, encoding="utf-8") + Path(tmp, "databases.json").write_text('{"databases": []}', encoding="utf-8") + proc = subprocess.run( + ["docker", "compose", "-p", "rendercheck", "config", "--quiet"], + cwd=tmp, + capture_output=True, + text=True, + check=False, + ) + if proc.returncode != 0: + raise Failure(f"{label}: docker compose config failed:\n{proc.stderr}\n{compose}") + print(f"ok {label}") + + +def agent_cases(repo: TemplateRepository) -> list[tuple[str, str, str]]: + ports = FixedPortAllocator() + cases = [] + # 1. empty agent, all toggles off + cases.append(("agent/empty", render_agent(repo, [], False, False, []), AGENT_ENV)) + # 2. toggles on, sqlite mount + cases.append( + ( + "agent/toggles", + render_agent(repo, [], True, True, [{"host": "./x.sqlite", "container": "/config/x.sqlite"}]), + AGENT_ENV, + ) + ) + # 3. one case per engine/variant + all_services, all_env = [], AGENT_ENV + for engine in registry: + if engine.template is None: + continue + for auth in (True, False) if engine.auth_variants else (True,): + spec = engine.generate(auth=auth, ports=ports, answers={}) + env = engine.env_vars(spec) + env_text = AGENT_ENV + "".join(f'{k}="{v}"\n' for k, v in env.items()) + body = repo.engine_template(engine.key).render(**engine.template_ctx(spec)) + service = {"name": spec.host, "volume": f"{spec.host}-data", "body": body} + cases.append((f"agent/{engine.key}{'/auth' if auth else '/noauth' if engine.auth_variants else ''}", + render_agent(repo, [service], False, False, []), env_text)) + all_services.append(service) + all_env += "".join(f'{k}="{v}"\n' for k, v in env.items()) + # 4. everything at once + cases.append(("agent/all", render_agent(repo, all_services, True, True, []), all_env)) + return cases + + +def render_agent(repo, services, host_gateway, docker_socket, mounts) -> str: + return repo.get("agent.yml.j2").render( + services=services, host_gateway=host_gateway, docker_socket=docker_socket, mounts=mounts, **AGENT_GLOBALS + ) + + +def dashboard_cases(repo: TemplateRepository) -> list[tuple[str, str, str]]: + return [ + (f"dashboard/{mode}", repo.get("dashboard.yml.j2").render(db_mode=mode, **DASHBOARD_VARS), DASHBOARD_ENV) + for mode in ("external", "internal", "custom") + ] + + +def engines_check(repo: TemplateRepository) -> None: + mapped = repo.manifest.engines + for engine in registry: + if engine.template is None: + continue + if mapped.get(engine.key) != engine.template: + raise Failure(f"engines.map.json: {engine.key} -> {mapped.get(engine.key)} but code says {engine.template}") + if engine.template not in repo.manifest.files: + raise Failure(f"{engine.key}: template {engine.template} not found") + for key in mapped: + if key not in registry: + raise Failure(f"engines.map.json maps unknown engine '{key}'") + print(f"ok engines-check ({len(mapped)} mapped)") + + +def main() -> int: + parser = argparse.ArgumentParser() + parser.add_argument("--templates", default=os.environ.get("PORTABASE_TEMPLATES_DIR", "templates")) + parser.add_argument("--no-compose", action="store_true", help="Skip docker compose config validation") + args = parser.parse_args() + + use_compose = not args.no_compose and shutil.which("docker") is not None + if not use_compose: + print("note: docker not available, YAML validation only") + repo = TemplateRepository(HttpClient(), GlobalConfig().cache_dir, "local", local_dir=Path(args.templates)) + try: + engines_check(repo) + for label, compose, env_text in agent_cases(repo) + dashboard_cases(repo): + validate(label, compose, env_text, use_compose) + except Failure as e: + print(f"FAIL {e}", file=sys.stderr) + return 1 + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) +``` + +- [ ] **Step 2: Exécuter** + +Run: `uv run python scripts/render_check.py` +Expected: `ok engines-check (9 mapped)` puis une ligne `ok` par cas : `agent/empty`, `agent/toggles`, `agent/postgresql`, `agent/postgresql-cluster`, `agent/mysql`, `agent/mariadb`, `agent/firebird`, `agent/mongodb/auth`, `agent/mongodb/noauth`, `agent/redis/auth`, `agent/redis/noauth`, `agent/valkey/auth`, `agent/valkey/noauth`, `agent/mssql`, `agent/all`, `dashboard/external`, `dashboard/internal`, `dashboard/custom`. Exit 0. + +Si `docker compose config` échoue sur un cas : lire l'erreur, corriger le template (pas le script). + +- [ ] **Step 3: Ruff sur le script** + +Run: `uv run ruff check scripts/ && uv run ruff format scripts/` + +- [ ] **Step 4: Commit** + +```bash +git add scripts/render_check.py +git commit -m "ci: add render_check script validating every template with fixtures" +``` + +--- + +### Task 8 : CI — `render-check`, `engines-check`, manifest à l'upload, hotfix + +**Files:** +- Modify: `.github/workflows/ci.yml` +- Modify: `.github/workflows/templates-upload.yml` +- Create: `.github/workflows/templates-hotfix.yml` + +- [ ] **Step 1: Ajouter le job `render-check` à `ci.yml`** (après `test`) + +```yaml + render-check: + name: render-check + runs-on: ubuntu-latest + permissions: + contents: read + steps: + - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4 + - uses: astral-sh/setup-uv@caf0cab7a618c569241d31dcd442f54681755d39 # v3 + - name: Install + run: uv sync --frozen --all-groups + - name: Render and validate templates (YAML + docker compose config) + run: uv run python scripts/render_check.py --templates templates +``` + +Le job `engines-check` de la spec est couvert par la fonction `engines_check()` du même script (une seule exécution, deux vérifications). Pas de job séparé. + +- [ ] **Step 2: `templates-upload.yml` — source et manifest** + +Remplacer les deux étapes d'upload par : + +```yaml + - name: Generate manifest + env: + VERSION: ${{ inputs.version }} + run: | + set -euo pipefail + CLEAN_VERSION="${VERSION#v}" + cd templates + FILES=$(find . -name '*.j2' -type f | sort | while read -r f; do + rel="${f#./}" + printf '{"%s":{"sha256":"%s","size":%s}}\n' "$rel" "$(sha256sum "$f" | cut -d' ' -f1)" "$(stat -c%s "$f")" + done | jq -s 'add') + jq -n \ + --arg version "$CLEAN_VERSION" \ + --arg commit "$GITHUB_SHA" \ + --arg date "$(date -u +%FT%TZ)" \ + --argjson files "$FILES" \ + --argjson engines "$(cat engines.map.json)" \ + '{schema:1, version:$version, generated_at:$date, commit:$commit, files:$files, engines:$engines}' \ + > manifest.json + cat manifest.json + + - name: Upload versioned templates + env: + VERSION: ${{ inputs.version }} + run: | + CLEAN_VERSION="${VERSION#v}" + s3cmd $S3CMD_ARGS sync templates/ \ + "s3://${{ secrets.S3_BUCKET }}/cli/public/templates/${CLEAN_VERSION}/" --acl-public --delete-removed + + - name: Upload latest templates (stable only, legacy fallback) + if: ${{ !inputs.is_prerelease }} + run: | + s3cmd $S3CMD_ARGS sync templates/ \ + "s3://${{ secrets.S3_BUCKET }}/cli/public/templates/latest/" --acl-public +``` + +`latest/` reste alimenté pour les vieux binaires (fallback legacy). Le nouveau code ne le lit jamais. À retirer quand plus aucune version legacy n'est supportée. + +- [ ] **Step 3: `templates-hotfix.yml`** + +```yaml +name: Templates hotfix + +on: + workflow_dispatch: + inputs: + version: + description: "Existing CLI version to re-publish templates for (e.g. 26.09.0). Templates must stay compatible with that version's code." + required: true + type: string + +permissions: {} + +jobs: + hotfix: + uses: ./.github/workflows/templates-upload.yml + with: + version: ${{ inputs.version }} + is_prerelease: true # never touch latest/ from a hotfix + secrets: + S3_ENDPOINT: ${{ secrets.S3_ENDPOINT }} + S3_ACCESS_KEY: ${{ secrets.S3_ACCESS_KEY }} + S3_SECRET_KEY: ${{ secrets.S3_SECRET_KEY }} + S3_BUCKET: ${{ secrets.S3_BUCKET }} +``` + +- [ ] **Step 4: Valider les YAML** + +Run: `for f in .github/workflows/ci.yml .github/workflows/templates-upload.yml .github/workflows/templates-hotfix.yml; do uv run python -c "import yaml,sys; yaml.safe_load(open('$f')); print('ok $f')"; done` + +- [ ] **Step 5: Tester la génération du manifest en local** + +Run: `cd templates && FILES=$(find . -name '*.j2' -type f | sort | while read -r f; do rel="${f#./}"; printf '{"%s":{"sha256":"%s","size":%s}}\n' "$rel" "$(sha256sum "$f" | cut -d' ' -f1)" "$(stat -c%s "$f")"; done | jq -s 'add') && jq -n --arg version 0.0.0 --arg commit x --arg date now --argjson files "$FILES" --argjson engines "$(cat engines.map.json)" '{schema:1, version:$version, generated_at:$date, commit:$commit, files:$files, engines:$engines}' | uv run python -c "import json,sys; from services.templates import Manifest; m = Manifest.from_json(json.load(sys.stdin)); print(len(m.files), 'files,', len(m.engines), 'engines')"; cd ..` +Expected: `9 files, 9 engines`. + +- [ ] **Step 6: Commit** + +```bash +git add .github/workflows/ +git commit -m "ci: render-check job, manifest generation on template upload, hotfix workflow" +``` + +--- + +### Task 9 : PR et release candidate + +- [ ] **Step 1: Lint complet et PR** + +Run: `uv run ruff check . && uv run ruff format --check . && uv run python scripts/render_check.py --no-compose` +Puis : +```bash +git checkout -b refactor/templates-engines +git push -u origin refactor/templates-engines +``` +PR « feat: Jinja2 templates, engine registry, template repository ». Checks attendus verts : `lint`, `test`, `render-check`, `gitleaks`, `plumber`, `build-smoke`. + +- [ ] **Step 2: Release candidate** + +Après merge : Bump version `26.08.0rc2` (ou suivant), channel `rc`. Vérifier sur S3 que `cli/public/templates/26.08.0rc2/` contient `manifest.json`, `agent.yml.j2`, `engines/`, **et** `agent.yml` / `dashboard.yml` legacy. Installer le binaire rc et lancer `portabase agent test-rc` (code legacy) : doit fonctionner comme avant (fetch `agent.yml` sous la version exacte). + +--- + +## Self-review + +**Spec coverage :** +- §5.4 `TemplateRepository` : résolution version/env/dev ✔, cache ✔, manifest sha256+size ✔, suppression fichiers obsolètes ✔, pas de `latest` côté client ✔, Jinja2 `StrictUndefined` ✔, schéma inconnu → `TemplateError` ✔, version ≠ → `TemplateError` ✔. +- §5.6 templates : `agent.yml.j2` avec `mounts`, `docker_socket`, `host_gateway`, `services`, `volumes` ✔ ; moteurs avec `{% if auth %}` ✔ ; `dashboard.yml.j2` avec `db_mode` ✔. +- §6 moteurs : hiérarchie, hooks, registre imports explicites ✔ ; `agent_database` hook ✔ ; Redis/Valkey séparés ✔ ; `describe` pour `db list` (Plan 4). +- §6.1 options : `option_fields`, `non_default_options`, projection ✔ ; parsing `-o` et prompts → Plan 4 (flow). +- §9.1 `render-check` ✔, `engines-check` (fusionné dans le script) ✔. §9.3 manifest ✔, hotfix ✔. +- §10 D : shippable en rc, legacy intact ✔. + +**Placeholders :** aucun. + +**Cohérence :** `DbEngine.template_ctx` produit `name/volume/auth/*_var` = variables consommées par tous les `.j2` ✔ ; `FirebirdEngine.template_ctx` ajoute `root_password_var` consommé par `firebird.yml.j2` ✔ ; `render_check.render_agent` passe `AGENT_GLOBALS` = variables `*_var` de `agent.yml.j2` ✔ ; `Manifest.engines` clé → `TemplateRepository.engine_template` ✔ ; `FixedPortAllocator` défini Task 1, utilisé Task 7 ✔. + +**Écarts connus :** +- `DatabaseSpec.host_port` est `None` pour les specs chargées depuis une install legacy tant que Plan 4 ne lit pas `.env` ; sans effet ici. From 333db61daa232d5351f670a4e1e02ca8f94445fa Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Th=C3=A9o=20LAGACHE?= Date: Fri, 11 Sep 2026 10:33:50 +0200 Subject: [PATCH 06/24] docs: add plan 4 (declarative rendering, flag-driven commands, build) Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_017ACLkgaiP48pBMyi7kZZ4a --- .../2026-09-11-plan-4-render-commands.md | 1797 +++++++++++++++++ 1 file changed, 1797 insertions(+) create mode 100644 docs/superpowers/plans/2026-09-11-plan-4-render-commands.md diff --git a/docs/superpowers/plans/2026-09-11-plan-4-render-commands.md b/docs/superpowers/plans/2026-09-11-plan-4-render-commands.md new file mode 100644 index 0000000..c4c183b --- /dev/null +++ b/docs/superpowers/plans/2026-09-11-plan-4-render-commands.md @@ -0,0 +1,1797 @@ +# Plan 4 — Rendu déclaratif et commandes agent / dashboard / db / build (chantiers E + F) + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Remplacer le code legacy (`agent.py`, `db.py`, `dashboard.py`, chirurgie texte du compose) par un rendu complet depuis l'état (`.env` + `databases.json` + faits du compose), un flux d'ajout de base partagé, des commandes entièrement pilotables par flags, et la commande `build`. Fin de la refonte : plus aucun `LegacyCommand`, plus de `console` global, plus de fallback `latest`. + +**Architecture:** `AgentProject`/`DashboardProject` chargent l'état depuis le dossier ; `ComposeRenderer` produit `docker-compose.yml` (+ `databases.json`) via Jinja2 ; `RenderResult.write()` valide, sauvegarde un compose legacy en `.legacy.yml`, écrit atomiquement. `AddDatabaseFlow` collecte un `DatabaseSpec` (flags → prompts → défauts) et mute le projet ; `AgentCommand` et `DbAddCommand` l'utilisent tous deux. + +**Tech Stack:** Python 3.12, Typer, Jinja2, PyYAML, questionary/Rich via `ui/`. + +**Spec:** `docs/superpowers/specs/2026-09-11-cli-refactor-design.md` — sections 4.2, 4.3, 5.1–5.3, 5.5, 5.7, 6.1, 7.2 (Summary, DataTable, Diff), 10 (E, F). + +## Global Constraints + +- Prérequis : Plans 1–3 exécutés. +- Règle de dépendance descendante (`commands → services, engines, ui, core` ; `services → engines, core` ; `flows` dans `commands/`). +- `.env` ne contient que des variables consommées par les conteneurs. Aucune clé `PORTABASE_*`. +- `databases.json` conserve exactement le format legacy (projection par `DbEngine.agent_entry`). +- Détection `managed` : `.env` contient `{PREFIX}_PORT` pour le `host` de l'entrée. +- Le compose généré porte l'en-tête `# Generated by Portabase CLI . Do not edit — use docker-compose.override.yml.` ; un compose sans cet en-tête est sauvegardé en `docker-compose.legacy.yml` avant la première réécriture (une seule fois). +- Toute commande mutante : `templates.resolve()` **avant** de muter quoi que ce soit. +- Pas de tests unitaires. Vérifications exécutables par commande, en interactif et en `--non-interactive`, sur une install neuve et sur une install legacy générée avec le binaire 26.07.6. +- Ce plan supprime : `commands/common.py` (déjà), `core/network.py`, `core/docker.py`, `templates/compose.py`, `templates/__init__.py`, `templates/agent.yml`, `templates/dashboard.yml`, `LegacyCommand`, les fonctions legacy de `core/config.py`, `console`/`print_banner`/`HINTS`/`check_system`/`start_docker`/`validate_work_dir`/`get_free_port`/`get_random_hint` de `core/utils.py`, toutes les `per-file-ignores` ruff. + +--- + +## File Structure + +| Fichier | Action | Responsabilité | +|---|---|---| +| `engines/base.py` | modifier | `label_default` | +| `services/envfile.py` | créer | `EnvFile` | +| `services/compose_facts.py` | créer | `ComposeFacts` | +| `services/project.py` | créer | `AgentProject`, `DashboardProject`, `ProjectKind`, `detect_kind`, `spec_from_entry` | +| `services/renderer.py` | créer | `ComposeRenderer`, `RenderResult`, `WriteReport` | +| `services/docker.py` | modifier | `remove_volume` | +| `ui/components/summary.py`, `table.py`, `diff.py` | créer | composants | +| `ui/__init__.py` | modifier | `summary`, `table`, `diff` | +| `commands/flows/__init__.py`, `add_database.py` | créer | `AddDatabaseFlow` | +| `commands/agent.py` | réécrire | `AgentCommand` | +| `commands/dashboard.py` | réécrire | `DashboardCommand` | +| `commands/db.py` | réécrire | `DbCommands` (`add`, `remove`, `list`) | +| `commands/build.py` | créer | `BuildCommand` | +| `commands/base.py` | modifier | retirer `LegacyCommand` | +| `main.py` | modifier | câblage final | +| `core/config.py`, `core/utils.py` | modifier | retirer le legacy | +| `scripts/render_check.py` | modifier | utiliser `ComposeRenderer` | +| `.github/workflows/ci.yml` | modifier | `build-smoke` avec `agent --non-interactive` | +| `pyproject.toml` | modifier | retirer `per-file-ignores` | +| `README.md` | modifier | note migration | + +--- + +### Task 1 : `EnvFile` + +**Files:** +- Create: `services/envfile.py` + +**Interfaces:** +- Produces: `EnvFile(path)` : `load() -> EnvFile` (classmethod `EnvFile.load(path)`), `get(key, default=None)`, `set(key, value)`, `merge(mapping)`, `remove(key)`, `remove_prefix(prefix)`, `as_dict() -> dict[str, str]`, `save()`, `exists`. Préserve ordre, commentaires, lignes vides. + +- [ ] **Step 1: Écrire le module** + +```python +"""Dotenv file kept as a list of lines so comments and order survive rewrites. + +Only container runtime variables live here. Values are always written double-quoted. +""" + +from __future__ import annotations + +import os +import re +from collections.abc import Mapping +from dataclasses import dataclass, field +from pathlib import Path + +_LINE = re.compile(r"""^\s*(?:export\s+)?([A-Za-z_][A-Za-z0-9_]*)\s*=\s*(.*?)\s*$""") + + +def _unquote(raw: str) -> str: + if len(raw) >= 2 and raw[0] == raw[-1] and raw[0] in "\"'": + inner = raw[1:-1] + if raw[0] == '"': + return inner.replace('\\"', '"').replace("\\\\", "\\") + return inner + # unquoted: strip trailing comment + return raw.split(" #", 1)[0].rstrip() + + +def _quote(value: str) -> str: + return '"' + value.replace("\\", "\\\\").replace('"', '\\"') + '"' + + +@dataclass +class EnvFile: + path: Path + _lines: list[str] = field(default_factory=list) # raw lines, without newline + _index: dict[str, int] = field(default_factory=dict) # key -> line number + + @classmethod + def load(cls, path: Path) -> EnvFile: + env = cls(path) + if path.exists(): + text = path.read_text(encoding="utf-8") + env._lines = text.splitlines() + for i, line in enumerate(env._lines): + m = _LINE.match(line) + if m and not line.lstrip().startswith("#"): + env._index[m.group(1)] = i + return env + + @property + def exists(self) -> bool: + return self.path.exists() + + def get(self, key: str, default: str | None = None) -> str | None: + i = self._index.get(key) + if i is None: + return default + m = _LINE.match(self._lines[i]) + return _unquote(m.group(2)) if m else default + + def as_dict(self) -> dict[str, str]: + return {k: self.get(k) or "" for k in self._index} + + def set(self, key: str, value: str) -> None: + line = f"{key}={_quote(str(value))}" + i = self._index.get(key) + if i is None: + self._lines.append(line) + self._index[key] = len(self._lines) - 1 + else: + self._lines[i] = line + + def merge(self, mapping: Mapping[str, str]) -> None: + for k, v in mapping.items(): + self.set(k, v) + + def remove(self, key: str) -> None: + i = self._index.pop(key, None) + if i is None: + return + del self._lines[i] + self._index = {k: (n - 1 if n > i else n) for k, n in self._index.items()} + + def remove_prefix(self, prefix: str) -> None: + for key in [k for k in self._index if k.startswith(prefix + "_")]: + self.remove(key) + + def save(self) -> None: + self.path.parent.mkdir(parents=True, exist_ok=True) + tmp = self.path.with_suffix(".env.tmp") + tmp.write_text("\n".join(self._lines) + "\n", encoding="utf-8") + os.replace(tmp, self.path) +``` + +- [ ] **Step 2: Vérifier** + +Run: `uv run python -c " +import tempfile; from pathlib import Path +from services.envfile import EnvFile +p = Path(tempfile.mkdtemp())/'.env' +p.write_text('# header\nTZ=\"UTC\"\nEDGE_KEY=\"a=b\"\n\nDB_PG_A1_PORT=\"5433\"\nDB_PG_A1_PASS=\"p\\\"q\"\nCUSTOM=plain # note\n') +e = EnvFile.load(p); print(e.get('TZ'), e.get('EDGE_KEY'), e.get('DB_PG_A1_PASS'), e.get('CUSTOM')) +e.set('TZ','Europe/Paris'); e.merge({'NEW':'x'}); e.remove_prefix('DB_PG_A1'); e.save() +print(p.read_text())"` +Expected: `UTC a=b p"q plain` puis le fichier avec `# header`, `TZ="Europe/Paris"`, `EDGE_KEY`, ligne vide conservée, `CUSTOM` réécrit tel quel, `NEW="x"` en fin, plus aucune `DB_PG_A1_*`. + +- [ ] **Step 3: Commit** + +```bash +git add services/envfile.py +git commit -m "feat(services): add EnvFile preserving order and comments" +``` + +--- + +### Task 2 : `ComposeFacts`, `project.py`, `label_default` + +**Files:** +- Create: `services/compose_facts.py` +- Create: `services/project.py` +- Modify: `engines/base.py` (ajouter `label_default = "External DB"` ; `DockerVolumeEngine.label_default = "Docker Volume"` dans `engines/docker_volume.py`) + +**Interfaces:** +- Produces: + - `ComposeFacts(path)` : `exists`, `is_generated` (en-tête présent), `host_gateway -> bool`, `raw -> dict`. + - `ProjectKind = Literal["agent", "dashboard"]`, `detect_kind(path) -> ProjectKind` (`ConfigError` sinon). + - `spec_from_entry(entry: dict, env: EnvFile) -> DatabaseSpec`. + - `AgentProject(path, env, databases, host_gateway)` : `load(path)`, `create(path, env_vars: dict, host_gateway)`, `managed`, `needs_docker_socket`, `sqlite_mounts`, `add(spec, engine)`, `remove(spec, engine)`, `find(id_or_name) -> DatabaseSpec`, `save_state()` (écrit `.env` seulement ; `databases.json` est rendu). + - `DashboardProject(path, env)` : `load(path)`, `create(path, env_vars)`, `db_mode`, `save_state()`. + +- [ ] **Step 1: `services/compose_facts.py`** + +```python +"""Read-only structural facts from an existing docker-compose.yml. Never writes.""" + +from __future__ import annotations + +from pathlib import Path + +import yaml + +GENERATED_MARKER = "# Generated by Portabase CLI" + + +class ComposeFacts: + def __init__(self, path: Path) -> None: + self.path = path + self.raw: dict = {} + self.text = "" + if path.exists(): + try: + self.text = path.read_text(encoding="utf-8") + loaded = yaml.safe_load(self.text) + self.raw = loaded if isinstance(loaded, dict) else {} + except (OSError, yaml.YAMLError): + self.raw = {} + + @property + def exists(self) -> bool: + return self.path.exists() + + @property + def is_generated(self) -> bool: + return self.text.startswith(GENERATED_MARKER) + + def _service(self, name: str) -> dict: + services = self.raw.get("services") or {} + svc = services.get(name) if isinstance(services, dict) else None + return svc if isinstance(svc, dict) else {} + + @property + def host_gateway(self) -> bool: + extra = self._service("agent").get("extra_hosts") + if isinstance(extra, list): + return any("host-gateway" in str(x) for x in extra) + if isinstance(extra, dict): + return any("host-gateway" in str(v) for v in extra.values()) + return False +``` + +- [ ] **Step 2: `services/project.py`** + +```python +"""Project state loaded from .env + databases.json + compose facts. Nothing else is stored.""" + +from __future__ import annotations + +import json +from dataclasses import dataclass, field +from pathlib import Path +from typing import Any, Literal + +from core.errors import ConfigError, ValidationError +from core.specs import DatabaseSpec +from engines.base import DbEngine +from engines.sqlite import SqliteEngine +from services.compose_facts import ComposeFacts +from services.envfile import EnvFile + +ProjectKind = Literal["agent", "dashboard"] +DATABASES_FILE = "databases.json" +COMPOSE_FILE = "docker-compose.yml" +ENV_FILE = ".env" + + +def detect_kind(path: Path) -> ProjectKind: + if (path / DATABASES_FILE).exists(): + return "agent" + env = EnvFile.load(path / ENV_FILE) + if env.get("PROJECT_SECRET") is not None: + return "dashboard" + raise ConfigError( + f"{path} is not a Portabase agent or dashboard folder.", + hint="Expected databases.json (agent) or a .env with PROJECT_SECRET (dashboard).", + ) + + +def spec_from_entry(entry: dict[str, Any], env: EnvFile) -> DatabaseSpec: + engine = str(entry.get("type", "")) + host = entry.get("host") + managed, host_port, root_password = False, None, None + if host: + prefix = str(host).upper().replace("-", "_") + raw_port = env.get(f"{prefix}_PORT") + if raw_port and raw_port.isdigit(): + managed, host_port = True, int(raw_port) + root_password = env.get(f"{prefix}_ROOT_PASS") + return DatabaseSpec( + id=str(entry.get("generated_id") or DbEngine.new_id()), + engine=engine, + name=str(entry.get("name", "")), + managed=managed, + host=str(host) if host else None, + port=int(entry["port"]) if entry.get("port") not in (None, "") else None, + host_port=host_port, + database=str(entry["database"]) if entry.get("database") is not None else None, + username=str(entry["username"]) if entry.get("username") is not None else None, + password=str(entry["password"]) if entry.get("password") not in (None, "") else None, + root_password=root_password, + path=str(entry["database"]) if engine == "sqlite" and entry.get("database") else None, + volume=str(entry["volume_name"]) if entry.get("volume_name") else None, + container=str(entry["container_name"]) if entry.get("container_name") else None, + options=dict(entry.get("options") or {}), + ) + + +@dataclass +class AgentProject: + path: Path + env: EnvFile + databases: list[DatabaseSpec] = field(default_factory=list) + host_gateway: bool = False + + # ---- construction ----------------------------------------------------- + + @classmethod + def load(cls, path: Path) -> AgentProject: + path = path.resolve() + env_path, db_path = path / ENV_FILE, path / DATABASES_FILE + if not env_path.exists() or not db_path.exists(): + raise ConfigError( + f"Not a Portabase agent folder: {path}", + hint=f"Expected {ENV_FILE} and {DATABASES_FILE}.", + ) + env = EnvFile.load(env_path) + try: + data = json.loads(db_path.read_text(encoding="utf-8")) + except (OSError, ValueError) as e: + raise ConfigError(f"{db_path} is not valid JSON.", cause=e) from e + entries = data.get("databases", []) if isinstance(data, dict) else [] + databases = [spec_from_entry(e, env) for e in entries if isinstance(e, dict)] + project = cls(path, env, databases, ComposeFacts(path / COMPOSE_FILE).host_gateway) + project.validate() + return project + + @classmethod + def create(cls, path: Path, env_vars: dict[str, str], *, host_gateway: bool) -> AgentProject: + path.mkdir(parents=True, exist_ok=True) + env = EnvFile.load(path / ENV_FILE) + env.merge(env_vars) + return cls(path, env, [], host_gateway) + + # ---- derived facts ---------------------------------------------------- + + @property + def managed(self) -> list[DatabaseSpec]: + return [d for d in self.databases if d.managed] + + @property + def needs_docker_socket(self) -> bool: + return any(d.engine == "docker-volume" for d in self.databases) + + @property + def sqlite_mounts(self) -> list[tuple[str, str]]: + mounts = [] + for d in self.databases: + if d.engine == "sqlite": + m = SqliteEngine.mount_for(d) + if m and m not in mounts: + mounts.append(m) + return mounts + + def validate(self) -> None: + seen: set[str] = set() + for d in self.managed: + if d.host in seen: + raise ConfigError(f"Two managed databases share the service name '{d.host}'.") + seen.add(d.host or "") + + # ---- mutation --------------------------------------------------------- + + def add(self, spec: DatabaseSpec, engine: DbEngine) -> None: + if spec.managed: + self.env.merge(engine.env_vars(spec)) + self.databases.append(spec) + self.validate() + + def remove(self, spec: DatabaseSpec, engine: DbEngine) -> None: + self.databases = [d for d in self.databases if d.id != spec.id] + if spec.managed and spec.host: + self.env.remove_prefix(spec.env_prefix) + + def find(self, id_or_name: str) -> DatabaseSpec: + matches = [d for d in self.databases if d.id == id_or_name or d.id.startswith(id_or_name) or d.name == id_or_name] + if not matches: + raise ValidationError(f"No database matching '{id_or_name}'.", hint="See: portabase db list") + if len(matches) > 1: + raise ValidationError(f"'{id_or_name}' matches several databases; use the id.") + return matches[0] + + def save_state(self) -> None: + self.env.save() + + +@dataclass +class DashboardProject: + path: Path + env: EnvFile + + @classmethod + def load(cls, path: Path) -> DashboardProject: + path = path.resolve() + env = EnvFile.load(path / ENV_FILE) + if env.get("PROJECT_SECRET") is None: + raise ConfigError(f"Not a Portabase dashboard folder: {path}", hint="Expected a .env with PROJECT_SECRET.") + return cls(path, env) + + @classmethod + def create(cls, path: Path, env_vars: dict[str, str]) -> DashboardProject: + path.mkdir(parents=True, exist_ok=True) + env = EnvFile.load(path / ENV_FILE) + env.merge(env_vars) + return cls(path, env) + + @property + def db_mode(self) -> Literal["external", "internal", "custom"]: + host = self.env.get("POSTGRES_HOST") + if host is None: + return "internal" + return "external" if host == "db" else "custom" + + @property + def project_name(self) -> str: + return self.env.get("PROJECT_NAME") or self.path.name + + def save_state(self) -> None: + self.env.save() +``` + +- [ ] **Step 3: `label_default`** + +Dans `engines/base.py`, après `has_modes: bool = True` : `label_default: str = "External DB"`. Dans `engines/docker_volume.py`, après `has_modes = False` : `label_default = "Docker Volume"`. Remplacer dans `from_existing` de `base.py` et `sqlite.py` `or "External DB"` par `or self.label_default`, et dans `docker_volume.py` `or "Docker Volume"` par `or self.label_default`. + +- [ ] **Step 4: Vérifier sur une install legacy réelle** + +Générer une install avec le binaire 26.07.6 (télécharger depuis la release GitHub) ou avec `git stash`/checkout du tag : + +```bash +cd /tmp && rm -rf legacy-agent && git -C /home/soluce/Documents/PROJETS/Portabase/cli stash -u -q; git -C /home/soluce/Documents/PROJETS/Portabase/cli checkout -q 26.07.6 +uv --directory /home/soluce/Documents/PROJETS/Portabase/cli run python /home/soluce/Documents/PROJETS/Portabase/cli/main.py agent legacy-agent --key "$(printf '{"serverUrl":"http://x","agentId":"a","masterKeyB64":"k"}' | base64 -w0)" +``` +Au wizard : tz `UTC`, polling `5`, extra_hosts `y`, puis `database` → `new` → `postgresql` (ownership `n`, clean `clean`), puis `database` → `new` → `redis` → `with-auth`, puis `database` → `existing` → `sqlite` → `Display` / path `ext.sqlite`, puis `docker-volume` (`Vol`, `myvol`, container vide), puis `done`, ne pas démarrer. + +```bash +git -C /home/soluce/Documents/PROJETS/Portabase/cli checkout -q main; git -C /home/soluce/Documents/PROJETS/Portabase/cli stash pop -q +uv --directory /home/soluce/Documents/PROJETS/Portabase/cli run python -c " +from pathlib import Path +from services.project import AgentProject, detect_kind +p = AgentProject.load(Path('/tmp/legacy-agent')) +print(detect_kind(p.path), 'gateway:', p.host_gateway, 'socket:', p.needs_docker_socket, 'mounts:', p.sqlite_mounts) +for d in p.databases: print(f'{d.engine:14} managed={d.managed!s:5} host={d.host} host_port={d.host_port} db={d.database} opts={d.options}')" +``` +Expected: `agent gateway: True socket: True mounts: [('./ext.sqlite', '/config/ext.sqlite')]` ; postgresql `managed=True host=db-pg-xxxx host_port=` ; redis `managed=True host=db-redis-auth-xxxx` ; sqlite `managed=False` ; docker-volume `managed=False`. + +- [ ] **Step 5: Commit** + +```bash +git add services/compose_facts.py services/project.py engines/base.py engines/sqlite.py engines/docker_volume.py +git commit -m "feat(services): add ComposeFacts and project state loaded from .env, databases.json and compose" +``` + +--- + +### Task 3 : `ComposeRenderer` et `RenderResult` + +**Files:** +- Create: `services/renderer.py` +- Modify: `services/docker.py` (ajouter `remove_volume`) + +**Interfaces:** +- Consumes: `TemplateRepository`, `EngineRegistry`, `AgentProject`, `DashboardProject`, `SqliteEngine.mount_for`. +- Produces: + - `ComposeRenderer(templates, engines, cli_version)` : `render_agent(project, *, inline=False) -> RenderResult`, `render_dashboard(project, *, inline=False) -> RenderResult`. + - `RenderResult(compose: str, databases: list[dict] | None)` : `validate()` (`TemplateError`), `write(path) -> WriteReport`, `diff_against(path) -> str`. + - `WriteReport(backed_up: Path | None, wrote: list[Path])`. + - `DockerRunner.remove_volume(name) -> None`. + +- [ ] **Step 1: `services/renderer.py`** + +```python +"""State → docker-compose.yml (+ databases.json). The only writer of those files.""" + +from __future__ import annotations + +import difflib +import json +import os +import shutil +from dataclasses import dataclass, field +from pathlib import Path +from typing import Any + +import jinja2 +import yaml + +from core.errors import TemplateError +from core.specs import DatabaseSpec +from engines.registry import EngineRegistry +from services.compose_facts import GENERATED_MARKER, ComposeFacts +from services.envfile import EnvFile +from services.project import COMPOSE_FILE, DATABASES_FILE, AgentProject, DashboardProject +from services.templates import TemplateRepository + +LEGACY_BACKUP = "docker-compose.legacy.yml" + + +@dataclass +class WriteReport: + backed_up: Path | None = None + wrote: list[Path] = field(default_factory=list) + + +@dataclass +class RenderResult: + compose: str + databases: list[dict[str, Any]] | None = None + + def validate(self) -> None: + try: + doc = yaml.safe_load(self.compose) + except yaml.YAMLError as e: + raise TemplateError("Rendered compose is not valid YAML; templates are broken.", cause=e) from e + if not isinstance(doc, dict) or "services" not in doc: + raise TemplateError("Rendered compose has no 'services' section; templates are broken.") + + def write(self, path: Path) -> WriteReport: + self.validate() + report = WriteReport() + compose_path = path / COMPOSE_FILE + facts = ComposeFacts(compose_path) + if facts.exists and not facts.is_generated: + backup = path / LEGACY_BACKUP + if not backup.exists(): + shutil.copy2(compose_path, backup) + report.backed_up = backup + _atomic_write(compose_path, self.compose) + report.wrote.append(compose_path) + if self.databases is not None: + db_path = path / DATABASES_FILE + _atomic_write(db_path, json.dumps({"databases": self.databases}, indent=2) + "\n") + try: + os.chmod(db_path, 0o666) # agent container may run as another uid (legacy behaviour) + except OSError: + pass + report.wrote.append(db_path) + return report + + def diff_against(self, path: Path) -> str: + current = (path / COMPOSE_FILE).read_text(encoding="utf-8") if (path / COMPOSE_FILE).exists() else "" + return "".join( + difflib.unified_diff( + current.splitlines(keepends=True), + self.compose.splitlines(keepends=True), + fromfile=f"{COMPOSE_FILE} (current)", + tofile=f"{COMPOSE_FILE} (rendered)", + ) + ) + + +def _atomic_write(path: Path, content: str) -> None: + path.parent.mkdir(parents=True, exist_ok=True) + tmp = path.with_suffix(path.suffix + ".tmp") + tmp.write_text(content, encoding="utf-8") + os.replace(tmp, path) + + +class ComposeRenderer: + def __init__(self, templates: TemplateRepository, engines: EngineRegistry, cli_version: str) -> None: + self.templates = templates + self.engines = engines + self.cli_version = cli_version + + def header(self) -> str: + return f"{GENERATED_MARKER} {self.cli_version}. Do not edit — use docker-compose.override.yml.\n" + + # ---- agent ------------------------------------------------------------ + + def render_agent(self, project: AgentProject, *, inline: bool = False) -> RenderResult: + env = project.env + ctx = { + "host_gateway": project.host_gateway, + "docker_socket": project.needs_docker_socket, + "mounts": [{"host": h, "container": c} for h, c in project.sqlite_mounts], + "services": [self._service(spec, inline) for spec in project.managed], + "tz_var": _var(env, "TZ", inline), + "edge_key_var": _var(env, "EDGE_KEY", inline), + "log_level_var": _var(env, "LOG_LEVEL", inline), + "polling_var": _var(env, "POLLING", inline), + } + compose = self.header() + self._render("agent.yml.j2", ctx) + databases = [self.engines.get(d.engine).agent_entry(d) for d in project.databases] + return RenderResult(compose=compose, databases=databases) + + def _service(self, spec: DatabaseSpec, inline: bool) -> dict[str, str]: + engine = self.engines.get(spec.engine) + body = self._render_template(self.templates.engine_template(spec.engine), engine.template_ctx(spec, inline=inline)) + return {"name": spec.host or "", "volume": f"{spec.host}-data", "body": body} + + # ---- dashboard -------------------------------------------------------- + + def render_dashboard(self, project: DashboardProject, *, inline: bool = False) -> RenderResult: + env = project.env + ctx = { + "db_mode": project.db_mode, + "project_name_var": project.project_name, # literal, as the legacy CLI did + "host_port_var": _var(env, "HOST_PORT", inline), + "tz_var": _var(env, "TZ", inline), + "log_level_var": _var(env, "LOG_LEVEL", inline), + "project_secret_var": _var(env, "PROJECT_SECRET", inline), + "project_url_var": _var(env, "PROJECT_URL", inline), + "pg_port_var": _var(env, "PG_PORT", inline), + "postgres_db_var": _var(env, "POSTGRES_DB", inline), + "postgres_user_var": _var(env, "POSTGRES_USER", inline), + "postgres_password_var": _var(env, "POSTGRES_PASSWORD", inline), + } + return RenderResult(compose=self.header() + self._render("dashboard.yml.j2", ctx), databases=None) + + # ---- internals -------------------------------------------------------- + + def _render(self, name: str, ctx: dict[str, Any]) -> str: + return self._render_template(self.templates.get(name), ctx) + + @staticmethod + def _render_template(template: jinja2.Template, ctx: dict[str, Any]) -> str: + try: + return template.render(**ctx) + except jinja2.TemplateError as e: + raise TemplateError(f"Template rendering failed: {e}", cause=e) from e + + +def _var(env: EnvFile, key: str, inline: bool) -> str: + return (env.get(key) or "") if inline else f"${{{key}}}" +``` + +- [ ] **Step 2: `DockerRunner.remove_volume`** (dans `services/docker.py`, après `ensure_network`) + +```python + def remove_volume(self, name: str) -> bool: + """True if removed, False if it did not exist. Raises on other failures.""" + proc = subprocess.run([self.binary, "volume", "rm", name], capture_output=True, text=True, check=False) + if proc.returncode == 0: + return True + if "no such volume" in (proc.stderr or "").lower(): + return False + raise DockerError(f"Could not remove volume '{name}': {proc.stderr.strip()}") +``` + +- [ ] **Step 3: Vérifier le rendu sur l'install legacy et le `--diff`** + +Run: `uv run python -c " +from pathlib import Path +from services.project import AgentProject +from services.renderer import ComposeRenderer +from services.templates import TemplateRepository +from services.http import HttpClient +from core.config import GlobalConfig +from engines import registry +import yaml +repo = TemplateRepository.from_environment(HttpClient(), GlobalConfig()) +r = ComposeRenderer(repo, registry, '0.0.0-dev') +p = AgentProject.load(Path('/tmp/legacy-agent')) +res = r.render_agent(p); res.validate() +doc = yaml.safe_load(res.compose) +print(sorted(doc['services']), doc['services']['agent']['volumes'], doc['services']['agent'].get('extra_hosts')) +print(res.diff_against(p.path)[:1200]) +print(len(res.databases), [d['type'] for d in res.databases])"` +Expected: services = `agent` + le service postgres + le service redis (mêmes noms que le compose legacy) ; volumes agent = `databases.json`, `./ext.sqlite:/config/ext.sqlite`, socket ; `extra_hosts` présent ; diff limité à l'en-tête, l'ordre des lignes, `restart: unless-stopped` sur redis ; `4 ['postgresql', 'redis', 'sqlite', 'docker-volume']`. + +Comparer aussi `res.databases` à `/tmp/legacy-agent/databases.json` : mêmes clés et valeurs par entrée (à l'ordre des clés près). + +- [ ] **Step 4: Commit** + +```bash +git add services/renderer.py services/docker.py +git commit -m "feat(services): add ComposeRenderer with validation, atomic write and legacy backup" +``` + +--- + +### Task 4 : Composants `Summary`, `DataTable`, `Diff` + +**Files:** +- Create: `ui/components/summary.py`, `ui/components/table.py`, `ui/components/diff.py` +- Modify: `ui/__init__.py` + +**Interfaces:** +- Produces: `UI.summary(rows: list[tuple[str, str]], *, title: str | None = None)`, `UI.table(columns: list[str], rows: list[list[str]], *, title: str | None = None)`, `UI.diff(text: str)`. + +- [ ] **Step 1: `ui/components/summary.py`** + +```python +from __future__ import annotations + +import re + +from rich.panel import Panel +from rich.table import Table + +from ui.components.base import Component + +_SENSITIVE = re.compile(r"(password|secret|key|token)", re.I) +_URL_CREDS = re.compile(r"://([^:/@]+):([^@/]+)@") + + +def mask(label: str, value: str) -> str: + if _SENSITIVE.search(label): + return "••••••••" + return _URL_CREDS.sub(r"://\1:****@", value) + + +class Summary(Component): + def __call__(self, rows: list[tuple[str, str]], *, title: str | None = None) -> None: + table = Table(show_header=False, box=None, padding=(0, 2)) + table.add_column("Property", style="bold cyan") + table.add_column("Value", style="white") + for label, value in rows: + table.add_row(label, mask(label, str(value))) + self.console.print("") + self.console.print(Panel(table, title=f"[bold white]{title}[/bold white]" if title else None, border_style="bold blue", expand=False)) +``` + +- [ ] **Step 2: `ui/components/table.py`** + +```python +from __future__ import annotations + +from rich.table import Table + +from ui.components.base import Component + +_STYLES = ["cyan", "blue", "magenta", "green", "white", "dim"] + + +class DataTable(Component): + def __call__(self, columns: list[str], rows: list[list[str]], *, title: str | None = None) -> None: + table = Table(title=title) + for i, col in enumerate(columns): + table.add_column(col, style=_STYLES[i % len(_STYLES)]) + for row in rows: + table.add_row(*[str(c) for c in row]) + self.console.print(table) +``` + +- [ ] **Step 3: `ui/components/diff.py`** + +```python +from __future__ import annotations + +from rich.syntax import Syntax + +from ui.components.base import Component + + +class Diff(Component): + def __call__(self, text: str) -> None: + if not text.strip(): + self.console.print("[info]ℹ No changes.[/info]") + return + self.console.print(Syntax(text, "diff", theme="ansi_dark", word_wrap=False)) +``` + +- [ ] **Step 4: Façade** — ajouter à `ui/__init__.py` les imports et méthodes : + +```python +from ui.components.diff import Diff +from ui.components.summary import Summary +from ui.components.table import DataTable + + def summary(self, rows: list[tuple[str, str]], *, title: str | None = None) -> None: + Summary(self.console)(rows, title=title) + + def table(self, columns: list[str], rows: list[list[str]], *, title: str | None = None) -> None: + DataTable(self.console)(columns, rows, title=title) + + def diff(self, text: str) -> None: + Diff(self.console)(text) +``` + +- [ ] **Step 5: Vérifier** + +Run: `uv run python -c " +from ui import UI +ui = UI() +ui.summary([('Name','x'),('Password','hunter2'),('Connection URL','postgresql://u:p@h:5432/d')], title='PROPOSED') +ui.table(['A','B'], [['1','2']], title='T') +ui.diff('--- a\n+++ b\n@@ -1 +1 @@\n-old\n+new\n'); ui.diff('')"` +Expected: panneau avec `••••••••` et `:****@`, table, diff colorisé, `ℹ No changes.`. + +- [ ] **Step 6: Commit** + +```bash +git add ui/ +git commit -m "feat(ui): add Summary, DataTable and Diff components" +``` + +--- + +### Task 5 : `AddDatabaseFlow` + +**Files:** +- Create: `commands/flows/__init__.py` (vide) +- Create: `commands/flows/add_database.py` + +**Interfaces:** +- Consumes: `UI`, `EngineRegistry`, `PortAllocator`, `Form`, `Field`, `DatabaseSpec`, `AgentProject`. +- Produces: `AddDatabaseFlow(ui, engines, ports)` : `collect(values: dict) -> tuple[DatabaseSpec, DbEngine]`, `apply(project, spec, engine) -> None`, `parse_options(items: list[str]) -> dict[str, str]` (static). + +- [ ] **Step 1: Écrire le module** + +```python +"""Shared 'add a database' wizard. Flags fill `values`; anything missing is prompted or errors.""" + +from __future__ import annotations + +from typing import Any + +from core.errors import ValidationError +from core.fields import Field +from core.specs import DatabaseSpec +from engines.base import DbEngine +from engines.registry import EngineRegistry +from services.ports import PortAllocator +from services.project import AgentProject +from ui import UI + +FLOW_KEYS = {"engine", "mode", "auth", "label", "options"} + + +class AddDatabaseFlow: + def __init__(self, ui: UI, engines: EngineRegistry, ports: PortAllocator) -> None: + self.ui = ui + self.engines = engines + self.ports = ports + + # ---- public ----------------------------------------------------------- + + @staticmethod + def parse_options(items: list[str] | None) -> dict[str, str]: + out: dict[str, str] = {} + for item in items or []: + if "=" not in item: + raise ValidationError(f"Invalid option '{item}'.", hint="Use -o KEY=VALUE") + key, value = item.split("=", 1) + out[key.strip()] = value.strip() + return out + + def collect(self, values: dict[str, Any]) -> tuple[DatabaseSpec, DbEngine]: + form = self.ui.form() + engine = self.engines.get( + form.choice("Select Database Engine", self.engines.choices(), value=values.get("engine"), name="engine") + ) + if engine.warning: + self.ui.warning(engine.warning) + + mode = "new" + if engine.has_modes: + mode = form.choice("Configuration Mode", ["new", "existing"], value=values.get("mode"), default="new", name="mode") + + auth = True + if mode == "new" and engine.auth_variants: + raw = values.get("auth") + if raw is None: + auth = form.choice("Variant", ["with-auth", "no-auth"], name="auth") == "with-auth" + else: + auth = bool(raw) + + fields = list(engine.fields_new() if mode == "new" else engine.fields_existing()) + if mode == "existing" or not engine.has_modes: + fields.insert(0, Field("label", "Display Name", "text", default=engine.label_default)) + + self._reject_irrelevant(values, fields, engine, mode) + + if mode == "existing": + self.ui.info(f"{engine.display} — existing database") + answers = form.collect(fields, values) + answers["options"] = self._collect_options(form, engine, values.get("options") or {}) + + if mode == "new": + spec = engine.generate(auth=auth, ports=self.ports, answers=answers) + else: + spec = engine.from_existing(answers) + return spec.with_options(answers["options"]), engine + + def apply(self, project: AgentProject, spec: DatabaseSpec, engine: DbEngine) -> None: + project.add(spec, engine) + + # ---- internals -------------------------------------------------------- + + def _collect_options(self, form, engine: DbEngine, provided: dict[str, str]) -> dict[str, Any]: + option_fields = engine.option_fields() + known = {f.name for f in option_fields} + unknown = set(provided) - known + if unknown: + raise ValidationError( + f"Unknown option(s) for {engine.key}: {', '.join(sorted(unknown))}.", + hint=("Valid options: " + ", ".join(sorted(known))) if known else f"{engine.key} has no options.", + ) + if not option_fields: + return {} + return form.collect(option_fields, provided) + + @staticmethod + def _reject_irrelevant(values: dict[str, Any], fields: list[Field], engine: DbEngine, mode: str) -> None: + relevant = {f.name for f in fields} | FLOW_KEYS + extra = sorted(k for k, v in values.items() if v is not None and k not in relevant) + if extra: + raise ValidationError( + f"Option(s) not applicable to {engine.key} in '{mode}' mode: {', '.join('--' + k.replace('_', '-') for k in extra)}.", + hint="Applicable: " + ", ".join(f"--{f.name.replace('_', '-')}" for f in fields) if fields else "No extra input needed.", + ) +``` + +- [ ] **Step 2: Vérifier en non-interactif** + +Run: `uv run python -c " +from ui import UI +from engines import registry +from services.ports import FixedPortAllocator +from commands.flows.add_database import AddDatabaseFlow +from core.errors import ValidationError +f = AddDatabaseFlow(UI(non_interactive=True), registry, FixedPortAllocator()) +s, e = f.collect({'engine':'postgresql','mode':'new','options': f.parse_options(['clean_mode=none'])}); print(e.key, s.managed, s.options) +s, e = f.collect({'engine':'redis','mode':'new','auth':False}); print(s.host[:9], s.auth) +s, e = f.collect({'engine':'sqlite','mode':'existing','path':'x.sqlite'}); print(s.name, s.database) +s, e = f.collect({'engine':'docker-volume','volume':'v'}); print(s.name, s.volume, s.container) +for bad in ({'engine':'postgresql','mode':'existing'}, {'engine':'redis','mode':'new','host':'h'}, {'engine':'mysql','mode':'new','options':{'clean_mode':'x'}}, {'engine':'nope'}): + try: f.collect(bad) + except ValidationError as err: print('ERR', err.message)"` +Expected: +``` +postgresql True {'clean_mode': 'none', 'keep_ownership': False} +db-redis- False +External DB /config/x.sqlite +Docker Volume v None +ERR Missing --host +ERR Option(s) not applicable to redis in 'new' mode: --host. +ERR Unknown option(s) for mysql: clean_mode. +ERR Unknown engine 'nope'. +``` + +- [ ] **Step 3: Commit** + +```bash +git add commands/flows/ +git commit -m "feat(commands): add AddDatabaseFlow shared by agent and db add" +``` + +--- + +### Task 6 : `commands/db.py` — `add`, `remove`, `list` + +**Files:** +- Modify: `commands/db.py` (réécriture complète) + +**Interfaces:** +- Produces: `DbCommands(ui, telemetry, engines, ports, templates, renderer, docker)` groupe `db` avec `DbAddCommand`, `DbRemoveCommand`, `DbListCommand`. + +- [ ] **Step 1: Réécrire le module** + +```python +"""db add / remove / list.""" + +from __future__ import annotations + +from pathlib import Path +from typing import Annotated + +import typer + +from commands.base import Command, CommandGroup +from commands.flows.add_database import AddDatabaseFlow +from core.errors import ValidationError +from engines.registry import EngineRegistry +from services.docker import DockerRunner +from services.ports import PortAllocator +from services.project import AgentProject +from services.renderer import ComposeRenderer, WriteReport +from services.telemetry import Telemetry +from services.templates import TemplateRepository +from ui import UI + +NameArg = Annotated[Path, typer.Argument(help="Agent folder")] + + +def report_write(ui: UI, report: WriteReport) -> None: + if report.backed_up: + ui.warning(f"Legacy compose backed up to {report.backed_up.name}. Manual edits belong in docker-compose.override.yml.") + + +class _DbCommand(Command): + panel = "Configuration" + no_args_is_help = True + + def __init__(self, ui, telemetry, engines: EngineRegistry, ports: PortAllocator, templates: TemplateRepository, renderer: ComposeRenderer, docker: DockerRunner) -> None: + super().__init__(ui, telemetry) + self.engines, self.ports, self.templates, self.renderer, self.docker = engines, ports, templates, renderer, docker + + def render_and_write(self, project: AgentProject) -> None: + with self.ui.status("Rendering configuration..."): + result = self.renderer.render_agent(project) + project.save_state() + report = result.write(project.path) + report_write(self.ui, report) + + +class DbAddCommand(_DbCommand): + name, help = "add", "Add a database to an agent." + + def run( + self, + name: NameArg, + engine: Annotated[str | None, typer.Option("--engine", "-e", help="Database engine")] = None, + mode: Annotated[str | None, typer.Option("--mode", help="new (container) or existing")] = None, + auth: Annotated[bool | None, typer.Option("--auth/--no-auth", help="Auth variant for mongodb/redis/valkey")] = None, + label: Annotated[str | None, typer.Option("--label", help="Display name")] = None, + host: Annotated[str | None, typer.Option("--host")] = None, + port: Annotated[int | None, typer.Option("--port")] = None, + database: Annotated[str | None, typer.Option("--database")] = None, + user: Annotated[str | None, typer.Option("--user")] = None, + password: Annotated[str | None, typer.Option("--password", help="Prefer --password-stdin")] = None, + password_stdin: Annotated[bool, typer.Option("--password-stdin", help="Read password from stdin")] = False, + path: Annotated[str | None, typer.Option("--path", help="SQLite file path (existing)")] = None, + db_name: Annotated[str | None, typer.Option("--name", help="SQLite file name (new)")] = None, + volume: Annotated[str | None, typer.Option("--volume", help="Docker volume name")] = None, + container: Annotated[str | None, typer.Option("--container", help="Container to restart after restore")] = None, + option: Annotated[list[str] | None, typer.Option("--option", "-o", help="Engine option KEY=VALUE (repeatable)")] = None, + ) -> None: + if password_stdin: + import sys + + password = sys.stdin.readline().rstrip("\n") + elif password is not None: + self.ui.warning("--password is visible in shell history; prefer --password-stdin.") + + project_path = self.require_project_dir(name) + self.templates.resolve() + project = AgentProject.load(project_path) + + flow = AddDatabaseFlow(self.ui, self.engines, self.ports) + values = { + "engine": engine, "mode": mode, "auth": auth, "label": label, "host": host, "port": port, + "database": database, "username": user, "password": password, "path": path, "name": db_name, + "volume": volume, "container": container, "options": flow.parse_options(option), + } + spec, eng = flow.collect(values) + flow.apply(project, spec, eng) + self.render_and_write(project) + + self.ui.success(f"Added {eng.display} database '{spec.name}' ({eng.describe(spec)}).") + self.ui.info(f"Restart the agent to apply changes: portabase restart {project_path.name}") + + +class DbRemoveCommand(_DbCommand): + name, help = "remove", "Remove a database from an agent." + + def run( + self, + name: NameArg, + target: Annotated[str | None, typer.Option("--id", "--name", "-i", help="Database id (or prefix) or display name")] = None, + purge_volume: Annotated[bool, typer.Option("--purge-volume", help="Also delete the Docker volume of a managed database")] = False, + yes: Annotated[bool, typer.Option("--yes", "-y", help="Skip confirmation")] = False, + ) -> None: + project_path = self.require_project_dir(name) + self.templates.resolve() + project = AgentProject.load(project_path) + if not project.databases: + self.ui.warning("No databases to remove.") + return + + if target is None: + choices = [f"{d.name} ({d.engine}) [{d.id[:8]}]" for d in project.databases] + picked = self.ui.form().choice("Which database to remove?", choices, name="id") + spec = project.databases[choices.index(picked)] + else: + spec = project.find(target) + engine = self.engines.get(spec.engine) + + if not yes: + extra = " and its Docker volume" if (purge_volume and spec.managed) else "" + self.confirm_or_abort(f"Remove '{spec.name}' ({engine.describe(spec)}){extra}?", default=False) + + project.remove(spec, engine) + self.render_and_write(project) + self.ui.success(f"Removed {spec.name}") + + if spec.managed: + volume_name = f"{self.docker.project_name(project_path)}_{spec.host}-data" + if purge_volume: + self.require_docker(self.docker) + removed = self.docker.remove_volume(volume_name) + self.ui.success(f"Deleted volume {volume_name}" if removed else f"Volume {volume_name} did not exist") + else: + self.ui.info(f"Data volume kept: {volume_name}. Delete it with: docker volume rm {volume_name}") + self.ui.info(f"Restart the agent to apply changes: portabase restart {project_path.name}") + + +class DbListCommand(_DbCommand): + name, help = "list", "List the databases of an agent." + + def run(self, name: NameArg) -> None: + project = AgentProject.load(self.require_project_dir(name)) + if not project.databases: + self.ui.warning("No databases configured.") + return + rows = [] + for d in project.databases: + engine = self.engines.get(d.engine) + opts = ", ".join(f"{k}={v}" for k, v in engine.non_default_options(d).items()) + rows.append([d.name, d.database or "", d.engine, engine.describe(d), d.username or "" if d.engine not in ("sqlite", "docker-volume") else "N/A", opts, d.id[:8] + "..."]) + self.ui.table(["Display Name", "Database", "Type", "Host:Port", "User", "Options", "ID"], rows, title=f"Databases for {project.path.name}") + + +class DbCommands(CommandGroup): + name, help, panel = "db", "Manage the databases of an agent.", "Configuration" + + def __init__(self, ui: UI, telemetry: Telemetry, engines: EngineRegistry, ports: PortAllocator, templates: TemplateRepository, renderer: ComposeRenderer, docker: DockerRunner) -> None: + super().__init__(ui, telemetry) + self._deps = (ui, telemetry, engines, ports, templates, renderer, docker) + + @property + def commands(self) -> list[Command]: + return [DbAddCommand(*self._deps), DbRemoveCommand(*self._deps), DbListCommand(*self._deps)] +``` + +Note : `sys` importé localement dans `run` pour `--password-stdin` ; déplacer en tête de module si ruff le demande. + +- [ ] **Step 2: Commit** (vérification à Task 9, une fois `main.py` câblé) + +```bash +git add commands/db.py +git commit -m "feat(commands): rewrite db add/remove/list on the declarative renderer" +``` + +--- + +### Task 7 : `commands/agent.py` et `commands/dashboard.py` + +**Files:** +- Modify: `commands/agent.py` (réécriture complète) +- Modify: `commands/dashboard.py` (réécriture complète) + +**Interfaces:** +- Produces: `AgentCommand(ui, telemetry, docker, templates, renderer, engines, ports)` ; `DashboardCommand(ui, telemetry, docker, templates, renderer, ports)`. + +- [ ] **Step 1: `commands/agent.py`** + +```python +"""portabase agent NAME — create an agent folder. Databases are added by db add (or the interactive loop).""" + +from __future__ import annotations + +from pathlib import Path +from typing import Annotated + +import typer + +from commands.base import Command +from commands.db import report_write +from commands.flows.add_database import AddDatabaseFlow +from core.errors import ValidationError +from core.utils import validate_edge_key +from engines.registry import EngineRegistry +from services.docker import DockerRunner +from services.ports import PortAllocator +from services.project import AgentProject +from services.renderer import ComposeRenderer +from services.telemetry import Telemetry +from services.templates import TemplateRepository +from ui import UI + +NETWORK = "portabase_network" + + +def _edge_key(value: str) -> str: + if not validate_edge_key(value): + raise ValidationError("Invalid Edge Key.", hint="Expected Base64 or JSON with serverUrl, agentId, masterKeyB64.") + return value + + +class AgentCommand(Command): + name, help, panel = "agent", "Create a new Portabase Agent instance.", "Creation" + no_args_is_help = True + + def __init__(self, ui: UI, telemetry: Telemetry, docker: DockerRunner, templates: TemplateRepository, renderer: ComposeRenderer, engines: EngineRegistry, ports: PortAllocator) -> None: + super().__init__(ui, telemetry) + self.docker, self.templates, self.renderer, self.engines, self.ports = docker, templates, renderer, engines, ports + + def run( + self, + name: Annotated[str, typer.Argument(help="Agent name (creates a folder)")], + key: Annotated[str | None, typer.Option("--key", "-k", help="Edge Key")] = None, + tz: Annotated[str | None, typer.Option("--tz", help="Timezone")] = None, + polling: Annotated[int | None, typer.Option("--polling", help="Polling frequency in seconds")] = None, + host_gateway: Annotated[bool | None, typer.Option("--host-gateway/--no-host-gateway", help="Map localhost to host-gateway")] = None, + start: Annotated[bool, typer.Option("--start", "-s", help="Start immediately")] = False, + force: Annotated[bool, typer.Option("--force", "-f", help="Overwrite an existing folder")] = False, + ) -> None: + self.ui.banner() + self.require_docker(self.docker) + self.docker.ensure_network(NETWORK) + self.templates.resolve() + + path = Path(name).resolve() + if path.exists() and not force: + self.ui.warning(f"Directory '{name}' already exists.") + self.confirm_or_abort("Overwrite?", default=False) + + form = self.ui.form() + env_vars = { + "EDGE_KEY": form.text("Edge Key", value=key, validator=_edge_key, name="key"), + "TZ": form.text("Timezone", value=tz, default="UTC", name="tz"), + "POLLING": str(form.integer("Polling frequency (seconds)", value=polling, default=5, name="polling")), + "LOG_LEVEL": "info", + } + gateway = form.confirm("Add extra_hosts mapping (localhost -> host-gateway)?", value=host_gateway, default=False, name="host_gateway") + + project = AgentProject.create(path, env_vars, host_gateway=gateway) + self._write(project) + self.ui.success(f"Agent '{name}' created in {path}") + + if not self.ui.non_interactive: + self.ui.section("Database Setup") + flow = AddDatabaseFlow(self.ui, self.engines, self.ports) + while self.ui.confirm("Add a database?", default=True): + spec, engine = flow.collect({}) + flow.apply(project, spec, engine) + self._write(project) + self.ui.success(f"Added {engine.display} '{spec.name}' ({engine.describe(spec)})") + else: + self.ui.hint(f"Add databases with: portabase db add {name} --engine postgresql --mode new") + + if start or (not self.ui.non_interactive and self.ui.confirm("Start agent now?", default=False)): + with self.ui.status("Starting agent..."): + self.docker.compose(path, ["up", "-d"]) + self.ui.success("Agent started.") + else: + self.ui.info(f"Run: portabase start {name}") + + def _write(self, project: AgentProject) -> None: + with self.ui.status("Rendering configuration..."): + result = self.renderer.render_agent(project) + project.save_state() + report = result.write(project.path) + report_write(self.ui, report) +``` + +- [ ] **Step 2: `commands/dashboard.py`** + +```python +"""portabase dashboard NAME — create a dashboard folder.""" + +from __future__ import annotations + +import secrets +from pathlib import Path +from typing import Annotated +from urllib.parse import quote + +import typer + +from commands.base import Command +from commands.db import report_write +from core.utils import generate_password, slugify_project_name +from services.docker import DockerRunner +from services.ports import PortAllocator +from services.project import DashboardProject +from services.renderer import ComposeRenderer +from services.telemetry import Telemetry +from services.templates import TemplateRepository +from ui import UI + +DB_MODES = ("external", "internal", "custom") +MODE_LABELS = { + "external": "Dedicated Docker Container (Recommended)", + "internal": "Embedded Database (In-container)", + "custom": "Custom/Existing Database", +} + + +class DashboardCommand(Command): + name, help, panel = "dashboard", "Create a new Portabase Dashboard instance.", "Creation" + no_args_is_help = True + + def __init__(self, ui: UI, telemetry: Telemetry, docker: DockerRunner, templates: TemplateRepository, renderer: ComposeRenderer, ports: PortAllocator) -> None: + super().__init__(ui, telemetry) + self.docker, self.templates, self.renderer, self.ports = docker, templates, renderer, ports + + def run( + self, + name: Annotated[str, typer.Argument(help="Dashboard name (creates a folder)")], + port: Annotated[int | None, typer.Option("--port", help="Web port")] = None, + db_mode: Annotated[str | None, typer.Option("--db-mode", help="external | internal | custom")] = None, + db_host: Annotated[str | None, typer.Option("--db-host")] = None, + db_port: Annotated[int | None, typer.Option("--db-port")] = None, + db_name: Annotated[str | None, typer.Option("--db-name")] = None, + db_user: Annotated[str | None, typer.Option("--db-user")] = None, + db_password_stdin: Annotated[bool, typer.Option("--db-password-stdin", help="Read the custom DB password from stdin")] = False, + tz: Annotated[str | None, typer.Option("--tz")] = None, + start: Annotated[bool, typer.Option("--start", "-s")] = False, + force: Annotated[bool, typer.Option("--force", "-f")] = False, + yes: Annotated[bool, typer.Option("--yes", "-y", help="Skip the configuration confirmation")] = False, + ) -> None: + self.ui.banner() + self.require_docker(self.docker) + self.templates.resolve() + + path = Path(name).resolve() + if path.exists() and not force: + self.ui.warning(f"Directory '{name}' already exists.") + self.confirm_or_abort("Overwrite?", default=False) + + form = self.ui.form() + web_port = form.integer("Web Port", value=port, default=8887, name="port") + mode = form.choice("Database Setup", list(DB_MODES), value=db_mode, default="external", name="db_mode") + project_name = slugify_project_name(path.name) + + env_vars = { + "HOST_PORT": str(web_port), + "PROJECT_SECRET": secrets.token_hex(32), + "PROJECT_URL": f"http://localhost:{web_port}", + "PROJECT_NAME": project_name, + "TZ": form.text("Timezone", value=tz, default="Europe/Paris", name="tz"), + "LOG_LEVEL": "info", + } + rows = [("Dashboard Name", name), ("Path", str(path)), ("Access URL", env_vars["PROJECT_URL"]), ("Database Setup", MODE_LABELS[mode])] + + if mode == "external": + pg_pass, pg_port = generate_password(16), self.ports.free() + env_vars.update(self._pg_env("portabase", "portabase", pg_pass, "db", 5432, pg_port)) + rows.append(("Internal Port", str(pg_port))) + elif mode == "custom": + self.ui.info("External Database Configuration") + host = form.text("Host", value=db_host, default="localhost", name="db_host") + dport = form.integer("Port", value=db_port, default=5432, name="db_port") + dbname = form.text("Database Name", value=db_name, default="portabase", name="db_name") + user = form.text("Username", value=db_user, name="db_user") + if db_password_stdin: + import sys + + password = sys.stdin.readline().rstrip("\n") + else: + password = form.secret("Password", name="db_password") + env_vars.update(self._pg_env(dbname, user, password, host, dport, dport)) + rows += [("DB Host", host), ("DB Name", dbname), ("Connection URL", env_vars["DATABASE_URL"])] + + rows.append(("Files to Create", "docker-compose.yml, .env")) + self.ui.summary(rows, title="PROPOSED CONFIGURATION") + if not yes: + self.confirm_or_abort("Apply this configuration and generate files?", default=True) + + project = DashboardProject.create(path, env_vars) + with self.ui.status("Rendering configuration..."): + result = self.renderer.render_dashboard(project) + project.save_state() + report = result.write(path) + report_write(self.ui, report) + self.ui.success(f"Dashboard '{name}' created in {path}") + + if start or (not self.ui.non_interactive and self.ui.confirm("Start dashboard now?", default=False)): + with self.ui.status("Starting..."): + self.docker.compose(path, ["up", "-d"]) + self.ui.success(f"Live at: {env_vars['PROJECT_URL']}") + else: + self.ui.info(f"Run: portabase start {name}") + + @staticmethod + def _pg_env(db: str, user: str, password: str, host: str, port: int, host_port: int) -> dict[str, str]: + return { + "POSTGRES_DB": db, + "POSTGRES_USER": user, + "POSTGRES_PASSWORD": password, + "POSTGRES_HOST": host, + "DATABASE_URL": f"postgresql://{quote(user, safe='')}:{quote(password, safe='')}@{host}:{port}/{db}?schema=public", + "PG_PORT": str(host_port), + } +``` + +`--yes` en non-interactif : `confirm_or_abort(default=True)` renvoie `True` sans prompt, donc `--yes` n'est nécessaire que pour sauter l'affichage ; conservé pour la lisibilité des scripts. + +- [ ] **Step 3: Commit** + +```bash +git add commands/agent.py commands/dashboard.py +git commit -m "feat(commands): rewrite agent and dashboard on the declarative renderer" +``` + +--- + +### Task 8 : `commands/build.py` + +**Files:** +- Create: `commands/build.py` + +**Interfaces:** +- Produces: `BuildCommand(ui, telemetry, templates, renderer)`. + +- [ ] **Step 1: Écrire le module** + +```python +"""portabase build PATH — re-render compose from state. Also the legacy migration entry point.""" + +from __future__ import annotations + +from pathlib import Path +from typing import Annotated + +import typer + +from commands.base import Command +from commands.db import report_write +from core.errors import ValidationError +from services.project import COMPOSE_FILE, DATABASES_FILE, ENV_FILE, AgentProject, DashboardProject, detect_kind +from services.renderer import ComposeRenderer, RenderResult +from services.telemetry import Telemetry +from services.templates import TemplateRepository +from ui import UI + + +class BuildCommand(Command): + name, help, panel = "build", "Re-render docker-compose.yml from the component's configuration.", "Configuration" + no_args_is_help = True + + def __init__(self, ui: UI, telemetry: Telemetry, templates: TemplateRepository, renderer: ComposeRenderer) -> None: + super().__init__(ui, telemetry) + self.templates, self.renderer = templates, renderer + + def run( + self, + path: Annotated[Path, typer.Argument(help="Component folder")], + diff: Annotated[bool, typer.Option("--diff", help="Show the diff, write nothing")] = False, + stdout: Annotated[bool, typer.Option("--stdout", help="Print the compose, write nothing")] = False, + inline_env: Annotated[bool, typer.Option("--inline-env", help="Substitute values instead of ${VAR} references")] = False, + output: Annotated[Path | None, typer.Option("--output", "-o", help="Write files to another directory")] = None, + ) -> None: + if sum([diff, stdout, output is not None]) > 1: + raise ValidationError("Use only one of --diff, --stdout, --output.") + path = self.require_project_dir(path) + self.templates.resolve() + kind = detect_kind(path) + + if kind == "agent": + project = AgentProject.load(path) + result: RenderResult = self.renderer.render_agent(project, inline=inline_env) + else: + project = DashboardProject.load(path) + result = self.renderer.render_dashboard(project, inline=inline_env) + result.validate() + + if inline_env and not stdout: + self.ui.warning("--inline-env writes secrets in clear text into the compose file.") + + if stdout: + self.ui.console.print(result.compose, end="", markup=False, highlight=False) + return + if diff: + self.ui.diff(result.diff_against(path)) + return + + target = (output or path).resolve() + if output is not None: + target.mkdir(parents=True, exist_ok=True) + (target / ENV_FILE).write_text((path / ENV_FILE).read_text(encoding="utf-8"), encoding="utf-8") + report = result.write(target) + report_write(self.ui, report) + self.ui.success(f"Rendered {', '.join(p.name for p in report.wrote)} in {target}") + if kind == "agent" and output is None: + self.ui.info(f"Restart to apply: portabase restart {path.name}") +``` + +`--stdout` imprime via `console.print(markup=False)` pour qu'aucun `[x]` du compose ne soit interprété comme balise Rich. Pour un pipe propre, `main.py` doit **ne pas** afficher la notification de mise à jour quand `--stdout` est présent (déjà géré : non-interactif ou stdin non-TTY ; ajouter `"--stdout" in sys.argv` à `_notify_update` par sécurité, Task 9). + +- [ ] **Step 2: Commit** + +```bash +git add commands/build.py +git commit -m "feat(commands): add build command (re-render, --diff, --stdout, --inline-env, --output)" +``` + +--- + +### Task 9 : Câblage final, suppression du legacy + +**Files:** +- Modify: `main.py` +- Modify: `commands/base.py` (retirer `LegacyCommand`) +- Delete: `core/network.py`, `core/docker.py`, `templates/compose.py`, `templates/__init__.py`, `templates/agent.yml`, `templates/dashboard.yml` +- Modify: `core/config.py` (retirer les fonctions legacy, garder `GlobalConfig`, `TEMPLATE_BASE_URL`, `GLOBAL_CONFIG_DIR/FILE`) +- Modify: `core/utils.py` (garder uniquement `generate_password`, `slugify_project_name`, `validate_edge_key`, et l'import `current_version` re-exporté peut disparaître) +- Modify: `pyproject.toml` (retirer `per-file-ignores`) +- Modify: `.github/workflows/templates-upload.yml` (retirer l'étape `latest`) +- Modify: `scripts/render_check.py` + +- [ ] **Step 1: `main.py` — remplacer les `LegacyCommand` et `legacy_db`** + +Imports à remplacer : + +```python +from commands.agent import AgentCommand +from commands.build import BuildCommand +from commands.dashboard import DashboardCommand +from commands.db import DbCommands +from engines import registry as engine_registry +from services.ports import PortAllocator +from services.renderer import ComposeRenderer +from services.templates import TemplateRepository +``` + +(supprimer `from commands import agent as legacy_agent`, `dashboard as legacy_dashboard`, `db as legacy_db`, `from commands.base import LegacyCommand`.) + +Dans `build_app`, après `updater = Updater(http, version)` : + +```python + templates = TemplateRepository.from_environment(http, config) + ports = PortAllocator() + renderer = ComposeRenderer(templates, engine_registry, version) +``` + +Liste `commands` : + +```python + commands = [ + AgentCommand(ui, telemetry, docker, templates, renderer, engine_registry, ports), + DashboardCommand(ui, telemetry, docker, templates, renderer, ports), + StartCommand(ui, telemetry, docker), + StopCommand(ui, telemetry, docker), + RestartCommand(ui, telemetry, docker), + LogsCommand(ui, telemetry, docker), + UninstallCommand(ui, telemetry, docker), + BuildCommand(ui, telemetry, templates, renderer), + UpdateCommand(ui, telemetry, checker, updater), + ] + for cmd in commands: + cmd.register(app) + DbCommands(ui, telemetry, engine_registry, ports, templates, renderer, docker).register(app) + ConfigCommands(ui, telemetry, config).register(app) +``` + +Dans `_notify_update`, condition étendue : `or "--stdout" in sys.argv`. + +Retirer du catcher les branches `click.exceptions.Exit` et `click.exceptions.Abort` ? **Non** : `--help` lève toujours `Exit(0)` et `version_callback` lève `typer.Exit()`. Garder `Exit` ; retirer `Abort` (plus de `typer.confirm`). + +- [ ] **Step 2: Nettoyage** + +```bash +git rm core/network.py core/docker.py templates/compose.py templates/__init__.py templates/agent.yml templates/dashboard.yml +``` + +`commands/base.py` : supprimer la classe `LegacyCommand` et l'import `Callable` s'il devient inutilisé. + +`core/config.py` : ne garder que les constantes (`TEMPLATE_BASE_URL`, `GLOBAL_CONFIG_DIR`, `GLOBAL_CONFIG_FILE`), les imports nécessaires et `GlobalConfig`. Supprimer `write_file`, `write_env_file`, `load_global_config`, `save_global_config`, `get_config_value`, `set_config_value`, `load_db_config`, `save_db_config`, `add_db_to_json`. + +`core/utils.py` : ne garder que `generate_password`, `slugify_project_name`, `validate_edge_key` et leurs imports (`base64`, `binascii`, `json`, `re`, `secrets`, `string`). Supprimer `questionary_style`, `custom_theme`, `HINTS`, `get_random_hint`, `console`, `BANNER`, `print_banner`, `get_free_port`, `start_docker`, `check_system`, `validate_work_dir`, le re-export `current_version`. + +Vérifier qu'aucune référence ne subsiste : +Run: `grep -rn "core.network\|core.docker\|templates.compose\|LegacyCommand\|get_random_hint\|print_banner\|check_system\|validate_work_dir\|get_free_port\|load_db_config\|add_db_to_json\|write_env_file\|get_config_value" --include=*.py . | grep -v ".venv"` +Expected: aucune sortie. + +`pyproject.toml` : supprimer entièrement `[tool.ruff.lint.per-file-ignores]` ; retirer `"templates"` de `known-first-party`. + +`.github/workflows/templates-upload.yml` : supprimer l'étape `Upload latest templates (stable only, legacy fallback)` — **seulement si** plus aucune version legacy n'est supportée. Sinon la garder ; par défaut la garder et ouvrir une issue « retirer latest/ ». Décision utilisateur. + +- [ ] **Step 3: `scripts/render_check.py` via `ComposeRenderer`** + +Remplacer `agent_cases`, `render_agent`, `dashboard_cases` par une construction de projets en mémoire : + +```python +from core.specs import DatabaseSpec # noqa: E402 +from services.envfile import EnvFile # noqa: E402 +from services.project import AgentProject, DashboardProject # noqa: E402 +from services.renderer import ComposeRenderer # noqa: E402 + + +def agent_project(tmp: Path, specs: list, engines_for, host_gateway=False, sqlite=False) -> AgentProject: + env = EnvFile(tmp / ".env") + env.merge({"TZ": "UTC", "EDGE_KEY": "x", "LOG_LEVEL": "info", "POLLING": "5"}) + project = AgentProject(tmp, env, [], host_gateway) + for spec, engine in zip(specs, engines_for): + project.add(spec, engine) + if sqlite: + sq = registry.get("sqlite") + project.add(sq.generate(auth=False, ports=FixedPortAllocator(), answers={"name": "x"}), sq) + return project + + +def agent_cases(renderer: ComposeRenderer) -> list[tuple[str, str, str]]: + ports = FixedPortAllocator() + tmp = Path(tempfile.mkdtemp()) + cases = [] + empty = agent_project(tmp, [], []) + cases.append(("agent/empty", renderer.render_agent(empty).compose, env_text(empty))) + toggles = agent_project(tmp, [], [], host_gateway=True, sqlite=True) + dv = registry.get("docker-volume") + toggles.add(dv.from_existing({"volume": "v"}), dv) + cases.append(("agent/toggles", renderer.render_agent(toggles).compose, env_text(toggles))) + all_specs, all_engines = [], [] + for engine in registry: + if engine.template is None: + continue + for auth in (True, False) if engine.auth_variants else (True,): + spec = engine.generate(auth=auth, ports=ports, answers={}) + one = agent_project(tmp, [spec], [engine]) + label = f"agent/{engine.key}" + ("/auth" if auth else "/noauth" if engine.auth_variants else "") + cases.append((label, renderer.render_agent(one).compose, env_text(one))) + all_specs.append(spec); all_engines.append(engine) + everything = agent_project(tmp, all_specs, all_engines, host_gateway=True, sqlite=True) + cases.append(("agent/all", renderer.render_agent(everything).compose, env_text(everything))) + return cases + + +def env_text(project) -> str: + return "".join(f'{k}="{v}"\n' for k, v in project.env.as_dict().items()) + + +def dashboard_cases(renderer: ComposeRenderer) -> list[tuple[str, str, str]]: + tmp = Path(tempfile.mkdtemp()) + base = {"HOST_PORT": "8887", "PROJECT_SECRET": "s", "PROJECT_URL": "http://localhost:8887", "PROJECT_NAME": "pb", "TZ": "UTC", "LOG_LEVEL": "info"} + pg = {"POSTGRES_DB": "pb", "POSTGRES_USER": "pb", "POSTGRES_PASSWORD": "p", "PG_PORT": "5433", "DATABASE_URL": "postgresql://pb:p@db:5432/pb"} + variants = {"external": {**base, **pg, "POSTGRES_HOST": "db"}, "internal": base, "custom": {**base, **pg, "POSTGRES_HOST": "remote"}} + cases = [] + for mode, vars_ in variants.items(): + env = EnvFile(tmp / f".env.{mode}"); env.merge(vars_) + project = DashboardProject(tmp, env) + assert project.db_mode == mode, (project.db_mode, mode) + cases.append((f"dashboard/{mode}", renderer.render_dashboard(project).compose, env_text(project))) + return cases +``` + +et dans `main()` : + +```python + repo = TemplateRepository(HttpClient(), GlobalConfig().cache_dir, "local", local_dir=Path(args.templates)) + renderer = ComposeRenderer(repo, registry, "render-check") + try: + engines_check(repo) + for label, compose, env_text_ in agent_cases(renderer) + dashboard_cases(renderer): + validate(label, compose, env_text_, use_compose) +``` + +Supprimer `AGENT_GLOBALS`, `AGENT_ENV`, `DASHBOARD_VARS`, `DASHBOARD_ENV`, `render_agent` devenus inutiles. Ajouter `import tempfile` déjà présent. + +Run: `uv run python scripts/render_check.py` +Expected: mêmes cas qu'au Plan 3, tous `ok`, dont `agent/toggles` avec socket + mount + extra_hosts. + +- [ ] **Step 4: Lint** + +Run: `uv run ruff check . && uv run ruff format --check .` +Expected: passe **sans aucune** exception par fichier. + +- [ ] **Step 5: Commit** + +```bash +git add -A +git commit -m "refactor: remove legacy commands and helpers; wire agent, dashboard, db, build on the renderer" +``` + +--- + +### Task 10 : Vérification de bout en bout + +Docker requis. `K` = edge key de test : `K=$(printf '{"serverUrl":"http://x","agentId":"a","masterKeyB64":"k"}' | base64 -w0)`. + +- [ ] **Step 1: Agent non-interactif + db add** + +```bash +cd /tmp && rm -rf ni-agent +M="uv --directory /home/soluce/Documents/PROJETS/Portabase/cli run python /home/soluce/Documents/PROJETS/Portabase/cli/main.py" +$M --non-interactive agent ni-agent --key "$K" --tz Europe/Paris --polling 7 --host-gateway; echo "exit=$?" +$M --non-interactive db add ni-agent --engine postgresql --mode new -o clean_mode=drop_schemas; echo "exit=$?" +$M --non-interactive db add ni-agent --engine redis --mode new --no-auth; echo "exit=$?" +printf 'secret\n' | $M --non-interactive db add ni-agent --engine mysql --mode existing --label Prod --host db.example --port 3306 --database app --user app --password-stdin; echo "exit=$?" +$M --non-interactive db add ni-agent --engine sqlite --mode new --name local; echo "exit=$?" +$M --non-interactive db add ni-agent --engine docker-volume --volume some_vol; echo "exit=$?" +$M db list ni-agent +cat ni-agent/.env; head -3 ni-agent/docker-compose.yml; python3 -c "import json; print([ (d['type'], d.get('options')) for d in json.load(open('ni-agent/databases.json'))['databases']])" +(cd ni-agent && docker compose config --quiet && echo "compose config OK") +``` +Expected: 6 × `exit=0` ; table à 6 lignes ; `.env` avec `TZ="Europe/Paris"`, `POLLING="7"`, `DB_PG_*` (4 vars), `DB_REDIS_*_PORT` seul ; en-tête `# Generated by Portabase CLI` ; options `{'clean_mode': 'drop_schemas'}` sur postgres, `None` ailleurs ; `compose config OK`. + +- [ ] **Step 2: Erreurs non-interactives** + +```bash +$M --non-interactive db add ni-agent --engine postgresql --mode existing; echo "exit=$?" +$M --non-interactive db add ni-agent --engine redis --mode new --host x; echo "exit=$?" +$M --non-interactive db add ni-agent --engine postgresql --mode new -o nope=1; echo "exit=$?" +$M --non-interactive agent ni-agent --key "$K"; echo "exit=$?" +``` +Expected: `Missing --host` exit 2 ; `not applicable` exit 2 ; `Unknown option(s)` exit 2 ; `Directory 'ni-agent' already exists` → confirm par défaut False → `Cancelled.` exit 130. + +- [ ] **Step 3: db remove, build** + +```bash +ID=$(python3 -c "import json; print([d for d in json.load(open('ni-agent/databases.json'))['databases'] if d['type']=='redis'][0]['generated_id'])") +$M db remove ni-agent --id "$ID" --yes; echo "exit=$?" +grep -c "db-redis" ni-agent/docker-compose.yml ni-agent/.env +$M build ni-agent --diff +$M build ni-agent --stdout --inline-env | head -20 +$M build ni-agent --output /tmp/ni-export && ls /tmp/ni-export +``` +Expected: `Removed`, `0` occurrences de `db-redis` dans les deux fichiers, diff « No changes. », compose inline avec valeurs littérales (pas de `${`), export contenant `docker-compose.yml`, `.env`, `databases.json`. + +- [ ] **Step 4: Lifecycle réel** + +```bash +$M start ni-agent && sleep 5 && $M logs ni-agent --no-follow | tail -5 && $M stop ni-agent && $M uninstall ni-agent --force +``` +Expected: services `agent` et `db-pg-*` démarrent (`docker compose ps` pendant le `sleep` si besoin), puis arrêt et suppression. + +- [ ] **Step 5: Install legacy** + +Réutiliser `/tmp/legacy-agent` (Task 2 step 4 ; le recréer sinon). + +```bash +cp -r /tmp/legacy-agent /tmp/legacy-copy +$M build /tmp/legacy-copy --diff +$M db add /tmp/legacy-copy --engine valkey --mode new --auth --non-interactive; echo "exit=$?" +ls /tmp/legacy-copy; head -1 /tmp/legacy-copy/docker-compose.yml +(cd /tmp/legacy-copy && docker compose config --quiet && echo "compose config OK") +diff <(python3 -c "import json; print(sorted((d['type'], d.get('host')) for d in json.load(open('/tmp/legacy-agent/databases.json'))['databases']))") <(python3 -c "import json; print(sorted((d['type'], d.get('host')) for d in json.load(open('/tmp/legacy-copy/databases.json'))['databases'] if d['type']!='valkey'))") && echo "entries preserved" +$M start /tmp/legacy-copy && $M stop /tmp/legacy-copy && $M uninstall /tmp/legacy-copy --force +``` +Expected: diff montre en-tête + `restart:` sur redis ; `db add` exit 0 avec le warning `Legacy compose backed up to docker-compose.legacy.yml` ; `docker-compose.legacy.yml` présent ; en-tête `# Generated` ; `compose config OK` ; `entries preserved` ; les anciens services démarrent avec leurs volumes existants (noms de service inchangés). + +- [ ] **Step 6: Dashboard** + +```bash +cd /tmp && rm -rf ni-dash +$M --non-interactive dashboard ni-dash --port 8899 --db-mode external --yes; echo "exit=$?" +(cd ni-dash && docker compose config --quiet && echo OK && grep -c "db:" docker-compose.yml) +$M --non-interactive dashboard ni-dash2 --port 8898 --db-mode internal --yes && (cd ni-dash2 && grep -c "postgres" docker-compose.yml) +printf 'pw\n' | $M --non-interactive dashboard ni-dash3 --port 8897 --db-mode custom --db-host pg.example --db-user u --db-password-stdin --yes && grep DATABASE_URL ni-dash3/.env +$M build ni-dash3 --diff +``` +Expected: external `OK 1` ; internal `0` ; custom `.env` avec `DATABASE_URL="postgresql://u:pw@pg.example:5432/portabase?schema=public"` ; diff vide. + +- [ ] **Step 7: Interactif** + +`$M agent int-agent` sans flags : bannière, prompts Edge Key / Timezone / Polling / extra_hosts, création, boucle « Add a database? » → ajouter un postgres (prompts options avec aide affichée) puis un redis (variante), répondre non, ne pas démarrer. Puis `$M db remove int-agent` avec sélection interactive. Ctrl-C au milieu d'un prompt → `Cancelled.` exit 130 sans traceback, fichiers cohérents (`docker compose config` passe). + +- [ ] **Step 8: CI `build-smoke`** + +Dans `ci.yml`, étape `Smoke` du job `build-smoke` : + +```yaml + - name: Smoke + env: + PORTABASE_TEMPLATES_DIR: ${{ github.workspace }}/templates + run: | + ./dist/portabase_smoke --version + cd "$(mktemp -d)" + K=$(printf '{"serverUrl":"http://x","agentId":"a","masterKeyB64":"k"}' | base64 -w0) + "$GITHUB_WORKSPACE/dist/portabase_smoke" --non-interactive agent smoke --key "$K" + "$GITHUB_WORKSPACE/dist/portabase_smoke" --non-interactive db add smoke --engine postgresql --mode new + "$GITHUB_WORKSPACE/dist/portabase_smoke" --non-interactive build smoke --diff + cd smoke && docker compose config --quiet +``` + +Le runner GitHub a Docker ; `agent` appelle `require_docker` + `ensure_network` → fonctionne. Si le daemon n'est pas disponible sur un runner donné, l'étape échoue explicitement (`E_DOCKER`) — c'est voulu. + +- [ ] **Step 9: README** + +Ajouter une section « Upgrading from 26.07 or earlier » : le premier `db add`/`build` re-génère `docker-compose.yml` (sauvegarde `docker-compose.legacy.yml`), les personnalisations vont dans `docker-compose.override.yml`, `portabase build --diff` montre les changements avant. + +- [ ] **Step 10: Commit, PR, rc** + +```bash +git add -A +git commit -m "ci: smoke-test agent creation and rendering with the built binary; document migration" +git checkout -b refactor/render-commands && git push -u origin refactor/render-commands +``` +PR « refactor: declarative rendering, flag-driven commands, build » → merge → Bump `26.09.0rc1` → tester le binaire rc sur une vraie install legacy → Bump `26.09.0` stable. + +--- + +## Self-review + +**Spec coverage :** +- §4.2 inventaire commandes : `agent` (flags, boucle interactive, aucune DB en non-interactif) ✔ ; `dashboard` (flags, modes, `--yes`) ✔ ; `db add` (tous flags + `-o`, rejet des flags non pertinents) ✔ ; `db remove` (`--id/--name`, `--purge-volume`, volume conservé par défaut) ✔ ; `db list` (options non-défaut) ✔ ; `build` (`--diff`, `--stdout`, `--inline-env`, `--output`) ✔ ; détection `kind` ✔ ; `back` supprimé ✔. +- §4.3 `AddDatabaseFlow.collect/apply`, séquence, usage par les deux commandes, rendu après chaque ajout ✔. +- §5.1 `DatabaseSpec.from_json` → `spec_from_entry` ; `AgentProject` (managed, socket, mounts, validate doublons) ✔ ; `DashboardProject.db_mode` ✔ ; `host` managé sans `_PORT` → traité externe (warning non émis : ajouter `ui.warning` dans `DbListCommand` si souhaité — non bloquant). +- §5.2 `EnvFile` ✔ (ordre, commentaires, quotes, merge, remove_prefix, atomique). +- §5.3 `ComposeFacts.host_gateway` liste/dict, jamais d'erreur ✔. +- §5.5 renderer, contexte, `*_var`, validation avant écriture, en-tête, `templates.resolve()` avant mutation ✔. +- §5.7 legacy : backup `.legacy.yml` une fois, `start/stop/logs` sans migration, `build --diff` ✔. +- §6.1 options : `-o`, validation clés, prompts avec `help`, projection non-défaut ✔. +- §7.2 `Summary` (masque), `DataTable`, `Diff` ✔. +- §10 F : `build-smoke` étendu, rc obligatoire ✔. + +**Placeholders :** aucun. + +**Cohérence :** `report_write` défini dans `commands/db.py`, importé par `agent`, `dashboard`, `build` ✔ ; `DbEngine.describe/label_default/non_default_options` utilisés ✔ ; `SqliteEngine.mount_for` (Plan 3) utilisé par `AgentProject.sqlite_mounts` ✔ ; `RenderResult.write/diff_against/validate` utilisés par les 4 commandes ✔ ; `DockerRunner.remove_volume` ajouté et utilisé ✔ ; `UI.summary/table/diff` ajoutés et utilisés ✔ ; `TemplateRepository.engine_template` (Plan 3) utilisé par `_service` ✔. + +**Écarts connus :** +- `templates-upload.yml` `latest/` : décision utilisateur (Task 9 step 2). +- Import local de `sys` dans deux `run` : ruff peut préférer un import de module ; déplacer. From 394b25d811a4267e2e2c3c668030e6b9adc68e87 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Th=C3=A9o=20LAGACHE?= Date: Fri, 11 Sep 2026 14:15:03 +0200 Subject: [PATCH 07/24] refactoring --- .github/CONTRIBUTING.md | 14 +- .github/actions/setup/action.yml | 13 + .github/assets/templates/agent.yml | 22 - .github/assets/templates/dashboard.yml | 46 - .github/dependabot.yml | 21 + .github/workflows/build.yml | 62 + .github/workflows/bump.yml | 85 + .github/workflows/ci.yml | 65 + .github/workflows/discord.yml | 81 + .github/workflows/github.yml | 147 - .github/workflows/plumber.yml | 28 + .github/workflows/publish.yml | 68 + .github/workflows/python.yml | 54 - .github/workflows/release-candidate.yml | 44 - .github/workflows/release.yml | 77 +- .github/workflows/security.yml | 46 + .github/workflows/templates-upload.yml | 50 - .gitignore | 6 +- .gitleaks.toml | 19 + CITATION.cff | 2 + README.md | 13 +- commands/agent.py | 1072 +------- commands/base.py | 105 + commands/build.py | 98 + commands/common.py | 68 - commands/config.py | 107 +- commands/dashboard.py | 377 ++- commands/db.py | 990 ++----- commands/decrypt.py | 203 +- {templates => commands/flows}/__init__.py | 0 commands/flows/add_database.py | 131 + commands/lifecycle.py | 104 + commands/update.py | 65 + core/config.py | 106 +- core/crypto.py | 70 +- core/docker.py | 19 - core/errors.py | 61 + core/fields.py | 22 + core/network.py | 26 - core/specs.py | 36 + core/updater.py | 291 -- core/utils.py | 174 +- core/version.py | 37 + .../plans/2026-09-11-plan-1-ci-hygiene.md | 954 +++++++ ...2026-09-11-plan-2-foundations-lifecycle.md | 2385 +++++++++++++++++ .../2026-09-11-plan-3-templates-engines.md | 1944 ++++++++++++++ .../2026-09-11-plan-4-render-commands.md | 1935 +++++++++++++ .../specs/2026-09-11-cli-refactor-design.md | 592 ++++ engines/__init__.py | 65 + engines/base.py | 175 ++ engines/docker_volume.py | 65 + engines/firebird.py | 52 + engines/mariadb.py | 8 + engines/mongodb.py | 40 + engines/mssql.py | 33 + engines/mysql.py | 8 + engines/postgresql.py | 49 + engines/redis.py | 51 + engines/sqlite.py | 70 + engines/valkey.py | 51 + main.py | 290 +- portabase.spec | 49 + pyproject.toml | 64 +- release | 96 - scripts/render_check.py | 215 ++ services/__init__.py | 0 services/compose_facts.py | 43 + services/docker.py | 126 + services/envfile.py | 85 + services/http.py | 79 + services/ports.py | 29 + services/project.py | 198 ++ services/renderer.py | 167 ++ services/telemetry.py | 39 + services/templates.py | 63 + services/updater.py | 222 ++ templates/agent.yml.j2 | 36 + templates/compose.py | 203 -- templates/dashboard.yml.j2 | 52 + templates/engines/firebird.yml.j2 | 20 + templates/engines/mariadb.yml.j2 | 19 + templates/engines/mongodb.yml.j2 | 23 + templates/engines/mssql.yml.j2 | 17 + templates/engines/mysql.yml.j2 | 19 + templates/engines/postgresql-cluster.yml.j2 | 18 + templates/engines/postgresql.yml.j2 | 18 + templates/engines/redis.yml.j2 | 22 + templates/engines/valkey.yml.j2 | 21 + ui/__init__.py | 105 + ui/components/__init__.py | 0 ui/components/banner.py | 18 + ui/components/base.py | 8 + ui/components/diff.py | 13 + ui/components/hints.py | 39 + ui/components/message.py | 35 + ui/components/progress.py | 34 + ui/components/prompt.py | 45 + ui/components/section.py | 11 + ui/components/status.py | 12 + ui/components/summary.py | 37 + ui/components/table.py | 19 + ui/form.py | 119 + ui/theme.py | 41 + uv.lock | 961 +++++++ 104 files changed, 13841 insertions(+), 3491 deletions(-) create mode 100644 .github/actions/setup/action.yml delete mode 100644 .github/assets/templates/agent.yml delete mode 100644 .github/assets/templates/dashboard.yml create mode 100644 .github/dependabot.yml create mode 100644 .github/workflows/build.yml create mode 100644 .github/workflows/bump.yml create mode 100644 .github/workflows/ci.yml create mode 100644 .github/workflows/discord.yml delete mode 100644 .github/workflows/github.yml create mode 100644 .github/workflows/plumber.yml create mode 100644 .github/workflows/publish.yml delete mode 100644 .github/workflows/python.yml delete mode 100644 .github/workflows/release-candidate.yml create mode 100644 .github/workflows/security.yml delete mode 100644 .github/workflows/templates-upload.yml create mode 100644 .gitleaks.toml create mode 100644 commands/base.py create mode 100644 commands/build.py delete mode 100644 commands/common.py rename {templates => commands/flows}/__init__.py (100%) create mode 100644 commands/flows/add_database.py create mode 100644 commands/lifecycle.py create mode 100644 commands/update.py delete mode 100644 core/docker.py create mode 100644 core/errors.py create mode 100644 core/fields.py delete mode 100644 core/network.py create mode 100644 core/specs.py delete mode 100644 core/updater.py create mode 100644 core/version.py create mode 100644 docs/superpowers/plans/2026-09-11-plan-1-ci-hygiene.md create mode 100644 docs/superpowers/plans/2026-09-11-plan-2-foundations-lifecycle.md create mode 100644 docs/superpowers/plans/2026-09-11-plan-3-templates-engines.md create mode 100644 docs/superpowers/plans/2026-09-11-plan-4-render-commands.md create mode 100644 docs/superpowers/specs/2026-09-11-cli-refactor-design.md create mode 100644 engines/__init__.py create mode 100644 engines/base.py create mode 100644 engines/docker_volume.py create mode 100644 engines/firebird.py create mode 100644 engines/mariadb.py create mode 100644 engines/mongodb.py create mode 100644 engines/mssql.py create mode 100644 engines/mysql.py create mode 100644 engines/postgresql.py create mode 100644 engines/redis.py create mode 100644 engines/sqlite.py create mode 100644 engines/valkey.py create mode 100644 portabase.spec delete mode 100755 release create mode 100644 scripts/render_check.py create mode 100644 services/__init__.py create mode 100644 services/compose_facts.py create mode 100644 services/docker.py create mode 100644 services/envfile.py create mode 100644 services/http.py create mode 100644 services/ports.py create mode 100644 services/project.py create mode 100644 services/renderer.py create mode 100644 services/telemetry.py create mode 100644 services/templates.py create mode 100644 services/updater.py create mode 100644 templates/agent.yml.j2 delete mode 100644 templates/compose.py create mode 100644 templates/dashboard.yml.j2 create mode 100644 templates/engines/firebird.yml.j2 create mode 100644 templates/engines/mariadb.yml.j2 create mode 100644 templates/engines/mongodb.yml.j2 create mode 100644 templates/engines/mssql.yml.j2 create mode 100644 templates/engines/mysql.yml.j2 create mode 100644 templates/engines/postgresql-cluster.yml.j2 create mode 100644 templates/engines/postgresql.yml.j2 create mode 100644 templates/engines/redis.yml.j2 create mode 100644 templates/engines/valkey.yml.j2 create mode 100644 ui/__init__.py create mode 100644 ui/components/__init__.py create mode 100644 ui/components/banner.py create mode 100644 ui/components/base.py create mode 100644 ui/components/diff.py create mode 100644 ui/components/hints.py create mode 100644 ui/components/message.py create mode 100644 ui/components/progress.py create mode 100644 ui/components/prompt.py create mode 100644 ui/components/section.py create mode 100644 ui/components/status.py create mode 100644 ui/components/summary.py create mode 100644 ui/components/table.py create mode 100644 ui/form.py create mode 100644 ui/theme.py create mode 100644 uv.lock diff --git a/.github/CONTRIBUTING.md b/.github/CONTRIBUTING.md index cf829ed..76e16d0 100644 --- a/.github/CONTRIBUTING.md +++ b/.github/CONTRIBUTING.md @@ -237,4 +237,16 @@ If you encounter a bug or have a suggestion for improvement, follow these steps: Thank you for contributing! 🙌 ---- \ No newline at end of file +--- +## Releasing + +Releases are cut from GitHub Actions, never from a local machine. + +1. Open **Actions → Bump version → Run workflow**. +2. Pick the branch (`main` for stable, any branch for a release candidate). +3. Enter the version without a leading `v` (`26.09.0` for stable, `26.09.0rc1` for a candidate) and the matching channel. +4. The workflow commits `chore(release): `, creates the tag and pushes. The tag triggers the build, the GitHub release, the Discord notification and the template upload. + +Stable versions must match `X.Y.Z` and can only be cut from `main`. + +The workflow pushes with the `RELEASE_TOKEN` repository secret (a fine-grained PAT with *Contents: read and write*). A tag pushed with the default `GITHUB_TOKEN` would not trigger the release workflows. diff --git a/.github/actions/setup/action.yml b/.github/actions/setup/action.yml new file mode 100644 index 0000000..e51372c --- /dev/null +++ b/.github/actions/setup/action.yml @@ -0,0 +1,13 @@ +name: Set up Python toolchain +description: Install uv and sync the locked environment, including dev dependencies. + +runs: + using: composite + steps: + - uses: astral-sh/setup-uv@caf0cab7a618c569241d31dcd442f54681755d39 # v3 + with: + enable-cache: true + + - name: Install dependencies + shell: bash + run: uv sync --frozen --all-groups diff --git a/.github/assets/templates/agent.yml b/.github/assets/templates/agent.yml deleted file mode 100644 index 4a06bc1..0000000 --- a/.github/assets/templates/agent.yml +++ /dev/null @@ -1,22 +0,0 @@ -services: - agent: - restart: unless-stopped - image: portabase/agent:latest - volumes: - - ./databases.json:/config/config.json - environment: - TZ: "${TZ}" - EDGE_KEY: "${EDGE_KEY}" - LOG_LEVEL: "${LOG_LEVEL}" - POLLING: "${POLLING}" - networks: - - portabase - -{{EXTRA_SERVICES}} - -{{EXTRA_VOLUMES}} - -networks: - portabase: - name: portabase_network - external: true diff --git a/.github/assets/templates/dashboard.yml b/.github/assets/templates/dashboard.yml deleted file mode 100644 index 5d27087..0000000 --- a/.github/assets/templates/dashboard.yml +++ /dev/null @@ -1,46 +0,0 @@ -name: ${PROJECT_NAME} -services: - portabase: - container_name: ${PROJECT_NAME}-app - image: portabase/portabase:latest - restart: unless-stopped - env_file: - - .env - ports: - - "${HOST_PORT}:80" - environment: - - TZ=${TZ} - - LOG_LEVEL=${LOG_LEVEL} - - PROJECT_SECRET=${PROJECT_SECRET} - - PROJECT_URL=${PROJECT_URL} - volumes: - - portabase-data:/data - depends_on: - db: - condition: service_healthy - healthcheck: - test: ["CMD-SHELL", "curl -f http://localhost/api/health"] - interval: 30s - timeout: 5s - retries: 3 - start_period: 60s - db: - container_name: ${PROJECT_NAME}-pg - image: postgres:17-alpine - restart: unless-stopped - ports: - - "${PG_PORT}:5432" - volumes: - - postgres-data:/var/lib/postgresql/data - environment: - - POSTGRES_DB=${POSTGRES_DB} - - POSTGRES_USER=${POSTGRES_USER} - - POSTGRES_PASSWORD=${POSTGRES_PASSWORD} - healthcheck: - test: ["CMD-SHELL", "pg_isready -U ${POSTGRES_USER} -d ${POSTGRES_DB}"] - interval: 10s - timeout: 5s - retries: 5 -volumes: - postgres-data: - portabase-data: diff --git a/.github/dependabot.yml b/.github/dependabot.yml new file mode 100644 index 0000000..acd57de --- /dev/null +++ b/.github/dependabot.yml @@ -0,0 +1,21 @@ +version: 2 +updates: + - package-ecosystem: github-actions + directory: / + schedule: + interval: weekly + groups: + actions: + patterns: ["*"] + commit-message: + prefix: "ci" + + - package-ecosystem: uv + directory: / + schedule: + interval: weekly + groups: + python: + patterns: ["*"] + commit-message: + prefix: "build" diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml new file mode 100644 index 0000000..7942dbf --- /dev/null +++ b/.github/workflows/build.yml @@ -0,0 +1,62 @@ +name: Build binaries + +on: + workflow_call: + +permissions: {} + +jobs: + build: + name: ${{ matrix.os }}-${{ matrix.arch }} + runs-on: ${{ matrix.runner }} + timeout-minutes: 20 + permissions: + contents: read + id-token: write + attestations: write + strategy: + fail-fast: false + matrix: + include: + - os: linux + arch: amd64 + runner: ubuntu-latest + - os: linux + arch: arm64 + runner: ubuntu-24.04-arm + - os: macos + arch: arm64 + runner: macos-latest + - os: macos + arch: amd64 + runner: macos-15-intel + - os: windows + arch: amd64 + runner: windows-latest + ext: .exe + defaults: + run: + shell: bash + env: + NAME: portabase_${{ matrix.os }}_${{ matrix.arch }} + BINARY: portabase_${{ matrix.os }}_${{ matrix.arch }}${{ matrix.ext }} + steps: + - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4 + - uses: ./.github/actions/setup + + - name: Build + id: build + uses: ./.github/actions/build + with: + name: ${{ env.NAME }} + + - name: Attest provenance + uses: actions/attest-build-provenance@e8998f949152b193b063cb0ec769d69d929409be # v2 + with: + subject-path: ${{ steps.build.outputs.path }} + + - name: Upload + uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4 + with: + name: ${{ env.BINARY }} + path: ${{ steps.build.outputs.path }} diff --git a/.github/workflows/bump.yml b/.github/workflows/bump.yml new file mode 100644 index 0000000..f8fe0f8 --- /dev/null +++ b/.github/workflows/bump.yml @@ -0,0 +1,85 @@ +name: Bump version + +on: + workflow_dispatch: + inputs: + version: + description: "Version (e.g. 26.09.0 or 26.09.0rc1). No leading v." + required: true + type: string + channel: + description: "stable: only from main. rc: any branch." + required: true + type: choice + options: [stable, rc] + default: rc + +permissions: {} + +jobs: + bump: + runs-on: ubuntu-latest + timeout-minutes: 10 + permissions: + contents: write + steps: + - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4 + with: + fetch-depth: 0 + token: ${{ secrets.RELEASE_TOKEN }} + + - name: Validate version against channel + env: + VERSION: ${{ inputs.version }} + CHANNEL: ${{ inputs.channel }} + REF: ${{ github.ref_name }} + run: | + set -euo pipefail + if [[ "$VERSION" == v* ]]; then + echo "::error::Version must not start with 'v'"; exit 1 + fi + if [[ "$CHANNEL" == "stable" ]]; then + if [[ "$REF" != "main" ]]; then + echo "::error::stable releases are only allowed from main (got $REF)"; exit 1 + fi + if [[ ! "$VERSION" =~ ^[0-9]+\.[0-9]+\.[0-9]+$ ]]; then + echo "::error::stable version must match X.Y.Z"; exit 1 + fi + else + if [[ ! "$VERSION" =~ ^[0-9]+\.[0-9]+\.[0-9]+([-.]?(rc|alpha|beta|a|b)[0-9]*(\.[0-9]+)?)$ ]]; then + echo "::error::rc version must match X.Y.Z(rc|a|b|alpha|beta)N"; exit 1 + fi + fi + if git rev-parse "$VERSION" >/dev/null 2>&1; then + echo "::error::Tag $VERSION already exists"; exit 1 + fi + + - name: Update version files + env: + VERSION: ${{ inputs.version }} + run: | + set -euo pipefail + DATE=$(date -u +%F) + sed -i "s/^version = \".*\"/version = \"$VERSION\"/" pyproject.toml + if [ -f CITATION.cff ]; then + sed -i "s/^version: .*/version: $VERSION/" CITATION.cff + sed -i "s/^date-released: .*/date-released: \"$DATE\"/" CITATION.cff + fi + git diff --stat + + - name: Commit, tag, push + env: + VERSION: ${{ inputs.version }} + run: | + set -euo pipefail + git config user.name "github-actions[bot]" + git config user.email "41898282+github-actions[bot]@users.noreply.github.com" + git add pyproject.toml CITATION.cff + if git diff --cached --quiet; then + echo "No version change to commit" + else + git commit -m "chore(release): $VERSION" + fi + git tag -a "$VERSION" -m "Release $VERSION" + git push origin HEAD + git push origin "$VERSION" diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml new file mode 100644 index 0000000..cae7298 --- /dev/null +++ b/.github/workflows/ci.yml @@ -0,0 +1,65 @@ +name: CI + +on: + pull_request: + push: + branches: [main] + +permissions: {} + +concurrency: + group: ci-${{ github.ref }} + cancel-in-progress: true + +jobs: + checks: + name: ${{ matrix.name }} + runs-on: ubuntu-latest + timeout-minutes: 10 + permissions: + contents: read + strategy: + fail-fast: false + matrix: + include: + - name: lint + run: uv run ruff check . --output-format=github + - name: format + run: uv run ruff format --check . + - name: types + run: uv run mypy + - name: templates + run: uv run python scripts/render_check.py --templates templates + steps: + - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4 + - uses: ./.github/actions/setup + - run: ${{ matrix.run }} + + binary: + name: binary + runs-on: ubuntu-latest + timeout-minutes: 15 + permissions: + contents: read + steps: + - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4 + - uses: ./.github/actions/setup + + - name: Build + id: build + uses: ./.github/actions/build + + # No PORTABASE_TEMPLATES_DIR: the binary must run on its own bundled copy. + - name: End-to-end agent creation + env: + BINARY: ${{ github.workspace }}/${{ steps.build.outputs.path }} + EDGE_KEY: eyJzZXJ2ZXJVcmwiOiJodHRwOi8veCIsImFnZW50SWQiOiJhIiwibWFzdGVyS2V5QjY0IjoiayJ9 + run: | + set -euo pipefail + cd "$(mktemp -d)" + "$BINARY" --help > /dev/null + "$BINARY" --non-interactive agent smoke --key "$EDGE_KEY" + "$BINARY" --non-interactive db add smoke --engine postgresql --mode new + "$BINARY" --non-interactive db list smoke + "$BINARY" --non-interactive build smoke --diff + cd smoke && docker compose config --quiet diff --git a/.github/workflows/discord.yml b/.github/workflows/discord.yml new file mode 100644 index 0000000..9bc6de4 --- /dev/null +++ b/.github/workflows/discord.yml @@ -0,0 +1,81 @@ +name: Discord Notification + +on: + workflow_call: + inputs: + release_tag: + required: true + type: string + discord_title: + required: true + type: string + discord_color: + required: true + type: number + discord_footer: + required: false + type: string + default: Portabase + secrets: + DISCORD_WEBHOOK: + required: true + +permissions: {} + +jobs: + notify: + name: notify + runs-on: ubuntu-latest + timeout-minutes: 5 + permissions: + contents: read + steps: + - name: Send Discord notification + env: + DISCORD_WEBHOOK: ${{ secrets.DISCORD_WEBHOOK }} + GH_TOKEN: ${{ github.token }} + REPOSITORY: ${{ github.repository }} + RELEASE_TAG: ${{ inputs.release_tag }} + DISCORD_TITLE: ${{ inputs.discord_title }} + DISCORD_COLOR: ${{ inputs.discord_color }} + DISCORD_FOOTER: ${{ inputs.discord_footer }} + run: | + set -euo pipefail + + RELEASE_INFO=$(gh release view "$RELEASE_TAG" -R "$REPOSITORY" --json name,url,body) + + RELEASE_TITLE=$(echo "$RELEASE_INFO" | jq -r '.name // empty') + if [ -z "$RELEASE_TITLE" ]; then + RELEASE_TITLE="$RELEASE_TAG" + fi + RELEASE_URL=$(echo "$RELEASE_INFO" | jq -r '.url // empty') + RELEASE_BODY=$(echo "$RELEASE_INFO" | jq -r '.body // ""') + + # Discord rejects the whole payload if any field is over its limit. + jq -n \ + --arg title "$RELEASE_TITLE" \ + --arg description "$RELEASE_BODY" \ + --arg url "$RELEASE_URL" \ + --arg icon "https://github.com/Portabase.png" \ + --arg discord_title "$DISCORD_TITLE" \ + --arg discord_footer "$DISCORD_FOOTER" \ + --argjson discord_color "$DISCORD_COLOR" \ + 'def clip($max): if (. | length) > $max then (.[0:$max - 3] + "...") else . end; + { + content: ($discord_title | clip(2000)), + embeds: [{ + title: ($title | clip(256)), + url: $url, + description: ($description | clip(4096)), + color: $discord_color, + author: { name: "Portabase", icon_url: $icon }, + footer: { text: ($discord_footer | clip(2048)) } + }] + }' > payload.json + + jq . payload.json + + curl --fail-with-body -sS \ + -H "Content-Type: application/json" \ + -d @payload.json \ + "$DISCORD_WEBHOOK?wait=true" diff --git a/.github/workflows/github.yml b/.github/workflows/github.yml deleted file mode 100644 index bc49fbf..0000000 --- a/.github/workflows/github.yml +++ /dev/null @@ -1,147 +0,0 @@ -name: GitHub Release - -on: - workflow_call: - inputs: - artifact_name: - required: false - type: string - default: "" - prerelease: - required: false - type: boolean - default: false - make_latest: - required: false - type: boolean - default: false - discord_title: - required: true - type: string - discord_color: - required: true - type: number - discord_footer: - required: true - type: string - secrets: - DISCORD_WEBHOOK: - required: true - GH_TOKEN: - required: true - -jobs: - create-release: - runs-on: ubuntu-latest - permissions: - contents: write - steps: - - name: Check out the repo - uses: actions/checkout@v4 - with: - fetch-depth: 0 - - - name: Download artifacts - if: inputs.artifact_name != '' - uses: actions/download-artifact@v4 - with: - pattern: ${{ inputs.artifact_name }} - path: dist - merge-multiple: true - - - name: Generate Checksums - if: inputs.artifact_name != '' - working-directory: dist - run: | - sha256sum * > checksums.txt - - - name: Build Changelog - id: build_changelog - uses: mikepenz/release-changelog-builder-action@v5 - with: - mode: "COMMIT" - configurationJson: | - { - "template": "#{{CHANGELOG}}", - "categories": [ - { - "title": "## Feature", - "labels": ["feat", "feature"] - }, - { - "title": "## Fix", - "labels": ["fix", "bug"] - }, - { - "title": "## Other", - "labels": [] - } - ], - "label_extractor": [ - { - "pattern": "^(build|chore|ci|docs|feat|fix|perf|refactor|revert|style|test){1}(\([\w\-\.]+\))?(!)?: ([\w ])+([\s\S]*)", - "on_property": "title", - "target": "$1" - } - ] - } - env: - GITHUB_TOKEN: ${{ secrets.GH_TOKEN }} - - - name: Create GitHub Release - uses: softprops/action-gh-release@v2 - with: - files: dist/* - generate_release_notes: false - body: ${{ steps.build_changelog.outputs.changelog }} - prerelease: ${{ inputs.prerelease }} - make_latest: ${{ inputs.make_latest }} - env: - GITHUB_TOKEN: ${{ secrets.GH_TOKEN }} - - - name: Send Discord Notification - env: - DISCORD_WEBHOOK: ${{ secrets.DISCORD_WEBHOOK }} - GH_TOKEN: ${{ secrets.GH_TOKEN }} - run: | - RELEASE_INFO=$(gh release view "${{ github.ref_name }}" -R ${{ github.repository }} --json name,url,body,author) - - RELEASE_TITLE=$(echo "$RELEASE_INFO" | jq -r .name) - if [ -z "$RELEASE_TITLE" ] || [ "$RELEASE_TITLE" = "null" ]; then RELEASE_TITLE="${{ github.ref_name }}"; fi - - RELEASE_URL=$(echo "$RELEASE_INFO" | jq -r .url) - RELEASE_BODY=$(echo "$RELEASE_INFO" | jq -r .body) - - AUTHOR_NAME="Portabase" - AUTHOR_ICON="https://github.com/Portabase.png" - - PAYLOAD=$(jq -n \ - --arg title "$RELEASE_TITLE" \ - --arg description "$RELEASE_BODY" \ - --arg url "$RELEASE_URL" \ - --arg author "$AUTHOR_NAME" \ - --arg icon "$AUTHOR_ICON" \ - --arg discord_title "${{ inputs.discord_title }}" \ - --arg discord_footer "${{ inputs.discord_footer }}" \ - --argjson discord_color ${{ inputs.discord_color }} \ - '{ - content: $discord_title, - embeds: [{ - title: $title, - url: $url, - description: $description, - color: $discord_color, - author: { - name: $author, - icon_url: $icon - }, - footer: { - text: $discord_footer - } - }] - }' - ) - - curl -H "Content-Type: application/json" \ - -d "$PAYLOAD" \ - "$DISCORD_WEBHOOK" diff --git a/.github/workflows/plumber.yml b/.github/workflows/plumber.yml new file mode 100644 index 0000000..5dc10b4 --- /dev/null +++ b/.github/workflows/plumber.yml @@ -0,0 +1,28 @@ +name: Plumber + +on: + pull_request: + push: + branches: [main] + +permissions: {} + +concurrency: + group: plumber-${{ github.ref }} + cancel-in-progress: true + +jobs: + audit: + name: audit + runs-on: ubuntu-24.04 + timeout-minutes: 10 + permissions: + contents: read + security-events: write + steps: + - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4 + - uses: getplumber/plumber@7ad9d267ee5a00163cec9e5c749a088d5f565167 # v0.4.26 + with: + score-push: true + upload-sarif: true + soft-fail: true diff --git a/.github/workflows/publish.yml b/.github/workflows/publish.yml new file mode 100644 index 0000000..07edf57 --- /dev/null +++ b/.github/workflows/publish.yml @@ -0,0 +1,68 @@ +name: Publish release + +on: + workflow_call: + inputs: + prerelease: + required: true + type: boolean + +permissions: {} + +jobs: + publish: + name: publish + runs-on: ubuntu-latest + timeout-minutes: 10 + permissions: + contents: write + steps: + - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4 + with: + fetch-depth: 0 + + - name: Download binaries + uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4 + with: + pattern: portabase_* + path: dist + merge-multiple: true + + - name: Generate checksums + working-directory: dist + run: sha256sum * > checksums.txt + + - name: Build changelog + id: changelog + uses: mikepenz/release-changelog-builder-action@c9dc8369bccbc41e0ac887f8fd674f5925d315f7 # v5 + with: + mode: COMMIT + configurationJson: | + { + "template": "#{{CHANGELOG}}", + "categories": [ + { "title": "## Feature", "labels": ["feat", "feature"] }, + { "title": "## Fix", "labels": ["fix", "bug"] }, + { "title": "## Other", "labels": [] } + ], + "label_extractor": [ + { + "pattern": "^(build|chore|ci|docs|feat|fix|perf|refactor|revert|style|test){1}(\\([\\w\\-\\.]+\\))?(!)?: ([\\w ])+([\\s\\S]*)", + "on_property": "title", + "target": "$1" + } + ] + } + env: + GITHUB_TOKEN: ${{ github.token }} + + - name: Create GitHub release + uses: softprops/action-gh-release@3bb12739c298aeb8a4eeaf626c5b8d85266b0e65 # v2 + with: + files: dist/* + generate_release_notes: false + body: ${{ steps.changelog.outputs.changelog }} + prerelease: ${{ inputs.prerelease }} + make_latest: ${{ !inputs.prerelease }} + env: + GITHUB_TOKEN: ${{ github.token }} diff --git a/.github/workflows/python.yml b/.github/workflows/python.yml deleted file mode 100644 index 320173f..0000000 --- a/.github/workflows/python.yml +++ /dev/null @@ -1,54 +0,0 @@ -name: Build Python Binaries - -on: - workflow_call: - -jobs: - build: - name: Build for ${{ matrix.os }} (${{ matrix.arch }}) - runs-on: ${{ matrix.runner }} - strategy: - matrix: - include: - - os: linux - arch: amd64 - runner: ubuntu-latest - - os: linux - arch: arm64 - runner: ubuntu-24.04-arm - - os: macos - arch: arm64 - runner: macos-latest - - os: macos - arch: amd64 - runner: macos-15-intel - - steps: - - uses: actions/checkout@v4 - - - name: Install uv - uses: astral-sh/setup-uv@v3 - - - name: Set up Python - run: uv python install - - - name: Build binary - run: | - rm -rf build dist *.spec - uv run pyinstaller \ - --onefile \ - --name portabase_${{ matrix.os }}_${{ matrix.arch }} \ - --paths=. \ - --collect-all rich \ - --collect-all requests \ - --collect-data certifi \ - --add-data "pyproject.toml:." \ - main.py - - - - name: Upload artifacts - uses: actions/upload-artifact@v4 - with: - name: portabase_${{ matrix.os }}_${{ matrix.arch }} - path: dist/portabase_${{ matrix.os }}_${{ matrix.arch }} - diff --git a/.github/workflows/release-candidate.yml b/.github/workflows/release-candidate.yml deleted file mode 100644 index c64c173..0000000 --- a/.github/workflows/release-candidate.yml +++ /dev/null @@ -1,44 +0,0 @@ -name: Publish Python binaries for release candidate - -on: - push: - tags: - - '*.*.*a*' - - '*.*.*b*' - - '*.*.*rc*' - - '*.*.*alpha*' - - '*.*.*beta*' - -permissions: - contents: write - packages: write - -jobs: - python_build: - uses: ./.github/workflows/python.yml - - github_release: - needs: python_build - uses: ./.github/workflows/github.yml - with: - artifact_name: "portabase_*" - prerelease: true - make_latest: false - discord_title: "||@release-cli|| New release candidate published" - discord_color: 16776960 - discord_footer: "Portabase" - secrets: - DISCORD_WEBHOOK: ${{ secrets.DISCORD_WEBHOOK }} - GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} - - template_upload: - needs: python_build - uses: ./.github/workflows/templates-upload.yml - with: - version: ${{ github.ref_name }} - is_prerelease: true - secrets: - S3_ENDPOINT: ${{ secrets.S3_ENDPOINT }} - S3_ACCESS_KEY: ${{ secrets.S3_ACCESS_KEY }} - S3_SECRET_KEY: ${{ secrets.S3_SECRET_KEY }} - S3_BUCKET: ${{ secrets.S3_BUCKET }} diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 1a9a422..c170c0c 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -1,46 +1,59 @@ -name: Publish Python binaries for release +name: Release on: push: tags: - '*.*.*' - - '!*-*' - - '!*.*.*a*' - - '!*.*.*b*' - - '!*.*.*rc*' - - '!*.*.*alpha*' - - '!*.*.*beta*' permissions: contents: write - packages: write + id-token: write + attestations: write + +concurrency: + group: release-${{ github.ref }} + cancel-in-progress: false jobs: - python_build: - uses: ./.github/workflows/python.yml + channel: + name: channel + runs-on: ubuntu-latest + timeout-minutes: 5 + outputs: + prerelease: ${{ steps.detect.outputs.prerelease }} + steps: + - id: detect + env: + TAG: ${{ github.ref_name }} + run: | + set -euo pipefail + if [[ "$TAG" =~ ^v?[0-9]+\.[0-9]+\.[0-9]+([-.]?(rc|alpha|beta|a|b)[0-9]*(\.[0-9]+)?)$ ]]; then + echo "prerelease=true" >> "$GITHUB_OUTPUT" + elif [[ "$TAG" =~ ^v?[0-9]+\.[0-9]+\.[0-9]+$ ]]; then + echo "prerelease=false" >> "$GITHUB_OUTPUT" + else + echo "::error::tag '$TAG' is neither X.Y.Z nor X.Y.ZN" + exit 1 + fi + + build: + needs: channel + uses: ./.github/workflows/build.yml - github_release: - needs: python_build - uses: ./.github/workflows/github.yml + publish: + needs: [channel, build] + uses: ./.github/workflows/publish.yml with: - artifact_name: "portabase_*" - prerelease: false - make_latest: true - discord_title: "||@release-cli|| New release published" - discord_color: 5814783 - discord_footer: "Portabase" - secrets: - DISCORD_WEBHOOK: ${{ secrets.DISCORD_WEBHOOK }} - GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} + prerelease: ${{ needs.channel.outputs.prerelease == 'true' }} - template_upload: - needs: python_build - uses: ./.github/workflows/templates-upload.yml + announce: + needs: [channel, publish] + uses: ./.github/workflows/discord.yml with: - version: ${{ github.ref_name }} - is_prerelease: false - secrets: - S3_ENDPOINT: ${{ secrets.S3_ENDPOINT }} - S3_ACCESS_KEY: ${{ secrets.S3_ACCESS_KEY }} - S3_SECRET_KEY: ${{ secrets.S3_SECRET_KEY }} - S3_BUCKET: ${{ secrets.S3_BUCKET }} + release_tag: ${{ github.ref_name }} + discord_title: >- + ${{ needs.channel.outputs.prerelease == 'true' + && '||@release-cli|| New release candidate published' + || '||@release-cli|| New release published' }} + discord_color: ${{ needs.channel.outputs.prerelease == 'true' && 16776960 || 5814783 }} + secrets: inherit diff --git a/.github/workflows/security.yml b/.github/workflows/security.yml new file mode 100644 index 0000000..7a83a05 --- /dev/null +++ b/.github/workflows/security.yml @@ -0,0 +1,46 @@ +name: Security + +on: + pull_request: + push: + branches: [main] + +permissions: + contents: read + +concurrency: + group: security-${{ github.ref }} + cancel-in-progress: true + +jobs: + dependencies: + name: dependencies + runs-on: ubuntu-latest + timeout-minutes: 10 + steps: + - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4 + - uses: aquasecurity/trivy-action@57a97c7e7821a5776cebc9bb87c984fa69cba8f1 # v0.35.0 + with: + scan-type: fs + format: table + severity: CRITICAL,HIGH + ignore-unfixed: true + exit-code: '1' + + secrets: + name: secrets + # Forks cannot read secrets, so the action would fail for reasons unrelated + # to the code under review. + if: github.event.pull_request.head.repo.fork == false + runs-on: ubuntu-latest + timeout-minutes: 10 + steps: + - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4 + with: + fetch-depth: 0 + - uses: gitleaks/gitleaks-action@ff98106e4c7b2bc287b24eaf42907196329070c7 # v2 + env: + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + GITLEAKS_LICENSE: ${{ secrets.GITLEAKS_LICENSE }} + with: + config-path: .gitleaks.toml diff --git a/.github/workflows/templates-upload.yml b/.github/workflows/templates-upload.yml deleted file mode 100644 index 3963b47..0000000 --- a/.github/workflows/templates-upload.yml +++ /dev/null @@ -1,50 +0,0 @@ -name: Upload Templates to S3 - -on: - workflow_call: - inputs: - version: - required: true - type: string - is_prerelease: - required: true - type: boolean - secrets: - S3_ENDPOINT: - required: true - S3_ACCESS_KEY: - required: true - S3_SECRET_KEY: - required: true - S3_BUCKET: - required: true - -jobs: - upload: - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v4 - - - name: Install S3cmd - run: sudo apt-get update && sudo apt-get install -y s3cmd - - - name: Configure S3cmd - run: | - cat < ~/.s3cfg - [default] - access_key = ${{ secrets.S3_ACCESS_KEY }} - secret_key = ${{ secrets.S3_SECRET_KEY }} - host_base = ${{ secrets.S3_ENDPOINT }} - host_bucket = %(bucket)s.${{ secrets.S3_ENDPOINT }} - use_https = True - EOF - - - name: Upload Versioned Templates - run: | - CLEAN_VERSION=$(echo "${{ inputs.version }}" | sed 's/^v//') - s3cmd sync .github/assets/templates/ s3://${{ secrets.S3_BUCKET }}/cli/public/templates/$CLEAN_VERSION/ --acl-public - - - name: Upload Latest Templates (Stable Only) - if: ${{ !inputs.is_prerelease }} - run: | - s3cmd sync .github/assets/templates/ s3://${{ secrets.S3_BUCKET }}/cli/public/templates/latest/ --acl-public diff --git a/.gitignore b/.gitignore index ecc0d2a..e2556d1 100644 --- a/.gitignore +++ b/.gitignore @@ -2,9 +2,11 @@ dist/ build/ .venv/ __pycache__/ -uv.lock +# Keep portabase.spec: it is the build definition. *.spec +!portabase.spec .claude .codex/ -/docs \ No newline at end of file + +test/ diff --git a/.gitleaks.toml b/.gitleaks.toml new file mode 100644 index 0000000..f0136f6 --- /dev/null +++ b/.gitleaks.toml @@ -0,0 +1,19 @@ +title = "portabase-cli" + +[extend] +useDefault = true + +[allowlist] +description = "Known false positives" +paths = [ + '''templates/.*''', + '''uv\.lock''', +] +regexes = [ + '''\$\{[A-Z0-9_]+\}''', + '''"masterKeyB64"''', +] +commits = [ + "074f436a291a49b8b9353bb5f7b536c41d9ecf52", + "c00f128f8e40964c4afe3acb77fcbd6670a9d5fa", +] diff --git a/CITATION.cff b/CITATION.cff index f51f2dd..69b7e28 100644 --- a/CITATION.cff +++ b/CITATION.cff @@ -7,6 +7,8 @@ authors: given-names: Charles - family-names: Larcher given-names: Killian + - family-names: Lagache + given-names: Théo repository-code: https://github.com/Portabase/cli url: https://portabase.io abstract: "Portabase CLI is a command-line interface tool designed to streamline and enhance the management of Portabase services, providing developers with efficient access to core functionalities directly from the terminal." diff --git a/README.md b/README.md index a5b1f67..d59ef84 100644 --- a/README.md +++ b/README.md @@ -10,7 +10,7 @@ The official command line interface (CLI) for managing and deploying Portabase instances with ease.

- +[![Plumber Score](https://score.getplumber.io/github.com/Portabase/cli.svg)](https://score.getplumber.io/github.com/Portabase/cli) [![License: Apache](https://img.shields.io/badge/License-apache-yellow.svg)](LICENSE) [![Platform](https://img.shields.io/badge/platform-linux%20%7C%20macos%20%7C%20windows-lightgrey)](https://github.com/Portabase/portabase) @@ -63,3 +63,14 @@ Distributed under the Apache License. See `LICENSE.txt` for more details. + +## Upgrading from 26.08 or earlier + +From this release the CLI owns `docker-compose.yml`: it is re-rendered from your +`.env` and `databases.json` whenever you run `portabase db add`, `db remove` or +`build`. The first time that happens on an older install, the existing file is +copied to `docker-compose.legacy.yml` first. + +- Preview the change before applying it: `portabase build --diff` +- Keep your own customisations in `docker-compose.override.yml`; Docker Compose + merges it automatically and the CLI never touches it. diff --git a/commands/agent.py b/commands/agent.py index b03c402..d06162e 100644 --- a/commands/agent.py +++ b/commands/agent.py @@ -1,949 +1,145 @@ -import os -import secrets -import uuid +from __future__ import annotations + from pathlib import Path -from typing import Optional +from typing import Annotated -import questionary import typer -from rich.panel import Panel -from rich.prompt import Confirm, IntPrompt, Prompt -from rich.table import Table - -from core.config import add_db_to_json, load_db_config, write_env_file, write_file -from core.docker import ensure_network, run_compose -from core.network import fetch_template -from core.utils import ( - check_system, - console, - generate_password, - get_free_port, - get_random_hint, - print_banner, - questionary_style, - validate_edge_key, -) -from templates.compose import ( - AGENT_FIREBIRD_SNIPPET, - AGENT_MARIADB_SNIPPET, - AGENT_MONGODB_AUTH_SNIPPET, - AGENT_MONGODB_SNIPPET, - AGENT_MSSQL_SNIPPET, - AGENT_POSTGRES_SNIPPET, - AGENT_REDIS_AUTH_SNIPPET, - AGENT_REDIS_SNIPPET, - AGENT_VALKEY_AUTH_SNIPPET, - AGENT_VALKEY_SNIPPET, -) - - -def agent( - name: str = typer.Argument(..., help="Name of the agent (creates a folder)"), - key: Optional[str] = typer.Option(None, "--key", "-k", help="Edge Key"), - tz: str = typer.Option("UTC", "--tz", help="Timezone"), - polling: int = typer.Option(5, "--polling", help="Polling frequency in seconds"), - start: bool = typer.Option(False, "--start", "-s", help="Start immediately"), -): - - print_banner() - check_system() - ensure_network("portabase_network") - - path = Path(name).resolve() - if path.exists(): - console.print(f"[warning]Directory '{name}' already exists.[/warning]") - if not Confirm.ask("Overwrite?"): - raise typer.Exit() - - path.mkdir(parents=True, exist_ok=True) - - if not key: - key = Prompt.ask("[key]Edge Key[/key]") - if not validate_edge_key(key): - console.print( - "[danger]✖ Invalid Edge Key. Please check the format (Base64 or JSON).[/danger]" +from commands.base import Command +from commands.db import report_write +from commands.flows.add_database import AddDatabaseFlow +from core.errors import ValidationError +from core.utils import validate_edge_key +from engines import EngineRegistry +from services.docker import DockerRunner +from services.ports import PortAllocator +from services.project import AgentProject +from services.renderer import ComposeRenderer +from services.telemetry import Telemetry +from services.templates import TemplateRepository +from ui import UI + +NETWORK = "portabase_network" + + +def _edge_key(value: str) -> str: + if not validate_edge_key(value): + raise ValidationError( + "Invalid Edge Key.", + hint="Expected Base64 or JSON with serverUrl, agentId, masterKeyB64.", + ) + return value + + +class AgentCommand(Command): + name, help, panel = "agent", "Create a new Portabase Agent instance.", "Creation" + no_args_is_help = True + + def __init__( + self, + ui: UI, + telemetry: Telemetry, + docker: DockerRunner, + templates: TemplateRepository, + renderer: ComposeRenderer, + engines: EngineRegistry, + ports: PortAllocator, + ) -> None: + super().__init__(ui, telemetry) + self.docker = docker + self.templates = templates + self.renderer = renderer + self.engines = engines + self.ports = ports + + def run( + self, + name: Annotated[str, typer.Argument(help="Agent name (creates a folder)")], + key: Annotated[str | None, typer.Option("--key", "-k", help="Edge Key")] = None, + tz: Annotated[str | None, typer.Option("--tz", help="Timezone")] = None, + polling: Annotated[ + int | None, typer.Option("--polling", help="Polling frequency in seconds") + ] = None, + host_gateway: Annotated[ + bool | None, + typer.Option( + "--host-gateway/--no-host-gateway", + help="Map localhost to host-gateway", + ), + ] = None, + start: Annotated[ + bool, typer.Option("--start", "-s", help="Start immediately") + ] = False, + force: Annotated[ + bool, typer.Option("--force", "-f", help="Overwrite an existing folder") + ] = False, + ) -> None: + self.ui.banner() + self.require_docker(self.docker) + self.docker.ensure_network(NETWORK) + self.templates.resolve() + + path = Path(name).resolve() + if path.exists() and not force: + self.ui.warning(f"Directory '{name}' already exists.") + self.confirm_or_abort("Overwrite?", default=False) + + form = self.ui.form() + env_vars = { + "EDGE_KEY": form.text( + "Edge Key", value=key, validator=_edge_key, name="key" + ), + "TZ": form.text("Timezone", value=tz, default="UTC", name="tz"), + "POLLING": str( + form.integer( + "Polling frequency (seconds)", + value=polling, + default=5, + name="polling", + ) + ), + "LOG_LEVEL": "info", + } + gateway = form.confirm( + "Add extra_hosts mapping (localhost -> host-gateway)?", + value=host_gateway, + default=False, + name="host_gateway", ) - raise typer.Exit(1) - - if not tz or tz == "UTC": - tz = Prompt.ask("Timezone", default="UTC") - - if polling == 5: - polling = IntPrompt.ask("Polling frequency (seconds)", default=5) - - add_host_gateway = Confirm.ask( - "Add extra_hosts mapping (localhost -> host-gateway)?", default=False - ) - - raw_template = fetch_template("agent.yml") - if "{{EXTRA_SERVICES}}" not in raw_template: - if "\nnetworks:" in raw_template: - raw_template = raw_template.replace( - "\nnetworks:", "\n\n{{EXTRA_SERVICES}}\n\nnetworks:" - ) - else: - raw_template += "\n\n{{EXTRA_SERVICES}}\n" + project = AgentProject.create(path, env_vars, host_gateway=gateway) + self._write(project) + self.ui.success(f"Agent '{name}' created in {path}") - if "{{EXTRA_VOLUMES}}" not in raw_template: - if "\nnetworks:" in raw_template: - raw_template = raw_template.replace( - "\nnetworks:", "\n\n{{EXTRA_VOLUMES}}\n\nnetworks:" + if self.ui.non_interactive: + self.ui.hint( + f"Add databases with: portabase db add {name} " + "--engine postgresql --mode new" ) else: - raw_template += "\n\n{{EXTRA_VOLUMES}}\n" - - env_vars = { - "EDGE_KEY": key, - "TZ": tz, - "POLLING": str(polling), - "LOG_LEVEL": "info", - } - - extra_services = "" - extra_volumes = "" - app_volumes = ["./databases.json:/config/config.json"] - volumes_list = [] - - json_path = path / "databases.json" - if not json_path.exists(): - write_file(json_path, '{"databases": []}') - try: - os.chmod(json_path, 0o666) - except: - pass - - console.print("") - console.print(Panel("[bold]Database Setup[/bold]", style="cyan")) - - while True: - storage_kind = questionary.select( - "What do you want to configure?", - choices=["done", "database", "docker-volume"], - default="database", - style=questionary_style, - ).ask() - - if storage_kind in (None, "done"): - break - - if storage_kind == "docker-volume": - console.print( - "[warning]⚠ Requires the Docker socket. It will be mounted " - "on the agent ([bold]/var/run/docker.sock[/bold]).[/warning]" - ) - friendly_name = Prompt.ask("Display Name", default="Docker Volume") - volume_name = Prompt.ask("Volume Name (e.g. databases_sqlite-data)").strip() - while not volume_name: - console.print("[danger]✖ Volume Name is required.[/danger]") - volume_name = Prompt.ask( - "Volume Name (e.g. databases_sqlite-data)" - ).strip() - container_name = Prompt.ask( - "Container Name (optional, enables auto-restart after restore)", - default="", - ) - dv_entry = { - "name": friendly_name, - "type": "docker-volume", - "volume_name": volume_name, - "generated_id": str(uuid.uuid4()), - } - if container_name: - dv_entry["container_name"] = container_name - - sock_mount = "/var/run/docker.sock:/var/run/docker.sock" - if sock_mount not in app_volumes: - app_volumes.append(sock_mount) - - add_db_to_json(path, dv_entry) - console.print("[success]✔ Added to config[/success]") - continue - - while True: - mode = Prompt.ask( - "Configuration Mode", choices=["new", "existing", "back"], default="new" - ) - - if mode == "back": - break - - if mode == "existing": - console.print("[info]External/Existing Database Configuration[/info]") - db_type = questionary.select( - "Select Database Type", - choices=[ - "back", - "postgresql", - "postgresql-cluster", - "mysql", - "mariadb", - "sqlite", - "firebird", - "mongodb", - "redis", - "valkey", - "mssql", - ], - style=questionary_style, - ).ask() - - if db_type == "back": - continue - - if not db_type: - raise typer.Exit() - - if db_type == "postgresql-cluster": - console.print( - "[warning]⚠ Postgres Cluster requires a superuser. " - "Cluster backup/restore uses pg_dumpall, which dumps all " - "databases and global objects (roles, tablespaces). " - "The provided user must be a Postgres superuser.[/warning]" - ) - - friendly_name = Prompt.ask("Display Name", default="External DB") - - if db_type == "sqlite": - db_name = Prompt.ask("Database Path (relative or absolute)") - if not db_name.startswith("/"): - app_volumes.append(f"./{db_name}:/config/{db_name}") - container_path = f"/config/{db_name}" - else: - container_path = db_name - - add_db_to_json( - path, - { - "name": friendly_name, - "database": container_path, - "type": db_type, - "generated_id": str(uuid.uuid4()), - }, - ) - else: - db_name = Prompt.ask("Database Name") - host = Prompt.ask("Host", default="localhost") - port = IntPrompt.ask( - "Port", - default=5432 - if db_type in ["postgresql", "postgresql-cluster"] - else ( - 3050 - if db_type == "firebird" - else ( - 1433 - if db_type == "mssql" - else ( - 3306 if db_type in ["mysql", "mariadb"] else 27017 - ) - ) - ), - ) - user = Prompt.ask("Username") - password = questionary.password( - "Password", style=questionary_style - ).ask() - if password is None: - raise typer.Exit() - - ext_entry = { - "name": friendly_name, - "database": db_name, - "type": db_type, - "username": user, - "password": password, - "port": port, - "host": host, - "generated_id": str(uuid.uuid4()), - } - if db_type == "postgresql": - console.print( - "[info]ℹ When enabled, omits [bold]--no-owner[/bold] and " - "[bold]--no-privileges[/bold] from the dump. Ownership and role " - "assignments are preserved in the output. By default, these flags " - "are applied to keep restores portable across different users and " - "environments, for example when migrating from one database " - "instance to another.[/info]" - ) - - keep_ownership = Confirm.ask("Keep ownership?", default=False) - - if keep_ownership is None: - raise typer.Exit() - - console.print( - "[info]ℹ Controls how the target database is cleaned before " - "a restore. [bold]pg_restore --clean[/bold] only drops " - "objects listed in the backup's own table of contents, so " - "anything already present in the target that the dump does " - "not know about survives and can make the restore " - "fail.[/info]" - ) - clean_mode = questionary.select( - "Clean mode", - choices=[ - questionary.Choice( - "clean - pg_restore --clean --if-exists (default)", - value="clean", - ), - questionary.Choice( - "none - no pre-clean, restore into an empty database", - value="none", - ), - questionary.Choice( - "drop_schemas - drop every non-system schema CASCADE " - "(recommended, works on managed Postgres)", - value="drop_schemas", - ), - questionary.Choice( - "drop_database - DROP DATABASE + CREATE DATABASE " - "(full reset)", - value="drop_database", - ), - ], - default="clean", - style=questionary_style, - ).ask() - if clean_mode is None: - raise typer.Exit() - if clean_mode == "drop_database": - console.print( - "[warning]⚠ drop_database drops the whole target " - "database before restoring. The user must have CREATEDB " - "and own the database, or be a superuser. Most managed " - "Postgres providers do not allow it.[/warning]" - ) - - pg_options = {} - if keep_ownership: - pg_options["keep_ownership"] = True - if clean_mode != "clean": - pg_options["clean_mode"] = clean_mode - if pg_options: - ext_entry["options"] = pg_options - - add_db_to_json(path, ext_entry) - console.print("[success]✔ Added to config[/success]") - break - - else: - console.print("[info]New Local Docker Container[/info]") - db_engine = questionary.select( - "Select Database Engine", - choices=[ - "back", - "postgresql", - "postgresql-cluster", - "mysql", - "mariadb", - "sqlite", - "firebird", - "mongodb", - "redis", - "valkey", - "mssql", - ], - style=questionary_style, - ).ask() - - if db_engine == "back": - continue - - if not db_engine: - raise typer.Exit() - - db_variant = "no-auth" - if db_engine in ["mongodb", "redis", "valkey"]: - engine_display = { - "mongodb": "MongoDB", - "redis": "Redis", - "valkey": "Valkey", - }[db_engine] - db_variant = questionary.select( - f"Select {engine_display} Variant", - choices=["back", "no-auth", "with-auth"], - style=questionary_style, - ).ask() - - if db_variant == "back": - continue - - if not db_variant: - raise typer.Exit() - - if db_engine == "sqlite": - db_name = Prompt.ask("Database Name", default="local") - if not db_name.endswith(".sqlite"): - db_name += ".sqlite" - - app_volumes.append(f"./{db_name}:/config/{db_name}") - - add_db_to_json( - path, - { - "name": db_name, - "database": f"/config/{db_name}", - "type": "sqlite", - "generated_id": str(uuid.uuid4()), - }, - ) - console.print( - f"[success]✔ Added SQLite database ({db_name})[/success]" - ) - - elif db_engine in ["postgresql", "postgresql-cluster"]: - pg_port = get_free_port() - db_user = "admin" - db_pass = generate_password(16) - db_name = f"pg_{secrets.token_hex(4)}" - service_name = f"db-pg-{secrets.token_hex(2)}" - - var_prefix = service_name.upper().replace("-", "_") - env_vars[f"{var_prefix}_PORT"] = str(pg_port) - env_vars[f"{var_prefix}_DB"] = db_name - env_vars[f"{var_prefix}_USER"] = db_user - env_vars[f"{var_prefix}_PASS"] = db_pass - - snippet = ( - AGENT_POSTGRES_SNIPPET.replace("${SERVICE_NAME}", service_name) - .replace("${PORT}", f"${{{var_prefix}_PORT}}") - .replace("${VOL_NAME}", f"{service_name}-data") - .replace("${DB_NAME}", f"${{{var_prefix}_DB}}") - .replace("${USER}", f"${{{var_prefix}_USER}}") - .replace("${PASSWORD}", f"${{{var_prefix}_PASS}}") - ) - - extra_services += snippet - volumes_list.append(f"{service_name}-data") - - pg_entry = { - "name": db_name, - "database": db_name, - "type": db_engine, - "username": db_user, - "password": db_pass, - "port": 5432, - "host": service_name, - "generated_id": str(uuid.uuid4()), - } - if db_engine == "postgresql": - console.print( - "[info]ℹ When enabled, omits [bold]--no-owner[/bold] and " - "[bold]--no-privileges[/bold] from the dump. Ownership and role " - "assignments are preserved in the output. By default, these flags " - "are applied to keep restores portable across different users and " - "environments, for example when migrating from one database " - "instance to another.[/info]" - ) - keep_ownership = Confirm.ask("Keep ownership?", default=False) - - if keep_ownership is None: - raise typer.Exit() - - console.print( - "[info]ℹ Controls how the target database is cleaned before " - "a restore. [bold]pg_restore --clean[/bold] only drops " - "objects listed in the backup's own table of contents, so " - "anything already present in the target that the dump does " - "not know about survives and can make the restore " - "fail.[/info]" - ) - clean_mode = questionary.select( - "Clean mode", - choices=[ - questionary.Choice( - "clean - pg_restore --clean --if-exists (default)", - value="clean", - ), - questionary.Choice( - "none - no pre-clean, restore into an empty database", - value="none", - ), - questionary.Choice( - "drop_schemas - drop every non-system schema CASCADE " - "(recommended, works on managed Postgres)", - value="drop_schemas", - ), - questionary.Choice( - "drop_database - DROP DATABASE + CREATE DATABASE " - "(full reset)", - value="drop_database", - ), - ], - default="clean", - style=questionary_style, - ).ask() - if clean_mode is None: - raise typer.Exit() - if clean_mode == "drop_database": - console.print( - "[warning]⚠ drop_database drops the whole target " - "database before restoring. The user must have CREATEDB " - "and own the database, or be a superuser. Most managed " - "Postgres providers do not allow it.[/warning]" - ) - - pg_options = {} - if keep_ownership: - pg_options["keep_ownership"] = True - if clean_mode != "clean": - pg_options["clean_mode"] = clean_mode - if pg_options: - pg_entry["options"] = pg_options - - add_db_to_json(path, pg_entry) - - label = ( - "Postgres Cluster" - if db_engine == "postgresql-cluster" - else "Postgres" - ) - console.print( - f"[success]✔ Added {label} container (Port {pg_port})[/success]" - ) - - elif db_engine == "mariadb" or db_engine == "mysql": - mysql_port = get_free_port() - db_user = "admin" - db_pass = generate_password(16) - db_name = f"mysql_{secrets.token_hex(4)}" - service_name = f"db-mariadb-{secrets.token_hex(2)}" - - var_prefix = service_name.upper().replace("-", "_") - env_vars[f"{var_prefix}_PORT"] = str(mysql_port) - env_vars[f"{var_prefix}_DB"] = db_name - env_vars[f"{var_prefix}_USER"] = db_user - env_vars[f"{var_prefix}_PASS"] = db_pass - - snippet = ( - AGENT_MARIADB_SNIPPET.replace("${SERVICE_NAME}", service_name) - .replace("${PORT}", f"${{{var_prefix}_PORT}}") - .replace("${VOL_NAME}", f"{service_name}-data") - .replace("${DB_NAME}", f"${{{var_prefix}_DB}}") - .replace("${USER}", f"${{{var_prefix}_USER}}") - .replace("${PASSWORD}", f"${{{var_prefix}_PASS}}") - ) - - extra_services += snippet - volumes_list.append(f"{service_name}-data") - - add_db_to_json( - path, - { - "name": db_name, - "database": db_name, - "type": db_engine, - "username": db_user, - "password": db_pass, - "port": 3306, - "host": service_name, - "generated_id": str(uuid.uuid4()), - }, - ) - console.print( - f"[success]✔ Added MariaDB container (Port {mysql_port})[/success]" - ) - - elif db_engine == "mongodb": - if db_variant == "with-auth": - mongo_port = get_free_port() - db_user = "admin" - db_pass = generate_password(16) - db_name = f"mongo_{secrets.token_hex(4)}" - service_name = f"db-mongo-auth-{secrets.token_hex(2)}" - - var_prefix = service_name.upper().replace("-", "_") - env_vars[f"{var_prefix}_PORT"] = str(mongo_port) - env_vars[f"{var_prefix}_DB"] = db_name - env_vars[f"{var_prefix}_USER"] = db_user - env_vars[f"{var_prefix}_PASS"] = db_pass - - snippet = ( - AGENT_MONGODB_AUTH_SNIPPET.replace( - "${SERVICE_NAME}", service_name - ) - .replace("${PORT}", f"${{{var_prefix}_PORT}}") - .replace("${VOL_NAME}", f"{service_name}-data") - .replace("${DB_NAME}", f"${{{var_prefix}_DB}}") - .replace("${USER}", f"${{{var_prefix}_USER}}") - .replace("${PASSWORD}", f"${{{var_prefix}_PASS}}") - ) - - extra_services += snippet - volumes_list.append(f"{service_name}-data") - - add_db_to_json( - path, - { - "name": db_name, - "database": db_name, - "type": "mongodb", - "username": db_user, - "password": db_pass, - "port": 27017, - "host": service_name, - "generated_id": str(uuid.uuid4()), - }, - ) - console.print( - f"[success]✔ Added MongoDB Auth container (Port {mongo_port})[/success]" - ) - else: - mongo_port = get_free_port() - db_name = f"mongo_{secrets.token_hex(4)}" - service_name = f"db-mongo-{secrets.token_hex(2)}" - - var_prefix = service_name.upper().replace("-", "_") - env_vars[f"{var_prefix}_PORT"] = str(mongo_port) - env_vars[f"{var_prefix}_DB"] = db_name - - snippet = ( - AGENT_MONGODB_SNIPPET.replace( - "${SERVICE_NAME}", service_name - ) - .replace("${PORT}", f"${{{var_prefix}_PORT}}") - .replace("${VOL_NAME}", f"{service_name}-data") - .replace("${DB_NAME}", f"${{{var_prefix}_DB}}") - ) - extra_services += snippet - volumes_list.append(f"{service_name}-data") - - add_db_to_json( - path, - { - "name": db_name, - "database": db_name, - "type": "mongodb", - "username": "", - "password": "", - "port": 27017, - "host": service_name, - "generated_id": str(uuid.uuid4()), - }, - ) - - console.print( - f"[success]✔ Added MongoDB container (Port {mongo_port})[/success]" - ) - - elif db_engine == "redis": - if db_variant == "with-auth": - redis_port = get_free_port() - db_name = f"redis_{secrets.token_hex(4)}" - service_name = f"db-redis-auth-{secrets.token_hex(2)}" - db_pass = generate_password(16) - - var_prefix = service_name.upper().replace("-", "_") - env_vars[f"{var_prefix}_PORT"] = str(redis_port) - env_vars[f"{var_prefix}_PASS"] = db_pass - - snippet = ( - AGENT_REDIS_AUTH_SNIPPET.replace( - "${SERVICE_NAME}", service_name - ) - .replace("${PORT}", f"${{{var_prefix}_PORT}}") - .replace("${VOL_NAME}", f"{service_name}-data") - .replace("${PASSWORD}", f"${{{var_prefix}_PASS}}") - ) - - extra_services += snippet - volumes_list.append(f"{service_name}-data") - - add_db_to_json( - path, - { - "name": db_name, - "database": "0", - "type": "redis", - "username": "", - "password": db_pass, - "port": 6379, - "host": service_name, - "generated_id": str(uuid.uuid4()), - }, - ) - console.print( - f"[success]✔ Added Redis Auth container (Port {redis_port})[/success]" - ) - else: - redis_port = get_free_port() - db_name = f"redis_{secrets.token_hex(4)}" - service_name = f"db-redis-{secrets.token_hex(2)}" - - var_prefix = service_name.upper().replace("-", "_") - env_vars[f"{var_prefix}_PORT"] = str(redis_port) - - snippet = ( - AGENT_REDIS_SNIPPET.replace("${SERVICE_NAME}", service_name) - .replace("${PORT}", f"${{{var_prefix}_PORT}}") - .replace("${VOL_NAME}", f"{service_name}-data") - ) - - extra_services += snippet - volumes_list.append(f"{service_name}-data") - - add_db_to_json( - path, - { - "name": db_name, - "database": "0", - "type": "redis", - "username": "", - "password": "", - "port": 6379, - "host": service_name, - "generated_id": str(uuid.uuid4()), - }, - ) - console.print( - f"[success]✔ Added Redis container (Port {redis_port})[/success]" - ) - - elif db_engine == "valkey": - if db_variant == "with-auth": - valkey_port = get_free_port() - db_name = f"valkey_{secrets.token_hex(4)}" - service_name = f"db-valkey-auth-{secrets.token_hex(2)}" - db_pass = generate_password(16) - - var_prefix = service_name.upper().replace("-", "_") - env_vars[f"{var_prefix}_PORT"] = str(valkey_port) - env_vars[f"{var_prefix}_PASS"] = db_pass - - snippet = ( - AGENT_VALKEY_AUTH_SNIPPET.replace( - "${SERVICE_NAME}", service_name - ) - .replace("${PORT}", f"${{{var_prefix}_PORT}}") - .replace("${VOL_NAME}", f"{service_name}-data") - .replace("${PASSWORD}", f"${{{var_prefix}_PASS}}") - ) - - extra_services += snippet - volumes_list.append(f"{service_name}-data") - - add_db_to_json( - path, - { - "name": db_name, - "database": "0", - "type": "valkey", - "username": "", - "password": db_pass, - "port": 6379, - "host": service_name, - "generated_id": str(uuid.uuid4()), - }, - ) - console.print( - f"[success]✔ Added Valkey Auth container (Port {valkey_port})[/success]" - ) - else: - valkey_port = get_free_port() - db_name = f"valkey_{secrets.token_hex(4)}" - service_name = f"db-valkey-{secrets.token_hex(2)}" - - var_prefix = service_name.upper().replace("-", "_") - env_vars[f"{var_prefix}_PORT"] = str(valkey_port) - - snippet = ( - AGENT_VALKEY_SNIPPET.replace( - "${SERVICE_NAME}", service_name - ) - .replace("${PORT}", f"${{{var_prefix}_PORT}}") - .replace("${VOL_NAME}", f"{service_name}-data") - ) - - extra_services += snippet - volumes_list.append(f"{service_name}-data") - - add_db_to_json( - path, - { - "name": db_name, - "database": "0", - "type": "valkey", - "username": "", - "password": "", - "port": 6379, - "host": service_name, - "generated_id": str(uuid.uuid4()), - }, - ) - console.print( - f"[success]✔ Added Valkey container (Port {valkey_port})[/success]" - ) - - elif db_engine == "firebird": - fb_port = get_free_port() - db_user = "alice" - db_pass = generate_password(16) - db_root_pass = generate_password(16) - db_name = "mirror.fdb" - db_container_path = f"/var/lib/firebird/data/{db_name}" - service_name = f"db-firebird-{secrets.token_hex(2)}" - - var_prefix = service_name.upper().replace("-", "_") - env_vars[f"{var_prefix}_PORT"] = str(fb_port) - env_vars[f"{var_prefix}_DB"] = db_name - env_vars[f"{var_prefix}_USER"] = db_user - env_vars[f"{var_prefix}_PASS"] = db_pass - env_vars[f"{var_prefix}_ROOT_PASS"] = db_root_pass - - snippet = ( - AGENT_FIREBIRD_SNIPPET.replace("${SERVICE_NAME}", service_name) - .replace("${PORT}", f"${{{var_prefix}_PORT}}") - .replace("${VOL_NAME}", f"{service_name}-data") - .replace("${DB_NAME}", f"${{{var_prefix}_DB}}") - .replace("${USER}", f"${{{var_prefix}_USER}}") - .replace("${PASSWORD}", f"${{{var_prefix}_PASS}}") - .replace("${ROOT_PASSWORD}", f"${{{var_prefix}_ROOT_PASS}}") - ) - - extra_services += snippet - volumes_list.append(f"{service_name}-data") - - add_db_to_json( - path, - { - "name": db_name, - "database": db_container_path, - "type": "firebird", - "username": db_user, - "password": db_pass, - "port": 3050, - "host": service_name, - "generated_id": str(uuid.uuid4()), - }, - ) - console.print( - f"[success]✔ Added Firebird container (Port {fb_port})[/success]" - ) - - elif db_engine == "mssql": - mssql_port = get_free_port() - db_pass = generate_password(16) - db_name = "master" - service_name = f"db-mssql-{secrets.token_hex(2)}" - - var_prefix = service_name.upper().replace("-", "_") - env_vars[f"{var_prefix}_PORT"] = str(mssql_port) - env_vars[f"{var_prefix}_PASS"] = db_pass - - snippet = ( - AGENT_MSSQL_SNIPPET.replace("${SERVICE_NAME}", service_name) - .replace("${PORT}", f"${{{var_prefix}_PORT}}") - .replace("${VOL_NAME}", f"{service_name}-data") - .replace("${PASSWORD}", f"${{{var_prefix}_PASS}}") - ) - - extra_services += snippet - volumes_list.append(f"{service_name}-data") - - add_db_to_json( - path, - { - "name": "MSSQL", - "database": db_name, - "type": "mssql", - "username": "sa", - "password": db_pass, - "port": 1433, - "host": service_name, - "generated_id": str(uuid.uuid4()), - }, - ) - console.print( - f"[success]✔ Added MSSQL container (Port {mssql_port})[/success]" - ) - break - - if volumes_list: - extra_volumes = "volumes:\n" - for vol in volumes_list: - extra_volumes += f" {vol}:\n" - - final_compose = raw_template.replace("{{EXTRA_SERVICES}}", extra_services) - final_compose = final_compose.replace("{{EXTRA_VOLUMES}}", extra_volumes) - - vols_str = "\n".join([f" - {v}" for v in app_volumes]) - final_compose = final_compose.replace( - " - ./databases.json:/config/config.json", vols_str - ) - - if add_host_gateway: - final_compose = final_compose.replace( - " image: portabase/agent:latest\n", - " image: portabase/agent:latest\n" - " extra_hosts:\n" - ' - "localhost:host-gateway"\n', - ) - - summary = Table(show_header=False, box=None, padding=(0, 2)) - summary.add_column("Property", style="bold cyan") - summary.add_column("Value", style="white") - - summary.add_row("Agent Name", name) - summary.add_row("Path", str(path)) - summary.add_row("Edge Key", f"{key[:10]}...{key[-10:]}" if len(key) > 20 else key) - summary.add_row("Timezone", tz) - summary.add_row("Polling", f"{polling}s") - summary.add_row("Host Gateway", "Yes" if add_host_gateway else "No") - - db_config = load_db_config(path) - dbs = db_config.get("databases", []) - if dbs: - db_details = [] - for db in dbs: - if db.get("type") == "sqlite": - db_details.append(f"• {db['name']} (sqlite: {db['database']})") - elif db.get("type") == "docker-volume": - db_details.append( - f"• {db['name']} (docker-volume: {db.get('volume_name', 'N/A')})" + self.ui.section("Database Setup") + flow = AddDatabaseFlow(self.ui, self.engines, self.ports) + while self.ui.confirm("Add a database?", default=True): + spec, engine = flow.collect({}) + flow.apply(project, spec, engine) + self._write(project) + self.ui.success( + f"Added {engine.display} '{spec.name}' ({engine.describe(spec)})" ) - else: - db_details.append( - f"• {db['name']} ({db['type']} on port {db.get('port', 'N/A')})" - ) - - summary.add_row("Databases", "\n".join(db_details)) - else: - summary.add_row("Databases", "[dim]None configured[/dim]") - summary.add_row("Files to Create", "• docker-compose.yml\n• .env\n• databases.json") - - console.print("") - console.print( - Panel( - summary, - title="[bold white]PROPOSED CONFIGURATION[/bold white]", - border_style="bold blue", - expand=False, - ) - ) - console.print( - "[dim]The agent will be configured in the directory above and ready for deployment.[/dim]\n" - ) - - if not Confirm.ask( - "[bold]Apply this configuration and generate files?[/bold]", default=True - ): - console.print("[warning]Configuration cancelled.[/warning]") - raise typer.Exit() - - write_file(path / "docker-compose.yml", final_compose) - write_env_file(path, env_vars) - - console.print( - Panel(f"[bold white]AGENT READY: {name}[/bold white]", style="bold #5f00d7") - ) - - if start or Confirm.ask("Start agent now?", default=False): - status_msg = f"[bold magenta]Starting...[/bold magenta]\n{get_random_hint()}" - with console.status(status_msg, spinner="earth"): - run_compose(path, ["up", "-d"]) - console.print(f"[bold green]✔ Agent {name} is running[/bold green]") - else: - console.print(f"[info]Run: portabase start {name}[/info]") + if start or ( + not self.ui.non_interactive + and self.ui.confirm("Start agent now?", default=False) + ): + with self.ui.status("Starting agent..."): + self.docker.compose(path, ["up", "-d"]) + self.ui.success("Agent started.") + else: + self.ui.info(f"Run: portabase start {name}") + + def _write(self, project: AgentProject) -> None: + with self.ui.status("Rendering configuration..."): + result = self.renderer.render_agent(project) + project.save_state() + report = result.write(project.path) + report_write(self.ui, report) diff --git a/commands/base.py b/commands/base.py new file mode 100644 index 0000000..8aabba8 --- /dev/null +++ b/commands/base.py @@ -0,0 +1,105 @@ +from __future__ import annotations + +import functools +from abc import ABC, abstractmethod +from collections.abc import Callable +from pathlib import Path + +import typer + +from core.errors import ConfigError, DockerError, UserAbort +from services.docker import DockerRunner +from services.telemetry import Telemetry +from ui import UI + + +class Command(ABC): + name: str + help: str + panel: str = "General" + no_args_is_help: bool = False + + def __init__(self, ui: UI, telemetry: Telemetry) -> None: + self.ui = ui + self.telemetry = telemetry + + def register(self, app: typer.Typer) -> None: + app.command( + self.name, + help=self.help, + rich_help_panel=self.panel, + no_args_is_help=self.no_args_is_help, + )(self._traced(self.run)) + + def _traced(self, fn: Callable) -> Callable: + @functools.wraps(fn) + def wrapper(*args, **kwargs): + with self.telemetry.span(f"command.{self.name}"): + return fn(*args, **kwargs) + + return wrapper + + @abstractmethod + def run(self, *args, **kwargs) -> None: ... + + def require_docker(self, docker: DockerRunner) -> None: + if not docker.available(): + raise DockerError( + "Docker not found (binary missing).", + hint="Install Docker: https://docs.docker.com/get-docker/", + ) + if docker.daemon_running(): + return + self.ui.warning("Docker is installed but the daemon is not running.") + if self.ui.confirm("Do you want to try starting Docker?", default=False): + with self.ui.status("Waiting for Docker to start..."): + started = docker.start_daemon() + if started: + self.ui.success("Docker started successfully.") + return + raise DockerError( + "Docker is required to continue.", + hint="Start the Docker daemon and retry.", + ) + + @staticmethod + def require_project_dir(path: Path) -> Path: + path = path.resolve() + if not (path / "docker-compose.yml").exists(): + raise ConfigError( + f"No Portabase configuration found in: {path}", + hint=( + "Expected a docker-compose.yml created by " + "'portabase agent' or 'portabase dashboard'." + ), + ) + return path + + def confirm_or_abort( + self, question: str, *, default: bool = False, value: bool | None = None + ) -> None: + if not self.ui.confirm(question, default=default, value=value): + raise UserAbort() + + +class CommandGroup(ABC): + name: str + help: str + panel: str = "General" + + def __init__(self, ui: UI, telemetry: Telemetry) -> None: + self.ui = ui + self.telemetry = telemetry + + @property + @abstractmethod + def commands(self) -> list[Command]: ... + + def build_typer(self) -> typer.Typer: + sub = typer.Typer(help=self.help, no_args_is_help=True) + for cmd in self.commands: + cmd.register(sub) + return sub + + def register(self, app: typer.Typer) -> None: + app.add_typer(self.build_typer(), name=self.name, rich_help_panel=self.panel) diff --git a/commands/build.py b/commands/build.py new file mode 100644 index 0000000..4b5465e --- /dev/null +++ b/commands/build.py @@ -0,0 +1,98 @@ +from __future__ import annotations + +from pathlib import Path +from typing import Annotated + +import typer + +from commands.base import Command +from commands.db import report_write +from core.errors import ValidationError +from services.project import ( + ENV_FILE, + AgentProject, + DashboardProject, + detect_kind, +) +from services.renderer import ComposeRenderer, RenderResult +from services.telemetry import Telemetry +from services.templates import TemplateRepository +from ui import UI + + +class BuildCommand(Command): + name = "build" + help = "Re-render docker-compose.yml from the component's configuration." + panel = "Configuration" + no_args_is_help = True + + def __init__( + self, + ui: UI, + telemetry: Telemetry, + templates: TemplateRepository, + renderer: ComposeRenderer, + ) -> None: + super().__init__(ui, telemetry) + self.templates = templates + self.renderer = renderer + + def run( + self, + path: Annotated[Path, typer.Argument(help="Component folder")], + diff: Annotated[ + bool, typer.Option("--diff", help="Show the diff, write nothing") + ] = False, + stdout: Annotated[ + bool, typer.Option("--stdout", help="Print the compose, write nothing") + ] = False, + inline_env: Annotated[ + bool, + typer.Option( + "--inline-env", help="Substitute values instead of ${VAR} references" + ), + ] = False, + output: Annotated[ + Path | None, + typer.Option("--output", "-o", help="Write files to another directory"), + ] = None, + ) -> None: + if sum([diff, stdout, output is not None]) > 1: + raise ValidationError("Use only one of --diff, --stdout, --output.") + path = self.require_project_dir(path) + self.templates.resolve() + kind = detect_kind(path) + + if kind == "agent": + agent = AgentProject.load(path) + result: RenderResult = self.renderer.render_agent(agent, inline=inline_env) + else: + dashboard = DashboardProject.load(path) + result = self.renderer.render_dashboard(dashboard, inline=inline_env) + result.validate() + + if inline_env and not stdout: + self.ui.warning( + "--inline-env writes secrets in clear text into the compose file." + ) + + if stdout: + self.ui.out(result.compose) + return + if diff: + self.ui.diff(result.diff_against(path)) + return + + target = (output or path).resolve() + if output is not None: + target.mkdir(parents=True, exist_ok=True) + (target / ENV_FILE).write_text( + (path / ENV_FILE).read_text(encoding="utf-8"), encoding="utf-8" + ) + report = result.write(target) + report_write(self.ui, report) + self.ui.success( + f"Rendered {', '.join(p.name for p in report.wrote)} in {target}" + ) + if kind == "agent" and output is None: + self.ui.info(f"Restart to apply: portabase restart {path.name}") diff --git a/commands/common.py b/commands/common.py deleted file mode 100644 index 3a8fefd..0000000 --- a/commands/common.py +++ /dev/null @@ -1,68 +0,0 @@ -import typer -import subprocess -import shutil -from pathlib import Path -from rich.prompt import Confirm -from core.utils import console, validate_work_dir, get_random_hint, slugify_project_name -from core.docker import run_compose - -def start(path: Path = typer.Argument(..., help="Path to component folder")): - path = path.resolve() - validate_work_dir(path) - status_msg = f"[bold magenta]Starting {path.name}...[/bold magenta]\n{get_random_hint()}" - with console.status(status_msg): - run_compose(path, ["up", "-d"]) - console.print("[success]✔ Started[/success]") - -def stop(path: Path = typer.Argument(..., help="Path to component folder")): - path = path.resolve() - validate_work_dir(path) - status_msg = f"[bold magenta]Stopping {path.name}...[/bold magenta]\n{get_random_hint()}" - with console.status(status_msg): - run_compose(path, ["stop"]) - console.print("[success]✔ Stopped[/success]") - -def restart(path: Path = typer.Argument(..., help="Path to component folder")): - path = path.resolve() - validate_work_dir(path) - status_msg = f"[bold magenta]Restarting {path.name}...[/bold magenta]\n{get_random_hint()}" - with console.status(status_msg): - run_compose(path, ["restart"]) - console.print("[success]✔ Restarted[/success]") - -def logs( - path: Path = typer.Argument(..., help="Path to component folder"), - follow: bool = typer.Option(True, "--follow/--no-follow", "-f") -): - path = path.resolve() - validate_work_dir(path) - args = ["logs"] - if follow: - args.append("-f") - try: - project_name = slugify_project_name(path.name) - subprocess.run(["docker", "compose", "-p", project_name] + args, cwd=path) - except KeyboardInterrupt: - pass - -def uninstall( - path: Path = typer.Argument(..., help="Path to component folder"), - force: bool = typer.Option(False, "--force", "-f") -): - path = path.resolve() - validate_work_dir(path) - - if not force: - console.print(f"[danger]⚠ WARNING: This will delete containers and data in {path}.[/danger]") - if not Confirm.ask("Are you sure?"): - raise typer.Exit() - - status_msg = f"[bold red]Uninstalling...[/bold red]\n{get_random_hint()}" - with console.status(status_msg): - run_compose(path, ["down", "-v"]) - try: - shutil.rmtree(path) - except Exception as e: - console.print(f"[warning]Could not remove directory: {e}[/warning]") - - console.print(f"[success]✔ Uninstalled[/success]") \ No newline at end of file diff --git a/commands/config.py b/commands/config.py index 7660f52..c781241 100644 --- a/commands/config.py +++ b/commands/config.py @@ -1,28 +1,97 @@ +from __future__ import annotations + +from typing import Annotated + import typer -from core.config import get_config_value, set_config_value -from core.utils import console +from commands.base import Command, CommandGroup +from core.config import GlobalConfig +from core.errors import ValidationError +from services.telemetry import Telemetry +from ui import UI + +CHANNELS = ("stable", "beta") + + +class _ConfigCommand(Command): + panel = "Configuration" + + def __init__(self, ui: UI, telemetry: Telemetry, config: GlobalConfig) -> None: + super().__init__(ui, telemetry) + self.config = config + + +class ConfigShow(_ConfigCommand): + name, help = "show", "Show the current configuration." -app = typer.Typer(help="Manage global CLI configuration.") + def run(self) -> None: + data = self.config.all() + self.ui.info(f"Configuration file: {self.config.path}") + for key in GlobalConfig.KNOWN_KEYS: + value = data.get(key, "[hint]unset[/hint]") + self.ui.print(f" [key]{key}[/key]: {value}") + for key in sorted(set(data) - set(GlobalConfig.KNOWN_KEYS)): + self.ui.print( + f" [key]{key}[/key]: {data[key]} [hint](unknown key)[/hint]" + ) -@app.command() -def channel( - name: str = typer.Argument(..., help="Update channel name (stable or beta)"), -): - name = name.lower() - if name not in ["stable", "beta"]: - console.print( - "[danger]✖ Invalid channel. Choose either 'stable' or 'beta'.[/danger]" +class ConfigGet(_ConfigCommand): + name, help = "get", "Print one configuration value." + no_args_is_help = True + + def run( + self, key: Annotated[str, typer.Argument(help="Configuration key")] + ) -> None: + value = self.config.get(key) + if value is None: + raise ValidationError( + f"'{key}' is not set.", + hint="Known keys: " + ", ".join(GlobalConfig.KNOWN_KEYS), + ) + self.ui.print(str(value)) + + +class ConfigSet(_ConfigCommand): + name, help = "set", "Set a configuration value." + no_args_is_help = True + + def run( + self, + key: Annotated[str, typer.Argument(help="Configuration key")], + value: Annotated[str, typer.Argument(help="Value")], + ) -> None: + if key == "update_channel" and value not in CHANNELS: + raise ValidationError( + f"Invalid channel '{value}'.", hint="Choose 'stable' or 'beta'." + ) + self.config.set(key, value) + self.ui.success(f"{key} = {value}") + + +class ConfigChannel(_ConfigCommand): + name, help = "channel", "Set the update channel (stable or beta)." + no_args_is_help = True + + def run(self, name: Annotated[str, typer.Argument(help="stable or beta")]) -> None: + ConfigSet(self.ui, self.telemetry, self.config).run( + "update_channel", name.lower() ) - raise typer.Exit(1) - set_config_value("update_channel", name) - console.print(f"[success]✔ Update channel set to: [bold]{name}[/bold][/success]") +class ConfigCommands(CommandGroup): + name, help, panel = "config", "Manage global CLI configuration.", "Configuration" + + def __init__(self, ui: UI, telemetry: Telemetry, config: GlobalConfig) -> None: + super().__init__(ui, telemetry) + self.config = config -@app.command() -def show(): - channel = get_config_value("update_channel", "auto (based on current version)") - console.print(f"[info]Current Configuration:[/info]") - console.print(f" [bold]Update Channel:[/bold] {channel}") + @property + def commands(self) -> list[Command]: + deps = (self.ui, self.telemetry, self.config) + return [ + ConfigShow(*deps), + ConfigGet(*deps), + ConfigSet(*deps), + ConfigChannel(*deps), + ] diff --git a/commands/dashboard.py b/commands/dashboard.py index 050f719..0a58dcc 100644 --- a/commands/dashboard.py +++ b/commands/dashboard.py @@ -1,205 +1,188 @@ -import re +from __future__ import annotations + import secrets +import sys from pathlib import Path +from typing import Annotated from urllib.parse import quote -import questionary import typer -from rich.panel import Panel -from rich.prompt import Confirm, IntPrompt, Prompt -from rich.table import Table - -from core.config import write_env_file, write_file -from core.docker import run_compose -from core.network import fetch_template -from core.utils import ( - check_system, - console, - generate_password, - get_free_port, - get_random_hint, - print_banner, - questionary_style, - slugify_project_name, -) - - -def dashboard( - name: str = typer.Argument(..., help="Name of the dashboard (creates a folder)"), - port: str = typer.Option("8887", help="Web Port"), - start: bool = typer.Option(False, "--start", "-s", help="Start immediately"), -): - print_banner() - check_system() - - path = Path(name).resolve() - if path.exists(): - console.print(f"[warning]Directory '{name}' already exists.[/warning]") - if not Confirm.ask("Overwrite?"): - raise typer.Exit() - - path.mkdir(parents=True, exist_ok=True) - project_name = slugify_project_name(path.name) - - raw_template = fetch_template("dashboard.yml") - - auth_secret = secrets.token_hex(32) - base_url = f"http://localhost:{port}" - - env_vars = { - "HOST_PORT": port, - "PROJECT_SECRET": auth_secret, - "PROJECT_URL": base_url, - "PROJECT_NAME": project_name, - "TZ": "Europe/Paris", - "LOG_LEVEL": "info", - } - - mode = questionary.select( - "Database Setup", - choices=[ - questionary.Choice( - "external: create a dedicated container in the same docker-compose.yml (recommended)", - value="external", - ), - questionary.Choice( - "internal: use the database embedded in the Portabase container", - value="internal", - ), - questionary.Choice( - "custom: provide credentials of an existing database", - value="custom", - ), - ], - style=questionary_style, - ).ask() - - if not mode: - raise typer.Exit() - - if mode == "external": - pg_port = get_free_port() - pg_pass = generate_password(16) - env_vars.update( - { - "POSTGRES_DB": "portabase", - "POSTGRES_USER": "portabase", - "POSTGRES_PASSWORD": pg_pass, - "POSTGRES_HOST": "db", - "DATABASE_URL": f"postgresql://portabase:{quote(pg_pass, safe='')}@db:5432/portabase?schema=public", - "PG_PORT": str(pg_port), - } - ) - final_compose = raw_template.replace("${PROJECT_NAME}", project_name) - elif mode == "custom": - console.print("[info]External Database Configuration[/info]") - db_host = Prompt.ask("Host", default="localhost") - db_port = IntPrompt.ask("Port", default=5432) - db_name = Prompt.ask("Database Name", default="portabase") - db_user = Prompt.ask("Username") - db_pass = questionary.password("Password", style=questionary_style).ask() - if db_pass is None: - raise typer.Exit() - - env_vars.update( - { - "POSTGRES_DB": db_name, - "POSTGRES_USER": db_user, - "POSTGRES_PASSWORD": db_pass, - "POSTGRES_HOST": db_host, - "DATABASE_URL": f"postgresql://{quote(db_user, safe='')}:{quote(db_pass, safe='')}@{db_host}:{db_port}/{db_name}?schema=public", - "PG_PORT": str(db_port), - } - ) - final_compose = re.sub( - r"[ ]{4}depends_on:\n[ ]{6}db:\n[ ]{8}condition: service_healthy\n", - "", - raw_template, - ) - final_compose = re.sub( - r"[ ]{2}db:.*?retries: 5\n", "", final_compose, flags=re.DOTALL - ) - final_compose = re.sub(r"[ ]{2}postgres-data:\n", "", final_compose) - final_compose = final_compose.replace("${PROJECT_NAME}", project_name) - else: - final_compose = re.sub( - r"[ ]{4}depends_on:\n[ ]{6}db:\n[ ]{8}condition: service_healthy\n", - "", - raw_template, - ) - final_compose = re.sub( - r"[ ]{2}db:.*?retries: 5\n", "", final_compose, flags=re.DOTALL - ) - final_compose = re.sub(r"[ ]{2}postgres-data:\n", "", final_compose) - final_compose = final_compose.replace("${PROJECT_NAME}", project_name) - - summary = Table(show_header=False, box=None, padding=(0, 2)) - summary.add_column("Property", style="bold cyan") - summary.add_column("Value", style="white") - - summary.add_row("Dashboard Name", name) - summary.add_row("Path", str(path)) - summary.add_row("Access URL", f"[bold green]http://localhost:{port}[/bold green]") - - db_setup_label = { - "external": "Dedicated Docker Container (Recommended)", - "internal": "Embedded Database (In-container)", - "custom": "Custom/Existing Database", - } - summary.add_row("Database Setup", db_setup_label.get(mode)) - - if mode == "external": - summary.add_row("Internal Port", env_vars["PG_PORT"]) - elif mode == "custom": - summary.add_row("DB Host", env_vars["POSTGRES_HOST"]) - summary.add_row("DB Name", env_vars["POSTGRES_DB"]) - masked_url = re.sub(r":.*?@", ":****@", env_vars["DATABASE_URL"]) - summary.add_row("Connection URL", f"[dim]{masked_url}[/dim]") - - summary.add_row("Files to Create", "• docker-compose.yml\n• .env") - - console.print("") - console.print( - Panel( - summary, - title="[bold white]PROPOSED CONFIGURATION[/bold white]", - border_style="bold blue", - expand=False, +from commands.base import Command +from commands.db import report_write +from core.utils import generate_password, slugify_project_name +from services.docker import DockerRunner +from services.ports import PortAllocator +from services.project import DashboardProject +from services.renderer import ComposeRenderer +from services.telemetry import Telemetry +from services.templates import TemplateRepository +from ui import UI + +DB_MODES = ("external", "internal", "custom") +MODE_LABELS = { + "external": "Dedicated Docker Container (Recommended)", + "internal": "Embedded Database (In-container)", + "custom": "Custom/Existing Database", +} + + +class DashboardCommand(Command): + name = "dashboard" + help = "Create a new Portabase Dashboard instance." + panel = "Creation" + no_args_is_help = True + + def __init__( + self, + ui: UI, + telemetry: Telemetry, + docker: DockerRunner, + templates: TemplateRepository, + renderer: ComposeRenderer, + ports: PortAllocator, + ) -> None: + super().__init__(ui, telemetry) + self.docker = docker + self.templates = templates + self.renderer = renderer + self.ports = ports + + def run( + self, + name: Annotated[str, typer.Argument(help="Dashboard name (creates a folder)")], + port: Annotated[int | None, typer.Option("--port", help="Web port")] = None, + db_mode: Annotated[ + str | None, typer.Option("--db-mode", help="external | internal | custom") + ] = None, + db_host: Annotated[ + str | None, typer.Option("--db-host", help="Host of the existing database") + ] = None, + db_port: Annotated[ + int | None, typer.Option("--db-port", help="Port of the existing database") + ] = None, + db_name: Annotated[ + str | None, typer.Option("--db-name", help="Database name") + ] = None, + db_user: Annotated[ + str | None, typer.Option("--db-user", help="Username") + ] = None, + db_password_stdin: Annotated[ + bool, + typer.Option( + "--db-password-stdin", help="Read the custom DB password from stdin" + ), + ] = False, + tz: Annotated[str | None, typer.Option("--tz", help="Timezone")] = None, + start: Annotated[ + bool, typer.Option("--start", "-s", help="Start immediately") + ] = False, + force: Annotated[ + bool, typer.Option("--force", "-f", help="Overwrite an existing folder") + ] = False, + yes: Annotated[ + bool, + typer.Option("--yes", "-y", help="Skip the configuration confirmation"), + ] = False, + ) -> None: + self.ui.banner() + self.require_docker(self.docker) + self.templates.resolve() + + path = Path(name).resolve() + if path.exists() and not force: + self.ui.warning(f"Directory '{name}' already exists.") + self.confirm_or_abort("Overwrite?", default=False) + + form = self.ui.form() + web_port = form.integer("Web Port", value=port, default=8887, name="port") + mode = form.choice( + "Database Setup", + list(DB_MODES), + value=db_mode, + default="external", + name="db_mode", ) - ) - console.print( - "[dim]The dashboard will be set up with the parameters above.[/dim]\n" - ) - - if not Confirm.ask( - "[bold]Apply this configuration and generate files?[/bold]", default=True - ): - console.print("[warning]Configuration cancelled.[/warning]") - raise typer.Exit() - - write_file(path / "docker-compose.yml", final_compose) - write_env_file(path, env_vars) - - db_info = "" - if mode == "external": - db_info = f"\n[dim]DB Port: {env_vars.get('PG_PORT')}[/dim]" - elif mode == "custom": - db_info = f"\n[dim]External DB: {env_vars.get('POSTGRES_HOST')}[/dim]" - else: - db_info = "\n[dim]Embedded Database[/dim]" - - console.print( - Panel( - f"[bold white]DASHBOARD CREATED: {name}[/bold white]\n[dim]Path: {path}[/dim]{db_info}", - style="bold #5f00d7", + project_name = slugify_project_name(path.name) + + env_vars = { + "HOST_PORT": str(web_port), + "PROJECT_SECRET": secrets.token_hex(32), + "PROJECT_URL": f"http://localhost:{web_port}", + "PROJECT_NAME": project_name, + "TZ": form.text("Timezone", value=tz, default="Europe/Paris", name="tz"), + "LOG_LEVEL": "info", + } + rows = [ + ("Dashboard Name", name), + ("Path", str(path)), + ("Access URL", env_vars["PROJECT_URL"]), + ("Database Setup", MODE_LABELS[mode]), + ] + + if mode == "external": + pg_pass, pg_port = generate_password(16), self.ports.free() + env_vars.update( + self._pg_env("portabase", "portabase", pg_pass, "db", 5432, pg_port) + ) + rows.append(("Internal Port", str(pg_port))) + elif mode == "custom": + self.ui.info("External Database Configuration") + host = form.text("Host", value=db_host, default="localhost", name="db_host") + dport = form.integer("Port", value=db_port, default=5432, name="db_port") + dbname = form.text( + "Database Name", value=db_name, default="portabase", name="db_name" + ) + user = form.text("Username", value=db_user, name="db_user") + if db_password_stdin: + password = sys.stdin.readline().rstrip("\n") + else: + password = form.secret("Password", name="db_password") + env_vars.update(self._pg_env(dbname, user, password, host, dport, dport)) + rows += [ + ("DB Host", host), + ("DB Name", dbname), + ("Connection URL", env_vars["DATABASE_URL"]), + ] + + rows.append(("Files to Create", "docker-compose.yml, .env")) + self.ui.summary(rows, title="PROPOSED CONFIGURATION") + if not yes: + self.confirm_or_abort( + "Apply this configuration and generate files?", default=True + ) + + project = DashboardProject.create(path, env_vars) + with self.ui.status("Rendering configuration..."): + result = self.renderer.render_dashboard(project) + project.save_state() + report = result.write(path) + report_write(self.ui, report) + self.ui.success(f"Dashboard '{name}' created in {path}") + + if start or ( + not self.ui.non_interactive + and self.ui.confirm("Start dashboard now?", default=False) + ): + with self.ui.status("Starting..."): + self.docker.compose(path, ["up", "-d"]) + self.ui.success(f"Live at: {env_vars['PROJECT_URL']}") + else: + self.ui.info(f"Run: portabase start {name}") + + @staticmethod + def _pg_env( + db: str, user: str, password: str, host: str, port: int, host_port: int + ) -> dict[str, str]: + url = ( + f"postgresql://{quote(user, safe='')}:{quote(password, safe='')}" + f"@{host}:{port}/{db}?schema=public" ) - ) - - if start or Confirm.ask("Start dashboard now?", default=False): - status_msg = f"[bold magenta]Starting...[/bold magenta]\n{get_random_hint()}" - with console.status(status_msg, spinner="earth"): - run_compose(path, ["up", "-d"]) - console.print(f"[bold green]✔ Live at: http://localhost:{port}[/bold green]") - else: - console.print(f"[info]Run: portabase start {name}[/info]") + return { + "POSTGRES_DB": db, + "POSTGRES_USER": user, + "POSTGRES_PASSWORD": password, + "POSTGRES_HOST": host, + "DATABASE_URL": url, + "PG_PORT": str(host_port), + } diff --git a/commands/db.py b/commands/db.py index e670442..4c5118b 100644 --- a/commands/db.py +++ b/commands/db.py @@ -1,756 +1,282 @@ -import re -import secrets -import uuid +from __future__ import annotations + +import sys from pathlib import Path +from typing import Annotated -import questionary import typer -from rich.panel import Panel -from rich.prompt import IntPrompt, Prompt -from rich.table import Table - -from core.config import add_db_to_json, load_db_config, save_db_config, write_env_file -from core.docker import ensure_network -from core.utils import ( - console, - generate_password, - get_free_port, - questionary_style, - validate_work_dir, -) -from templates.compose import ( - AGENT_FIREBIRD_SNIPPET, - AGENT_MARIADB_SNIPPET, - AGENT_MONGODB_AUTH_SNIPPET, - AGENT_MONGODB_SNIPPET, - AGENT_MSSQL_SNIPPET, - AGENT_POSTGRES_SNIPPET, - AGENT_REDIS_AUTH_SNIPPET, - AGENT_REDIS_SNIPPET, - AGENT_VALKEY_AUTH_SNIPPET, - AGENT_VALKEY_SNIPPET, -) - -app = typer.Typer(help="Manage databases configuration.") - -DOCKER_SOCKET_MOUNT = "/var/run/docker.sock:/var/run/docker.sock" - -def ensure_docker_socket(path: Path): - """Mount the Docker socket on the agent's app service if not already present.""" - compose_path = path / "docker-compose.yml" - if not compose_path.exists(): - console.print( - "[warning]⚠ docker-compose.yml not found. Add " - f"[bold]{DOCKER_SOCKET_MOUNT}[/bold] to the agent volumes manually.[/warning]" +from commands.base import Command, CommandGroup +from commands.flows.add_database import AddDatabaseFlow +from engines import EngineRegistry +from services.docker import DockerRunner +from services.ports import PortAllocator +from services.project import AgentProject +from services.renderer import ComposeRenderer, WriteReport +from services.telemetry import Telemetry +from services.templates import TemplateRepository +from ui import UI + +NameArg = Annotated[Path, typer.Argument(help="Agent folder")] + + +def report_write(ui: UI, report: WriteReport) -> None: + if report.backed_up: + ui.warning( + f"Legacy compose backed up to {report.backed_up.name}. " + "Manual edits belong in docker-compose.override.yml." ) - return - content = compose_path.read_text() - if DOCKER_SOCKET_MOUNT in content: - return - anchor = "- ./databases.json:/config/config.json" - lines = content.splitlines(keepends=True) - new_lines = [] - inserted = False - for line in lines: - new_lines.append(line) - if not inserted and anchor in line: - indent = line[: len(line) - len(line.lstrip())] - new_lines.append(f"{indent}- {DOCKER_SOCKET_MOUNT}\n") - inserted = True +class _DbCommand(Command): + panel = "Configuration" + no_args_is_help = True + + def __init__( + self, + ui: UI, + telemetry: Telemetry, + engines: EngineRegistry, + ports: PortAllocator, + templates: TemplateRepository, + renderer: ComposeRenderer, + docker: DockerRunner, + ) -> None: + super().__init__(ui, telemetry) + self.engines = engines + self.ports = ports + self.templates = templates + self.renderer = renderer + self.docker = docker + + def render_and_write(self, project: AgentProject) -> None: + with self.ui.status("Rendering configuration..."): + result = self.renderer.render_agent(project) + project.save_state() + report = result.write(project.path) + report_write(self.ui, report) + + +class DbAddCommand(_DbCommand): + name, help = "add", "Add a database to an agent." + + def run( + self, + name: NameArg, + engine: Annotated[ + str | None, typer.Option("--engine", "-e", help="Database engine") + ] = None, + mode: Annotated[ + str | None, typer.Option("--mode", help="new (container) or existing") + ] = None, + auth: Annotated[ + bool | None, + typer.Option( + "--auth/--no-auth", help="Auth variant for mongodb/redis/valkey" + ), + ] = None, + label: Annotated[ + str | None, typer.Option("--label", help="Display name") + ] = None, + host: Annotated[ + str | None, typer.Option("--host", help="Host of an existing database") + ] = None, + port: Annotated[ + int | None, typer.Option("--port", help="Port of an existing database") + ] = None, + database: Annotated[ + str | None, typer.Option("--database", help="Database name") + ] = None, + user: Annotated[str | None, typer.Option("--user", help="Username")] = None, + password: Annotated[ + str | None, typer.Option("--password", help="Prefer --password-stdin") + ] = None, + password_stdin: Annotated[ + bool, typer.Option("--password-stdin", help="Read password from stdin") + ] = False, + path: Annotated[ + str | None, typer.Option("--path", help="SQLite file path (existing)") + ] = None, + db_name: Annotated[ + str | None, typer.Option("--name", help="SQLite file name (new)") + ] = None, + volume: Annotated[ + str | None, typer.Option("--volume", help="Docker volume name") + ] = None, + container: Annotated[ + str | None, + typer.Option("--container", help="Container to restart after restore"), + ] = None, + option: Annotated[ + list[str] | None, + typer.Option("--option", "-o", help="Engine option KEY=VALUE (repeatable)"), + ] = None, + ) -> None: + if password_stdin: + password = sys.stdin.readline().rstrip("\n") + elif password is not None: + self.ui.warning( + "--password is visible in shell history; prefer --password-stdin." + ) - if inserted: - compose_path.write_text("".join(new_lines)) - console.print( - f"[info]ℹ Mounted Docker socket ([bold]{DOCKER_SOCKET_MOUNT}[/bold]) " - "on the agent.[/info]" + project_path = self.require_project_dir(name) + self.templates.resolve() + project = AgentProject.load(project_path) + + flow = AddDatabaseFlow(self.ui, self.engines, self.ports) + values = { + "engine": engine, + "mode": mode, + "auth": auth, + "label": label, + "host": host, + "port": port, + "database": database, + "username": user, + "password": password, + "path": path, + "name": db_name, + "volume": volume, + "container": container, + "options": flow.parse_options(option), + } + spec, eng = flow.collect(values) + flow.apply(project, spec, eng) + self.render_and_write(project) + + self.ui.success( + f"Added {eng.display} database '{spec.name}' ({eng.describe(spec)})." ) - else: - console.print( - "[warning]⚠ Could not locate the agent volumes block. Add " - f"[bold]{DOCKER_SOCKET_MOUNT}[/bold] to docker-compose.yml manually.[/warning]" + self.ui.info( + f"Restart the agent to apply changes: portabase restart {project_path.name}" ) -@app.command("list") -def list_dbs(name: str = typer.Argument(..., help="Name of the agent")): - path = Path(name).resolve() - validate_work_dir(path) - - config = load_db_config(path) - dbs = config.get("databases", []) - - if not dbs: - console.print("[warning]No databases configured.[/warning]") - return - - table = Table(title=f"Databases for {name}") - table.add_column("Display Name", style="cyan") - table.add_column("Database", style="blue") - table.add_column("Type", style="magenta") - table.add_column("Host:Port", style="green") - table.add_column("User", style="white") - table.add_column("ID", style="dim") - - for db in dbs: - db_type = db.get("type", "N/A") - if db_type == "sqlite": - host_port = "Local File" - elif db_type == "docker-volume": - host_port = f"volume: {db.get('volume_name', 'N/A')}" +class DbRemoveCommand(_DbCommand): + name, help = "remove", "Remove a database from an agent." + + def run( + self, + name: NameArg, + target: Annotated[ + str | None, + typer.Option( + "--id", "--name", "-i", help="Database id (or prefix) or display name" + ), + ] = None, + purge_volume: Annotated[ + bool, + typer.Option( + "--purge-volume", + help="Also delete the Docker volume of a managed database", + ), + ] = False, + yes: Annotated[ + bool, typer.Option("--yes", "-y", help="Skip confirmation") + ] = False, + ) -> None: + project_path = self.require_project_dir(name) + self.templates.resolve() + project = AgentProject.load(project_path) + if not project.databases: + self.ui.warning("No databases to remove.") + return + + if target is None: + choices = [f"{d.name} ({d.engine}) [{d.id[:8]}]" for d in project.databases] + picked = self.ui.form().choice( + "Which database to remove?", choices, name="id" + ) + spec = project.databases[choices.index(picked)] else: - host_port = f"{db.get('host', 'N/A')}:{db.get('port', 'N/A')}" - username = ( - "N/A" - if db_type in ("sqlite", "docker-volume") - else db.get("username", "N/A") - ) - - table.add_row( - db.get("name", "N/A"), - db.get("database", db.get("name", "N/A")), - db_type, - host_port, - username, - db.get("generated_id", "")[:8] + "...", - ) - console.print(table) - + spec = project.find(target) + engine = self.engines.get(spec.engine) -@app.command("add") -def add_db(name: str = typer.Argument(..., help="Name of the agent")): - path = Path(name).resolve() - validate_work_dir(path) - ensure_network("portabase_network") - - console.print(Panel("Add Database to Agent", style="bold blue")) - - while True: - storage_kind = questionary.select( - "What do you want to configure?", - choices=["done", "database", "docker-volume"], - default="database", - style=questionary_style, - ).ask() - - if storage_kind in (None, "done"): - break - - if storage_kind == "docker-volume": - console.print( - "[warning]⚠ Requires the Docker socket mounted on the agent " - "([bold]/var/run/docker.sock[/bold]). It will be added to " - "docker-compose.yml automatically.[/warning]" - ) - friendly_name = Prompt.ask("Display Name", default="Docker Volume") - volume_name = Prompt.ask("Volume Name (e.g. databases_sqlite-data)").strip() - while not volume_name: - console.print("[danger]✖ Volume Name is required.[/danger]") - volume_name = Prompt.ask( - "Volume Name (e.g. databases_sqlite-data)" - ).strip() - container_name = Prompt.ask( - "Container Name (optional, enables auto-restart after restore)", - default="", + if not yes: + extra = " and its Docker volume" if (purge_volume and spec.managed) else "" + self.confirm_or_abort( + f"Remove '{spec.name}' ({engine.describe(spec)}){extra}?", default=False ) - entry = { - "name": friendly_name, - "type": "docker-volume", - "volume_name": volume_name, - "generated_id": str(uuid.uuid4()), - } - if container_name: - entry["container_name"] = container_name - ensure_docker_socket(path) - add_db_to_json(path, entry) - console.print("[success]✔ Added to config[/success]") - continue - mode = Prompt.ask( - "Configuration Mode", - choices=["new", "existing", "back"], - default="existing", - ) - - if mode == "back": - break - - if mode == "existing": - db_type = questionary.select( - "Select Database Type", - choices=[ - "back", - "postgresql", - "postgresql-cluster", - "mysql", - "mariadb", - "sqlite", - "firebird", - "mongodb", - "mssql", - ], - style=questionary_style, - ).ask() - - if db_type == "back": - continue - - if not db_type: - raise typer.Exit() - - if db_type == "postgresql-cluster": - console.print( - "[warning]⚠ Postgres Cluster requires a superuser. " - "Cluster backup/restore uses pg_dumpall, which dumps all " - "databases and global objects (roles, tablespaces). " - "The provided user must be a Postgres superuser.[/warning]" + project.remove(spec, engine) + self.render_and_write(project) + self.ui.success(f"Removed {spec.name}") + + if spec.managed: + volume_name = f"{self.docker.project_name(project_path)}_{spec.host}-data" + if purge_volume: + self.require_docker(self.docker) + removed = self.docker.remove_volume(volume_name) + self.ui.success( + f"Deleted volume {volume_name}" + if removed + else f"Volume {volume_name} did not exist" ) - - friendly_name = Prompt.ask("Display Name", default="External DB") - - if db_type == "sqlite": - db_name = Prompt.ask("Database Path (e.g. /data/db.sqlite)") - entry = { - "name": friendly_name, - "database": db_name, - "type": db_type, - "generated_id": str(uuid.uuid4()), - } else: - db_name = Prompt.ask("Database Name") - host = Prompt.ask("Host", default="localhost") - port = IntPrompt.ask( - "Port", - default=5432 - if db_type in ["postgresql", "postgresql-cluster"] - else ( - 3050 - if db_type == "firebird" - else ( - 1433 - if db_type == "mssql" - else (3306 if db_type in ["mysql", "mariadb"] else 27017) - ) - ), + self.ui.info( + f"Data volume kept: {volume_name}. " + f"Delete it with: docker volume rm {volume_name}" ) - user = Prompt.ask("Username") - password = questionary.password( - "Password", style=questionary_style - ).ask() - if password is None: - raise typer.Exit() - - entry = { - "name": friendly_name, - "database": db_name, - "type": db_type, - "username": user, - "password": password, - "port": port, - "host": host, - "generated_id": str(uuid.uuid4()), - } - - if db_type == "postgresql": - console.print( - "[info]ℹ When enabled, omits [bold]--no-owner[/bold] and " - "[bold]--no-privileges[/bold] from the dump. Ownership and role " - "assignments are preserved in the output. By default, these flags " - "are applied to keep restores portable across different users and " - "environments, for example when migrating from one database " - "instance to another.[/info]" - ) - keep_ownership = questionary.confirm( - "Keep ownership?", - default=False, - style=questionary_style, - ).ask() - if keep_ownership is None: - raise typer.Exit() - - console.print( - "[info]ℹ Controls how the target database is cleaned before a " - "restore. [bold]pg_restore --clean[/bold] only drops objects " - "listed in the backup's own table of contents, so anything " - "already present in the target that the dump does not know " - "about survives and can make the restore fail.[/info]" - ) - clean_mode = questionary.select( - "Clean mode", - choices=[ - questionary.Choice( - "clean - pg_restore --clean --if-exists (default)", - value="clean", - ), - questionary.Choice( - "none - no pre-clean, restore into an empty database", - value="none", - ), - questionary.Choice( - "drop_schemas - drop every non-system schema CASCADE " - "(recommended, works on managed Postgres)", - value="drop_schemas", - ), - questionary.Choice( - "drop_database - DROP DATABASE + CREATE DATABASE " - "(full reset)", - value="drop_database", - ), - ], - default="clean", - style=questionary_style, - ).ask() - if clean_mode is None: - raise typer.Exit() - if clean_mode == "drop_database": - console.print( - "[warning]⚠ drop_database drops the whole target database " - "before restoring. The user must have CREATEDB and own the " - "database, or be a superuser. Most managed Postgres " - "providers do not allow it.[/warning]" - ) - - pg_options = {} - if keep_ownership: - pg_options["keep_ownership"] = True - if clean_mode != "clean": - pg_options["clean_mode"] = clean_mode - if pg_options: - entry["options"] = pg_options - - add_db_to_json(path, entry) - break - else: - db_engine = questionary.select( - "Select Database Engine", - choices=[ - "back", - "postgresql", - "postgresql-cluster", - "mysql", - "mariadb", - "sqlite", - "firebird", - "mongodb", - "redis", - "valkey", - "mssql", - ], - style=questionary_style, - ).ask() - - if db_engine == "back": - continue - - if not db_engine: - raise typer.Exit() - - db_variant = "no-auth" - if db_engine in ["mongodb", "redis", "valkey"]: - engine_display = { - "mongodb": "MongoDB", - "redis": "Redis", - "valkey": "Valkey", - }[db_engine] - db_variant = questionary.select( - f"Select {engine_display} Variant", - choices=["back", "no-auth", "with-auth"], - default="no-auth", - style=questionary_style, - ).ask() - - if db_variant == "back": - continue - - if not db_variant: - raise typer.Exit() - - env_vars = {} - snippet = "" - service_name = "" - db_name = "" - db_container_path = "" - db_user = "" - db_pass = "" - db_port = 0 - pg_options = None - - if db_engine == "sqlite": - db_name = Prompt.ask("Database Name", default="local") - if not db_name.endswith(".sqlite"): - db_name += ".sqlite" - - compose_path = path / "docker-compose.yml" - if compose_path.exists(): - content = compose_path.read_text() - lines = content.splitlines(keepends=True) - new_lines = [] - in_app_service = False - inserted = False - - for line in lines: - new_lines.append(line) - if not inserted: - if re.search(r"^ app:", line): - in_app_service = True - elif in_app_service and re.search(r"^ volumes:", line): - new_lines.append( - f" - ./{db_name}:/config/{db_name}\n" - ) - in_app_service = False - inserted = True - elif in_app_service and re.search(r"^ [a-zA-Z]", line): - in_app_service = False - - with open(compose_path, "w") as f: - f.writelines(new_lines) - - add_db_to_json( - path, - { - "name": db_name, - "database": f"/config/{db_name}", - "type": "sqlite", - "generated_id": str(uuid.uuid4()), - }, - ) - console.print(f"[success]✔ Added SQLite database ({db_name})[/success]") - - elif db_engine in ["postgresql", "postgresql-cluster"]: - db_port = get_free_port() - db_user = "admin" - db_pass = generate_password(16) - db_name = f"pg_{secrets.token_hex(4)}" - service_name = f"db-pg-{secrets.token_hex(2)}" - var_prefix = service_name.upper().replace("-", "_") - env_vars[f"{var_prefix}_PORT"] = str(db_port) - env_vars[f"{var_prefix}_DB"] = db_name - env_vars[f"{var_prefix}_USER"] = db_user - env_vars[f"{var_prefix}_PASS"] = db_pass - snippet = ( - AGENT_POSTGRES_SNIPPET.replace("${SERVICE_NAME}", service_name) - .replace("${PORT}", f"${{{var_prefix}_PORT}}") - .replace("${VOL_NAME}", f"{service_name}-data") - .replace("${DB_NAME}", f"${{{var_prefix}_DB}}") - .replace("${USER}", f"${{{var_prefix}_USER}}") - .replace("${PASSWORD}", f"${{{var_prefix}_PASS}}") - ) - - if db_engine == "postgresql": - console.print( - "[info]ℹ When enabled, omits [bold]--no-owner[/bold] and " - "[bold]--no-privileges[/bold] from the dump. Ownership and role " - "assignments are preserved in the output. By default, these flags " - "are applied to keep restores portable across different users and " - "environments, for example when migrating from one database " - "instance to another.[/info]" - ) - keep_ownership = questionary.confirm( - "Keep ownership?", - default=False, - style=questionary_style, - ).ask() - if keep_ownership is None: - raise typer.Exit() - - console.print( - "[info]ℹ Controls how the target database is cleaned before a " - "restore. [bold]pg_restore --clean[/bold] only drops objects " - "listed in the backup's own table of contents, so anything " - "already present in the target that the dump does not know " - "about survives and can make the restore fail.[/info]" - ) - clean_mode = questionary.select( - "Clean mode", - choices=[ - questionary.Choice( - "clean - pg_restore --clean --if-exists (default)", - value="clean", - ), - questionary.Choice( - "none - no pre-clean, restore into an empty database", - value="none", - ), - questionary.Choice( - "drop_schemas - drop every non-system schema CASCADE " - "(recommended, works on managed Postgres)", - value="drop_schemas", - ), - questionary.Choice( - "drop_database - DROP DATABASE + CREATE DATABASE " - "(full reset)", - value="drop_database", - ), - ], - default="clean", - style=questionary_style, - ).ask() - if clean_mode is None: - raise typer.Exit() - if clean_mode == "drop_database": - console.print( - "[warning]⚠ drop_database drops the whole target database " - "before restoring. The user must have CREATEDB and own the " - "database, or be a superuser. Most managed Postgres " - "providers do not allow it.[/warning]" - ) - - options = {} - if keep_ownership: - options["keep_ownership"] = True - if clean_mode != "clean": - options["clean_mode"] = clean_mode - if options: - pg_options = options - - elif db_engine in ["mysql", "mariadb"]: - db_port = get_free_port() - db_user = "admin" - db_pass = generate_password(16) - db_name = f"mysql_{secrets.token_hex(4)}" - service_name = f"db-mariadb-{secrets.token_hex(2)}" - var_prefix = service_name.upper().replace("-", "_") - env_vars[f"{var_prefix}_PORT"] = str(db_port) - env_vars[f"{var_prefix}_DB"] = db_name - env_vars[f"{var_prefix}_USER"] = db_user - env_vars[f"{var_prefix}_PASS"] = db_pass - snippet = ( - AGENT_MARIADB_SNIPPET.replace("${SERVICE_NAME}", service_name) - .replace("${PORT}", f"${{{var_prefix}_PORT}}") - .replace("${VOL_NAME}", f"{service_name}-data") - .replace("${DB_NAME}", f"${{{var_prefix}_DB}}") - .replace("${USER}", f"${{{var_prefix}_USER}}") - .replace("${PASSWORD}", f"${{{var_prefix}_PASS}}") - ) - - elif db_engine == "mongodb": - db_port = get_free_port() - db_name = f"mongo_{secrets.token_hex(4)}" - if db_variant == "with-auth": - db_user = "admin" - db_pass = generate_password(16) - service_name = f"db-mongo-auth-{secrets.token_hex(2)}" - var_prefix = service_name.upper().replace("-", "_") - env_vars[f"{var_prefix}_PORT"] = str(db_port) - env_vars[f"{var_prefix}_DB"] = db_name - env_vars[f"{var_prefix}_USER"] = db_user - env_vars[f"{var_prefix}_PASS"] = db_pass - snippet = ( - AGENT_MONGODB_AUTH_SNIPPET.replace( - "${SERVICE_NAME}", service_name - ) - .replace("${PORT}", f"${{{var_prefix}_PORT}}") - .replace("${VOL_NAME}", f"{service_name}-data") - .replace("${DB_NAME}", f"${{{var_prefix}_DB}}") - .replace("${USER}", f"${{{var_prefix}_USER}}") - .replace("${PASSWORD}", f"${{{var_prefix}_PASS}}") - ) - else: - service_name = f"db-mongo-{secrets.token_hex(2)}" - var_prefix = service_name.upper().replace("-", "_") - env_vars[f"{var_prefix}_PORT"] = str(db_port) - env_vars[f"{var_prefix}_DB"] = db_name - snippet = ( - AGENT_MONGODB_SNIPPET.replace("${SERVICE_NAME}", service_name) - .replace("${PORT}", f"${{{var_prefix}_PORT}}") - .replace("${VOL_NAME}", f"{service_name}-data") - .replace("${DB_NAME}", f"${{{var_prefix}_DB}}") - ) - - elif db_engine == "firebird": - db_port = get_free_port() - db_user = "alice" - db_pass = generate_password(16) - db_root_pass = generate_password(16) - db_name = "mirror.fdb" - db_container_path = f"/var/lib/firebird/data/{db_name}" - service_name = f"db-firebird-{secrets.token_hex(2)}" - var_prefix = service_name.upper().replace("-", "_") - env_vars[f"{var_prefix}_PORT"] = str(db_port) - env_vars[f"{var_prefix}_DB"] = db_name - env_vars[f"{var_prefix}_USER"] = db_user - env_vars[f"{var_prefix}_PASS"] = db_pass - env_vars[f"{var_prefix}_ROOT_PASS"] = db_root_pass - snippet = ( - AGENT_FIREBIRD_SNIPPET.replace("${SERVICE_NAME}", service_name) - .replace("${PORT}", f"${{{var_prefix}_PORT}}") - .replace("${VOL_NAME}", f"{service_name}-data") - .replace("${DB_NAME}", f"${{{var_prefix}_DB}}") - .replace("${USER}", f"${{{var_prefix}_USER}}") - .replace("${PASSWORD}", f"${{{var_prefix}_PASS}}") - .replace("${ROOT_PASSWORD}", f"${{{var_prefix}_ROOT_PASS}}") - ) - - elif db_engine == "redis": - db_port = get_free_port() - db_name = f"redis_{secrets.token_hex(4)}" - if db_variant == "with-auth": - db_pass = generate_password(16) - service_name = f"db-redis-auth-{secrets.token_hex(2)}" - var_prefix = service_name.upper().replace("-", "_") - env_vars[f"{var_prefix}_PORT"] = str(db_port) - env_vars[f"{var_prefix}_PASS"] = db_pass - snippet = ( - AGENT_REDIS_AUTH_SNIPPET.replace( - "${SERVICE_NAME}", service_name - ) - .replace("${PORT}", f"${{{var_prefix}_PORT}}") - .replace("${VOL_NAME}", f"{service_name}-data") - .replace("${PASSWORD}", f"${{{var_prefix}_PASS}}") - ) - else: - service_name = f"db-redis-{secrets.token_hex(2)}" - var_prefix = service_name.upper().replace("-", "_") - env_vars[f"{var_prefix}_PORT"] = str(db_port) - snippet = ( - AGENT_REDIS_SNIPPET.replace("${SERVICE_NAME}", service_name) - .replace("${PORT}", f"${{{var_prefix}_PORT}}") - .replace("${VOL_NAME}", f"{service_name}-data") - ) - - elif db_engine == "valkey": - db_port = get_free_port() - db_name = f"valkey_{secrets.token_hex(4)}" - if db_variant == "with-auth": - db_pass = generate_password(16) - service_name = f"db-valkey-auth-{secrets.token_hex(2)}" - var_prefix = service_name.upper().replace("-", "_") - env_vars[f"{var_prefix}_PORT"] = str(db_port) - env_vars[f"{var_prefix}_PASS"] = db_pass - snippet = ( - AGENT_VALKEY_AUTH_SNIPPET.replace( - "${SERVICE_NAME}", service_name - ) - .replace("${PORT}", f"${{{var_prefix}_PORT}}") - .replace("${VOL_NAME}", f"{service_name}-data") - .replace("${PASSWORD}", f"${{{var_prefix}_PASS}}") - ) - else: - service_name = f"db-valkey-{secrets.token_hex(2)}" - var_prefix = service_name.upper().replace("-", "_") - env_vars[f"{var_prefix}_PORT"] = str(db_port) - snippet = ( - AGENT_VALKEY_SNIPPET.replace("${SERVICE_NAME}", service_name) - .replace("${PORT}", f"${{{var_prefix}_PORT}}") - .replace("${VOL_NAME}", f"{service_name}-data") - ) - - elif db_engine == "mssql": - db_port = get_free_port() - db_pass = generate_password(16) - db_name = "master" - service_name = f"db-mssql-{secrets.token_hex(2)}" - var_prefix = service_name.upper().replace("-", "_") - env_vars[f"{var_prefix}_PORT"] = str(db_port) - env_vars[f"{var_prefix}_PASS"] = db_pass - snippet = ( - AGENT_MSSQL_SNIPPET.replace("${SERVICE_NAME}", service_name) - .replace("${PORT}", f"${{{var_prefix}_PORT}}") - .replace("${VOL_NAME}", f"{service_name}-data") - .replace("${PASSWORD}", f"${{{var_prefix}_PASS}}") - ) - - compose_path = path / "docker-compose.yml" - if compose_path.exists(): - content = compose_path.read_text() - - vol_match = re.search(r"^volumes:", content, re.MULTILINE) - net_match = re.search(r"^networks:", content, re.MULTILINE) - - if vol_match: - insert_pos = vol_match.start() - elif net_match: - insert_pos = net_match.start() - else: - insert_pos = len(content) - - content = content[:insert_pos] + snippet + "\n" + content[insert_pos:] - - vol_match = re.search(r"^volumes:", content, re.MULTILINE) - net_match = re.search(r"^networks:", content, re.MULTILINE) - vol_entry = f" {service_name}-data:\n" - - if vol_match: - if net_match and net_match.start() > vol_match.start(): - content = ( - content[: net_match.start()] - + vol_entry - + content[net_match.start() :] - ) - else: - if not content.endswith("\n"): - content += "\n" - content += vol_entry - else: - if not content.endswith("\n"): - content += "\n" - content += "\nvolumes:\n" + vol_entry - - with open(compose_path, "w") as f: - f.write(content) - - if db_engine != "sqlite": - write_env_file(path, env_vars) - new_entry = { - "name": "mirror.fdb" if db_engine == "firebird" else db_name, - "database": db_container_path - if db_engine == "firebird" - else ("0" if db_engine in ["redis", "valkey"] else db_name), - "type": db_engine, - "username": "sa" if db_engine == "mssql" else db_user, - "password": db_pass, - "port": 5432 - if db_engine in ["postgresql", "postgresql-cluster"] - else ( - 3050 - if db_engine == "firebird" - else ( - 3306 - if db_engine in ["mysql", "mariadb"] - else ( - 1433 - if db_engine == "mssql" - else (6379 if db_engine in ["redis", "valkey"] else 27017) - ) - ) - ), - "host": service_name, - "generated_id": str(uuid.uuid4()), - } - if pg_options is not None: - new_entry["options"] = pg_options - add_db_to_json(path, new_entry) - break - - console.print("[success]✔ Database added to configuration.[/success]") - console.print( - "[info]Restart the agent to apply changes: [/info]" - + f"portabase restart {name}" - ) - - -@app.command("remove") -def remove_db(name: str = typer.Argument(..., help="Name of the agent")): - path = Path(name).resolve() - validate_work_dir(path) - - config = load_db_config(path) - dbs = config.get("databases", []) + self.ui.info( + f"Restart the agent to apply changes: portabase restart {project_path.name}" + ) - if not dbs: - console.print("[warning]No databases to remove.[/warning]") - return - options = [f"{db['name']} ({db['type']})" for db in dbs] - choice = Prompt.ask("Which database to remove?", choices=options) +class DbListCommand(_DbCommand): + name, help = "list", "List an agent's databases." - index = options.index(choice) - removed = dbs.pop(index) + def run(self, name: NameArg) -> None: + project = AgentProject.load(self.require_project_dir(name)) + if not project.databases: + self.ui.warning("No databases configured.") + return + rows = [] + for d in project.databases: + engine = self.engines.get(d.engine) + opts = ", ".join( + f"{k}={v}" for k, v in engine.non_default_options(d).items() + ) + user = ( + "N/A" if d.engine in ("sqlite", "docker-volume") else (d.username or "") + ) + rows.append( + [ + d.name, + d.database or "", + d.engine, + engine.describe(d), + user, + opts, + d.id[:8] + "...", + ] + ) + self.ui.table( + ["Display Name", "Database", "Type", "Host:Port", "User", "Options", "ID"], + rows, + title=f"Databases for {project.path.name}", + ) - config["databases"] = dbs - save_db_config(path, config) - console.print(f"[success]✔ Removed {removed['name']}[/success]") - console.print("[info]Restart the agent to apply changes.[/info]") +class DbCommands(CommandGroup): + name, help, panel = "db", "Manage an agent's databases.", "Configuration" + + def __init__( + self, + ui: UI, + telemetry: Telemetry, + engines: EngineRegistry, + ports: PortAllocator, + templates: TemplateRepository, + renderer: ComposeRenderer, + docker: DockerRunner, + ) -> None: + super().__init__(ui, telemetry) + self._deps = (ui, telemetry, engines, ports, templates, renderer, docker) + + @property + def commands(self) -> list[Command]: + return [ + DbAddCommand(*self._deps), + DbRemoveCommand(*self._deps), + DbListCommand(*self._deps), + ] diff --git a/commands/decrypt.py b/commands/decrypt.py index d14ddfc..0b06a9b 100644 --- a/commands/decrypt.py +++ b/commands/decrypt.py @@ -1,8 +1,11 @@ +from __future__ import annotations + from pathlib import Path -from typing import Optional +from typing import Annotated import typer +from commands.base import Command from core.crypto import ( ENC_SUFFIX, DecryptionError, @@ -10,124 +13,102 @@ default_output_for, load_master_key, ) -from core.utils import console +from core.errors import ConfigError, ValidationError def _looks_like_dir(path: Path) -> bool: - """True when ``path`` is an existing directory or clearly names one.""" if path.exists(): return path.is_dir() - # A trailing separator or no suffix is treated as a directory hint. return str(path).endswith(("/", "\\")) or path.suffix == "" -def decrypt( - input_path: Path = typer.Argument( - ..., - help="A .enc file, or a folder containing .enc files.", - ), - output_path: Optional[Path] = typer.Argument( - None, - help="Output file or folder (must match the input type). " - "Defaults to the same directory as the input.", - ), - key: Optional[Path] = typer.Option( - None, - "--key", - "-k", - help="Path to the master key file. Defaults to 'master_key.bin' in the " - "current directory.", - ), -): - """Decrypt Portabase AES-256-GCM ``.enc`` backup files.""" - input_path = input_path.resolve() - - if not input_path.exists(): - console.print(f"[danger]✖ Input path not found: {input_path}[/danger]") - raise typer.Exit(1) - - try: +class DecryptCommand(Command): + name = "decrypt" + help = "Decrypt Portabase .enc backup files (single file or folder)." + panel = "Configuration" + no_args_is_help = True + + def run( + self, + input_path: Annotated[ + Path, + typer.Argument(help="A .enc file, or a folder containing .enc files."), + ], + output_path: Annotated[ + Path | None, + typer.Argument( + help="Output file or folder (must match the input type). " + "Defaults to the input directory." + ), + ] = None, + key: Annotated[ + Path | None, + typer.Option( + "--key", "-k", help="Master key file. Defaults to ./master_key.bin" + ), + ] = None, + ) -> None: + input_path = input_path.resolve() + if not input_path.exists(): + raise ConfigError(f"Input path not found: {input_path}") master_key = load_master_key(key.resolve() if key else None) - except DecryptionError as exc: - console.print(f"[danger]✖ {exc}[/danger]") - raise typer.Exit(1) - - if input_path.is_dir(): - _decrypt_folder(input_path, output_path, master_key) - else: - _decrypt_single(input_path, output_path, master_key) - - -def _decrypt_single( - enc_path: Path, output_path: Optional[Path], master_key: bytes -) -> None: - if enc_path.suffix != ENC_SUFFIX: - console.print( - f"[warning]⚠ {enc_path.name} does not end with {ENC_SUFFIX}; " - "decrypting anyway.[/warning]" + if input_path.is_dir(): + self._folder(input_path, output_path, master_key) + else: + self._single(input_path, output_path, master_key) + + def _single( + self, enc_path: Path, output_path: Path | None, master_key: bytes + ) -> None: + if enc_path.suffix != ENC_SUFFIX: + self.ui.warning( + f"{enc_path.name} does not end with {ENC_SUFFIX}; decrypting anyway." + ) + if output_path is None: + out = enc_path.parent / default_output_for(enc_path) + elif _looks_like_dir(output_path): + out = output_path.resolve() / default_output_for(enc_path) + else: + out = output_path.resolve() + try: + decrypt_enc_file(enc_path, out, master_key) + except OSError as e: + raise DecryptionError(f"I/O error on {enc_path.name}: {e}", cause=e) from e + self.ui.success(f"Decrypted {enc_path.name} → {out}") + + def _folder( + self, in_dir: Path, output_path: Path | None, master_key: bytes + ) -> None: + enc_files = sorted( + p for p in in_dir.iterdir() if p.is_file() and p.suffix == ENC_SUFFIX ) - - if output_path is None: - out_path = enc_path.parent / default_output_for(enc_path) - elif _looks_like_dir(output_path): - out_path = output_path.resolve() / default_output_for(enc_path) - else: - out_path = output_path.resolve() - - try: - decrypt_enc_file(enc_path, out_path, master_key) - except DecryptionError as exc: - console.print(f"[danger]✖ Failed to decrypt {enc_path.name}: {exc}[/danger]") - raise typer.Exit(1) - except OSError as exc: - console.print(f"[danger]✖ I/O error on {enc_path.name}: {exc}[/danger]") - raise typer.Exit(1) - - console.print(f"[success]✔ Decrypted[/success] {enc_path.name} → {out_path}") - - -def _decrypt_folder( - in_dir: Path, output_path: Optional[Path], master_key: bytes -) -> None: - enc_files = sorted(p for p in in_dir.iterdir() if p.is_file() and p.suffix == ENC_SUFFIX) - - if not enc_files: - console.print(f"[warning]No {ENC_SUFFIX} files found in {in_dir}.[/warning]") - raise typer.Exit() - - if output_path is None: - out_dir = in_dir - elif _looks_like_dir(output_path): - out_dir = output_path.resolve() - else: - console.print( - "[danger]✖ Input is a folder, so the output must be a folder too.[/danger]" + if not enc_files: + self.ui.warning(f"No {ENC_SUFFIX} files found in {in_dir}.") + return + if output_path is None: + out_dir = in_dir + elif _looks_like_dir(output_path): + out_dir = output_path.resolve() + else: + raise ValidationError( + "Input is a folder, so the output must be a folder too." + ) + out_dir.mkdir(parents=True, exist_ok=True) + + failures: list[tuple[str, str]] = [] + with self.ui.status(f"Decrypting {len(enc_files)} file(s)..."): + for enc_path in enc_files: + out = out_dir / default_output_for(enc_path) + try: + decrypt_enc_file(enc_path, out, master_key) + except (DecryptionError, OSError) as e: + failures.append((enc_path.name, str(e))) + succeeded = len(enc_files) - len(failures) + self.ui.info( + f"Done: {succeeded} succeeded, {len(failures)} failed " + f"of {len(enc_files)} file(s)." ) - raise typer.Exit(1) - - out_dir.mkdir(parents=True, exist_ok=True) - - succeeded = 0 - failures: list[tuple[str, str]] = [] - - with console.status(f"[bold magenta]Decrypting {len(enc_files)} file(s)...[/bold magenta]"): - for enc_path in enc_files: - out_path = out_dir / default_output_for(enc_path) - try: - decrypt_enc_file(enc_path, out_path, master_key) - except (DecryptionError, OSError) as exc: - failures.append((enc_path.name, str(exc))) - console.print(f"[danger]✖ {enc_path.name}: {exc}[/danger]") - continue - succeeded += 1 - console.print(f"[success]✔[/success] {enc_path.name} → {out_path.name}") - - console.print( - f"\n[info]Done: {succeeded} succeeded, {len(failures)} failed " - f"of {len(enc_files)} file(s).[/info]" - ) - if failures: - console.print("[warning]Failed files:[/warning]") - for name, reason in failures: - console.print(f" [danger]•[/danger] {name}: {reason}") - raise typer.Exit(1) + if failures: + for name, reason in failures: + self.ui.print(f" [danger]•[/danger] {name}: {reason}") + raise DecryptionError(f"{len(failures)} file(s) failed to decrypt.") diff --git a/templates/__init__.py b/commands/flows/__init__.py similarity index 100% rename from templates/__init__.py rename to commands/flows/__init__.py diff --git a/commands/flows/add_database.py b/commands/flows/add_database.py new file mode 100644 index 0000000..878c4eb --- /dev/null +++ b/commands/flows/add_database.py @@ -0,0 +1,131 @@ +from __future__ import annotations + +from typing import Any + +from core.errors import ValidationError +from core.fields import Field +from core.specs import DatabaseSpec +from engines import EngineRegistry +from engines.base import DbEngine +from services.ports import PortAllocator +from services.project import AgentProject +from ui import UI +from ui.form import Form + +FLOW_KEYS = {"engine", "mode", "auth", "label", "options"} + + +class AddDatabaseFlow: + def __init__(self, ui: UI, engines: EngineRegistry, ports: PortAllocator) -> None: + self.ui = ui + self.engines = engines + self.ports = ports + + @staticmethod + def parse_options(items: list[str] | None) -> dict[str, str]: + out: dict[str, str] = {} + for item in items or []: + if "=" not in item: + raise ValidationError( + f"Invalid option '{item}'.", hint="Use -o KEY=VALUE" + ) + key, value = item.split("=", 1) + out[key.strip()] = value.strip() + return out + + def collect(self, values: dict[str, Any]) -> tuple[DatabaseSpec, DbEngine]: + form = self.ui.form() + engine_key = form.choice( + "Select Database Engine", + self.engines.choices(), + value=values.get("engine"), + name="engine", + ) + engine = self.engines.get(engine_key) + if engine.warning: + self.ui.warning(engine.warning) + + mode = "new" + if engine.has_modes: + mode = form.choice( + "Configuration Mode", + ["new", "existing"], + value=values.get("mode"), + default="new", + name="mode", + ) + + fields = list( + engine.fields_new() if mode == "new" else engine.fields_existing() + ) + if mode == "existing" or not engine.has_modes: + fields.insert( + 0, Field("label", "Display Name", "text", default=engine.label_default) + ) + + self._reject_irrelevant(values, fields, engine, mode) + + auth = True + if mode == "new" and engine.auth_variants: + raw = values.get("auth") + if raw is None: + picked = form.choice("Variant", ["with-auth", "no-auth"], name="auth") + auth = picked == "with-auth" + else: + auth = bool(raw) + + if mode == "existing": + self.ui.info(f"{engine.display} — existing database") + answers = form.collect(fields, values) + answers["options"] = self._collect_options( + form, engine, values.get("options") or {} + ) + + if mode == "new": + spec = engine.generate(auth=auth, ports=self.ports, answers=answers) + else: + spec = engine.from_existing(answers) + return spec.with_options(answers["options"]), engine + + def apply( + self, project: AgentProject, spec: DatabaseSpec, engine: DbEngine + ) -> None: + project.add(spec, engine) + + def _collect_options( + self, form: Form, engine: DbEngine, provided: dict[str, str] + ) -> dict[str, Any]: + option_fields = engine.option_fields() + known = {f.name for f in option_fields} + unknown = set(provided) - known + if unknown: + raise ValidationError( + f"Unknown option(s) for {engine.key}: {', '.join(sorted(unknown))}.", + hint=( + ("Valid options: " + ", ".join(sorted(known))) + if known + else f"{engine.key} has no options." + ), + ) + if not option_fields: + return {} + return form.collect(option_fields, provided) + + @staticmethod + def _reject_irrelevant( + values: dict[str, Any], fields: list[Field], engine: DbEngine, mode: str + ) -> None: + relevant = {f.name for f in fields} | FLOW_KEYS + extra = sorted( + k for k, v in values.items() if v is not None and k not in relevant + ) + if not extra: + return + flags = ", ".join("--" + k.replace("_", "-") for k in extra) + applicable = ", ".join("--" + f.name.replace("_", "-") for f in fields) + raise ValidationError( + f"Option(s) not applicable to {engine.key} in '{mode}' mode: {flags}.", + hint=f"Applicable: {applicable}" + if applicable + else "No extra input needed.", + ) diff --git a/commands/lifecycle.py b/commands/lifecycle.py new file mode 100644 index 0000000..dc2adab --- /dev/null +++ b/commands/lifecycle.py @@ -0,0 +1,104 @@ +from __future__ import annotations + +import contextlib +import shutil +from pathlib import Path +from typing import Annotated + +import typer + +from commands.base import Command +from services.docker import DockerRunner +from services.telemetry import Telemetry +from ui import UI + +PathArg = Annotated[Path, typer.Argument(help="Path to the component folder")] + + +class _ComposeCommand(Command): + panel = "Lifecycle" + no_args_is_help = True + verb: str + compose_args: list[str] + done: str + + def __init__(self, ui: UI, telemetry: Telemetry, docker: DockerRunner) -> None: + super().__init__(ui, telemetry) + self.docker = docker + + def run(self, path: PathArg) -> None: + path = self.require_project_dir(path) + self.require_docker(self.docker) + with self.ui.status(f"{self.verb} {path.name}..."): + self.docker.compose(path, self.compose_args) + self.ui.success(self.done) + + +class StartCommand(_ComposeCommand): + name, help = "start", "Start a Portabase component." + verb, compose_args, done = "Starting", ["up", "-d"], "Started" + + +class StopCommand(_ComposeCommand): + name, help = "stop", "Stop a Portabase component." + verb, compose_args, done = "Stopping", ["stop"], "Stopped" + + +class RestartCommand(_ComposeCommand): + name, help = "restart", "Restart a Portabase component." + verb, compose_args, done = "Restarting", ["restart"], "Restarted" + + +class LogsCommand(Command): + name, help, panel = "logs", "Show the logs of a Portabase component.", "Lifecycle" + no_args_is_help = True + + def __init__(self, ui: UI, telemetry: Telemetry, docker: DockerRunner) -> None: + super().__init__(ui, telemetry) + self.docker = docker + + def run( + self, + path: PathArg, + follow: Annotated[ + bool, typer.Option("--follow/--no-follow", "-f", help="Follow log output") + ] = True, + ) -> None: + path = self.require_project_dir(path) + self.require_docker(self.docker) + args = ["logs", "-f"] if follow else ["logs"] + with contextlib.suppress(KeyboardInterrupt): + self.docker.compose(path, args, check=False) + + +class UninstallCommand(Command): + name = "uninstall" + help = "Uninstall and delete a Portabase component." + panel = "Lifecycle" + no_args_is_help = True + + def __init__(self, ui: UI, telemetry: Telemetry, docker: DockerRunner) -> None: + super().__init__(ui, telemetry) + self.docker = docker + + def run( + self, + path: PathArg, + force: Annotated[ + bool, typer.Option("--force", "-f", help="Skip confirmation") + ] = False, + ) -> None: + path = self.require_project_dir(path) + self.require_docker(self.docker) + if not force: + self.ui.warning( + f"This will delete containers, volumes and all data in {path}." + ) + self.confirm_or_abort("Are you sure?", default=False) + with self.ui.status("Uninstalling..."): + self.docker.compose(path, ["down", "-v"]) + try: + shutil.rmtree(path) + except OSError as e: + self.ui.warning(f"Could not remove directory: {e}") + self.ui.success("Uninstalled") diff --git a/commands/update.py b/commands/update.py new file mode 100644 index 0000000..dd6cafb --- /dev/null +++ b/commands/update.py @@ -0,0 +1,65 @@ +from __future__ import annotations + +from commands.base import Command +from core.errors import NetworkError, UpdateError +from core.version import UNKNOWN, parse_version +from services.telemetry import Telemetry +from services.updater import Release, UpdateChecker, Updater, is_frozen +from ui import UI + + +class UpdateCommand(Command): + name, help, panel = "update", "Update the CLI to the latest version.", "System" + + def __init__( + self, ui: UI, telemetry: Telemetry, checker: UpdateChecker, updater: Updater + ) -> None: + super().__init__(ui, telemetry) + self.checker = checker + self.updater = updater + + def run(self) -> None: + if not is_frozen(): + self.ui.warning( + "The update command is only available for the binary version " + "of Portabase CLI." + ) + self.ui.info( + "If you installed from source, use [bold]git pull[/bold] to update." + ) + return + + current = self.checker.current + release = self._latest() + if release.tag == current: + self.ui.success(f"Portabase CLI is already up to date ({current}).") + return + if current != UNKNOWN and parse_version(release.tag) < parse_version(current): + self.ui.warning( + f"Current version ({current}) is newer than the latest remote " + f"version ({release.tag})." + ) + self.confirm_or_abort("Continue with the downgrade?", default=False) + + target = self.updater.target_path() + self.ui.info(f"Updating Portabase CLI from {current} to {release.tag}") + self.ui.info(f"Target installation path: {target}") + + total = self.updater.expected_size(release) or 0 + with self.ui.progress().download( + f"Downloading {release.tag}...", total + ) as advance: + tmp = self.updater.download(release, advance) + self.updater.install(tmp, target) + self.ui.success(f"Successfully updated to {release.tag}!") + + def _latest(self) -> Release: + try: + release = self.checker.fetch_latest() + except NetworkError as e: + raise UpdateError( + "Could not fetch latest release data from GitHub.", cause=e + ) from e + if release is None: + raise UpdateError("No release found for this channel.") + return release diff --git a/core/config.py b/core/config.py index 6f47004..1e4c418 100644 --- a/core/config.py +++ b/core/config.py @@ -1,82 +1,40 @@ import json import os -import uuid from pathlib import Path -TEMPLATE_BASE_URL = "https://s3.eu-central-3.ionoscloud.com/portabase-software/cli/public/templates" GLOBAL_CONFIG_DIR = Path.home() / ".portabase" GLOBAL_CONFIG_FILE = GLOBAL_CONFIG_DIR / "config.json" -def write_file(path: Path, content: str): - path.parent.mkdir(parents=True, exist_ok=True) - with open(path, "w") as f: - f.write(content) -def write_env_file(work_dir: Path, env_vars: dict): - existing = {} - env_path = work_dir / ".env" - if env_path.exists(): - with open(env_path, "r") as f: - for line in f: - if "=" in line: - k, v = line.strip().split("=", 1) - existing[k] = v.strip('"') - - existing.update(env_vars) - content = "" - for k, v in existing.items(): - content += f'{k}="{v}"\n' - write_file(env_path, content) - -def load_global_config() -> dict: - if not GLOBAL_CONFIG_FILE.exists(): - return {} - try: - with open(GLOBAL_CONFIG_FILE, "r") as f: - return json.load(f) - except: - return {} - -def save_global_config(config: dict): - GLOBAL_CONFIG_DIR.mkdir(parents=True, exist_ok=True) - with open(GLOBAL_CONFIG_FILE, "w") as f: - json.dump(config, f, indent=2) - -def get_config_value(key: str, default=None): - config = load_global_config() - return config.get(key, default) - -def set_config_value(key: str, value): - config = load_global_config() - config[key] = value - save_global_config(config) - -def load_db_config(path: Path) -> dict: - json_path = path / "databases.json" - if not json_path.exists(): - return {"databases": []} - try: - with open(json_path, "r") as f: - return json.load(f) - except: - return {"databases": []} - -def save_db_config(path: Path, config: dict): - json_path = path / "databases.json" - with open(json_path, "w") as f: - json.dump(config, f, indent=2) - try: - os.chmod(json_path, 0o666) - except: - pass - -def add_db_to_json(path: Path, db_entry: dict): - config = load_db_config(path) - if "databases" not in config: - config["databases"] = [] - - if "generated_id" not in db_entry: - db_entry["generated_id"] = str(uuid.uuid4()) - - config["databases"].append(db_entry) - save_db_config(path, config) \ No newline at end of file +class GlobalConfig: + KNOWN_KEYS = ("update_channel",) + + def __init__(self, path: Path = GLOBAL_CONFIG_FILE) -> None: + self.path = path + self.cache_dir = path.parent / "cache" + + def all(self) -> dict: + if not self.path.exists(): + return {} + try: + with open(self.path, encoding="utf-8") as f: + data = json.load(f) + except (OSError, json.JSONDecodeError): + return {} + return data if isinstance(data, dict) else {} + + def get(self, key: str, default=None): + return self.all().get(key, default) + + def set(self, key: str, value) -> None: + data = self.all() + data[key] = value + self.path.parent.mkdir(parents=True, exist_ok=True) + tmp = self.path.with_suffix(".json.tmp") + with open(tmp, "w", encoding="utf-8") as f: + json.dump(data, f, indent=2) + os.replace(tmp, self.path) + + @property + def update_channel(self) -> str | None: + return self.get("update_channel") diff --git a/core/crypto.py b/core/crypto.py index 016eee2..dfb4b2d 100644 --- a/core/crypto.py +++ b/core/crypto.py @@ -1,22 +1,5 @@ -"""AES-256-GCM decryption for Portabase ``.enc`` backup files. - -The ``.enc`` format is produced by the Rust encryption pipeline. Layout:: - - {"version":1,"cipher":"AES-256-GCM","chunk_size":16777216,"base_nonce":[...8 bytes...]}\n - [u32 big-endian ciphertext length][ciphertext + 16-byte GCM tag] # chunk 0 - [u32 big-endian ciphertext length][ciphertext + 16-byte GCM tag] # chunk 1 - ... - -* First line is a compact JSON header terminated by ``\n``. -* Each subsequent chunk is a 4-byte big-endian length prefix followed by the - AES-256-GCM ciphertext with the authentication tag appended (as emitted by - the Rust ``aes-gcm`` crate). -* The 12-byte nonce for chunk ``i`` is ``base_nonce (8 bytes) || i (u32 BE)``. -* The master key is the raw 32-byte AES-256 key. No key derivation is applied; - the Rust side base64-decodes ``masterKeyB64`` to obtain these same 32 bytes. -""" - import base64 +import contextlib import json import os import struct @@ -25,6 +8,8 @@ from cryptography.exceptions import InvalidTag from cryptography.hazmat.primitives.ciphers.aead import AESGCM +from core.errors import PortabaseError + ENC_SUFFIX = ".enc" DEFAULT_KEY_FILENAME = "master_key.bin" CIPHER_NAME = "AES-256-GCM" @@ -32,26 +17,16 @@ _LEN_PREFIX_LEN = 4 _AES_256_KEY_LEN = 32 _TAG_LEN = 16 -# Hard ceiling on a single chunk's plaintext, used when the header's declared -# chunk_size is missing or implausible. Bounds per-chunk allocation so a corrupt -# length prefix can never force an arbitrarily large read on a multi-GB file. _MAX_CHUNK_PLAINTEXT = 256 * 1024 * 1024 -# Copy plaintext to disk in bounded slices so a large chunk is never handed to -# the OS write path as one giant buffer. _WRITE_SLICE = 4 * 1024 * 1024 -class DecryptionError(Exception): - """Raised when a ``.enc`` file cannot be decrypted.""" +class DecryptionError(PortabaseError): + code = "E_CRYPTO" + exit_code = 8 def load_master_key(key_path: Path | None) -> bytes: - """Load the raw 32-byte AES-256 key from ``key_path``. - - When ``key_path`` is ``None`` the key file is looked up as - ``master_key.bin`` in the current working directory. The file may hold - either the raw 32 key bytes or the STANDARD base64 encoding of them. - """ if key_path is None: key_path = Path.cwd() / DEFAULT_KEY_FILENAME @@ -62,14 +37,12 @@ def load_master_key(key_path: Path | None) -> bytes: raw = key_path.read_bytes() - # Raw 32-byte key (as produced by the test/backup tooling). if len(raw) == _AES_256_KEY_LEN: return raw - # Otherwise try to interpret the file as base64 text (masterKeyB64). try: decoded = base64.standard_b64decode(raw.strip()) - except Exception: # binascii.Error / ValueError on malformed base64 + except ValueError: decoded = b"" if len(decoded) == _AES_256_KEY_LEN: return decoded @@ -81,12 +54,6 @@ def load_master_key(key_path: Path | None) -> bytes: def _read_header(handle) -> tuple[bytes, int]: - """Read and validate the JSON header line. - - Returns ``(base_nonce, chunk_size)`` where ``chunk_size`` is the declared - plaintext chunk size, clamped to a safe ceiling. It bounds how many bytes a - single chunk read may allocate regardless of total file size. - """ header_line = handle.readline() if not header_line: raise DecryptionError("File is empty: missing header.") @@ -97,7 +64,9 @@ def _read_header(handle) -> tuple[bytes, int]: cipher = header.get("cipher") if cipher != CIPHER_NAME: - raise DecryptionError(f"Unsupported cipher: {cipher!r} (expected {CIPHER_NAME}).") + raise DecryptionError( + f"Unsupported cipher: {cipher!r} (expected {CIPHER_NAME})." + ) base_nonce = bytes(header.get("base_nonce", [])) if len(base_nonce) != _BASE_NONCE_LEN: @@ -107,19 +76,11 @@ def _read_header(handle) -> tuple[bytes, int]: chunk_size = header.get("chunk_size") if not isinstance(chunk_size, int) or not 0 < chunk_size <= _MAX_CHUNK_PLAINTEXT: - # Unknown or implausible declared size: fall back to the hard ceiling as - # the per-chunk allocation limit rather than trusting the file. chunk_size = _MAX_CHUNK_PLAINTEXT return base_nonce, chunk_size def decrypt_enc_file(enc_path: Path, out_path: Path, key: bytes) -> None: - """Decrypt a single ``.enc`` file to ``out_path``. - - The plaintext is written to a temporary sibling file first and atomically - renamed on success, so a failure never leaves a partial output behind. - Raises :class:`DecryptionError` on any format or authentication failure. - """ aesgcm = AESGCM(key) tmp_path = out_path.with_name(out_path.name + ".part") @@ -134,13 +95,11 @@ def decrypt_enc_file(enc_path: Path, out_path: Path, key: bytes) -> None: while True: len_buf = src.read(_LEN_PREFIX_LEN) if not len_buf: - break # clean end of stream + break if len(len_buf) != _LEN_PREFIX_LEN: raise DecryptionError("Truncated chunk length prefix.") chunk_len = struct.unpack(">I", len_buf)[0] - # Bound the allocation before reading: a corrupt prefix must - # not be able to trigger a multi-GB read on a large file. if chunk_len < _TAG_LEN: raise DecryptionError( f"Chunk {chunk_index} length {chunk_len} is smaller than " @@ -168,8 +127,6 @@ def decrypt_enc_file(enc_path: Path, out_path: Path, key: bytes) -> None: "(wrong key or corrupt data)." ) from exc - # Release the ciphertext buffer before writing the plaintext, - # then write in bounded slices to keep the footprint flat. del ciphertext for start in range(0, len(plaintext), _WRITE_SLICE): dst.write(plaintext[start : start + _WRITE_SLICE]) @@ -177,15 +134,12 @@ def decrypt_enc_file(enc_path: Path, out_path: Path, key: bytes) -> None: os.replace(tmp_path, out_path) except BaseException: - try: + with contextlib.suppress(OSError): tmp_path.unlink(missing_ok=True) - except OSError: - pass raise def default_output_for(enc_path: Path) -> str: - """Return the plaintext filename for ``enc_path`` (strips a trailing ``.enc``).""" name = enc_path.name if name.endswith(ENC_SUFFIX): return name[: -len(ENC_SUFFIX)] diff --git a/core/docker.py b/core/docker.py deleted file mode 100644 index b66c1e0..0000000 --- a/core/docker.py +++ /dev/null @@ -1,19 +0,0 @@ -import subprocess -import typer -from core.utils import console, slugify_project_name -from pathlib import Path - -def ensure_network(name: str): - try: - subprocess.run(["docker", "network", "inspect", name], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, check=True) - except subprocess.CalledProcessError: - subprocess.run(["docker", "network", "create", name], stdout=subprocess.DEVNULL, check=True) - -def run_compose(cwd: Path, args: list): - try: - project_name = slugify_project_name(cwd.resolve().name) - cmd = ["docker", "compose", "-p", project_name] + args - subprocess.run(cmd, cwd=cwd, check=True) - except subprocess.CalledProcessError: - console.print("[danger]Command failed.[/danger]") - raise typer.Exit(1) \ No newline at end of file diff --git a/core/errors.py b/core/errors.py new file mode 100644 index 0000000..8d0612d --- /dev/null +++ b/core/errors.py @@ -0,0 +1,61 @@ +from __future__ import annotations + + +class PortabaseError(Exception): + code: str = "E_GENERIC" + exit_code: int = 1 + + def __init__( + self, + message: str, + *, + hint: str | None = None, + cause: BaseException | None = None, + ) -> None: + super().__init__(message) + self.message = message + self.hint = hint + self.cause = cause + if cause is not None: + self.__cause__ = cause + + def __str__(self) -> str: + return self.message + + +class UserAbort(PortabaseError): + code = "E_ABORT" + exit_code = 130 + + def __init__(self, message: str = "Canceled.", **kwargs) -> None: + super().__init__(message, **kwargs) + + +class ValidationError(PortabaseError): + code = "E_VALIDATION" + exit_code = 2 + + +class ConfigError(PortabaseError): + code = "E_CONFIG" + exit_code = 3 + + +class DockerError(PortabaseError): + code = "E_DOCKER" + exit_code = 4 + + +class TemplateError(PortabaseError): + code = "E_TEMPLATE" + exit_code = 5 + + +class NetworkError(PortabaseError): + code = "E_NETWORK" + exit_code = 6 + + +class UpdateError(PortabaseError): + code = "E_UPDATE" + exit_code = 7 diff --git a/core/fields.py b/core/fields.py new file mode 100644 index 0000000..cd01959 --- /dev/null +++ b/core/fields.py @@ -0,0 +1,22 @@ +from __future__ import annotations + +from collections.abc import Callable +from dataclasses import dataclass +from typing import Any, Literal + +FieldKind = Literal["text", "int", "secret", "bool", "choice", "path"] + + +@dataclass(frozen=True) +class Field: + name: str + prompt: str + kind: FieldKind = "text" + default: Any = None + choices: tuple[str, ...] = () + help: str | None = None + validator: Callable[[Any], Any] | None = None + + @property + def flag(self) -> str: + return "--" + self.name.replace("_", "-") diff --git a/core/network.py b/core/network.py deleted file mode 100644 index 9eeeb84..0000000 --- a/core/network.py +++ /dev/null @@ -1,26 +0,0 @@ -import requests -import typer -from rich.console import Console -from core.config import TEMPLATE_BASE_URL -from core.utils import current_version, get_random_hint - -console = Console() - -def fetch_template(filename: str) -> str: - version = current_version() - url = f"{TEMPLATE_BASE_URL}/{version if version != 'unknown' else 'latest'}/{filename}" - - try: - status_msg = f"[dim]Fetching template...[/dim]\n{get_random_hint()}" - with console.status(status_msg): - response = requests.get(url, timeout=10) - if response.status_code in [403, 404] and version != "unknown": - url = f"{TEMPLATE_BASE_URL}/latest/{filename}" - response = requests.get(url, timeout=10) - - response.raise_for_status() - return response.text - except requests.RequestException as e: - console.print(f"[bold red] Error fetching template:[/bold red] {e}") - console.print("[dim]Check your internet connection or the template URL.[/dim]") - raise typer.Exit(1) \ No newline at end of file diff --git a/core/specs.py b/core/specs.py new file mode 100644 index 0000000..82ec84c --- /dev/null +++ b/core/specs.py @@ -0,0 +1,36 @@ +from __future__ import annotations + +from dataclasses import dataclass, field, replace +from typing import Any + + +@dataclass(frozen=True) +class DatabaseSpec: + id: str + engine: str + name: str + managed: bool = False + host: str | None = None + port: int | None = None + host_port: int | None = None + database: str | None = None + username: str | None = None + password: str | None = None + root_password: str | None = None + path: str | None = None + volume: str | None = None + container: str | None = None + options: dict[str, Any] = field(default_factory=dict) + + @property + def env_prefix(self) -> str: + if not self.host: + raise ValueError("env_prefix requires a host/service name") + return self.host.upper().replace("-", "_") + + @property + def auth(self) -> bool: + return bool(self.password) + + def with_options(self, options: dict[str, Any]) -> DatabaseSpec: + return replace(self, options=dict(options)) diff --git a/core/updater.py b/core/updater.py deleted file mode 100644 index be187fe..0000000 --- a/core/updater.py +++ /dev/null @@ -1,291 +0,0 @@ -import json -import os -import platform -import shutil -import subprocess -import sys -import tempfile -import time -from pathlib import Path - -import requests -import typer - -from core.config import get_config_value -from core.utils import console, current_version, get_random_hint - -GITHUB_REPO = "Portabase/cli" -GITHUB_API_BASE_URL = f"https://api.github.com/repos/{GITHUB_REPO}/releases" -CACHE_FILE = Path.home() / ".portabase" / "update_cache.json" - - -def is_prerelease(version: str) -> bool: - v = version.lower() - return any(x in v for x in ["a", "b", "rc", "alpha", "beta"]) - - -def get_platform_info(): - system = platform.system().lower() - if system == "darwin": - system = "macos" - machine = platform.machine().lower() - - arch = "amd64" - if machine in ["arm64", "aarch64"]: - arch = "arm64" - elif machine in ["x86_64", "amd64"]: - arch = "amd64" - - return system, arch - - -def get_latest_release_data(pre=False): - try: - if not pre: - response = requests.get(f"{GITHUB_API_BASE_URL}/latest", timeout=5) - response.raise_for_status() - return response.json() - else: - response = requests.get(GITHUB_API_BASE_URL, timeout=5) - response.raise_for_status() - releases = response.json() - return releases[0] if releases else None - except Exception: - return None - - -def check_for_updates(force=False): - if ( - not force - and not getattr(sys, "frozen", False) - and platform.system().lower() != "windows" - ): - return None - - current = current_version() - if current == "unknown": - return None - - channel = get_config_value("update_channel") - if channel: - include_pre = channel == "beta" - else: - include_pre = is_prerelease(current) - - latest_tag = None - - try: - CACHE_FILE.parent.mkdir(parents=True, exist_ok=True) - if not force and CACHE_FILE.exists(): - with open(CACHE_FILE, "r") as f: - cache = json.load(f) - if time.time() - cache.get("last_check", 0) < 86400: - latest_tag = cache.get("latest_version") - except Exception: - pass - - if latest_tag is None: - data = get_latest_release_data(pre=include_pre) - if data: - latest_tag = data.get("tag_name", "").lstrip("v") - try: - with open(CACHE_FILE, "w") as f: - json.dump( - {"last_check": time.time(), "latest_version": latest_tag}, f - ) - except Exception: - pass - - if not latest_tag: - return None - - if latest_tag != current: - console.print( - f"\n[warning]⚠ A new version of Portabase CLI is available: [bold]{latest_tag}[/bold] (current: {current})[/warning]" - ) - console.print("[info]Run [bold]portabase update[/bold] to update.[/info]\n") - return latest_tag - return None - - -def update_cli(): - if not getattr(sys, "frozen", False) and platform.system().lower() != "windows": - console.print( - "[warning]⚠ The update command is only available for the binary version of Portabase CLI.[/warning]" - ) - console.print( - "[info]If you installed via source, please use [bold]git pull[/bold] to update.[/info]" - ) - return - - current = current_version() - - channel = get_config_value("update_channel") - if channel: - pre = channel == "beta" - else: - pre = is_prerelease(current) if current != "unknown" else False - - data = get_latest_release_data(pre=pre) - if not data: - console.print( - "[danger]✖ Could not fetch latest release data from GitHub.[/danger]" - ) - return - - latest_tag = data.get("tag_name", "").lstrip("v") - - if latest_tag == current: - console.print( - f"[success]✔ Portabase CLI is already up to date ({current}).[/success]" - ) - return - - try: - if latest_tag < current and not ( - is_prerelease(current) and not is_prerelease(latest_tag) - ): - console.print( - f"[warning]⚠ Current version ({current}) appears to be older than the latest remote version ({latest_tag}).[/warning]" - ) - if not typer.confirm("Do you want to continue with the update ?"): - return - except Exception: - pass - - system, arch = get_platform_info() - asset_name = f"portabase_{system}_{arch}" - if system == "windows": - asset_name += ".exe" - - asset = next((a for a in data.get("assets", []) if a["name"] == asset_name), None) - - if not asset: - console.print( - f"[danger]✖ Could not find binary for your platform ({system}/{arch}) in the latest release.[/danger]" - ) - available_assets = [a["name"] for a in data.get("assets", [])] - console.print(f"[info]Target asset name: {asset_name}[/info]") - console.print(f"[info]Available assets: {', '.join(available_assets)}[/info]") - return - - console.print( - f"[info]Updating Portabase CLI from {current} to {latest_tag}...[/info]" - ) - - try: - if system == "windows": - default_bin_path = ( - Path(os.environ.get("APPDATA", "")) / "Portabase" / "portabase.exe" - ) - else: - default_bin_path = Path("/usr/local/bin/portabase") - - if getattr(sys, "frozen", False): - current_exe = Path(sys.executable) - else: - if default_bin_path.exists(): - current_exe = default_bin_path - else: - current_exe = ( - Path.home() - / ".local" - / "bin" - / ("portabase" if system != "windows" else "portabase.exe") - ) - - console.print(f"[info]Target installation path: {current_exe}[/info]") - - download_url = asset["browser_download_url"] - fd, temp_path = tempfile.mkstemp(prefix="portabase_update_") - temp_file = Path(temp_path) - os.close(fd) - - try: - response = requests.get(download_url, stream=True, timeout=15) - response.raise_for_status() - total_size = int(response.headers.get("content-length", 0)) - - from rich.progress import ( - BarColumn, - DownloadColumn, - Progress, - SpinnerColumn, - TextColumn, - TransferSpeedColumn, - ) - - with Progress( - SpinnerColumn(), - TextColumn( - "[progress.description]{task.description}\n[hint]" - + get_random_hint() - + "[/hint]" - ), - BarColumn(), - DownloadColumn(), - TransferSpeedColumn(), - console=console, - ) as progress: - task = progress.add_task( - f"Downloading {asset_name}...", total=total_size - ) - with open(temp_file, "wb") as f: - for chunk in response.iter_content(chunk_size=8192): - if chunk: - f.write(chunk) - progress.update(task, advance=len(chunk)) - except Exception as e: - if temp_file.exists(): - temp_file.unlink() - raise e - - if system != "windows": - temp_file.chmod(0o755) - - if system == "windows": - if current_exe.exists(): - old_exe = Path(f"{current_exe}.old") - if old_exe.exists(): - old_exe.unlink() - current_exe.rename(old_exe) - temp_file.rename(current_exe) - else: - need_sudo = not os.access(current_exe.parent, os.W_OK) or ( - current_exe.exists() and not os.access(current_exe, os.W_OK) - ) - - if need_sudo: - console.print( - "[info]Permissions required to install to /usr/local/bin. Using sudo...[/info]" - ) - if current_exe.exists(): - subprocess.run( - ["sudo", "mv", str(current_exe), f"{current_exe}.old"], - check=False, - ) - subprocess.run( - ["sudo", "mv", str(temp_file), str(current_exe)], check=True - ) - subprocess.run(["sudo", "chmod", "+x", str(current_exe)], check=True) - else: - if current_exe.exists(): - old_exe = Path(f"{current_exe}.old") - if old_exe.exists(): - old_exe.unlink() - current_exe.rename(old_exe) - current_exe.parent.mkdir(parents=True, exist_ok=True) - shutil.move(str(temp_file), str(current_exe)) - - try: - console.print(f"[success]✔ Successfully updated to {latest_tag}![/success]") - except Exception: - print(f"Successfully updated to {latest_tag}!") - - except Exception as e: - try: - console.print(f"[danger]✖ An error occurred during update: {e}[/danger]") - except Exception: - print(f"An error occurred during update: {e}") - if "temp_file" in locals() and temp_file.exists(): - temp_file.unlink() diff --git a/core/utils.py b/core/utils.py index a6ec75e..c676f18 100644 --- a/core/utils.py +++ b/core/utils.py @@ -1,69 +1,9 @@ import base64 import binascii import json -import platform -import random import re import secrets -import shutil -import socket import string -import subprocess -import time -from pathlib import Path - -import typer -from questionary import Style -from rich.align import Align -from rich.console import Console, Theme -from rich.prompt import Confirm - -questionary_style = Style( - [ - ("pointer", "fg:#ff8800 bold"), - ("highlighted", "fg:black bg:#ff8800 bold"), - ("selected", "fg:#ff8800 bold"), - ] -) - -custom_theme = Theme( - { - "info": "dim cyan", - "warning": "magenta", - "danger": "bold red", - "success": "bold green", - "title": "bold white on #5f00d7", - "key": "bold #ff6600", - "value": "white", - "hint": "italic dim white", - } -) - -HINTS = [ - "The Edge Key contains the connection details for dashboard and agent communication.", - "Portabase uses Docker Compose to isolate your databases.", - "You can list all configured databases using 'portabase db list '.", - "Running 'portabase stop' will gracefully shut down your containers.", - "The agent polls the github for configuration updates.", - "Logs can be viewed in real-time with 'portabase logs '.", - "Custom environment variables can be added to the generated .env file.", - "Need to update? Use 'portabase update' to get the latest version.", - "You can add multiple databases to a single agent during setup.", - "Portabase Dashboard provides a web interface to manage your infrastructure.", - "Is Docker not running? The CLI will offer to start it for you!", - "All configurations are stored locally in the component's folder.", - "The 'portabase restart' command is useful after manual .env modifications.", - "Portabase is open-source! Check our GitHub to contribute.", - "Using the --start flag with 'agent' or 'dashboard' skips the final prompt.", - "Internal databases are automatically backed up when using volumes.", - "The dashboard requires a PostgreSQL database to store its own data.", - "You can change the update channel to 'beta' in the config for early features.", - "Portabase network ensures secure communication between your containers.", - "Lost your Edge Key? You can find it in the dashboard.", - "The 'portabase uninstall' command safely removes containers and their data.", - "Use 'portabase --version' to check your current installation details.", - "The 'databases.json' file keeps track of all managed database instances.", -] def generate_password(length: int = 16) -> str: @@ -74,7 +14,7 @@ def generate_password(length: int = 16) -> str: lower = string.ascii_lowercase upper = string.ascii_uppercase digits = string.digits - symbols = "!@#$%^&*()-_=+[]{}|;:,.<>?" + symbols = "!@#%^&*()-_=+[]{}|;:,.<>?" password = [ secrets.choice(lower), @@ -99,99 +39,7 @@ def slugify_project_name(value: str, fallback: str = "portabase") -> str: return slug or fallback -def get_random_hint(): - return f"[hint]{random.choice(HINTS)}[/hint]" - - -console = Console(theme=custom_theme) - -BANNER = """ -[bold #ff6600]█▀█ █▀█ █▀█ ▀█▀ ▄▀█ █▄▄ ▄▀█ █▀ █▀▀[/bold #ff6600] -[bold #ff6600]█▀▀ █▄█ █▀▄  █  █▀█ █▄█ █▀█ ▄█ ██▄[/bold #ff6600] -[dim]Deploy your infrastructure anywhere.[/dim] -""" - - -def print_banner(): - console.print(Align.center(BANNER)) - console.print(Align.center(get_random_hint() + "\n")) - - -def get_free_port(): - with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s: - s.bind(("", 0)) - return s.getsockname()[1] - - -def start_docker(): - """Attempts to start the Docker daemon based on the OS.""" - os_type = platform.system() - - try: - if os_type == "Linux": - subprocess.run(["sudo", "systemctl", "start", "docker"], check=True) - elif os_type == "Darwin": - subprocess.run(["open", "--background", "-a", "Docker"], check=True) - elif os_type == "Windows": - subprocess.run(["start", "docker"], shell=True, check=True) - - console.print("[info]Waiting for Docker to start...[/info]") - for _ in range(10): - try: - subprocess.run( - ["docker", "info"], - stdout=subprocess.DEVNULL, - stderr=subprocess.DEVNULL, - check=True, - ) - console.print("[success]✔ Docker started successfully.[/success]") - return True - except: - time.sleep(2) - except Exception as e: - console.print(f"[danger]✖ Failed to start Docker:[/danger] {e}") - - return False - - -def check_system(): - docker_path = shutil.which("docker") - - if docker_path is None: - console.print("[danger]✖ Docker not found (binary missing).[/danger]") - raise typer.Exit(1) - - try: - subprocess.run( - [docker_path, "info"], - stdout=subprocess.DEVNULL, - stderr=subprocess.DEVNULL, - check=True, - ) - except subprocess.CalledProcessError: - console.print( - "[warning]⚠ Docker is installed but the Daemon is not running.[/warning]" - ) - if Confirm.ask("Do you want to try starting Docker?"): - if start_docker(): - return - - console.print("[danger]✖ Docker is required to continue.[/danger]") - raise typer.Exit(1) - except Exception as e: - console.print(f"[danger]✖ Critical Error executing Docker:[/danger] {e}") - raise typer.Exit(1) - - -def validate_work_dir(path: Path): - if not (path / "docker-compose.yml").exists(): - console.print(f"[danger]No Portabase configuration found in: {path}[/danger]") - raise typer.Exit(1) - return path - - def validate_edge_key(key: str) -> bool: - """Validates the integrity of the EDGE_KEY (Base64 or JSON).""" try: try: decoded_bytes = base64.b64decode(key, validate=True) @@ -205,23 +53,5 @@ def validate_edge_key(key: str) -> bool: required_fields = ["serverUrl", "agentId", "masterKeyB64"] return all(field in data for field in required_fields) - except Exception: + except TypeError: return False - - -def current_version() -> str: - - try: - import sys - import tomllib - from pathlib import Path - - if getattr(sys, "frozen", False): - base_path = Path(sys._MEIPASS) - else: - base_path = Path(__file__).parent.parent - with open(base_path / "pyproject.toml", "rb") as f: - __version__ = tomllib.load(f)["project"]["version"] - except (FileNotFoundError, KeyError, ImportError, AttributeError): - __version__ = "unknown" - return __version__ diff --git a/core/version.py b/core/version.py new file mode 100644 index 0000000..0ea8299 --- /dev/null +++ b/core/version.py @@ -0,0 +1,37 @@ +from __future__ import annotations + +import re +import sys +import tomllib +from functools import lru_cache +from pathlib import Path + +UNKNOWN = "unknown" +_PRE = re.compile(r"^(\d+)\.(\d+)\.(\d+)(?:[-.]?(rc|alpha|beta|a|b)(\d*))?$", re.I) + + +@lru_cache(maxsize=1) +def current_version() -> str: + try: + bundled = getattr(sys, "_MEIPASS", None) + base = Path(bundled) if bundled else Path(__file__).parent.parent + with open(base / "pyproject.toml", "rb") as f: + return tomllib.load(f)["project"]["version"] + except (FileNotFoundError, KeyError, tomllib.TOMLDecodeError, AttributeError): + return UNKNOWN + + +def is_prerelease(version: str) -> bool: + m = _PRE.match(version.strip().lstrip("v")) + return bool(m and m.group(4)) + + +def parse_version(version: str) -> tuple[int, int, int, int, int]: + m = _PRE.match(version.strip().lstrip("v")) + if not m: + return (0, 0, 0, 0, 0) + major, minor, patch = (int(m.group(i)) for i in (1, 2, 3)) + tag = (m.group(4) or "").lower() + rank = {"alpha": 0, "a": 0, "beta": 1, "b": 1, "rc": 2, "": 3}[tag] + num = int(m.group(5)) if m.group(5) else 0 + return (major, minor, patch, rank, num) diff --git a/docs/superpowers/plans/2026-09-11-plan-1-ci-hygiene.md b/docs/superpowers/plans/2026-09-11-plan-1-ci-hygiene.md new file mode 100644 index 0000000..9f0648a --- /dev/null +++ b/docs/superpowers/plans/2026-09-11-plan-1-ci-hygiene.md @@ -0,0 +1,954 @@ +# Plan 1 — CI, hygiène et release (chantier A) + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** CI de PR bloquante (lint, secrets, sécurité pipeline, build smoke), workflows durcis et pinnés, `./release` remplacé par `bump.yml` — sans changer une ligne de comportement du CLI. + +**Architecture:** Un workflow `ci.yml` sur PR/push main avec des jobs indépendants. Les workflows de release existants restent structurellement identiques, seulement pinnés par SHA et restreints en permissions. La configuration ruff vit dans `pyproject.toml` avec des exclusions explicites pour le code legacy qui sera supprimé aux plans 2–4. + +**Tech Stack:** GitHub Actions, uv 0.9, ruff 0.16, pytest, PyInstaller 6.17, gitleaks-action v2, getplumber/plumber, Dependabot. + +**Spec:** `docs/superpowers/specs/2026-09-11-cli-refactor-design.md` — sections 9 (CI, sécurité, release) et 10 (étape A). + +## Global Constraints + +- Python `>=3.12` (pyproject actuel). Ne pas changer. +- Aucune modification de comportement du CLI dans ce plan. Seuls `pyproject.toml`, `.gitignore`, `.github/**`, `.gitleaks.toml` et le formatage (`ruff format`) bougent. +- Toutes les `uses:` pinnées par SHA complet + commentaire `# vX.Y.Z`. +- `permissions: {}` au top de chaque workflow ; permissions explicites par job. +- Le job `test` existe mais ne collecte aucun test (réservé à la spec tests). +- Aucun test unitaire dans ce plan (consigne utilisateur). Chaque tâche a des étapes de vérification exécutables. +- Commits en Conventional Commits (`chore`, `ci`, `build`, `style`). +- `gh` n'est pas authentifié sur ce poste : les appels `gh api` sur dépôts publics fonctionnent, `gh` sur `Portabase/cli` (protection de branche, secrets) ne fonctionne pas. Vérifier ces points dans l'interface GitHub. + +SHAs résolus le 2026-09-11 (à réutiliser tels quels) : + +| Action | Tag | SHA | +|---|---|---| +| actions/checkout | v4 | `11d5960a326750d5838078e36cf38b85af677262` | +| actions/upload-artifact | v4 | `ea165f8d65b6e75b540449e92b4886f43607fa02` | +| actions/download-artifact | v4 | `d3f86a106a0bac45b974a628896c90dbdf5c8093` | +| actions/attest-build-provenance | v2 | `e8998f949152b193b063cb0ec769d69d929409be` | +| astral-sh/setup-uv | v3 | `caf0cab7a618c569241d31dcd442f54681755d39` | +| astral-sh/ruff-action | v3 | `4919ec5cf1f49eff0871dbcea0da843445b837e6` | +| softprops/action-gh-release | v2 | `3bb12739c298aeb8a4eeaf626c5b8d85266b0e65` | +| mikepenz/release-changelog-builder-action | v5 | `c9dc8369bccbc41e0ac887f8fd674f5925d315f7` | +| gitleaks/gitleaks-action | v2 | `ff98106e4c7b2bc287b24eaf42907196329070c7` | +| getplumber/plumber | (doc officielle) | `3feac69e925e9771f8a495f4177af754d568c1ad` | + +Pour re-résoudre un SHA : `gh api repos///git/ref/tags/ --jq .object.sha` (si `.object.type == "tag"`, résoudre encore via `repos///git/tags/ --jq .object.sha`). + +--- + +## File Structure + +| Fichier | Action | Responsabilité | +|---|---|---| +| `pyproject.toml` | modifier | deps runtime/dev, config ruff, config pytest | +| `.gitignore` | modifier | retirer `uv.lock` (tracké, requis par `--frozen`) | +| `commands/*.py`, `core/*.py`, `main.py` | reformater seulement | `ruff format` mécanique, aucun changement sémantique | +| `.gitleaks.toml` | créer | allowlist des faux positifs | +| `.github/workflows/ci.yml` | créer | lint, test, gitleaks, plumber, build-smoke | +| `.github/workflows/python.yml` | modifier | pin SHA, permissions, `--frozen`, attestation | +| `.github/workflows/github.yml` | modifier | pin SHA, permissions par job | +| `.github/workflows/templates-upload.yml` | modifier | pin SHA, permissions, s3cmd sans `~/.s3cfg` | +| `.github/workflows/release.yml`, `release-candidate.yml` | modifier | `permissions: {}` top-level, retirer `packages: write` | +| `.github/dependabot.yml` | créer | github-actions + uv hebdo | +| `.github/workflows/bump.yml` | créer | remplace `./release` | +| `release` | supprimer | — | +| `.github/CONTRIBUTING.md` | modifier | procédure de release | + +--- + +### Task 1 : `pyproject.toml` — dépendances, ruff, pytest + +**Files:** +- Modify: `pyproject.toml` +- Modify: `.gitignore` + +**Interfaces:** +- Produces: commandes `uv run ruff check .`, `uv run ruff format --check .`, `uv run pytest` utilisables localement et en CI ; groupe `dev` avec `pyinstaller`, `ruff`, `pytest`. + +- [ ] **Step 1: Réécrire `pyproject.toml`** + +Remplacer le contenu intégral par : + +```toml +[project] +name = "portabase-cli" +version = "26.07.6" +description = "The official command line interface (CLI) for managing and deploying Portabase instances with ease." +readme = "README.md" +requires-python = ">=3.12" +dependencies = [ + "typer>=0.20.0", + "rich>=14.2.0", + "questionary>=2.1.0", + "requests>=2.32.5", + "pyyaml>=6.0.3", +] + +[dependency-groups] +dev = [ + "pyinstaller>=6.17.0", + "ruff>=0.16.0", + "pytest>=8.3", +] + +[tool.ruff] +target-version = "py312" +line-length = 88 +extend-exclude = [".venv", "build", "dist"] + +[tool.ruff.lint] +select = [ + "E", "F", "W", # pycodestyle / pyflakes + "I", # isort + "UP", # pyupgrade + "B", # bugbear + "BLE", # blind except + "S110", # try-except-pass + "E722", # bare except + "TID251", # banned imports (activé au plan 2 : rich.prompt, typer.prompt) + "SIM", + "TRY201", + "PLW1510", # subprocess.run sans check= +] +ignore = [ + "B008", # typer.Argument(...) / typer.Option(...) en défaut : idiome Typer + "E501", # line length géré par ruff format +] + +# Code legacy supprimé aux plans 2-4. Ne pas étendre cette liste : tout nouveau +# fichier doit passer sans exception. +[tool.ruff.lint.per-file-ignores] +"commands/agent.py" = ["BLE001", "E722", "S110", "SIM102"] +"commands/db.py" = ["BLE001", "E722", "S110"] +"commands/dashboard.py" = ["BLE001"] +"commands/common.py" = ["BLE001", "PLW1510"] +"core/config.py" = ["BLE001", "E722", "S110"] +"core/utils.py" = ["BLE001", "E722", "S110", "PLR1730"] +"core/updater.py" = ["BLE001", "TRY201"] +"core/network.py" = ["BLE001"] + +[tool.ruff.lint.isort] +known-first-party = ["commands", "core", "templates"] + +[tool.pytest.ini_options] +testpaths = ["tests"] +addopts = "-q" +``` + +- [ ] **Step 2: Retirer `uv.lock` de `.gitignore`** + +`.gitignore` devient : + +``` +dist/ +build/ +.venv/ +__pycache__/ +*.spec +``` + +- [ ] **Step 3: Régénérer le lock et synchroniser** + +Run: `uv lock && uv sync --all-groups` +Expected: `uv.lock` mis à jour (pyinstaller passe en groupe dev, ruff et pytest ajoutés), `.venv` contient `ruff` et `pytest`. + +- [ ] **Step 4: Vérifier que le CLI démarre toujours** + +Run: `uv run python main.py --version` +Expected: `Portabase CLI version: 26.07.6` (plus éventuel message de mise à jour). + +- [ ] **Step 5: Vérifier le lint** + +Run: `uv run ruff check .` +Expected: `All checks passed!`. Si des erreurs subsistent, ajuster **uniquement** `per-file-ignores` pour les fichiers legacy listés ; ne pas modifier le code Python. + +- [ ] **Step 6: Commit** + +```bash +git add pyproject.toml uv.lock .gitignore +git commit -m "build: move pyinstaller to dev group, add ruff and pytest config + +Legacy files get per-file-ignores for bare/blind excepts; those files are +rewritten in later plans and the ignores are removed with them." +``` + +--- + +### Task 2 : Formatage mécanique + +**Files:** +- Modify: tous les fichiers signalés par `ruff format --check` (5 fichiers au 2026-09-11) + +**Interfaces:** +- Produces: `uv run ruff format --check .` passe. + +- [ ] **Step 1: Lister les fichiers à reformater** + +Run: `uv run ruff format --check .` +Expected: `5 files would be reformatted, 17 files already formatted` (nombres indicatifs). + +- [ ] **Step 2: Appliquer** + +Run: `uv run ruff format .` + +- [ ] **Step 3: Vérifier que rien de sémantique n'a changé** + +Run: `git diff --stat && uv run python main.py --help` +Expected: diff uniquement sur espaces/quotes/retours à la ligne ; `--help` affiche les commandes `agent`, `dashboard`, `start`, `stop`, `restart`, `logs`, `uninstall`, `db`, `config`, `update`. + +- [ ] **Step 4: Vérifier lint + format ensemble** + +Run: `uv run ruff check . && uv run ruff format --check .` +Expected: les deux passent. + +- [ ] **Step 5: Commit** + +```bash +git add -A commands core main.py +git commit -m "style: apply ruff format" +``` + +--- + +### Task 3 : `.gitleaks.toml` + +**Files:** +- Create: `.gitleaks.toml` + +**Interfaces:** +- Produces: config lue par `gitleaks/gitleaks-action` (Task 4) et par `gitleaks detect` en local. + +- [ ] **Step 1: Créer le fichier** + +```toml +# Gitleaks configuration for Portabase CLI. +# Extends the default ruleset; only adds allowlists for known false positives. + +title = "portabase-cli" + +[extend] +useDefault = true + +[allowlist] +description = "Known false positives" +paths = [ + # Compose templates contain PASSWORD=${...} placeholders, never real secrets. + '''templates/.*''', + '''\.github/assets/templates/.*''', + # Lock file: hashes only. + '''uv\.lock''', +] +regexes = [ + # Compose interpolation placeholders. + '''\$\{[A-Z0-9_]+\}''', + # Test/fixture edge keys are base64 JSON with these field names, not credentials. + '''"masterKeyB64"''', +] +``` + +- [ ] **Step 2: Scanner l'historique en local** + +Run: `uvx --from gitleaks gitleaks detect --source . --config .gitleaks.toml --redact --no-banner || docker run --rm -v "$PWD:/repo" -w /repo ghcr.io/gitleaks/gitleaks:v8 detect --source . --config .gitleaks.toml --redact --no-banner` + +(Le binaire gitleaks n'est pas distribué via PyPI ; la première commande échouera, la seconde via Docker fonctionne. Si aucun des deux n'est disponible, passer : la CI fera le scan à la Task 4.) + +Expected: `no leaks found`. Si des fuites réelles sont trouvées dans l'historique : **s'arrêter et le signaler** — ne pas allowlister, ne pas réécrire l'historique sans décision explicite. + +- [ ] **Step 3: Commit** + +```bash +git add .gitleaks.toml +git commit -m "ci: add gitleaks config with template placeholder allowlist" +``` + +--- + +### Task 4 : `ci.yml` — lint, test, gitleaks, plumber, build-smoke + +**Files:** +- Create: `.github/workflows/ci.yml` + +**Interfaces:** +- Consumes: config ruff/pytest de Task 1, `.gitleaks.toml` de Task 3. +- Produces: check requis `CI / lint`, `CI / test`, `CI / gitleaks`, `CI / plumber`, `CI / build-smoke` sur chaque PR. Le job `build-smoke` sera enrichi au Plan 4 (invocation `agent --non-interactive`). + +- [ ] **Step 1: Créer le workflow** + +```yaml +name: CI + +on: + pull_request: + push: + branches: [main] + +permissions: {} + +concurrency: + group: ci-${{ github.ref }} + cancel-in-progress: true + +jobs: + lint: + name: lint + runs-on: ubuntu-latest + permissions: + contents: read + steps: + - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4 + - uses: astral-sh/setup-uv@caf0cab7a618c569241d31dcd442f54681755d39 # v3 + - name: Install + run: uv sync --frozen --all-groups + - name: Ruff check + run: uv run ruff check . --output-format=github + - name: Ruff format + run: uv run ruff format --check . + + test: + name: test + runs-on: ubuntu-latest + permissions: + contents: read + steps: + - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4 + - uses: astral-sh/setup-uv@caf0cab7a618c569241d31dcd442f54681755d39 # v3 + - name: Install + run: uv sync --frozen --all-groups + - name: Pytest + # Exit code 5 = no tests collected. Accepted until the test suite exists. + run: | + set +e + uv run pytest + code=$? + set -e + if [ "$code" -ne 0 ] && [ "$code" -ne 5 ]; then exit "$code"; fi + + gitleaks: + name: gitleaks + runs-on: ubuntu-latest + permissions: + contents: read + steps: + - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4 + with: + fetch-depth: 0 + - uses: gitleaks/gitleaks-action@ff98106e4c7b2bc287b24eaf42907196329070c7 # v2 + env: + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + GITLEAKS_CONFIG: .gitleaks.toml + + plumber: + name: plumber + runs-on: ubuntu-24.04 + permissions: + contents: read + security-events: write + steps: + - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4 + - uses: getplumber/plumber@3feac69e925e9771f8a495f4177af754d568c1ad + with: + score-push: false + upload-sarif: true + # First run: observe only. Tighten to min-score once the baseline is known. + soft-fail: true + + build-smoke: + name: build-smoke + runs-on: ubuntu-latest + permissions: + contents: read + steps: + - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4 + - uses: astral-sh/setup-uv@caf0cab7a618c569241d31dcd442f54681755d39 # v3 + - name: Install + run: uv sync --frozen --all-groups + - name: Build binary + run: | + rm -rf build dist *.spec + uv run pyinstaller \ + --onefile \ + --name portabase_smoke \ + --paths=. \ + --collect-all rich \ + --collect-all requests \ + --collect-data certifi \ + --add-data "pyproject.toml:." \ + main.py + - name: Smoke + run: | + ./dist/portabase_smoke --version + ./dist/portabase_smoke --help +``` + +- [ ] **Step 2: Valider la syntaxe YAML localement** + +Run: `uv run python -c "import yaml,sys; yaml.safe_load(open('.github/workflows/ci.yml')); print('ok')"` +Expected: `ok`. + +- [ ] **Step 3: Vérifier localement ce que fera le job lint** + +Run: `uv sync --frozen --all-groups && uv run ruff check . --output-format=github && uv run ruff format --check .` +Expected: aucune sortie d'erreur. + +- [ ] **Step 4: Vérifier localement ce que fera le job test** + +Run: `uv run pytest; echo "exit=$?"` +Expected: `exit=5` (aucun test collecté). + +- [ ] **Step 5: Vérifier localement ce que fera build-smoke** + +Run: `rm -rf build dist *.spec && uv run pyinstaller --onefile --name portabase_smoke --paths=. --collect-all rich --collect-all requests --collect-data certifi --add-data "pyproject.toml:." main.py && ./dist/portabase_smoke --version` +Expected: `Portabase CLI version: 26.07.6`. Puis `rm -rf build dist *.spec`. + +- [ ] **Step 6: Commit** + +```bash +git add .github/workflows/ci.yml +git commit -m "ci: add PR workflow (lint, test, gitleaks, plumber, build smoke)" +``` + +--- + +### Task 5 : Durcir `python.yml` (build binaires) + +**Files:** +- Modify: `.github/workflows/python.yml` + +**Interfaces:** +- Consumes: appelé par `release.yml` / `release-candidate.yml` via `workflow_call`. +- Produces: artefacts `portabase__` inchangés + attestation de provenance. + +- [ ] **Step 1: Réécrire le workflow** + +```yaml +name: Build Python Binaries + +on: + workflow_call: + +permissions: {} + +jobs: + build: + name: Build for ${{ matrix.os }} (${{ matrix.arch }}) + runs-on: ${{ matrix.runner }} + permissions: + contents: read + id-token: write + attestations: write + strategy: + matrix: + include: + - os: linux + arch: amd64 + runner: ubuntu-latest + - os: linux + arch: arm64 + runner: ubuntu-24.04-arm + - os: macos + arch: arm64 + runner: macos-latest + - os: macos + arch: amd64 + runner: macos-15-intel + + steps: + - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4 + + - uses: astral-sh/setup-uv@caf0cab7a618c569241d31dcd442f54681755d39 # v3 + + - name: Install + run: uv sync --frozen --all-groups + + - name: Build binary + run: | + rm -rf build dist *.spec + uv run pyinstaller \ + --onefile \ + --name portabase_${{ matrix.os }}_${{ matrix.arch }} \ + --paths=. \ + --collect-all rich \ + --collect-all requests \ + --collect-data certifi \ + --add-data "pyproject.toml:." \ + main.py + + - name: Smoke + run: ./dist/portabase_${{ matrix.os }}_${{ matrix.arch }} --version + + - name: Attest provenance + uses: actions/attest-build-provenance@e8998f949152b193b063cb0ec769d69d929409be # v2 + with: + subject-path: dist/portabase_${{ matrix.os }}_${{ matrix.arch }} + + - name: Upload artifacts + uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4 + with: + name: portabase_${{ matrix.os }}_${{ matrix.arch }} + path: dist/portabase_${{ matrix.os }}_${{ matrix.arch }} +``` + +Changements par rapport à l'actuel : `uv python install` remplacé par `uv sync --frozen` (respecte `.python-version` et le lock) ; étape `Smoke` ; attestation ; permissions explicites. + +- [ ] **Step 2: Valider YAML** + +Run: `uv run python -c "import yaml; yaml.safe_load(open('.github/workflows/python.yml')); print('ok')"` +Expected: `ok`. + +- [ ] **Step 3: Commit** + +```bash +git add .github/workflows/python.yml +git commit -m "ci: pin actions, scope permissions, attest binaries in build workflow" +``` + +--- + +### Task 6 : Durcir `github.yml` (release GitHub + Discord) + +**Files:** +- Modify: `.github/workflows/github.yml` + +**Interfaces:** +- Consumes: artefacts de Task 5. +- Produces: release GitHub identique à aujourd'hui. + +- [ ] **Step 1: Modifier uniquement l'en-tête et les `uses:`** + +Remplacer le bloc `jobs:` d'en-tête et les trois `uses:` ; le reste (changelog config, script Discord) reste identique. + +En-tête (après le bloc `on:` existant, avant `jobs:`) — ajouter : + +```yaml +permissions: {} +``` + +Job : + +```yaml +jobs: + create-release: + runs-on: ubuntu-latest + permissions: + contents: write + steps: + - name: Check out the repo + uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4 + with: + fetch-depth: 0 + + - name: Download artifacts + if: inputs.artifact_name != '' + uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4 + with: + pattern: ${{ inputs.artifact_name }} + path: dist + merge-multiple: true +``` + +Et plus bas : + +```yaml + - name: Build Changelog + id: build_changelog + uses: mikepenz/release-changelog-builder-action@c9dc8369bccbc41e0ac887f8fd674f5925d315f7 # v5 +``` + +```yaml + - name: Create GitHub Release + uses: softprops/action-gh-release@3bb12739c298aeb8a4eeaf626c5b8d85266b0e65 # v2 +``` + +- [ ] **Step 2: Vérifier qu'aucun `@vN` non pinné ne reste** + +Run: `grep -nE 'uses: .*@v[0-9]' .github/workflows/github.yml` +Expected: aucune sortie. + +- [ ] **Step 3: Valider YAML** + +Run: `uv run python -c "import yaml; yaml.safe_load(open('.github/workflows/github.yml')); print('ok')"` +Expected: `ok`. + +- [ ] **Step 4: Commit** + +```bash +git add .github/workflows/github.yml +git commit -m "ci: pin actions and scope permissions in release workflow" +``` + +--- + +### Task 7 : Durcir `templates-upload.yml` (S3 sans fichier de credentials) + +**Files:** +- Modify: `.github/workflows/templates-upload.yml` + +**Interfaces:** +- Produces: même arborescence S3 qu'aujourd'hui (`cli/public/templates//` et `latest/`). La source reste `.github/assets/templates/` jusqu'au Plan 3 qui la déplace vers `templates/` et ajoute le manifest. + +- [ ] **Step 1: Réécrire le workflow** + +```yaml +name: Upload Templates to S3 + +on: + workflow_call: + inputs: + version: + required: true + type: string + is_prerelease: + required: true + type: boolean + secrets: + S3_ENDPOINT: + required: true + S3_ACCESS_KEY: + required: true + S3_SECRET_KEY: + required: true + S3_BUCKET: + required: true + +permissions: {} + +jobs: + upload: + runs-on: ubuntu-latest + permissions: + contents: read + env: + # s3cmd reads these flags; no config file is written to disk. + S3CMD_ARGS: >- + --access_key=${{ secrets.S3_ACCESS_KEY }} + --secret_key=${{ secrets.S3_SECRET_KEY }} + --host=${{ secrets.S3_ENDPOINT }} + --host-bucket=%(bucket)s.${{ secrets.S3_ENDPOINT }} + --ssl + steps: + - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4 + + - name: Install s3cmd + run: sudo apt-get update && sudo apt-get install -y s3cmd + + - name: Upload versioned templates + run: | + CLEAN_VERSION="${{ inputs.version }}" + CLEAN_VERSION="${CLEAN_VERSION#v}" + s3cmd $S3CMD_ARGS sync .github/assets/templates/ \ + "s3://${{ secrets.S3_BUCKET }}/cli/public/templates/${CLEAN_VERSION}/" --acl-public + + - name: Upload latest templates (stable only) + if: ${{ !inputs.is_prerelease }} + run: | + s3cmd $S3CMD_ARGS sync .github/assets/templates/ \ + "s3://${{ secrets.S3_BUCKET }}/cli/public/templates/latest/" --acl-public +``` + +Note : les secrets passés en arguments de ligne de commande sont masqués dans les logs par GitHub (`***`). C'est le compromis retenu ; l'alternative (`~/.s3cfg`) laisse les secrets en clair sur le disque du runner. + +- [ ] **Step 2: Valider YAML** + +Run: `uv run python -c "import yaml; yaml.safe_load(open('.github/workflows/templates-upload.yml')); print('ok')"` +Expected: `ok`. + +- [ ] **Step 3: Commit** + +```bash +git add .github/workflows/templates-upload.yml +git commit -m "ci: pass S3 credentials to s3cmd as flags instead of writing ~/.s3cfg" +``` + +--- + +### Task 8 : Permissions top-level sur `release.yml` et `release-candidate.yml` + +**Files:** +- Modify: `.github/workflows/release.yml:10-13` +- Modify: `.github/workflows/release-candidate.yml:12-15` + +**Interfaces:** +- Produces: workflows appelants avec permissions minimales ; les jobs `uses:` héritent des permissions déclarées dans les workflows appelés (Tasks 5–7). + +- [ ] **Step 1: Dans les deux fichiers, remplacer** + +```yaml +permissions: + contents: write + packages: write +``` + +par + +```yaml +permissions: + contents: write + id-token: write + attestations: write + security-events: write +``` + +Un workflow appelant doit déclarer au moins les permissions que les workflows appelés demandent (`contents: write` pour la release, `id-token`/`attestations` pour l'attestation). `packages: write` n'était utilisé par aucun job. + +- [ ] **Step 2: Vérifier** + +Run: `grep -n "packages" .github/workflows/*.yml` +Expected: aucune sortie. + +- [ ] **Step 3: Commit** + +```bash +git add .github/workflows/release.yml .github/workflows/release-candidate.yml +git commit -m "ci: drop unused packages permission, declare attestation permissions" +``` + +--- + +### Task 9 : Dependabot + +**Files:** +- Create: `.github/dependabot.yml` + +**Interfaces:** +- Produces: PRs hebdomadaires pour les SHAs d'actions et les dépendances uv. + +- [ ] **Step 1: Créer le fichier** + +```yaml +version: 2 +updates: + - package-ecosystem: github-actions + directory: / + schedule: + interval: weekly + groups: + actions: + patterns: ["*"] + commit-message: + prefix: "ci" + + - package-ecosystem: uv + directory: / + schedule: + interval: weekly + groups: + python: + patterns: ["*"] + commit-message: + prefix: "build" +``` + +- [ ] **Step 2: Valider YAML** + +Run: `uv run python -c "import yaml; yaml.safe_load(open('.github/dependabot.yml')); print('ok')"` +Expected: `ok`. + +- [ ] **Step 3: Commit** + +```bash +git add .github/dependabot.yml +git commit -m "ci: enable dependabot for actions and uv" +``` + +--- + +### Task 10 : `bump.yml` remplace `./release` + +**Files:** +- Create: `.github/workflows/bump.yml` +- Delete: `release` +- Modify: `.github/CONTRIBUTING.md` + +**Interfaces:** +- Produces: déclenchement manuel qui commit `chore(release): X`, tague `X` et pousse. Le push du tag déclenche `release.yml` ou `release-candidate.yml` selon le motif, exactement comme le script. + +- [ ] **Step 1: Créer le workflow** + +```yaml +name: Bump version + +on: + workflow_dispatch: + inputs: + version: + description: "Version (e.g. 26.09.0 or 26.09.0rc1). No leading v." + required: true + type: string + channel: + description: "stable: only from main. rc: any branch." + required: true + type: choice + options: [stable, rc] + default: rc + +permissions: {} + +jobs: + bump: + runs-on: ubuntu-latest + permissions: + contents: write + steps: + - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4 + with: + fetch-depth: 0 + # Use a PAT if branch protection blocks GITHUB_TOKEN pushes to main. + token: ${{ secrets.GITHUB_TOKEN }} + + - name: Validate version against channel + env: + VERSION: ${{ inputs.version }} + CHANNEL: ${{ inputs.channel }} + REF: ${{ github.ref_name }} + run: | + set -euo pipefail + if [[ "$VERSION" == v* ]]; then + echo "::error::Version must not start with 'v'"; exit 1 + fi + if [[ "$CHANNEL" == "stable" ]]; then + if [[ "$REF" != "main" ]]; then + echo "::error::stable releases are only allowed from main (got $REF)"; exit 1 + fi + if [[ ! "$VERSION" =~ ^[0-9]+\.[0-9]+\.[0-9]+$ ]]; then + echo "::error::stable version must match X.Y.Z"; exit 1 + fi + else + if [[ ! "$VERSION" =~ ^[0-9]+\.[0-9]+\.[0-9]+([-.]?(rc|alpha|beta|a|b)[0-9]*(\.[0-9]+)?)$ ]]; then + echo "::error::rc version must match X.Y.Z(rc|a|b|alpha|beta)N"; exit 1 + fi + fi + if git rev-parse "$VERSION" >/dev/null 2>&1; then + echo "::error::Tag $VERSION already exists"; exit 1 + fi + + - name: Update version files + env: + VERSION: ${{ inputs.version }} + run: | + set -euo pipefail + DATE=$(date -u +%F) + sed -i "s/^version = \".*\"/version = \"$VERSION\"/" pyproject.toml + if [ -f CITATION.cff ]; then + sed -i "s/^version: .*/version: $VERSION/" CITATION.cff + sed -i "s/^date-released: .*/date-released: \"$DATE\"/" CITATION.cff + fi + git diff --stat + + - name: Commit, tag, push + env: + VERSION: ${{ inputs.version }} + run: | + set -euo pipefail + git config user.name "github-actions[bot]" + git config user.email "41898282+github-actions[bot]@users.noreply.github.com" + git add pyproject.toml CITATION.cff + if git diff --cached --quiet; then + echo "No version change to commit" + else + git commit -m "chore(release): $VERSION" + fi + git tag -a "$VERSION" -m "Release $VERSION" + git push origin HEAD + git push origin "$VERSION" +``` + +Différences avec le script : pas de `package.json` / `Cargo.toml` (absents du dépôt) ; `git add .` remplacé par un `add` ciblé ; identité bot. + +Point à vérifier dans l'interface GitHub (Settings → Branches) : si `main` exige une PR, `git push origin HEAD` sera refusé pour `GITHUB_TOKEN`. Deux solutions : (a) autoriser `github-actions[bot]` à contourner la règle ; (b) créer un PAT fine-grained (Contents: write) stocké en secret `RELEASE_TOKEN` et remplacer `token: ${{ secrets.GITHUB_TOKEN }}` par `token: ${{ secrets.RELEASE_TOKEN }}`. Note : un push effectué avec `GITHUB_TOKEN` ne déclenche **pas** d'autres workflows par design GitHub — **le push du tag ne déclenchera donc pas `release.yml`**. Avec un PAT (`RELEASE_TOKEN`), il le déclenche. → **Utiliser un PAT est obligatoire** pour que le tag lance la release. Créer le secret avant le premier usage. + +- [ ] **Step 2: Remplacer le token par le PAT** + +Dans le workflow ci-dessus, `token: ${{ secrets.GITHUB_TOKEN }}` → `token: ${{ secrets.RELEASE_TOKEN }}` et supprimer le commentaire au-dessus. Le secret `RELEASE_TOKEN` (fine-grained PAT, dépôt `Portabase/cli`, permissions Contents: Read and write, Metadata: Read) doit être créé par un mainteneur dans Settings → Secrets → Actions. + +- [ ] **Step 3: Supprimer le script** + +Run: `git rm release` + +- [ ] **Step 4: Documenter dans CONTRIBUTING.md** + +Ajouter une section à la fin de `.github/CONTRIBUTING.md` : + +```markdown +## Releasing + +Releases are cut from GitHub Actions, never from a local machine. + +1. Open **Actions → Bump version → Run workflow**. +2. Pick the branch (`main` for stable, any branch for a release candidate). +3. Enter the version without a leading `v` (`26.09.0` for stable, `26.09.0rc1` for a candidate) and the matching channel. +4. The workflow commits `chore(release): `, creates the tag and pushes. The tag triggers the build, the GitHub release, the Discord notification and the template upload. + +Stable versions must match `X.Y.Z` and can only be cut from `main`. +``` + +- [ ] **Step 5: Valider YAML** + +Run: `uv run python -c "import yaml; yaml.safe_load(open('.github/workflows/bump.yml')); print('ok')"` +Expected: `ok`. + +- [ ] **Step 6: Commit** + +```bash +git add .github/workflows/bump.yml .github/CONTRIBUTING.md +git commit -m "ci: replace ./release script with bump workflow" +``` + +--- + +### Task 11 : Vérification de bout en bout sur GitHub + +**Files:** aucun. + +**Interfaces:** +- Consumes: tout ce qui précède. + +- [ ] **Step 1: Pousser une branche et ouvrir une PR** + +```bash +git checkout -b ci/hygiene +git push -u origin ci/hygiene +gh pr create --fill --title "ci: PR workflow, pinned actions, bump workflow" --body "Implements plan 1 (chantier A) of docs/superpowers/specs/2026-09-11-cli-refactor-design.md. No CLI behaviour change." +``` + +(`gh` non authentifié ici : créer la PR depuis l'interface si la commande échoue.) + +- [ ] **Step 2: Vérifier les checks** + +Expected dans l'onglet Checks : `lint`, `test`, `gitleaks`, `plumber`, `build-smoke` tous verts. `plumber` publie un rapport SARIF dans Security → Code scanning ; noter le score obtenu. + +- [ ] **Step 3: Si `plumber` remonte des findings sur les workflows** + +Les traiter dans la même PR si triviaux (permission manquante, action non pinnée oubliée). Sinon ouvrir une issue avec la liste et laisser `soft-fail: true`. + +- [ ] **Step 4: Créer le secret `RELEASE_TOKEN`** + +Settings → Secrets and variables → Actions → New repository secret. PAT fine-grained, dépôt `Portabase/cli`, Contents: Read and write, Metadata: Read. + +- [ ] **Step 5: Merger, puis tester `bump.yml` avec un rc jetable** + +Actions → Bump version → branche `main`, version `26.07.7rc1`, channel `rc`. Expected : commit `chore(release): 26.07.7rc1` sur `main`, tag créé, `release-candidate.yml` déclenché, binaires attestés publiés en pre-release, templates uploadés sous `templates/26.07.7rc1/`. + +- [ ] **Step 6: Rendre les checks requis** + +Settings → Branches → `main` → Require status checks : `lint`, `test`, `gitleaks`, `build-smoke`. Laisser `plumber` non requis tant que `soft-fail: true`. + +--- + +## Self-review + +**Spec coverage (§9, §10 A) :** +- 9.1 `ci.yml` : lint ✔ (T4), test vide ✔ (T4), gitleaks ✔ (T3, T4), plumber ✔ (T4), build-smoke `--version` ✔ (T4 ; l'invocation `agent --non-interactive` arrive au Plan 4), `render-check` et `engines-check` → Plan 3 (dépendent des templates `.j2` et du registre). +- 9.2 pin SHA ✔ (T4–T8), Dependabot ✔ (T9), `permissions: {}` ✔, `packages: write` retiré ✔ (T8), `~/.s3cfg` supprimé ✔ (T7), attestation ✔ (T5). +- 9.3 `bump.yml` ✔ (T10), `./release` supprimé ✔, pas de release-please ✔, question branch protection → T10/T11. `templates-hotfix.yml` et manifest → Plan 3. +- 9.4 `pyproject.toml` ✔ (T1) ; `jinja2` ajouté au Plan 3 quand il est utilisé. +- 10 A : shippable stable ✔ (T11 step 5 le prouve avec un rc). + +**Placeholder scan :** aucun TBD/TODO. Toutes les étapes ont leur contenu ou leur commande. + +**Type consistency :** noms de jobs identiques entre T4 et T11 (`lint`, `test`, `gitleaks`, `plumber`, `build-smoke`) ; secret `RELEASE_TOKEN` cohérent T10/T11 ; SHAs identiques entre tâches. + +**Écart connu :** T1 `per-file-ignores` liste des fichiers/règles déduits du run ruff du 2026-09-11 ; si ruff remonte une règle non listée sur un fichier legacy, l'ajouter à la liste de ce fichier (pas de correction de code). diff --git a/docs/superpowers/plans/2026-09-11-plan-2-foundations-lifecycle.md b/docs/superpowers/plans/2026-09-11-plan-2-foundations-lifecycle.md new file mode 100644 index 0000000..51f8cae --- /dev/null +++ b/docs/superpowers/plans/2026-09-11-plan-2-foundations-lifecycle.md @@ -0,0 +1,2385 @@ +# Plan 2 — Fondations et lifecycle (chantiers B + C) + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Poser les fondations POO (erreurs, `ui/`, services d'infrastructure, `Command`, catcher, télémétrie no-op, updater sans auto-update) et réécrire les commandes de lifecycle/config/update dessus, tout en gardant `agent`, `dashboard` et `db` sur l'ancien code via un adaptateur — le CLI reste shippable en stable à la fin. + +**Architecture:** `main.py` construit les dépendances (`UI`, `Telemetry`, `GlobalConfig`, `HttpClient`, `DockerRunner`) et les injecte dans des classes `Command` enregistrées sur Typer. Un seul `try` dans `main()` traduit `PortabaseError` en message + code de sortie. Les commandes legacy sont enregistrées telles quelles par `LegacyCommand` ; elles continuent d'importer `core.utils.console` jusqu'au Plan 4. + +**Tech Stack:** Python 3.12, Typer 0.25 / Click 8.4, Rich 15, questionary 2.1, requests. + +**Spec:** `docs/superpowers/specs/2026-09-11-cli-refactor-design.md` — sections 3, 4.1, 7, 8, 10 (B, C). + +## Global Constraints + +- Prérequis : Plan 1 exécuté (ruff configuré, CI en place). +- Règle de dépendance descendante : `commands → services, engines, ui, core` ; `services → engines, core` (jamais `ui`) ; `ui → core` ; `core → rien`. Vérifiée par ruff `TID251` (Task 12). +- `rich.prompt`, `typer.prompt`, `typer.confirm`, `print` interdits hors `ui/` (ruff `TID251`, activé Task 12 avec exceptions legacy). +- `typer.Exit` n'est levé nulle part hors des fichiers legacy ; le nouveau code lève `PortabaseError`. +- Pas de tests unitaires (consigne). Chaque tâche a des vérifications exécutables ; les commandes Docker sont vérifiées avec un dossier agent réel si Docker est disponible, sinon sur leurs chemins d'erreur. +- Ne pas toucher `commands/agent.py`, `commands/db.py`, `commands/dashboard.py`, `commands/decrypt.py`, `core/crypto.py`, `core/network.py`, `core/docker.py`, `templates/compose.py` (réécrits ou supprimés au Plan 4). `decrypt` (ajouté en 26.08.12) est enregistré via `LegacyCommand` comme `agent`/`dashboard`. `core/utils.py` : seulement retirer `current_version` (Task 2). +- Déviation spec assumée : `Field` vit dans `core/fields.py` (partagé par `ui.Form` et `engines`), pas dans `engines/base.py`. +- Nom de la clé de config existante conservé : `update_channel` (valeurs `stable` / `beta`). +- Commits Conventional Commits, un par tâche minimum. + +--- + +## File Structure + +| Fichier | Action | Responsabilité | +|---|---|---| +| `core/errors.py` | créer | hiérarchie `PortabaseError` | +| `core/version.py` | créer | `current_version()`, `parse_version()`, `is_prerelease()` | +| `core/utils.py` | modifier | retirer `current_version` (re-export pour legacy) | +| `core/config.py` | modifier | ajouter classe `GlobalConfig` ; fonctions legacy conservées | +| `core/fields.py` | créer | `Field` | +| `ui/theme.py` | créer | `PALETTE`, `RICH_THEME`, `QUESTIONARY_STYLE` | +| `ui/components/base.py` | créer | `Component` | +| `ui/components/hints.py` | créer | `HINTS`, `Hint` | +| `ui/components/message.py` | créer | `Message` | +| `ui/components/banner.py` | créer | `Banner` | +| `ui/components/section.py` | créer | `Section` | +| `ui/components/status.py` | créer | `Status` | +| `ui/components/progress.py` | créer | `Progress` (téléchargement) | +| `ui/components/prompt.py` | créer | `Prompt` (questionary) | +| `ui/form.py` | créer | `Form` | +| `ui/__init__.py` | créer | façade `UI` | +| `services/http.py` | créer | `HttpClient` | +| `services/docker.py` | créer | `DockerRunner` | +| `services/telemetry.py` | créer | `Telemetry`, `NoopTelemetry`, `ConsoleTelemetry`, `TelemetryHub`, `TelemetryFactory` | +| `services/updater.py` | créer | `Release`, `UpdateChecker`, `Updater` | +| `commands/base.py` | créer | `Command`, `CommandGroup`, `LegacyCommand` | +| `commands/lifecycle.py` | créer | `Start/Stop/Restart/Logs/Uninstall` | +| `commands/config.py` | réécrire | `ConfigCommands` | +| `commands/update.py` | créer | `UpdateCommand` | +| `main.py` | réécrire | `Settings`, `build_app`, `main` | +| `commands/common.py`, `core/updater.py` | supprimer | — | +| `pyproject.toml` | modifier | `TID251`, per-file-ignores mis à jour | + +--- + +### Task 1 : `core/errors.py` + +**Files:** +- Create: `core/errors.py` + +**Interfaces:** +- Produces: `PortabaseError(message, *, hint=None, cause=None)` avec attributs `message`, `hint`, `cause`, classe-attributs `code: str`, `exit_code: int` ; sous-classes `UserAbort`, `ValidationError`, `ConfigError`, `DockerError`, `TemplateError`, `NetworkError`, `UpdateError`. + +- [ ] **Step 1: Écrire le module** + +```python +"""Exception hierarchy. Every error the CLI reports to a user is one of these.""" + +from __future__ import annotations + + +class PortabaseError(Exception): + code: str = "E_GENERIC" + exit_code: int = 1 + + def __init__( + self, + message: str, + *, + hint: str | None = None, + cause: BaseException | None = None, + ) -> None: + super().__init__(message) + self.message = message + self.hint = hint + self.cause = cause + if cause is not None: + self.__cause__ = cause + + def __str__(self) -> str: + return self.message + + +class UserAbort(PortabaseError): + code = "E_ABORT" + exit_code = 130 + + def __init__(self, message: str = "Cancelled.", **kwargs) -> None: + super().__init__(message, **kwargs) + + +class ValidationError(PortabaseError): + code = "E_VALIDATION" + exit_code = 2 + + +class ConfigError(PortabaseError): + code = "E_CONFIG" + exit_code = 3 + + +class DockerError(PortabaseError): + code = "E_DOCKER" + exit_code = 4 + + +class TemplateError(PortabaseError): + code = "E_TEMPLATE" + exit_code = 5 + + +class NetworkError(PortabaseError): + code = "E_NETWORK" + exit_code = 6 + + +class UpdateError(PortabaseError): + code = "E_UPDATE" + exit_code = 7 +``` + +- [ ] **Step 2: Vérifier** + +Run: `uv run python -c "from core.errors import *; e = DockerError('daemon down', hint='start it'); print(e.code, e.exit_code, e, e.hint); assert isinstance(e, PortabaseError)"` +Expected: `E_DOCKER 4 daemon down start it`. + +- [ ] **Step 3: Commit** + +```bash +git add core/errors.py +git commit -m "feat(core): add PortabaseError hierarchy with stable codes and exit codes" +``` + +--- + +### Task 2 : `core/version.py` et `core/fields.py` + +**Files:** +- Create: `core/version.py` +- Create: `core/fields.py` +- Modify: `core/utils.py:197-214` (fonction `current_version`) + +**Interfaces:** +- Produces: `current_version() -> str` ; `parse_version(v: str) -> tuple` ; `is_prerelease(v: str) -> bool` ; `Field` dataclass. + +- [ ] **Step 1: Écrire `core/version.py`** + +```python +"""CLI version helpers. Version is read from the bundled pyproject.toml.""" + +from __future__ import annotations + +import re +import sys +import tomllib +from functools import lru_cache +from pathlib import Path + +UNKNOWN = "unknown" +_PRE = re.compile(r"^(\d+)\.(\d+)\.(\d+)(?:[-.]?(rc|alpha|beta|a|b)(\d*))?$", re.I) + + +@lru_cache(maxsize=1) +def current_version() -> str: + try: + base = Path(sys._MEIPASS) if getattr(sys, "frozen", False) else Path(__file__).parent.parent + with open(base / "pyproject.toml", "rb") as f: + return tomllib.load(f)["project"]["version"] + except (FileNotFoundError, KeyError, tomllib.TOMLDecodeError, AttributeError): + return UNKNOWN + + +def is_prerelease(version: str) -> bool: + m = _PRE.match(version.strip().lstrip("v")) + return bool(m and m.group(4)) + + +def parse_version(version: str) -> tuple[int, int, int, int, int]: + """Sortable tuple. Pre-releases sort before the final release of the same number. + + (major, minor, patch, pre_rank, pre_number) — pre_rank: 0 alpha/a, 1 beta/b, 2 rc, 3 final. + """ + m = _PRE.match(version.strip().lstrip("v")) + if not m: + return (0, 0, 0, 0, 0) + major, minor, patch = (int(m.group(i)) for i in (1, 2, 3)) + tag = (m.group(4) or "").lower() + rank = {"alpha": 0, "a": 0, "beta": 1, "b": 1, "rc": 2, "": 3}[tag] + num = int(m.group(5)) if m.group(5) else 0 + return (major, minor, patch, rank, num) +``` + +- [ ] **Step 2: Écrire `core/fields.py`** + +```python +"""Declarative input field. Used by ui.Form to prompt or validate a value.""" + +from __future__ import annotations + +from collections.abc import Callable +from dataclasses import dataclass +from typing import Any, Literal + +FieldKind = Literal["text", "int", "secret", "bool", "choice", "path"] + + +@dataclass(frozen=True) +class Field: + name: str + prompt: str + kind: FieldKind = "text" + default: Any = None + choices: tuple[str, ...] = () + help: str | None = None + validator: Callable[[Any], Any] | None = None + + @property + def flag(self) -> str: + return "--" + self.name.replace("_", "-") +``` + +- [ ] **Step 3: Retirer `current_version` de `core/utils.py`** + +Supprimer la fonction `current_version` (lignes ~197–214) et ajouter en tête des imports : + +```python +from core.version import current_version # noqa: F401 — re-export for legacy modules +``` + +`core/network.py` et `core/updater.py` importent `current_version` depuis `core.utils` ; le re-export les garde fonctionnels jusqu'à leur suppression. + +- [ ] **Step 4: Vérifier** + +Run: `uv run python -c "from core.version import *; print(current_version(), is_prerelease('26.09.0rc1'), parse_version('26.09.0rc1') < parse_version('26.09.0'), parse_version('26.10.0') > parse_version('26.9.9'))" && uv run python main.py --version` +Expected: `26.07.6 True True True` puis `Portabase CLI version: 26.07.6`. + +- [ ] **Step 5: Commit** + +```bash +git add core/version.py core/fields.py core/utils.py +git commit -m "feat(core): add version helpers and Field descriptor" +``` + +--- + +### Task 3 : `GlobalConfig` + +**Files:** +- Modify: `core/config.py` + +**Interfaces:** +- Produces: `GlobalConfig(path: Path = GLOBAL_CONFIG_FILE)` avec `get(key, default=None)`, `set(key, value)`, `all() -> dict`, `cache_dir: Path` (`~/.portabase/cache`) ; propriétés typées `update_channel: str | None`, `telemetry: bool`, `telemetry_endpoint: str | None`. +- Les fonctions module-level existantes restent (legacy). + +- [ ] **Step 1: Ajouter la classe en fin de `core/config.py`** + +```python +class GlobalConfig: + """~/.portabase/config.json. Unknown keys are preserved.""" + + KNOWN_KEYS = ("update_channel", "telemetry", "telemetry_endpoint") + + def __init__(self, path: Path = GLOBAL_CONFIG_FILE) -> None: + self.path = path + self.cache_dir = path.parent / "cache" + + def all(self) -> dict: + if not self.path.exists(): + return {} + try: + with open(self.path, encoding="utf-8") as f: + data = json.load(f) + except (OSError, json.JSONDecodeError): + return {} + return data if isinstance(data, dict) else {} + + def get(self, key: str, default=None): + return self.all().get(key, default) + + def set(self, key: str, value) -> None: + data = self.all() + data[key] = value + self.path.parent.mkdir(parents=True, exist_ok=True) + tmp = self.path.with_suffix(".json.tmp") + with open(tmp, "w", encoding="utf-8") as f: + json.dump(data, f, indent=2) + os.replace(tmp, self.path) + + @property + def update_channel(self) -> str | None: + return self.get("update_channel") + + @property + def telemetry(self) -> bool: + return str(self.get("telemetry", "false")).lower() in ("1", "true", "yes") + + @property + def telemetry_endpoint(self) -> str | None: + return self.get("telemetry_endpoint") +``` + +Ajouter au-dessus des fonctions legacy le commentaire : + +```python +# --- Legacy helpers below: used by commands/agent.py, db.py, dashboard.py, core/updater.py. +# --- Removed in plan 4. New code uses GlobalConfig. +``` + +- [ ] **Step 2: Vérifier** + +Run: `uv run python -c " +from pathlib import Path; import tempfile +from core.config import GlobalConfig +c = GlobalConfig(Path(tempfile.mkdtemp())/'config.json') +print(c.all(), c.telemetry); c.set('update_channel','beta'); c.set('telemetry', True) +print(c.update_channel, c.telemetry, c.all())"` +Expected: `{} False` puis `beta True {'update_channel': 'beta', 'telemetry': True}`. + +- [ ] **Step 3: Commit** + +```bash +git add core/config.py +git commit -m "feat(core): add GlobalConfig class over ~/.portabase/config.json" +``` + +--- + +### Task 4 : `ui/theme.py` et composants d'affichage + +**Files:** +- Create: `ui/__init__.py` (vide pour l'instant, rempli Task 6) +- Create: `ui/theme.py` +- Create: `ui/components/__init__.py` (vide) +- Create: `ui/components/base.py` +- Create: `ui/components/hints.py` +- Create: `ui/components/message.py` +- Create: `ui/components/banner.py` +- Create: `ui/components/section.py` +- Create: `ui/components/status.py` +- Create: `ui/components/progress.py` + +**Interfaces:** +- Produces: `Component(console)` ; `Hint(console).random() -> str` ; `Message(console).success/info/warning(text)`, `.error(exc: PortabaseError, *, verbose: bool, unexpected: bool)` ; `Banner(console)()` ; `Section(console)(title)` ; `Status(console)(text) -> ContextManager` ; `Progress(console).download(description, total) -> ContextManager[Callable[[int], None]]`. + +- [ ] **Step 1: `ui/theme.py`** + +```python +"""Single source of visual tokens. Rich theme and questionary style derive from PALETTE.""" + +from __future__ import annotations + +from questionary import Style +from rich.theme import Theme + +PALETTE = { + "brand": "#ff6600", + "accent": "#5f00d7", + "info": "cyan", + "warning": "magenta", + "danger": "red", + "success": "green", + "muted": "grey50", +} + +RICH_THEME = Theme( + { + "info": f"dim {PALETTE['info']}", + "warning": PALETTE["warning"], + "danger": f"bold {PALETTE['danger']}", + "success": f"bold {PALETTE['success']}", + "title": f"bold white on {PALETTE['accent']}", + "key": f"bold {PALETTE['brand']}", + "value": "white", + "hint": f"italic {PALETTE['muted']}", + "brand": f"bold {PALETTE['brand']}", + } +) + +QUESTIONARY_STYLE = Style( + [ + ("qmark", f"fg:{PALETTE['brand']} bold"), + ("question", "bold"), + ("pointer", f"fg:{PALETTE['brand']} bold"), + ("highlighted", f"fg:black bg:{PALETTE['brand']} bold"), + ("selected", f"fg:{PALETTE['brand']} bold"), + ("answer", f"fg:{PALETTE['brand']}"), + ] +) + +QUESTIONARY_STYLE_PLAIN = Style([]) +``` + +- [ ] **Step 2: `ui/components/base.py`** + +```python +from __future__ import annotations + +from rich.console import Console + + +class Component: + """Stateless renderable bound to a console. Instantiate per call.""" + + def __init__(self, console: Console) -> None: + self.console = console +``` + +- [ ] **Step 3: `ui/components/hints.py`** + +Reprendre la liste `HINTS` de `core/utils.py:42-66` telle quelle. + +```python +from __future__ import annotations + +import random + +from ui.components.base import Component + +HINTS = [ + "The Edge Key contains the connection details for dashboard and agent communication.", + "Portabase uses Docker Compose to isolate your databases.", + "You can list all configured databases using 'portabase db list '.", + "Running 'portabase stop' will gracefully shut down your containers.", + "The agent polls the github for configuration updates.", + "Logs can be viewed in real-time with 'portabase logs '.", + "Custom environment variables can be added to the generated .env file.", + "Need to update? Use 'portabase update' to get the latest version.", + "You can add multiple databases to a single agent during setup.", + "Portabase Dashboard provides a web interface to manage your infrastructure.", + "Is Docker not running? The CLI will offer to start it for you!", + "All configurations are stored locally in the component's folder.", + "The 'portabase restart' command is useful after manual .env modifications.", + "Portabase is open-source! Check our GitHub to contribute.", + "Using the --start flag with 'agent' or 'dashboard' skips the final prompt.", + "Internal databases are automatically backed up when using volumes.", + "The dashboard requires a PostgreSQL database to store its own data.", + "You can change the update channel to 'beta' in the config for early features.", + "Portabase network ensures secure communication between your containers.", + "Lost your Edge Key? You can find it in the dashboard.", + "The 'portabase uninstall' command safely removes containers and their data.", + "Use 'portabase --version' to check your current installation details.", + "The 'databases.json' file keeps track of all managed database instances.", +] + + +class Hint(Component): + def random(self) -> str: + return f"[hint]{random.choice(HINTS)}[/hint]" + + def __call__(self, text: str | None = None) -> None: + self.console.print(f"[hint]{text}[/hint]" if text else self.random()) +``` + +- [ ] **Step 4: `ui/components/message.py`** + +```python +from __future__ import annotations + +import traceback + +from core.errors import PortabaseError +from ui.components.base import Component + + +class Message(Component): + def success(self, text: str) -> None: + self.console.print(f"[success]✔ {text}[/success]") + + def info(self, text: str) -> None: + self.console.print(f"[info]ℹ {text}[/info]") + + def warning(self, text: str) -> None: + self.console.print(f"[warning]⚠ {text}[/warning]") + + def error(self, exc: PortabaseError, *, verbose: bool = False, unexpected: bool = False) -> None: + label = "Unexpected error" if unexpected else "Error" + self.console.print(f"[danger]✖ {label}:[/danger] {exc.message}") + if exc.hint: + self.console.print(f" [hint]↳ {exc.hint}[/hint]") + if verbose or unexpected: + self.console.print(f" [hint]code: {exc.code}[/hint]") + if verbose and exc.cause is not None: + self.console.print(f" [hint]cause: {type(exc.cause).__name__}: {exc.cause}[/hint]") + if verbose: + self.console.print("".join(traceback.format_exception(exc)), highlight=False, markup=False) +``` + +- [ ] **Step 5: `ui/components/banner.py`** + +```python +from __future__ import annotations + +from rich.align import Align + +from ui.components.base import Component +from ui.components.hints import Hint + +BANNER = """ +[brand]█▀█ █▀█ █▀█ ▀█▀ ▄▀█ █▄▄ ▄▀█ █▀ █▀▀[/brand] +[brand]█▀▀ █▄█ █▀▄ █ █▀█ █▄█ █▀█ ▄█ ██▄[/brand] +[hint]Deploy your infrastructure anywhere.[/hint] +""" + + +class Banner(Component): + def __call__(self) -> None: + self.console.print(Align.center(BANNER)) + self.console.print(Align.center(Hint(self.console).random() + "\n")) +``` + +- [ ] **Step 6: `ui/components/section.py`** + +```python +from __future__ import annotations + +from rich.panel import Panel + +from ui.components.base import Component + + +class Section(Component): + def __call__(self, title: str) -> None: + self.console.print("") + self.console.print(Panel(f"[bold]{title}[/bold]", style="cyan", expand=False)) +``` + +- [ ] **Step 7: `ui/components/status.py`** + +```python +from __future__ import annotations + +from contextlib import AbstractContextManager + +from ui.components.base import Component +from ui.components.hints import Hint + + +class Status(Component): + def __call__(self, text: str, *, spinner: str = "dots") -> AbstractContextManager: + message = f"[bold magenta]{text}[/bold magenta]\n{Hint(self.console).random()}" + return self.console.status(message, spinner=spinner) +``` + +- [ ] **Step 8: `ui/components/progress.py`** + +```python +from __future__ import annotations + +from collections.abc import Callable, Iterator +from contextlib import contextmanager + +from rich.progress import ( + BarColumn, + DownloadColumn, + Progress as RichProgress, + SpinnerColumn, + TextColumn, + TransferSpeedColumn, +) + +from ui.components.base import Component +from ui.components.hints import Hint + + +class Progress(Component): + @contextmanager + def download(self, description: str, total: int) -> Iterator[Callable[[int], None]]: + """Yields an advance(n_bytes) callable.""" + with RichProgress( + SpinnerColumn(), + TextColumn("[progress.description]{task.description}\n" + Hint(self.console).random()), + BarColumn(), + DownloadColumn(), + TransferSpeedColumn(), + console=self.console, + ) as progress: + task = progress.add_task(description, total=total or None) + yield lambda n: progress.update(task, advance=n) +``` + +- [ ] **Step 9: Vérifier le rendu** + +Run: `uv run python -c " +from rich.console import Console +from ui.theme import RICH_THEME +from ui.components.message import Message +from ui.components.banner import Banner +from ui.components.section import Section +from core.errors import DockerError +c = Console(theme=RICH_THEME) +Banner(c)(); Section(c)('Database Setup') +m = Message(c); m.success('ok'); m.info('note'); m.warning('careful') +m.error(DockerError('daemon down', hint='run: sudo systemctl start docker')) +m.error(DockerError('daemon down', hint='x', cause=RuntimeError('boom')), verbose=True)"` +Expected: bannière orange, panneau cyan, quatre messages avec icônes ✔ ℹ ⚠ ✖, hint indenté, puis le bloc verbose avec `code: E_DOCKER`, `cause: RuntimeError: boom` et une traceback. + +- [ ] **Step 10: Commit** + +```bash +git add ui/ +git commit -m "feat(ui): add theme tokens and display components" +``` + +--- + +### Task 5 : `ui/components/prompt.py` et `ui/form.py` + +**Files:** +- Create: `ui/components/prompt.py` +- Create: `ui/form.py` + +**Interfaces:** +- Consumes: `Field` (Task 2), `UserAbort`/`ValidationError` (Task 1), `QUESTIONARY_STYLE` (Task 4). +- Produces: `Prompt(console, style)` avec `text/integer/secret/confirm/select/path` renvoyant `None` sur Ctrl-C ; `Form(prompt, non_interactive)` avec `ask(field, value)`, `collect(fields, values) -> dict`, raccourcis `text/integer/secret/confirm/choice`. + +- [ ] **Step 1: `ui/components/prompt.py`** + +```python +from __future__ import annotations + +from collections.abc import Sequence + +import questionary +from questionary import Style +from rich.console import Console + +from ui.components.base import Component + + +class Prompt(Component): + """Thin wrapper over questionary. Every method returns None when the user aborts (Ctrl-C).""" + + def __init__(self, console: Console, style: Style) -> None: + super().__init__(console) + self.style = style + + def text(self, message: str, *, default: str | None = None) -> str | None: + return questionary.text(message, default=default or "", style=self.style).ask() + + def integer(self, message: str, *, default: int | None = None) -> int | None: + answer = questionary.text( + message, + default="" if default is None else str(default), + validate=lambda v: v.strip().lstrip("-").isdigit() or "Enter a whole number", + style=self.style, + ).ask() + return None if answer is None else int(answer) + + def secret(self, message: str) -> str | None: + return questionary.password(message, style=self.style).ask() + + def confirm(self, message: str, *, default: bool = False) -> bool | None: + return questionary.confirm(message, default=default, style=self.style).ask() + + def select(self, message: str, choices: Sequence[str], *, default: str | None = None) -> str | None: + return questionary.select(message, choices=list(choices), default=default, style=self.style).ask() + + def path(self, message: str, *, default: str | None = None) -> str | None: + return questionary.path(message, default=default or "", style=self.style).ask() +``` + +- [ ] **Step 2: `ui/form.py`** + +```python +"""Flag → prompt → default → error. The only place that knows about non-interactive mode.""" + +from __future__ import annotations + +from collections.abc import Callable, Sequence +from typing import Any + +from core.errors import UserAbort, ValidationError +from core.fields import Field +from ui.components.prompt import Prompt + +_TRUE = {"1", "true", "yes", "y", "on"} +_FALSE = {"0", "false", "no", "n", "off"} + + +class Form: + def __init__(self, prompt: Prompt, non_interactive: bool) -> None: + self.prompt = prompt + self.non_interactive = non_interactive + self._askers: dict[str, Callable[[Field], Any]] = { + "text": lambda f: self.prompt.text(f.prompt, default=f.default), + "int": lambda f: self.prompt.integer(f.prompt, default=f.default), + "secret": lambda f: self.prompt.secret(f.prompt), + "bool": lambda f: self.prompt.confirm(f.prompt, default=bool(f.default)), + "choice": lambda f: self.prompt.select(f.prompt, f.choices, default=f.default), + "path": lambda f: self.prompt.path(f.prompt, default=f.default), + } + + # ---- core ------------------------------------------------------------- + + def ask(self, field: Field, value: Any | None = None) -> Any: + if value is not None: + return self._coerce_and_validate(field, value) + if self.non_interactive: + if field.default is not None: + return self._coerce_and_validate(field, field.default) + raise ValidationError( + f"Missing {field.flag}", + hint=f"Required in non-interactive mode: {field.prompt}", + ) + return self._ask_until_valid(field) + + def collect(self, fields: Sequence[Field], values: dict[str, Any]) -> dict[str, Any]: + return {f.name: self.ask(f, values.get(f.name)) for f in fields} + + # ---- shortcuts -------------------------------------------------------- + + def text(self, prompt: str, *, value=None, default=None, validator=None, name="value") -> str: + return self.ask(Field(name, prompt, "text", default=default, validator=validator), value) + + def integer(self, prompt: str, *, value=None, default=None, validator=None, name="value") -> int: + return self.ask(Field(name, prompt, "int", default=default, validator=validator), value) + + def secret(self, prompt: str, *, value=None, validator=None, name="value") -> str: + return self.ask(Field(name, prompt, "secret", validator=validator), value) + + def confirm(self, prompt: str, *, value=None, default: bool = False, name="value") -> bool: + return self.ask(Field(name, prompt, "bool", default=default), value) + + def choice(self, prompt: str, choices: Sequence[str], *, value=None, default=None, name="value") -> str: + return self.ask(Field(name, prompt, "choice", default=default, choices=tuple(choices)), value) + + # ---- internals -------------------------------------------------------- + + def _ask_until_valid(self, field: Field) -> Any: + if field.help: + self.prompt.console.print(f"[info]ℹ {field.help}[/info]") + while True: + answer = self._askers[field.kind](field) + if answer is None: + raise UserAbort() + try: + return self._coerce_and_validate(field, answer) + except ValidationError as e: + self.prompt.console.print(f"[danger]✖ {e.message}[/danger]") + + def _coerce_and_validate(self, field: Field, value: Any) -> Any: + value = self._coerce(field, value) + if field.kind == "choice" and value not in field.choices: + raise ValidationError( + f"Invalid value for {field.flag}: {value!r}", + hint="Choices: " + ", ".join(field.choices), + ) + if field.validator is not None: + value = field.validator(value) # raises ValidationError + return value + + @staticmethod + def _coerce(field: Field, value: Any) -> Any: + if field.kind == "int" and not isinstance(value, int): + try: + return int(str(value).strip()) + except ValueError as e: + raise ValidationError(f"{field.flag} must be a whole number, got {value!r}") from e + if field.kind == "bool" and not isinstance(value, bool): + s = str(value).strip().lower() + if s in _TRUE: + return True + if s in _FALSE: + return False + raise ValidationError(f"{field.flag} must be true or false, got {value!r}") + if field.kind in ("text", "secret", "path", "choice"): + return str(value) + return value +``` + +- [ ] **Step 3: Vérifier le mode non-interactif (sans terminal)** + +Run: `uv run python -c " +from rich.console import Console +from ui.theme import QUESTIONARY_STYLE +from ui.components.prompt import Prompt +from ui.form import Form +from core.fields import Field +from core.errors import ValidationError +f = Form(Prompt(Console(), QUESTIONARY_STYLE), non_interactive=True) +print(f.text('Timezone', value=None, default='UTC'), f.integer('Polling', value='7'), f.confirm('Gateway?', value='yes')) +print(f.collect([Field('engine','Engine','choice',choices=('a','b')), Field('port','Port','int',default=5432)], {'engine':'a'})) +try: f.text('Edge key') +except ValidationError as e: print('OK:', e.message, '|', e.hint) +try: f.choice('Mode', ['new','existing'], value='bogus') +except ValidationError as e: print('OK:', e.message, '|', e.hint)"` +Expected : +``` +UTC 7 True +{'engine': 'a', 'port': 5432} +OK: Missing --value | Required in non-interactive mode: Edge key +OK: Invalid value for --value: 'bogus' | Choices: new, existing +``` + +- [ ] **Step 4: Vérifier le mode interactif (terminal requis)** + +Run: `uv run python -c " +from rich.console import Console +from ui.theme import QUESTIONARY_STYLE +from ui.components.prompt import Prompt +from ui.form import Form +f = Form(Prompt(Console(), QUESTIONARY_STYLE), non_interactive=False) +print(f.choice('Mode', ['new','existing'], default='new')) +print(f.integer('Port', default=5432))"` +Répondre aux deux prompts. Puis relancer et faire Ctrl-C au premier prompt. +Expected: valeurs saisies affichées ; sur Ctrl-C, traceback se terminant par `core.errors.UserAbort: Cancelled.` (le catcher n'est pas encore branché — attendu). + +- [ ] **Step 5: Commit** + +```bash +git add ui/components/prompt.py ui/form.py +git commit -m "feat(ui): add questionary Prompt and Form with non-interactive resolution" +``` + +--- + +### Task 6 : Façade `UI` + +**Files:** +- Modify: `ui/__init__.py` + +**Interfaces:** +- Produces: `UI(console=None, *, non_interactive=False, verbose=False, no_color=False)` ; `configure(**kwargs)` ; `banner()`, `success/info/warning(text)`, `error(exc, unexpected=False)`, `hint(text=None)`, `section(title)`, `status(text)`, `progress()`, `confirm(q, default=False, value=None) -> bool`, `form() -> Form`, `print(renderable)`. Attribut `console`. + +- [ ] **Step 1: Écrire la façade** + +```python +"""Facade: the only thing `commands/` imports from ui. Rich and questionary stay inside ui/.""" + +from __future__ import annotations + +from rich.console import Console + +from core.errors import PortabaseError +from ui.components.banner import Banner +from ui.components.hints import Hint +from ui.components.message import Message +from ui.components.progress import Progress +from ui.components.prompt import Prompt +from ui.components.section import Section +from ui.components.status import Status +from ui.form import Form +from ui.theme import QUESTIONARY_STYLE, QUESTIONARY_STYLE_PLAIN, RICH_THEME + + +class UI: + def __init__( + self, + console: Console | None = None, + *, + non_interactive: bool = False, + verbose: bool = False, + no_color: bool = False, + ) -> None: + self.non_interactive = non_interactive + self.verbose = verbose + self.no_color = no_color + self.console = console or self._make_console() + + def configure(self, *, non_interactive: bool | None = None, verbose: bool | None = None, no_color: bool | None = None) -> None: + if non_interactive is not None: + self.non_interactive = non_interactive + if verbose is not None: + self.verbose = verbose + if no_color is not None and no_color != self.no_color: + self.no_color = no_color + self.console = self._make_console() + + def _make_console(self) -> Console: + return Console(theme=RICH_THEME, no_color=self.no_color) + + # ---- output ----------------------------------------------------------- + + def print(self, renderable) -> None: + self.console.print(renderable) + + def banner(self) -> None: + Banner(self.console)() + + def success(self, text: str) -> None: + Message(self.console).success(text) + + def info(self, text: str) -> None: + Message(self.console).info(text) + + def warning(self, text: str) -> None: + Message(self.console).warning(text) + + def error(self, exc: PortabaseError, *, unexpected: bool = False) -> None: + Message(self.console).error(exc, verbose=self.verbose, unexpected=unexpected) + + def hint(self, text: str | None = None) -> None: + Hint(self.console)(text) + + def section(self, title: str) -> None: + Section(self.console)(title) + + def status(self, text: str): + return Status(self.console)(text) + + def progress(self) -> Progress: + return Progress(self.console) + + # ---- input ------------------------------------------------------------ + + def form(self) -> Form: + style = QUESTIONARY_STYLE_PLAIN if self.no_color else QUESTIONARY_STYLE + return Form(Prompt(self.console, style), self.non_interactive) + + def confirm(self, question: str, *, default: bool = False, value: bool | None = None) -> bool: + return self.form().confirm(question, value=value, default=default) +``` + +- [ ] **Step 2: Vérifier** + +Run: `uv run python -c " +from ui import UI +ui = UI(non_interactive=True) +ui.banner(); ui.section('Test'); ui.success('a'); ui.warning('b'); ui.hint() +print('confirm default:', ui.confirm('Really?', default=False)) +with ui.status('Working...'): import time; time.sleep(0.5) +ui.configure(no_color=True); ui.success('no color')"` +Expected: rendu, `confirm default: False` sans prompt, spinner 0,5 s, dernière ligne sans couleur. + +- [ ] **Step 3: Commit** + +```bash +git add ui/__init__.py +git commit -m "feat(ui): add UI facade" +``` + +--- + +### Task 7 : `services/http.py` et `services/docker.py` + +**Files:** +- Create: `services/__init__.py` (vide) +- Create: `services/http.py` +- Create: `services/docker.py` + +**Interfaces:** +- Produces: + - `HttpClient(timeout=10.0)` : `get_json(url) -> Any`, `get_text(url) -> str`, `download(url, dest: Path, on_progress: Callable[[int], None] | None = None, *, timeout=30.0) -> int` (octets), `head_content_length(url) -> int | None`. Lèvent `NetworkError`. + - `DockerRunner(docker_bin: str | None = None)` : `available() -> bool`, `daemon_running() -> bool`, `start_daemon() -> bool`, `ensure_network(name)`, `compose(cwd, args, *, check=True, capture=False) -> subprocess.CompletedProcess`, `project_name(cwd) -> str`. Lèvent `DockerError`. + +- [ ] **Step 1: `services/http.py`** + +```python +"""requests wrapper. Every failure becomes NetworkError; nothing else leaks out.""" + +from __future__ import annotations + +from collections.abc import Callable +from pathlib import Path +from typing import Any + +import requests + +from core.errors import NetworkError + +_HINT = "Check your internet connection or proxy settings." + + +class HttpClient: + def __init__(self, timeout: float = 10.0, user_agent: str = "portabase-cli") -> None: + self.timeout = timeout + self.session = requests.Session() + self.session.headers["User-Agent"] = user_agent + + def get_json(self, url: str) -> Any: + try: + r = self.session.get(url, timeout=self.timeout) + r.raise_for_status() + return r.json() + except requests.RequestException as e: + raise NetworkError(f"GET {url} failed: {e}", hint=_HINT, cause=e) from e + except ValueError as e: + raise NetworkError(f"GET {url}: response is not JSON", cause=e) from e + + def get_text(self, url: str) -> str: + try: + r = self.session.get(url, timeout=self.timeout) + r.raise_for_status() + return r.text + except requests.RequestException as e: + raise NetworkError(f"GET {url} failed: {e}", hint=_HINT, cause=e) from e + + def status(self, url: str) -> int: + """HTTP status without raising on 4xx/5xx. Network failure still raises.""" + try: + return self.session.get(url, timeout=self.timeout, stream=True).status_code + except requests.RequestException as e: + raise NetworkError(f"GET {url} failed: {e}", hint=_HINT, cause=e) from e + + def download( + self, + url: str, + dest: Path, + on_progress: Callable[[int], None] | None = None, + *, + timeout: float = 30.0, + ) -> int: + written = 0 + try: + with self.session.get(url, stream=True, timeout=timeout) as r: + r.raise_for_status() + with open(dest, "wb") as f: + for chunk in r.iter_content(chunk_size=64 * 1024): + if not chunk: + continue + f.write(chunk) + written += len(chunk) + if on_progress: + on_progress(len(chunk)) + except requests.RequestException as e: + dest.unlink(missing_ok=True) + raise NetworkError(f"Download of {url} failed: {e}", hint=_HINT, cause=e) from e + return written + + def content_length(self, url: str) -> int | None: + try: + r = self.session.head(url, timeout=self.timeout, allow_redirects=True) + value = r.headers.get("content-length") + return int(value) if value else None + except (requests.RequestException, ValueError): + return None +``` + +- [ ] **Step 2: `services/docker.py`** + +Reprend `core/docker.py` + `check_system`/`start_docker` de `core/utils.py`, sans aucune sortie terminal. + +```python +"""Docker CLI runner. No terminal output; callers decide what to show.""" + +from __future__ import annotations + +import platform +import shutil +import subprocess +import time +from pathlib import Path + +from core.errors import DockerError +from core.utils import slugify_project_name + +_START_COMMANDS = { + "Linux": ["sudo", "systemctl", "start", "docker"], + "Darwin": ["open", "--background", "-a", "Docker"], + "Windows": ["cmd", "/c", "start", "docker"], +} + + +class DockerRunner: + def __init__(self, docker_bin: str | None = None) -> None: + self._bin = docker_bin + + @property + def binary(self) -> str: + if self._bin is None: + found = shutil.which("docker") + if found is None: + raise DockerError( + "Docker not found (binary missing).", + hint="Install Docker: https://docs.docker.com/get-docker/", + ) + self._bin = found + return self._bin + + def available(self) -> bool: + return shutil.which("docker") is not None + + def daemon_running(self) -> bool: + try: + subprocess.run( + [self.binary, "info"], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, check=True + ) + return True + except (subprocess.CalledProcessError, OSError): + return False + + def start_daemon(self, *, wait_seconds: int = 20) -> bool: + cmd = _START_COMMANDS.get(platform.system()) + if cmd is None: + return False + try: + subprocess.run(cmd, check=True) + except (subprocess.CalledProcessError, OSError) as e: + raise DockerError(f"Failed to start Docker: {e}", cause=e) from e + deadline = time.monotonic() + wait_seconds + while time.monotonic() < deadline: + if self.daemon_running(): + return True + time.sleep(2) + return False + + def ensure_network(self, name: str) -> None: + inspect = subprocess.run( + [self.binary, "network", "inspect", name], + stdout=subprocess.DEVNULL, + stderr=subprocess.DEVNULL, + check=False, + ) + if inspect.returncode == 0: + return + try: + subprocess.run([self.binary, "network", "create", name], stdout=subprocess.DEVNULL, check=True) + except subprocess.CalledProcessError as e: + raise DockerError(f"Could not create Docker network '{name}'.", cause=e) from e + + @staticmethod + def project_name(cwd: Path) -> str: + return slugify_project_name(cwd.resolve().name) + + def compose( + self, + cwd: Path, + args: list[str], + *, + check: bool = True, + capture: bool = False, + ) -> subprocess.CompletedProcess: + cmd = [self.binary, "compose", "-p", self.project_name(cwd), *args] + try: + return subprocess.run( + cmd, + cwd=cwd, + check=check, + capture_output=capture, + text=capture, + ) + except subprocess.CalledProcessError as e: + raise DockerError( + f"docker compose {' '.join(args)} failed (exit {e.returncode}).", + hint=f"Run it manually in {cwd} to see the full output.", + cause=e, + ) from e +``` + +- [ ] **Step 3: Vérifier** + +Run: `uv run python -c " +from services.http import HttpClient +from services.docker import DockerRunner +from core.errors import NetworkError, DockerError +h = HttpClient(timeout=5) +print(type(h.get_json('https://api.github.com/repos/Portabase/cli')).__name__) +try: h.get_json('https://127.0.0.1:1/nope') +except NetworkError as e: print('NetworkError OK:', e.code) +d = DockerRunner(); print('docker available:', d.available(), '| daemon:', d.available() and d.daemon_running()) +try: DockerRunner(docker_bin='/nonexistent').compose(__import__('pathlib').Path('.'), ['version']) +except (DockerError, OSError) as e: print('error path OK:', type(e).__name__)"` +Expected: `dict`, `NetworkError OK: E_NETWORK`, état Docker local, `error path OK: FileNotFoundError` ou `DockerError` (les deux acceptables ici ; `OSError` est traité au niveau commande, Task 9). + +- [ ] **Step 4: Commit** + +```bash +git add services/ +git commit -m "feat(services): add HttpClient and DockerRunner" +``` + +--- + +### Task 8 : `services/telemetry.py` + +**Files:** +- Create: `services/telemetry.py` + +**Interfaces:** +- Produces: `Telemetry` ABC (`session(**attrs)`, `span(name, **attrs)`, `event(name, **attrs)`, `error(exc, unexpected=False)`, `flush()`) ; `NoopTelemetry` ; `ConsoleTelemetry(stream=sys.stderr)` ; `TelemetryHub(inner)` avec `.set(inner)` ; `TelemetryFactory.build(config: GlobalConfig, *, debug: bool) -> TelemetryHub`. + +- [ ] **Step 1: Écrire le module** + +```python +"""Telemetry contract. Noop by default; OTel exporter can be plugged later without touching callers. + +Never record: agent names, paths, keys, credentials, file contents. +""" + +from __future__ import annotations + +import sys +import time +from abc import ABC, abstractmethod +from collections.abc import Iterator +from contextlib import contextmanager +from typing import Any, TextIO + +from core.config import GlobalConfig + + +class Telemetry(ABC): + @abstractmethod + def session(self, **attrs: Any): + """Context manager: root span for one CLI invocation.""" + + @abstractmethod + def span(self, name: str, **attrs: Any): + """Context manager: child span.""" + + @abstractmethod + def event(self, name: str, **attrs: Any) -> None: ... + + @abstractmethod + def error(self, exc: BaseException, *, unexpected: bool = False) -> None: ... + + def flush(self) -> None: + return None + + +class NoopTelemetry(Telemetry): + @contextmanager + def session(self, **attrs: Any) -> Iterator[None]: + yield + + @contextmanager + def span(self, name: str, **attrs: Any) -> Iterator[None]: + yield + + def event(self, name: str, **attrs: Any) -> None: + return None + + def error(self, exc: BaseException, *, unexpected: bool = False) -> None: + return None + + +class ConsoleTelemetry(Telemetry): + """--debug: prints spans and events to stderr. Development aid, not an exporter.""" + + def __init__(self, stream: TextIO = sys.stderr) -> None: + self.stream = stream + self._depth = 0 + + def _log(self, line: str) -> None: + self.stream.write(" " * self._depth + f"[telemetry] {line}\n") + self.stream.flush() + + @contextmanager + def session(self, **attrs: Any) -> Iterator[None]: + with self.span("session", **attrs): + yield + + @contextmanager + def span(self, name: str, **attrs: Any) -> Iterator[None]: + self._log(f"▶ {name} {attrs}") + self._depth += 1 + start = time.perf_counter() + try: + yield + finally: + self._depth -= 1 + self._log(f"◀ {name} {time.perf_counter() - start:.3f}s") + + def event(self, name: str, **attrs: Any) -> None: + self._log(f"• {name} {attrs}") + + def error(self, exc: BaseException, *, unexpected: bool = False) -> None: + code = getattr(exc, "code", type(exc).__name__) + self._log(f"✖ error code={code} unexpected={unexpected}") + + +class TelemetryHub(Telemetry): + """Delegates to a swappable implementation. Commands hold the hub; main swaps the inner.""" + + def __init__(self, inner: Telemetry | None = None) -> None: + self.inner: Telemetry = inner or NoopTelemetry() + + def set(self, inner: Telemetry) -> None: + self.inner = inner + + def session(self, **attrs: Any): + return self.inner.session(**attrs) + + def span(self, name: str, **attrs: Any): + return self.inner.span(name, **attrs) + + def event(self, name: str, **attrs: Any) -> None: + self.inner.event(name, **attrs) + + def error(self, exc: BaseException, *, unexpected: bool = False) -> None: + self.inner.error(exc, unexpected=unexpected) + + def flush(self) -> None: + self.inner.flush() + + +class TelemetryFactory: + @staticmethod + def build(config: GlobalConfig, *, debug: bool = False) -> TelemetryHub: + if debug: + return TelemetryHub(ConsoleTelemetry()) + # Opt-in and endpoint present → OTel exporter (future plan). Until then: noop. + return TelemetryHub(NoopTelemetry()) +``` + +- [ ] **Step 2: Vérifier** + +Run: `uv run python -c " +from services.telemetry import * +from core.config import GlobalConfig +hub = TelemetryFactory.build(GlobalConfig(), debug=True) +with hub.session(cli_version='x'): + with hub.span('command.start', command='start'): + hub.event('compose', args='up') + hub.error(RuntimeError('boom'), unexpected=True) +hub.set(NoopTelemetry()) +with hub.span('silent'): pass +print('ok')"` +Expected: lignes `[telemetry]` imbriquées sur stderr pour session/command/event/error, rien pour `silent`, puis `ok`. + +- [ ] **Step 3: Commit** + +```bash +git add services/telemetry.py +git commit -m "feat(services): add Telemetry contract with noop, console and hub implementations" +``` + +--- + +### Task 9 : `commands/base.py` — `Command`, `CommandGroup`, `LegacyCommand` + +**Files:** +- Create: `commands/base.py` + +**Interfaces:** +- Consumes: `UI`, `Telemetry`, `DockerRunner`, erreurs. +- Produces: + - `Command(ui, telemetry)` : attributs de classe `name`, `help`, `panel`, `no_args_is_help=False` ; `register(app)` ; `run(...)` abstraite ; helpers `require_docker(docker)`, `require_project_dir(path) -> Path`. + - `CommandGroup(ui, telemetry)` : `name`, `help`, `commands: list[Command]`, `typer() -> typer.Typer`, `register(app)`. + - `LegacyCommand(ui, telemetry, fn, *, name, help, panel, no_args_is_help=True)`. + +- [ ] **Step 1: Écrire le module** + +```python +"""Command base classes. Typer registers bound `run` methods; dependencies come via constructors.""" + +from __future__ import annotations + +import functools +from abc import ABC, abstractmethod +from collections.abc import Callable +from pathlib import Path + +import typer + +from core.errors import ConfigError, DockerError, UserAbort +from services.docker import DockerRunner +from services.telemetry import Telemetry +from ui import UI + + +class Command(ABC): + name: str + help: str + panel: str = "General" + no_args_is_help: bool = False + + def __init__(self, ui: UI, telemetry: Telemetry) -> None: + self.ui = ui + self.telemetry = telemetry + + # ---- registration ----------------------------------------------------- + + def register(self, app: typer.Typer) -> None: + app.command( + self.name, + help=self.help, + rich_help_panel=self.panel, + no_args_is_help=self.no_args_is_help, + )(self._traced(self.run)) + + def _traced(self, fn: Callable) -> Callable: + @functools.wraps(fn) + def wrapper(*args, **kwargs): + with self.telemetry.span(f"command.{self.name}"): + return fn(*args, **kwargs) + + return wrapper + + @abstractmethod + def run(self, *args, **kwargs) -> None: ... + + # ---- shared helpers --------------------------------------------------- + + def require_docker(self, docker: DockerRunner) -> None: + """Binary present and daemon up, offering to start it when interactive.""" + if not docker.available(): + raise DockerError( + "Docker not found (binary missing).", + hint="Install Docker: https://docs.docker.com/get-docker/", + ) + if docker.daemon_running(): + return + self.ui.warning("Docker is installed but the daemon is not running.") + if self.ui.confirm("Do you want to try starting Docker?", default=False): + with self.ui.status("Waiting for Docker to start..."): + started = docker.start_daemon() + if started: + self.ui.success("Docker started successfully.") + return + raise DockerError("Docker is required to continue.", hint="Start the Docker daemon and retry.") + + @staticmethod + def require_project_dir(path: Path) -> Path: + path = path.resolve() + if not (path / "docker-compose.yml").exists(): + raise ConfigError( + f"No Portabase configuration found in: {path}", + hint="Expected a docker-compose.yml created by 'portabase agent' or 'portabase dashboard'.", + ) + return path + + def confirm_or_abort(self, question: str, *, default: bool = False, value: bool | None = None) -> None: + if not self.ui.confirm(question, default=default, value=value): + raise UserAbort() + + +class CommandGroup: + name: str + help: str + panel: str = "General" + + def __init__(self, ui: UI, telemetry: Telemetry) -> None: + self.ui = ui + self.telemetry = telemetry + + @property + @abstractmethod + def commands(self) -> list[Command]: ... + + def typer(self) -> typer.Typer: + sub = typer.Typer(help=self.help, no_args_is_help=True) + for cmd in self.commands: + cmd.register(sub) + return sub + + def register(self, app: typer.Typer) -> None: + app.add_typer(self.typer(), name=self.name, rich_help_panel=self.panel) + + +class LegacyCommand(Command): + """Adapter for the pre-refactor function-style commands. Removed in plan 4.""" + + def __init__( + self, + ui: UI, + telemetry: Telemetry, + fn: Callable, + *, + name: str, + help: str, + panel: str, + no_args_is_help: bool = True, + ) -> None: + super().__init__(ui, telemetry) + self.name, self.help, self.panel, self.no_args_is_help = name, help, panel, no_args_is_help + self._fn = fn + + def register(self, app: typer.Typer) -> None: + app.command( + self.name, + help=self.help, + rich_help_panel=self.panel, + no_args_is_help=self.no_args_is_help, + )(self._traced(self._fn)) + + def run(self, *args, **kwargs) -> None: + return self._fn(*args, **kwargs) +``` + +Note : `_traced` utilise `functools.wraps`, donc Typer voit la signature de la fonction d'origine (`__wrapped__`) — c'est ce qui permet d'envelopper sans casser l'introspection des paramètres. + +- [ ] **Step 2: Vérifier l'introspection Typer à travers `_traced`** + +Run: `uv run python -c " +from typing import Annotated +import typer +from commands.base import Command +from ui import UI +from services.telemetry import NoopTelemetry +class Hello(Command): + name, help, panel = 'hello', 'Say hello', 'Test' + def run(self, name: Annotated[str, typer.Argument()], loud: Annotated[bool, typer.Option('--loud')] = False): + print('hello', name.upper() if loud else name) +app = typer.Typer(add_completion=False) +@app.callback() +def root(): pass +Hello(UI(), NoopTelemetry()).register(app) +app(['hello', 'bob', '--loud'], standalone_mode=False)"` +Expected: `hello BOB`. + +- [ ] **Step 3: Commit** + +```bash +git add commands/base.py +git commit -m "feat(commands): add Command, CommandGroup and LegacyCommand base classes" +``` + +--- + +### Task 10 : `services/updater.py` + +**Files:** +- Create: `services/updater.py` + +**Interfaces:** +- Consumes: `HttpClient`, `GlobalConfig`, `core.version`. +- Produces: `Release(tag, assets: dict[str, str], prerelease: bool)` ; `UpdateChecker(http, config, current: str)` : `include_prerelease -> bool`, `latest(force=False) -> Release | None` (cache 24 h, `None` si réseau KO), `available() -> str | None` (tag plus récent ou `None`) ; `Updater(http, current: str)` : `asset_name() -> str`, `target_path() -> Path`, `download(release, on_progress) -> Path` (vérifie sha256 via `checksums.txt`), `install(tmp: Path, target: Path) -> None`. + +- [ ] **Step 1: Écrire le module** + +```python +"""Update check (notify only) and manual update with checksum verification.""" + +from __future__ import annotations + +import hashlib +import json +import os +import platform +import shutil +import subprocess +import sys +import tempfile +import time +from collections.abc import Callable +from dataclasses import dataclass +from pathlib import Path + +from core.config import GlobalConfig +from core.errors import NetworkError, UpdateError +from core.version import UNKNOWN, is_prerelease, parse_version +from services.http import HttpClient + +GITHUB_REPO = "Portabase/cli" +RELEASES_URL = f"https://api.github.com/repos/{GITHUB_REPO}/releases" +CACHE_TTL = 24 * 3600 + + +@dataclass(frozen=True) +class Release: + tag: str + assets: dict[str, str] # name -> browser_download_url + prerelease: bool + + @classmethod + def from_api(cls, data: dict) -> Release: + return cls( + tag=str(data.get("tag_name", "")).lstrip("v"), + assets={a["name"]: a["browser_download_url"] for a in data.get("assets", [])}, + prerelease=bool(data.get("prerelease", False)), + ) + + +def platform_asset_name() -> str: + system = platform.system().lower() + system = "macos" if system == "darwin" else system + machine = platform.machine().lower() + arch = "arm64" if machine in ("arm64", "aarch64") else "amd64" + name = f"portabase_{system}_{arch}" + return name + ".exe" if system == "windows" else name + + +def is_frozen() -> bool: + return bool(getattr(sys, "frozen", False)) + + +class UpdateChecker: + def __init__(self, http: HttpClient, config: GlobalConfig, current: str) -> None: + self.http = http + self.config = config + self.current = current + self.cache_file = config.cache_dir / "release.json" + + @property + def include_prerelease(self) -> bool: + channel = self.config.update_channel + if channel: + return channel == "beta" + return is_prerelease(self.current) + + def fetch_latest(self) -> Release | None: + """Network call. Returns None when nothing is published.""" + if self.include_prerelease: + releases = self.http.get_json(RELEASES_URL) + return Release.from_api(releases[0]) if releases else None + return Release.from_api(self.http.get_json(f"{RELEASES_URL}/latest")) + + def latest(self, *, force: bool = False) -> Release | None: + """Cached 24h. Returns None on any network failure — never raises.""" + if not force: + cached = self._read_cache() + if cached is not None: + return cached + try: + release = self.fetch_latest() + except NetworkError: + return None + if release is not None: + self._write_cache(release) + return release + + def available(self, *, force: bool = False) -> str | None: + if self.current == UNKNOWN: + return None + release = self.latest(force=force) + if release is None: + return None + if parse_version(release.tag) > parse_version(self.current): + return release.tag + return None + + def _read_cache(self) -> Release | None: + try: + with open(self.cache_file, encoding="utf-8") as f: + data = json.load(f) + if time.time() - float(data.get("checked_at", 0)) > CACHE_TTL: + return None + if data.get("channel_pre") != self.include_prerelease: + return None + return Release(tag=data["tag"], assets=data.get("assets", {}), prerelease=bool(data.get("prerelease"))) + except (OSError, ValueError, KeyError): + return None + + def _write_cache(self, release: Release) -> None: + try: + self.cache_file.parent.mkdir(parents=True, exist_ok=True) + with open(self.cache_file, "w", encoding="utf-8") as f: + json.dump( + { + "checked_at": time.time(), + "channel_pre": self.include_prerelease, + "tag": release.tag, + "assets": release.assets, + "prerelease": release.prerelease, + }, + f, + ) + except OSError: + pass + + +class Updater: + CHECKSUMS_ASSET = "checksums.txt" + + def __init__(self, http: HttpClient, current: str) -> None: + self.http = http + self.current = current + + def target_path(self) -> Path: + if is_frozen(): + return Path(sys.executable).resolve() + if platform.system().lower() == "windows": + return Path(os.environ.get("APPDATA", "")) / "Portabase" / "portabase.exe" + default = Path("/usr/local/bin/portabase") + return default if default.exists() else Path.home() / ".local" / "bin" / "portabase" + + def download(self, release: Release, on_progress: Callable[[int], None] | None = None) -> Path: + name = platform_asset_name() + url = release.assets.get(name) + if url is None: + raise UpdateError( + f"No binary for this platform ({name}) in release {release.tag}.", + hint="Available: " + ", ".join(sorted(release.assets)) if release.assets else None, + ) + fd, tmp = tempfile.mkstemp(prefix="portabase_update_") + os.close(fd) + tmp_path = Path(tmp) + try: + self.http.download(url, tmp_path, on_progress, timeout=60) + self._verify(release, name, tmp_path) + except Exception: + tmp_path.unlink(missing_ok=True) + raise + return tmp_path + + def expected_size(self, release: Release) -> int | None: + url = release.assets.get(platform_asset_name()) + return self.http.content_length(url) if url else None + + def _verify(self, release: Release, name: str, path: Path) -> None: + url = release.assets.get(self.CHECKSUMS_ASSET) + if url is None: + raise UpdateError(f"Release {release.tag} has no {self.CHECKSUMS_ASSET}; refusing to install.") + expected = None + for line in self.http.get_text(url).splitlines(): + parts = line.split() + if len(parts) == 2 and parts[1].lstrip("*") == name: + expected = parts[0].lower() + if expected is None: + raise UpdateError(f"{name} not listed in {self.CHECKSUMS_ASSET}; refusing to install.") + digest = hashlib.sha256() + with open(path, "rb") as f: + for chunk in iter(lambda: f.read(1 << 20), b""): + digest.update(chunk) + if digest.hexdigest() != expected: + raise UpdateError("Checksum mismatch for downloaded binary; refusing to install.") + + def install(self, tmp: Path, target: Path) -> None: + system = platform.system().lower() + if system != "windows": + tmp.chmod(0o755) + target.parent.mkdir(parents=True, exist_ok=True) + backup = target.with_name(target.name + ".old") + writable = os.access(target.parent, os.W_OK) and (not target.exists() or os.access(target, os.W_OK)) + try: + if writable or system == "windows": + if target.exists(): + backup.unlink(missing_ok=True) + target.rename(backup) + shutil.move(str(tmp), str(target)) + else: + if target.exists(): + subprocess.run(["sudo", "mv", str(target), str(backup)], check=True) + subprocess.run(["sudo", "mv", str(tmp), str(target)], check=True) + subprocess.run(["sudo", "chmod", "+x", str(target)], check=True) + except (OSError, subprocess.CalledProcessError) as e: + raise UpdateError(f"Could not install to {target}: {e}", cause=e) from e +``` + +- [ ] **Step 2: Vérifier le checker (réseau requis)** + +Run: `uv run python -c " +from pathlib import Path; import tempfile +from services.http import HttpClient +from services.updater import UpdateChecker, platform_asset_name +from core.config import GlobalConfig +cfg = GlobalConfig(Path(tempfile.mkdtemp())/'config.json') +c = UpdateChecker(HttpClient(), cfg, '0.0.1') +r = c.latest(force=True); print('latest:', r.tag, 'pre:', r.prerelease, 'assets:', len(r.assets)) +print('cached:', c.latest().tag == r.tag, '| available from 0.0.1:', c.available()) +print('asset for this machine:', platform_asset_name(), platform_asset_name() in r.assets)"` +Expected: tag de la dernière release stable (ex. `26.07.6`), `cached: True`, `available from 0.0.1: `, asset présent `True` sur linux/macos. + +- [ ] **Step 3: Commit** + +```bash +git add services/updater.py +git commit -m "feat(services): add UpdateChecker (notify, cached) and Updater with checksum verification" +``` + +--- + +### Task 11 : `commands/lifecycle.py`, `commands/config.py`, `commands/update.py` + +**Files:** +- Create: `commands/lifecycle.py` +- Modify: `commands/config.py` (réécriture complète) +- Create: `commands/update.py` +- Delete: `commands/common.py` (Task 12, après bascule de `main.py`) + +**Interfaces:** +- Consumes: `Command`, `CommandGroup`, `DockerRunner`, `UpdateChecker`, `Updater`, `GlobalConfig`. +- Produces: classes `StartCommand`, `StopCommand`, `RestartCommand`, `LogsCommand`, `UninstallCommand` (constructeur `(ui, telemetry, docker)`) ; `ConfigCommands(ui, telemetry, config)` groupe `config` avec `show`, `get`, `set`, `channel` ; `UpdateCommand(ui, telemetry, checker, updater)`. + +- [ ] **Step 1: `commands/lifecycle.py`** + +```python +"""start / stop / restart / logs / uninstall. No rendering: work on any folder with a compose file.""" + +from __future__ import annotations + +import shutil +from pathlib import Path +from typing import Annotated + +import typer + +from commands.base import Command +from services.docker import DockerRunner +from services.telemetry import Telemetry +from ui import UI + +PathArg = Annotated[Path, typer.Argument(help="Path to the component folder")] + + +class _ComposeCommand(Command): + panel = "Lifecycle" + no_args_is_help = True + verb: str + compose_args: list[str] + done: str + + def __init__(self, ui: UI, telemetry: Telemetry, docker: DockerRunner) -> None: + super().__init__(ui, telemetry) + self.docker = docker + + def run(self, path: PathArg) -> None: + path = self.require_project_dir(path) + self.require_docker(self.docker) + with self.ui.status(f"{self.verb} {path.name}..."): + self.docker.compose(path, self.compose_args) + self.ui.success(self.done) + + +class StartCommand(_ComposeCommand): + name, help = "start", "Start a Portabase component." + verb, compose_args, done = "Starting", ["up", "-d"], "Started" + + +class StopCommand(_ComposeCommand): + name, help = "stop", "Stop a Portabase component." + verb, compose_args, done = "Stopping", ["stop"], "Stopped" + + +class RestartCommand(_ComposeCommand): + name, help = "restart", "Restart a Portabase component." + verb, compose_args, done = "Restarting", ["restart"], "Restarted" + + +class LogsCommand(Command): + name, help, panel = "logs", "View logs of a Portabase component.", "Lifecycle" + no_args_is_help = True + + def __init__(self, ui: UI, telemetry: Telemetry, docker: DockerRunner) -> None: + super().__init__(ui, telemetry) + self.docker = docker + + def run( + self, + path: PathArg, + follow: Annotated[bool, typer.Option("--follow/--no-follow", "-f", help="Follow log output")] = True, + ) -> None: + path = self.require_project_dir(path) + self.require_docker(self.docker) + args = ["logs", "-f"] if follow else ["logs"] + try: + self.docker.compose(path, args, check=False) + except KeyboardInterrupt: + pass + + +class UninstallCommand(Command): + name, help, panel = "uninstall", "Uninstall and delete a Portabase component.", "Lifecycle" + no_args_is_help = True + + def __init__(self, ui: UI, telemetry: Telemetry, docker: DockerRunner) -> None: + super().__init__(ui, telemetry) + self.docker = docker + + def run( + self, + path: PathArg, + force: Annotated[bool, typer.Option("--force", "-f", help="Skip confirmation")] = False, + ) -> None: + path = self.require_project_dir(path) + self.require_docker(self.docker) + if not force: + self.ui.warning(f"This will delete containers, volumes and all data in {path}.") + self.confirm_or_abort("Are you sure?", default=False) + with self.ui.status("Uninstalling..."): + self.docker.compose(path, ["down", "-v"]) + try: + shutil.rmtree(path) + except OSError as e: + self.ui.warning(f"Could not remove directory: {e}") + self.ui.success("Uninstalled") +``` + +- [ ] **Step 2: `commands/config.py` (réécriture)** + +```python +"""Global configuration (~/.portabase/config.json).""" + +from __future__ import annotations + +from typing import Annotated + +import typer + +from commands.base import Command, CommandGroup +from core.config import GlobalConfig +from core.errors import ValidationError +from services.telemetry import Telemetry +from ui import UI + +CHANNELS = ("stable", "beta") +BOOL_KEYS = ("telemetry",) + + +class _ConfigCommand(Command): + panel = "Configuration" + + def __init__(self, ui: UI, telemetry: Telemetry, config: GlobalConfig) -> None: + super().__init__(ui, telemetry) + self.config = config + + +class ConfigShow(_ConfigCommand): + name, help = "show", "Show the current configuration." + + def run(self) -> None: + data = self.config.all() + self.ui.info(f"Configuration file: {self.config.path}") + for key in GlobalConfig.KNOWN_KEYS: + value = data.get(key, "[hint]unset[/hint]") + self.ui.print(f" [key]{key}[/key]: {value}") + for key in sorted(set(data) - set(GlobalConfig.KNOWN_KEYS)): + self.ui.print(f" [key]{key}[/key]: {data[key]} [hint](unknown key)[/hint]") + + +class ConfigGet(_ConfigCommand): + name, help = "get", "Print one configuration value." + no_args_is_help = True + + def run(self, key: Annotated[str, typer.Argument(help="Configuration key")]) -> None: + value = self.config.get(key) + if value is None: + raise ValidationError(f"'{key}' is not set.", hint="Known keys: " + ", ".join(GlobalConfig.KNOWN_KEYS)) + self.ui.print(str(value)) + + +class ConfigSet(_ConfigCommand): + name, help = "set", "Set a configuration value." + no_args_is_help = True + + def run( + self, + key: Annotated[str, typer.Argument(help="Configuration key")], + value: Annotated[str, typer.Argument(help="Value")], + ) -> None: + if key == "update_channel" and value not in CHANNELS: + raise ValidationError(f"Invalid channel '{value}'.", hint="Choose 'stable' or 'beta'.") + stored: object = value + if key in BOOL_KEYS: + lowered = value.lower() + if lowered not in ("true", "false", "1", "0", "yes", "no"): + raise ValidationError(f"'{key}' expects true or false.") + stored = lowered in ("true", "1", "yes") + self.config.set(key, stored) + self.ui.success(f"{key} = {stored}") + + +class ConfigChannel(_ConfigCommand): + """Kept for compatibility with the previous `config channel ` command.""" + + name, help = "channel", "Set the update channel (stable or beta)." + no_args_is_help = True + + def run(self, name: Annotated[str, typer.Argument(help="stable or beta")]) -> None: + ConfigSet(self.ui, self.telemetry, self.config).run("update_channel", name.lower()) + + +class ConfigCommands(CommandGroup): + name, help, panel = "config", "Manage global CLI configuration.", "Configuration" + + def __init__(self, ui: UI, telemetry: Telemetry, config: GlobalConfig) -> None: + super().__init__(ui, telemetry) + self.config = config + + @property + def commands(self) -> list[Command]: + deps = (self.ui, self.telemetry, self.config) + return [ConfigShow(*deps), ConfigGet(*deps), ConfigSet(*deps), ConfigChannel(*deps)] +``` + +- [ ] **Step 3: `commands/update.py`** + +```python +"""Manual update. Auto-update is gone; main.py only prints a notice after commands.""" + +from __future__ import annotations + +from commands.base import Command +from core.errors import UpdateError +from core.version import UNKNOWN, parse_version +from services.telemetry import Telemetry +from services.updater import Release, UpdateChecker, Updater, is_frozen +from ui import UI + + +class UpdateCommand(Command): + name, help, panel = "update", "Update the CLI to the latest version.", "System" + + def __init__(self, ui: UI, telemetry: Telemetry, checker: UpdateChecker, updater: Updater) -> None: + super().__init__(ui, telemetry) + self.checker = checker + self.updater = updater + + def run(self) -> None: + if not is_frozen(): + self.ui.warning("The update command is only available for the binary version of Portabase CLI.") + self.ui.info("If you installed from source, use [bold]git pull[/bold] to update.") + return + + current = self.checker.current + release = self._latest() + if release.tag == current: + self.ui.success(f"Portabase CLI is already up to date ({current}).") + return + if current != UNKNOWN and parse_version(release.tag) < parse_version(current): + self.ui.warning(f"Current version ({current}) is newer than the latest remote version ({release.tag}).") + self.confirm_or_abort("Continue with the downgrade?", default=False) + + target = self.updater.target_path() + self.ui.info(f"Updating Portabase CLI from {current} to {release.tag}") + self.ui.info(f"Target installation path: {target}") + + total = self.updater.expected_size(release) or 0 + with self.ui.progress().download(f"Downloading {release.tag}...", total) as advance: + tmp = self.updater.download(release, advance) + self.updater.install(tmp, target) + self.ui.success(f"Successfully updated to {release.tag}!") + + def _latest(self) -> Release: + try: + release = self.checker.fetch_latest() + except Exception as e: # NetworkError + raise UpdateError("Could not fetch latest release data from GitHub.", cause=e) from e + if release is None: + raise UpdateError("No release found for this channel.") + return release +``` + +- [ ] **Step 4: Vérifier le lint des nouveaux fichiers** + +Run: `uv run ruff check commands/lifecycle.py commands/config.py commands/update.py commands/base.py services ui core` +Expected: `All checks passed!`. (`except Exception` dans `_latest` : remplacer par `except NetworkError` en important `NetworkError` depuis `core.errors` si ruff `BLE001` se plaint — c'est de toute façon plus précis.) + +- [ ] **Step 5: Commit** + +```bash +git add commands/lifecycle.py commands/config.py commands/update.py +git commit -m "feat(commands): rewrite lifecycle, config and update commands as classes" +``` + +--- + +### Task 12 : `main.py` — câblage, catcher, bascule + +**Files:** +- Modify: `main.py` (réécriture complète) +- Delete: `commands/common.py`, `core/updater.py` +- Modify: `pyproject.toml` (`TID251`, per-file-ignores) + +**Interfaces:** +- Consumes: tout ce qui précède + fonctions legacy `commands.agent.agent`, `commands.dashboard.dashboard`, `commands.db.app`. +- Produces: `Settings`, `build_app(ui, telemetry, config, settings) -> tuple[typer.Typer, UpdateChecker]`, `main() -> None`. + +- [ ] **Step 1: Réécrire `main.py`** + +```python +"""Entry point. Builds dependencies, registers commands, owns the single error boundary.""" + +from __future__ import annotations + +import os +import platform +import sys +from dataclasses import dataclass +from typing import Annotated + +import click +import typer + +from commands import agent as legacy_agent +from commands import dashboard as legacy_dashboard +from commands import db as legacy_db +from commands import decrypt as legacy_decrypt +from commands.base import LegacyCommand +from commands.config import ConfigCommands +from commands.lifecycle import LogsCommand, RestartCommand, StartCommand, StopCommand, UninstallCommand +from commands.update import UpdateCommand +from core.config import GlobalConfig +from core.errors import PortabaseError, UserAbort, ValidationError +from core.version import current_version +from services.docker import DockerRunner +from services.http import HttpClient +from services.telemetry import ConsoleTelemetry, TelemetryFactory, TelemetryHub +from services.updater import UpdateChecker, Updater, is_frozen +from ui import UI + + +@dataclass +class Settings: + non_interactive: bool = False + verbose: bool = False + debug: bool = False + no_color: bool = False + + @classmethod + def from_env(cls) -> Settings: + return cls( + non_interactive=os.environ.get("PORTABASE_NON_INTERACTIVE", "").lower() in ("1", "true", "yes") + or not sys.stdin.isatty(), + no_color=bool(os.environ.get("NO_COLOR")), + ) + + +def build_app( + ui: UI, telemetry: TelemetryHub, config: GlobalConfig, settings: Settings +) -> tuple[typer.Typer, UpdateChecker]: + app = typer.Typer(no_args_is_help=True, add_completion=False, rich_markup_mode="rich") + http = HttpClient() + docker = DockerRunner() + version = current_version() + checker = UpdateChecker(http, config, version) + updater = Updater(http, version) + + def version_callback(value: bool) -> None: + if value: + ui.print(f"Portabase CLI version: {version}") + latest = checker.available(force=True) + if latest: + ui.warning(f"A new version is available: [bold]{latest}[/bold]") + raise typer.Exit() + + @app.callback() + def root( + _version: Annotated[ + bool | None, + typer.Option("--version", help="Show the version and exit.", callback=version_callback, is_eager=True), + ] = None, + verbose: Annotated[bool, typer.Option("--verbose", help="Show error causes and tracebacks.")] = False, + debug: Annotated[bool, typer.Option("--debug", help="Verbose plus telemetry trace on stderr.")] = False, + no_color: Annotated[bool, typer.Option("--no-color", help="Disable colours.")] = False, + non_interactive: Annotated[ + bool, + typer.Option("--non-interactive", envvar="PORTABASE_NON_INTERACTIVE", help="Never prompt; fail on missing input."), + ] = False, + ) -> None: + """Portabase CLI to manage agents, dashboards and databases.""" + settings.verbose = verbose or debug + settings.debug = debug + settings.no_color = settings.no_color or no_color + settings.non_interactive = settings.non_interactive or non_interactive + ui.configure(verbose=settings.verbose, no_color=settings.no_color, non_interactive=settings.non_interactive) + if debug: + telemetry.set(ConsoleTelemetry()) + + commands = [ + LegacyCommand(ui, telemetry, legacy_agent.agent, name="agent", help="Create a new Portabase Agent instance.", panel="Creation"), + LegacyCommand(ui, telemetry, legacy_dashboard.dashboard, name="dashboard", help="Create a new Portabase Dashboard instance.", panel="Creation"), + LegacyCommand(ui, telemetry, legacy_decrypt.decrypt, name="decrypt", help="Decrypt Portabase .enc backup files (single file or folder).", panel="Configuration"), + StartCommand(ui, telemetry, docker), + StopCommand(ui, telemetry, docker), + RestartCommand(ui, telemetry, docker), + LogsCommand(ui, telemetry, docker), + UninstallCommand(ui, telemetry, docker), + UpdateCommand(ui, telemetry, checker, updater), + ] + for cmd in commands: + cmd.register(app) + + app.add_typer(legacy_db.app, name="db", rich_help_panel="Configuration") # legacy, replaced in plan 4 + ConfigCommands(ui, telemetry, config).register(app) + + return app, checker + + +def _notify_update(ui: UI, checker: UpdateChecker, settings: Settings, invoked: str | None) -> None: + if not is_frozen() or settings.non_interactive or invoked in ("update", None): + return + latest = checker.available() + if latest: + ui.print("") + ui.warning(f"A new version of Portabase CLI is available: [bold]{latest}[/bold] (current: {checker.current})") + ui.info("Run [bold]portabase update[/bold] to update.") + + +def main() -> None: + settings = Settings.from_env() + config = GlobalConfig() + ui = UI(non_interactive=settings.non_interactive, no_color=settings.no_color) + telemetry = TelemetryFactory.build(config, debug=False) + app, checker = build_app(ui, telemetry, config, settings) + invoked = next((a for a in sys.argv[1:] if not a.startswith("-")), None) + exit_code = 0 + + try: + with telemetry.session(cli_version=current_version(), os=platform.system()): + app(standalone_mode=False) + except UserAbort as e: + ui.warning(e.message) + telemetry.event("abort") + exit_code = e.exit_code + except PortabaseError as e: + ui.error(e) + telemetry.error(e) + exit_code = e.exit_code + except click.exceptions.NoArgsIsHelpError: + exit_code = 0 # help already printed by Typer + except click.exceptions.Exit as e: # typer.Exit from legacy code or --help + exit_code = e.exit_code + except click.exceptions.Abort: # typer.Abort from legacy code + ui.warning("Cancelled.") + exit_code = 130 + except click.UsageError as e: + err = ValidationError(e.format_message(), hint="Run 'portabase --help' for usage.") + ui.error(err) + telemetry.error(err) + exit_code = err.exit_code + except KeyboardInterrupt: + ui.console.print("") + ui.warning("Cancelled.") + exit_code = 130 + except Exception as e: # noqa: BLE001 — last resort: a bug, not an expected error + wrapped = PortabaseError("Unexpected error: " + str(e), cause=e) + ui.error(wrapped, unexpected=True) + telemetry.error(e, unexpected=True) + exit_code = 1 + finally: + telemetry.flush() + + if exit_code == 0: + _notify_update(ui, checker, settings, invoked) + raise SystemExit(exit_code) + + +if __name__ == "__main__": + main() +``` + +- [ ] **Step 2: Supprimer les modules remplacés** + +Run: `git rm commands/common.py core/updater.py` + +Puis vérifier qu'aucun import ne subsiste : +Run: `grep -rn "commands.common\|core.updater\|check_for_updates\|update_cli" --include=*.py . | grep -v ".venv"` +Expected: aucune sortie. + +- [ ] **Step 3: Mettre à jour `pyproject.toml`** + +Remplacer le bloc `[tool.ruff.lint.per-file-ignores]` par : + +```toml +# Code legacy supprimé au plan 4. Ne pas étendre cette liste. +[tool.ruff.lint.per-file-ignores] +"commands/agent.py" = ["BLE001", "E722", "S110", "SIM102", "TID251"] +"commands/db.py" = ["BLE001", "E722", "S110", "TID251"] +"commands/dashboard.py" = ["BLE001", "TID251"] +"commands/decrypt.py" = ["B904", "TID251"] +"core/crypto.py" = ["BLE001", "SIM105"] +"core/config.py" = ["BLE001", "E722", "S110"] +"core/utils.py" = ["BLE001", "E722", "S110", "PLR1730", "TID251"] +"core/network.py" = ["BLE001", "TID251"] +"main.py" = ["TID251"] + +[tool.ruff.lint.flake8-tidy-imports.banned-api] +"rich.prompt".msg = "Use ui.form() / ui.confirm() instead." +"rich.console".msg = "Only ui/ may build a Console. Use the UI facade." +"typer.prompt".msg = "Use ui.form() instead." +"typer.confirm".msg = "Use ui.confirm() instead." +``` + +Et ajouter `"TID251"` dans `select` s'il n'y est pas déjà (il y est depuis Plan 1). `main.py` importe `click` et `typer.Exit`, pas de prompt : `TID251` sur `main.py` est là uniquement pour `ui.console.print` ? Non — `rich.console` n'y est pas importé. Retirer `"main.py" = ["TID251"]` si `ruff check` passe sans. + +Ajouter `known-first-party = ["commands", "core", "services", "ui", "templates"]` dans `[tool.ruff.lint.isort]`. + +- [ ] **Step 4: Lint complet** + +Run: `uv run ruff check . && uv run ruff format --check .` +Expected: passe. Sinon `uv run ruff format .` puis corriger les erreurs signalées **dans les nouveaux fichiers uniquement**. + +- [ ] **Step 5: Vérifier l'aide et les erreurs de saisie** + +Run: `uv run python main.py; echo "exit=$?"` +Expected: aide affichée, `exit=0`. + +Run: `uv run python main.py --help | head -30` +Expected: panneaux `Creation` (agent, dashboard), `Lifecycle` (start, stop, restart, logs, uninstall), `Configuration` (decrypt, db, config), `System` (update) ; options `--version`, `--verbose`, `--debug`, `--no-color`, `--non-interactive`. + +Run: `uv run python main.py start; echo "exit=$?"` +Expected: aide de `start` (no_args_is_help), `exit=0`. + +Run: `uv run python main.py bogus; echo "exit=$?"` +Expected: `✖ Error: No such command 'bogus'.` + hint, `exit=2`. + +Run: `uv run python main.py start /tmp/does-not-exist; echo "exit=$?"` +Expected: `✖ Error: No Portabase configuration found in: /tmp/does-not-exist` + hint, `exit=3`. + +Run: `uv run python main.py --verbose start /tmp/does-not-exist 2>&1 | grep -c "code: E_CONFIG"` +Expected: `1`. + +- [ ] **Step 6: Vérifier config** + +Run: `uv run python main.py config show && uv run python main.py config set update_channel beta && uv run python main.py config get update_channel && uv run python main.py config channel stable && uv run python main.py config set update_channel nope; echo "exit=$?"` +Expected: affichage, `✔ update_channel = beta`, `beta`, `✔ update_channel = stable`, puis `✖ Error: Invalid channel 'nope'.` `exit=2`. + +- [ ] **Step 7: Vérifier update et --version (non-frozen)** + +Run: `uv run python main.py update; echo "exit=$?"; uv run python main.py --version; echo "exit=$?"` +Expected: avertissement "only available for the binary version", `exit=0` ; version puis éventuellement "A new version is available", `exit=0`. + +- [ ] **Step 8: Vérifier le mode non-interactif et Ctrl-C** + +Run: `uv run python main.py --non-interactive uninstall /tmp/does-not-exist; echo "exit=$?"` +Expected: `E_CONFIG`, `exit=3` (l'erreur dossier précède la confirmation). + +Créer un faux projet : `mkdir -p /tmp/pb-fake && touch /tmp/pb-fake/docker-compose.yml`. +Run: `uv run python main.py --non-interactive uninstall /tmp/pb-fake; echo "exit=$?"` +Expected (Docker présent) : confirm par défaut `False` → `⚠ Cancelled.` `exit=130`, dossier intact. (Docker absent : `E_DOCKER`, `exit=4`.) + +Run: `uv run python main.py uninstall /tmp/pb-fake` puis Ctrl-C au prompt. +Expected: `⚠ Cancelled.`, `exit=130`, pas de traceback. + +- [ ] **Step 9: Vérifier les commandes legacy à travers le catcher** + +Run: `uv run python main.py agent; echo "exit=$?"` puis `uv run python main.py db list /tmp/does-not-exist; echo "exit=$?"` +Expected: aide de `agent` `exit=0` ; message legacy `No Portabase configuration found` (ancien style) et `exit=1` (via `typer.Exit(1)` → `click.exceptions.Exit`). + +- [ ] **Step 10: Vérifier le lifecycle réel (si Docker disponible)** + +```bash +cd /tmp && rm -rf pb-smoke && uv --directory /home/soluce/Documents/PROJETS/Portabase/cli run python /home/soluce/Documents/PROJETS/Portabase/cli/main.py dashboard pb-smoke --port 8899 +``` +Répondre `internal` au choix DB, `N` à "Start dashboard now?". Puis : + +```bash +M=/home/soluce/Documents/PROJETS/Portabase/cli/main.py +uv --directory /home/soluce/Documents/PROJETS/Portabase/cli run python $M start /tmp/pb-smoke +uv --directory /home/soluce/Documents/PROJETS/Portabase/cli run python $M logs /tmp/pb-smoke --no-follow | tail -3 +uv --directory /home/soluce/Documents/PROJETS/Portabase/cli run python $M restart /tmp/pb-smoke +uv --directory /home/soluce/Documents/PROJETS/Portabase/cli run python $M stop /tmp/pb-smoke +uv --directory /home/soluce/Documents/PROJETS/Portabase/cli run python $M uninstall /tmp/pb-smoke --force +ls /tmp/pb-smoke 2>&1 +``` +Expected: `✔ Started`, quelques lignes de logs, `✔ Restarted`, `✔ Stopped`, `✔ Uninstalled`, `No such file or directory`. + +- [ ] **Step 11: Commit** + +```bash +git add main.py pyproject.toml +git commit -m "refactor: wire commands through DI container and single error boundary + +Lifecycle, config and update run on the new Command classes; agent, +dashboard and db stay on legacy code behind LegacyCommand until plan 4. +Auto-update is replaced by a post-command notice." +``` + +--- + +### Task 13 : Build smoke, PR + +**Files:** aucun nouveau. + +- [ ] **Step 1: Binaire local** + +Run: `rm -rf build dist *.spec && uv run pyinstaller --onefile --name portabase_smoke --paths=. --collect-all rich --collect-all requests --collect-data certifi --add-data "pyproject.toml:." main.py && ./dist/portabase_smoke --version && ./dist/portabase_smoke config show && ./dist/portabase_smoke start /tmp/nope; echo "exit=$?"; rm -rf build dist *.spec` +Expected: version, config, `E_CONFIG` `exit=3`. Aucun `ModuleNotFoundError` (questionary, services, ui embarqués via `--paths=.`). + +- [ ] **Step 2: PR** + +```bash +git checkout -b refactor/foundations +git push -u origin refactor/foundations +``` +Ouvrir la PR « refactor: foundations (errors, ui, services, Command) + lifecycle rewrite ». Checks Plan 1 verts attendus. + +- [ ] **Step 3: Release candidate (optionnel mais recommandé)** + +Après merge : Actions → Bump version → `26.08.0rc1`, channel `rc`. Installer le binaire rc sur une machine avec une install existante et dérouler `start/stop/logs/restart` + `--version` (la notification de mise à jour après commande s'affiche seulement en binaire). + +--- + +## Self-review + +**Spec coverage :** +- §3 structure : `core/errors`, `core/version`, `core/config` (GlobalConfig), `ui/*`, `services/{http,docker,telemetry,updater}`, `commands/{base,lifecycle,config,update}`, `main.py` ✔. `services/{envfile,ports,templates,renderer,project,compose_facts}`, `engines/`, `commands/{agent,dashboard,build,db,flows}` → Plans 3–4. `core/fields.py` : déviation documentée. +- §4.1 `Command`, `register`, `_traced`, injection ✔ (T9). `Annotated` ✔. +- §7 ui : tokens ✔, composants Banner/Message/Section/Status/Hint/Prompt ✔ + Progress (appelant : update). `Summary`, `DataTable`, `Diff` → Plan 4 (appelants). `Form` ✔ avec flag→prompt→défaut→erreur, `UserAbort` sur `None` ✔. `NO_COLOR` ✔. Pas de prompt sous status : respecté dans lifecycle (confirm avant status). +- §8.1 hiérarchie et codes ✔. §8.2 catcher, `standalone_mode=False`, mapping click ✔ (T12). §8.3 télémétrie contrat + noop + console + hub ✔ ; opt-in config lu par `TelemetryFactory` (endpoint ignoré tant qu'aucun exporter — documenté). §8.4 updater : notif après commande, cache 24 h, silencieux offline, checksum ✔. +- §10 B+C : shippable, legacy via `LegacyCommand` ✔. + +**Placeholders :** aucun. + +**Cohérence des types :** `UI.confirm(question, *, default, value)` utilisé par `Command.confirm_or_abort` et `require_docker` ✔ ; `Telemetry.span` context manager utilisé par `_traced` ✔ ; `UpdateChecker.available(force=)` utilisé par `version_callback` et `_notify_update` ✔ ; `Updater.expected_size/download/install` utilisés par `UpdateCommand` ✔ ; `HttpClient.content_length` utilisé par `Updater.expected_size` ✔ (`head_content_length` cité dans l'interface T7 = `content_length` ; nom retenu : `content_length`). + +**Écarts connus :** +- `UpdateCommand._latest` : utiliser `except NetworkError` (T11 step 4). diff --git a/docs/superpowers/plans/2026-09-11-plan-3-templates-engines.md b/docs/superpowers/plans/2026-09-11-plan-3-templates-engines.md new file mode 100644 index 0000000..3b5be5d --- /dev/null +++ b/docs/superpowers/plans/2026-09-11-plan-3-templates-engines.md @@ -0,0 +1,1944 @@ +# Plan 3 — Templates Jinja2 et moteurs DB (chantier D) + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Introduire les templates Jinja2 versionnés (source `templates/` à la racine, manifest, cache, `TemplateRepository`), le registre de moteurs DB en classes, et les garde-fous CI (`render-check`, `engines-check`, manifest à l'upload, hotfix) — sans encore brancher le rendu sur les commandes (Plan 4). Le CLI reste fonctionnel : les commandes legacy continuent de lire `agent.yml` / `dashboard.yml` (conservés dans `templates/` jusqu'au Plan 4). + +**Architecture:** `TemplateRepository` résout une version → dossier local (`./templates` en dev, cache `~/.portabase/cache/templates//` en binaire), vérifie un `manifest.json` (sha256) et expose des `jinja2.Template`. Chaque `DbEngine` déclare ses champs, génère un `DatabaseSpec`, produit ses variables `.env`, son contexte de template et sa projection `databases.json`. `render_check.py` rend chaque template avec des fixtures et valide le YAML puis `docker compose config`. + +**Tech Stack:** Jinja2 3.1, PyYAML, Python 3.12, GitHub Actions, s3cmd, jq. + +**Spec:** `docs/superpowers/specs/2026-09-11-cli-refactor-design.md` — sections 5.4, 5.6, 6, 6.1, 9.1 (`render-check`, `engines-check`), 9.3 (hotfix, manifest), 10 (D). + +## Global Constraints + +- Prérequis : Plans 1 et 2 exécutés. +- Règle de dépendance : `engines → core` uniquement. `services → engines, core`. Vérifié par revue ; ruff ne le détecte pas. +- Déviations spec assumées : + - `DatabaseSpec` vit dans `core/specs.py` (produit par `engines`, consommé par `services`), pas dans `services/project.py`. + - Pas de `mysql.yml.j2` : le moteur `mysql` utilise `engines/mariadb.yml.j2`, comme le code legacy (image `mariadb:latest`). Changer d'image casserait les volumes des installs existantes. +- Les fichiers legacy `agent.yml` et `dashboard.yml` sont déplacés tels quels dans `templates/` et restent uploadés (le code legacy les fetch sous `/`). Supprimés au Plan 4. +- Conventions de nommage legacy conservées à l'identique (service `db-pg-`, `db-mongo-auth-`, db `pg_`, user `admin`, firebird user `alice` / `mirror.fdb`, mssql `sa` / `master` / name `MSSQL`, redis/valkey `database: "0"`), pour que les nouvelles installs ressemblent aux anciennes. +- `generate_password` retire `$` et `` ` `` des symboles (Task 1). +- Pas de tests unitaires. `render_check.py` est la vérification exécutable de ce plan et devient un job CI. +- Aucune commande utilisateur ne change dans ce plan. + +--- + +## File Structure + +| Fichier | Action | Responsabilité | +|---|---|---| +| `core/utils.py` | modifier | `generate_password` sans `$`/`` ` `` | +| `core/specs.py` | créer | `DatabaseSpec` | +| `services/ports.py` | créer | `PortAllocator` | +| `engines/__init__.py` | créer | `registry` | +| `engines/base.py` | créer | `DbEngine` | +| `engines/registry.py` | créer | `EngineRegistry` | +| `engines/sql.py` | créer | `StandardSqlEngine`, `PostgresEngine`, `PostgresClusterEngine`, `MySqlEngine`, `MariaDbEngine`, `MssqlEngine`, `FirebirdEngine` | +| `engines/redis.py` | créer | `RedisEngine` | +| `engines/valkey.py` | créer | `ValkeyEngine` | +| `engines/mongo.py` | créer | `MongoEngine` | +| `engines/sqlite.py` | créer | `SqliteEngine` | +| `engines/docker_volume.py` | créer | `DockerVolumeEngine` | +| `templates/agent.yml.j2`, `dashboard.yml.j2`, `engines/*.yml.j2` | créer | templates Jinja2 | +| `templates/engines.map.json` | créer | clé moteur → template | +| `templates/agent.yml`, `dashboard.yml` | déplacer depuis `.github/assets/templates/` | legacy | +| `services/templates.py` | créer | `Manifest`, `TemplateRepository` | +| `scripts/render_check.py` | créer | validation des templates | +| `.github/workflows/ci.yml` | modifier | jobs `render-check`, `engines-check` | +| `.github/workflows/templates-upload.yml` | modifier | source `templates/`, manifest | +| `.github/workflows/templates-hotfix.yml` | créer | re-upload d'une version | +| `pyproject.toml` | modifier | `jinja2` | +| `.gitleaks.toml` | modifier | chemin `templates/` déjà allowlisté ; retirer `.github/assets/templates` | + +--- + +### Task 1 : `core/specs.py`, `services/ports.py`, mot de passe + +**Files:** +- Create: `core/specs.py` +- Create: `services/ports.py` +- Modify: `core/utils.py:70-92` (`generate_password`) + +**Interfaces:** +- Produces: `DatabaseSpec` (frozen dataclass) avec `env_prefix`, `is_service`, `with_options()` ; `PortAllocator().free() -> int` ; `generate_password(length=16)` sans `$` ni `` ` ``. + +- [ ] **Step 1: `core/specs.py`** + +```python +"""Typed view of one databases.json entry plus what the CLI needs to render it.""" + +from __future__ import annotations + +from dataclasses import dataclass, field, replace +from typing import Any + + +@dataclass(frozen=True) +class DatabaseSpec: + id: str + engine: str + name: str + managed: bool = False # True: a Compose service is rendered for it + host: str | None = None # service name when managed, remote host otherwise + port: int | None = None # container/remote port (what the agent connects to) + host_port: int | None = None # published port on the Docker host (managed only) + database: str | None = None + username: str | None = None + password: str | None = None + root_password: str | None = None # firebird + path: str | None = None # sqlite + volume: str | None = None # docker-volume + container: str | None = None # docker-volume + options: dict[str, Any] = field(default_factory=dict) + + @property + def env_prefix(self) -> str: + if not self.host: + raise ValueError("env_prefix requires a host/service name") + return self.host.upper().replace("-", "_") + + @property + def auth(self) -> bool: + return bool(self.password) + + def with_options(self, options: dict[str, Any]) -> DatabaseSpec: + return replace(self, options=dict(options)) +``` + +- [ ] **Step 2: `services/ports.py`** + +```python +"""Free TCP port allocation. Remembers ports handed out during the process to avoid duplicates.""" + +from __future__ import annotations + +import socket + + +class PortAllocator: + def __init__(self) -> None: + self._given: set[int] = set() + + def free(self) -> int: + for _ in range(50): + with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s: + s.bind(("", 0)) + port = s.getsockname()[1] + if port not in self._given: + self._given.add(port) + return port + raise RuntimeError("Could not allocate a free port") + + +class FixedPortAllocator(PortAllocator): + """Deterministic ports for render checks and fixtures.""" + + def __init__(self, start: int = 40000) -> None: + super().__init__() + self._next = start + + def free(self) -> int: + port = self._next + self._next += 1 + return port +``` + +- [ ] **Step 3: Corriger `generate_password` dans `core/utils.py`** + +Remplacer la ligne `symbols = "!@#$%^&*()-_=+[]{}|;:,.<>?"` par : + +```python + # No '$' (Compose interpolation / shell), no '`' or quotes (shell command args in templates). + symbols = "!@#%^&*()-_=+[]{}|;:,.<>?" +``` + +- [ ] **Step 4: Vérifier** + +Run: `uv run python -c " +from core.specs import DatabaseSpec +from services.ports import PortAllocator, FixedPortAllocator +from core.utils import generate_password +s = DatabaseSpec(id='1', engine='postgresql', name='x', managed=True, host='db-pg-a1f2', password='p') +print(s.env_prefix, s.auth, s.with_options({'a':1}).options) +p = PortAllocator(); a, b = p.free(), p.free(); print(a != b, FixedPortAllocator().free()) +pw = generate_password(); print(len(pw), '\$' not in pw and '\`' not in pw)"` +Expected: `DB_PG_A1F2 True {'a': 1}`, `True 40000`, `16 True`. + +- [ ] **Step 5: Commit** + +```bash +git add core/specs.py services/ports.py core/utils.py +git commit -m "feat: add DatabaseSpec, PortAllocator; drop shell-unsafe symbols from generated passwords" +``` + +--- + +### Task 2 : `engines/base.py` et `engines/registry.py` + +**Files:** +- Create: `engines/__init__.py` (rempli Task 4) +- Create: `engines/base.py` +- Create: `engines/registry.py` + +**Interfaces:** +- Produces: `DbEngine` ABC : + - classe-attributs `key`, `display`, `default_port: int | None`, `template: str | None`, `auth_variants=False`, `warning=None`, `has_modes=True` + - `fields_existing() -> list[Field]`, `fields_new() -> list[Field]`, `option_fields() -> list[Field]` + - `generate(*, auth: bool, ports: PortAllocator, answers: dict) -> DatabaseSpec` + - `from_existing(answers: dict) -> DatabaseSpec` + - `env_vars(spec) -> dict[str, str]` + - `template_ctx(spec, *, inline: bool = False) -> dict` + - `agent_entry(spec) -> dict` + - `describe(spec) -> str` (pour `db list` : « host:port », « Local File », « volume: x ») + - helpers `new_id()`, `service_name(slug, auth)`, `var(spec, suffix, value, inline)` +- `EngineRegistry(engines)` : `get(key)`, `keys()`, `choices()`, `__iter__`. + +- [ ] **Step 1: `engines/base.py`** + +```python +"""DbEngine: everything the CLI needs to know about one database engine.""" + +from __future__ import annotations + +import secrets +import uuid +from abc import ABC, abstractmethod +from typing import Any + +from core.fields import Field +from core.specs import DatabaseSpec +from services.ports import PortAllocator + +STANDARD_EXISTING_FIELDS = ( + Field("host", "Host", "text", default="localhost"), + Field("port", "Port", "int"), # default filled per engine + Field("database", "Database Name", "text"), + Field("username", "Username", "text"), + Field("password", "Password", "secret"), +) + + +class DbEngine(ABC): + key: str + display: str + default_port: int | None = None + template: str | None = None # e.g. "engines/postgresql.yml.j2"; None: no Compose service + auth_variants: bool = False # offer with-auth / no-auth when creating a container + warning: str | None = None # shown before collecting answers + has_modes: bool = True # new/existing choice applies + + # ---- declarations ----------------------------------------------------- + + def fields_existing(self) -> list[Field]: + return [ + Field("port", "Port", "int", default=self.default_port) if f.name == "port" else f + for f in STANDARD_EXISTING_FIELDS + ] + + def fields_new(self) -> list[Field]: + return [] + + def option_fields(self) -> list[Field]: + return [] + + # ---- construction ----------------------------------------------------- + + @abstractmethod + def generate(self, *, auth: bool, ports: PortAllocator, answers: dict[str, Any]) -> DatabaseSpec: ... + + def from_existing(self, answers: dict[str, Any]) -> DatabaseSpec: + return DatabaseSpec( + id=self.new_id(), + engine=self.key, + name=answers.get("label") or "External DB", + managed=False, + host=answers["host"], + port=int(answers["port"]), + database=answers["database"], + username=answers["username"], + password=answers["password"], + ) + + # ---- rendering inputs ------------------------------------------------- + + def env_vars(self, spec: DatabaseSpec) -> dict[str, str]: + """Variables written to .env for a managed service. Default: PORT, DB, USER, PASS.""" + p = spec.env_prefix + return { + f"{p}_PORT": str(spec.host_port), + f"{p}_DB": spec.database or "", + f"{p}_USER": spec.username or "", + f"{p}_PASS": spec.password or "", + } + + def template_ctx(self, spec: DatabaseSpec, *, inline: bool = False) -> dict[str, Any]: + return { + "name": spec.host, + "volume": f"{spec.host}-data", + "auth": spec.auth, + "port_var": self.var(spec, "PORT", spec.host_port, inline), + "db_var": self.var(spec, "DB", spec.database, inline), + "user_var": self.var(spec, "USER", spec.username, inline), + "password_var": self.var(spec, "PASS", spec.password, inline), + } + + def agent_entry(self, spec: DatabaseSpec) -> dict[str, Any]: + """Projection to databases.json. Same shape as the legacy CLI.""" + entry: dict[str, Any] = { + "name": spec.name, + "database": self.agent_database(spec), + "type": self.key, + "username": spec.username or "", + "password": spec.password or "", + "port": spec.port, + "host": spec.host, + "generated_id": spec.id, + } + options = self.non_default_options(spec) + if options: + entry["options"] = options + return entry + + def agent_database(self, spec: DatabaseSpec) -> str: + return spec.database or "" + + def describe(self, spec: DatabaseSpec) -> str: + return f"{spec.host}:{spec.port}" + + # ---- helpers ---------------------------------------------------------- + + def non_default_options(self, spec: DatabaseSpec) -> dict[str, Any]: + defaults = {f.name: f.default for f in self.option_fields()} + return {k: v for k, v in spec.options.items() if k in defaults and v != defaults[k]} + + @staticmethod + def new_id() -> str: + return str(uuid.uuid4()) + + @staticmethod + def service_name(slug: str, auth: bool = False) -> str: + suffix = "auth-" if auth else "" + return f"db-{slug}-{suffix}{secrets.token_hex(2)}" + + @staticmethod + def var(spec: DatabaseSpec, suffix: str, value: Any, inline: bool) -> str: + return str(value if value is not None else "") if inline else f"${{{spec.env_prefix}_{suffix}}}" +``` + +- [ ] **Step 2: `engines/registry.py`** + +```python +from __future__ import annotations + +from collections.abc import Iterable, Iterator + +from core.errors import ValidationError +from engines.base import DbEngine + + +class EngineRegistry: + def __init__(self, engines: Iterable[DbEngine]) -> None: + self._by_key: dict[str, DbEngine] = {} + for engine in engines: + if engine.key in self._by_key: + raise ValueError(f"Duplicate engine key: {engine.key}") + self._by_key[engine.key] = engine + + def get(self, key: str) -> DbEngine: + try: + return self._by_key[key] + except KeyError: + raise ValidationError( + f"Unknown engine '{key}'.", + hint="Available: " + ", ".join(self.keys()), + ) from None + + def keys(self) -> list[str]: + return list(self._by_key) + + def choices(self) -> list[str]: + return self.keys() + + def __iter__(self) -> Iterator[DbEngine]: + return iter(self._by_key.values()) + + def __contains__(self, key: str) -> bool: + return key in self._by_key +``` + +- [ ] **Step 3: Vérifier** + +Run: `uv run python -c " +from engines.base import DbEngine +from engines.registry import EngineRegistry +from core.errors import ValidationError +print([f.name for f in DbEngine.fields_existing(type('E',(DbEngine,),{'key':'x','display':'X','default_port':1,'generate':lambda *a,**k: None})())]) +try: EngineRegistry([]).get('nope') +except ValidationError as e: print(e.message, '|', e.hint)"` +Expected: `['host', 'port', 'database', 'username', 'password']` puis `Unknown engine 'nope'. | Available: `. + +- [ ] **Step 4: Commit** + +```bash +git add engines/ +git commit -m "feat(engines): add DbEngine base class and EngineRegistry" +``` + +--- + +### Task 3 : Moteurs SQL (`engines/sql.py`) + +**Files:** +- Create: `engines/sql.py` + +**Interfaces:** +- Produces: `StandardSqlEngine` et sous-classes `PostgresEngine` (`postgresql`), `PostgresClusterEngine` (`postgresql-cluster`), `MySqlEngine` (`mysql`), `MariaDbEngine` (`mariadb`), `MssqlEngine` (`mssql`), `FirebirdEngine` (`firebird`). + +- [ ] **Step 1: Écrire le module** + +```python +"""SQL engines rendered as Compose services. Naming mirrors the legacy CLI.""" + +from __future__ import annotations + +import secrets +from typing import Any + +from core.fields import Field +from core.specs import DatabaseSpec +from core.utils import generate_password +from engines.base import DbEngine +from services.ports import PortAllocator + + +class StandardSqlEngine(DbEngine): + slug: str # service name fragment: db--xxxx + db_prefix: str # generated database name: _xxxxxxxx + default_user = "admin" + + def generate(self, *, auth: bool, ports: PortAllocator, answers: dict[str, Any]) -> DatabaseSpec: + db_name = f"{self.db_prefix}_{secrets.token_hex(4)}" + return DatabaseSpec( + id=self.new_id(), + engine=self.key, + name=db_name, + managed=True, + host=self.service_name(self.slug), + port=self.default_port, + host_port=ports.free(), + database=db_name, + username=self.default_user, + password=generate_password(16), + options=dict(answers.get("options", {})), + ) + + +class PostgresEngine(StandardSqlEngine): + key, display, default_port = "postgresql", "PostgreSQL", 5432 + template, slug, db_prefix = "engines/postgresql.yml.j2", "pg", "pg" + + def option_fields(self) -> list[Field]: + return [ + Field( + "keep_ownership", + "Keep ownership?", + "bool", + default=False, + help=( + "When enabled, omits --no-owner and --no-privileges from the dump. Ownership and role " + "assignments are preserved. By default these flags are applied to keep restores portable " + "across users and environments." + ), + ), + Field( + "clean_mode", + "Clean mode", + "choice", + default="clean", + choices=("clean", "none", "drop_schemas", "drop_database"), + help=( + "How the target database is cleaned before a restore. clean: pg_restore --clean --if-exists. " + "none: no pre-clean. drop_schemas: drop every non-system schema CASCADE (works on managed " + "Postgres). drop_database: DROP DATABASE + CREATE DATABASE — requires CREATEDB or superuser; " + "most managed providers do not allow it." + ), + ), + ] + + +class PostgresClusterEngine(StandardSqlEngine): + key, display, default_port = "postgresql-cluster", "PostgreSQL Cluster", 5432 + template, slug, db_prefix = "engines/postgresql.yml.j2", "pg", "pg" + warning = ( + "Postgres Cluster requires a superuser. Cluster backup/restore uses pg_dumpall, which dumps all " + "databases and global objects (roles, tablespaces). The provided user must be a Postgres superuser." + ) + + +class MariaDbEngine(StandardSqlEngine): + key, display, default_port = "mariadb", "MariaDB", 3306 + template, slug, db_prefix = "engines/mariadb.yml.j2", "mariadb", "mysql" + + +class MySqlEngine(MariaDbEngine): + """Legacy behaviour: a 'mysql' container is a MariaDB image. Kept for volume compatibility.""" + + key, display = "mysql", "MySQL" + + +class MssqlEngine(StandardSqlEngine): + key, display, default_port = "mssql", "Microsoft SQL Server", 1433 + template, slug, db_prefix = "engines/mssql.yml.j2", "mssql", "master" + + def generate(self, *, auth: bool, ports: PortAllocator, answers: dict[str, Any]) -> DatabaseSpec: + return DatabaseSpec( + id=self.new_id(), + engine=self.key, + name="MSSQL", + managed=True, + host=self.service_name(self.slug), + port=self.default_port, + host_port=ports.free(), + database="master", + username="sa", + password=generate_password(16), + ) + + def env_vars(self, spec: DatabaseSpec) -> dict[str, str]: + p = spec.env_prefix + return {f"{p}_PORT": str(spec.host_port), f"{p}_PASS": spec.password or ""} + + +class FirebirdEngine(StandardSqlEngine): + key, display, default_port = "firebird", "Firebird", 3050 + template, slug, db_prefix = "engines/firebird.yml.j2", "firebird", "fb" + DATA_DIR = "/var/lib/firebird/data" + + def generate(self, *, auth: bool, ports: PortAllocator, answers: dict[str, Any]) -> DatabaseSpec: + db_file = "mirror.fdb" + return DatabaseSpec( + id=self.new_id(), + engine=self.key, + name=db_file, + managed=True, + host=self.service_name(self.slug), + port=self.default_port, + host_port=ports.free(), + database=f"{self.DATA_DIR}/{db_file}", + username="alice", + password=generate_password(16), + root_password=generate_password(16), + ) + + def env_vars(self, spec: DatabaseSpec) -> dict[str, str]: + base = super().env_vars(spec) + # Compose template expects the bare file name; databases.json carries the container path. + base[f"{spec.env_prefix}_DB"] = (spec.database or "").rsplit("/", 1)[-1] + base[f"{spec.env_prefix}_ROOT_PASS"] = spec.root_password or "" + return base + + def template_ctx(self, spec: DatabaseSpec, *, inline: bool = False) -> dict[str, Any]: + ctx = super().template_ctx(spec, inline=inline) + ctx["db_var"] = self.var(spec, "DB", (spec.database or "").rsplit("/", 1)[-1], inline) + ctx["root_password_var"] = self.var(spec, "ROOT_PASS", spec.root_password, inline) + return ctx +``` + +- [ ] **Step 2: Vérifier** + +Run: `uv run python -c " +from engines.sql import * +from services.ports import FixedPortAllocator +p = FixedPortAllocator() +for E in (PostgresEngine, MySqlEngine, MssqlEngine, FirebirdEngine): + e = E(); s = e.generate(auth=True, ports=p, answers={'options': {'clean_mode': 'none'}}) + print(E.key, s.host[:9], sorted(e.env_vars(s)), e.agent_entry(s).get('options'), e.template_ctx(s)['port_var']) +print(PostgresEngine().agent_entry(PostgresEngine().generate(auth=True, ports=p, answers={})).get('options'))"` +Expected (hex variable) : +``` +postgresql db-pg-xxx ['DB_PG_XXXX_DB', 'DB_PG_XXXX_PASS', 'DB_PG_XXXX_PORT', 'DB_PG_XXXX_USER'] {'clean_mode': 'none'} ${DB_PG_XXXX_PORT} +mysql db-mariad [... 4 vars] None ... +mssql db-mssql- [..._PASS, ..._PORT] None ... +firebird db-fireb [..._DB, ..._PASS, ..._PORT, ..._ROOT_PASS, ..._USER] None ... +None +``` +La dernière ligne : options par défaut → pas de clé `options`. + +- [ ] **Step 3: Commit** + +```bash +git add engines/sql.py +git commit -m "feat(engines): add SQL engines (postgresql, cluster, mysql, mariadb, mssql, firebird)" +``` + +--- + +### Task 4 : Redis, Valkey, Mongo, SQLite, Docker volume, registre + +**Files:** +- Create: `engines/redis.py`, `engines/valkey.py`, `engines/mongo.py`, `engines/sqlite.py`, `engines/docker_volume.py` +- Modify: `engines/__init__.py` + +**Interfaces:** +- Produces: `RedisEngine`, `ValkeyEngine`, `MongoEngine`, `SqliteEngine`, `DockerVolumeEngine` ; `engines.registry: EngineRegistry` (instance module-level) ; `engines.ALL: tuple[DbEngine, ...]`. + +- [ ] **Step 1: `engines/redis.py`** + +```python +from __future__ import annotations + +import secrets +from typing import Any + +from core.fields import Field +from core.specs import DatabaseSpec +from core.utils import generate_password +from engines.base import DbEngine +from services.ports import PortAllocator + + +class RedisEngine(DbEngine): + key, display, default_port = "redis", "Redis", 6379 + template = "engines/redis.yml.j2" + auth_variants = True + + def fields_existing(self) -> list[Field]: + return [ + Field("host", "Host", "text", default="localhost"), + Field("port", "Port", "int", default=self.default_port), + Field("database", "Database index", "text", default="0"), + Field("username", "Username (empty if none)", "text", default=""), + Field("password", "Password (empty if none)", "text", default=""), + ] + + def generate(self, *, auth: bool, ports: PortAllocator, answers: dict[str, Any]) -> DatabaseSpec: + return DatabaseSpec( + id=self.new_id(), + engine=self.key, + name=f"redis_{secrets.token_hex(4)}", + managed=True, + host=self.service_name("redis", auth), + port=self.default_port, + host_port=ports.free(), + database="0", + username="", + password=generate_password(16) if auth else None, + ) + + def env_vars(self, spec: DatabaseSpec) -> dict[str, str]: + p = spec.env_prefix + out = {f"{p}_PORT": str(spec.host_port)} + if spec.auth: + out[f"{p}_PASS"] = spec.password or "" + return out + + def agent_database(self, spec: DatabaseSpec) -> str: + return spec.database or "0" +``` + +- [ ] **Step 2: `engines/valkey.py`** + +Identique à Redis sauf identité et template : + +```python +from __future__ import annotations + +import secrets +from typing import Any + +from core.fields import Field +from core.specs import DatabaseSpec +from core.utils import generate_password +from engines.base import DbEngine +from services.ports import PortAllocator + + +class ValkeyEngine(DbEngine): + key, display, default_port = "valkey", "Valkey", 6379 + template = "engines/valkey.yml.j2" + auth_variants = True + + def fields_existing(self) -> list[Field]: + return [ + Field("host", "Host", "text", default="localhost"), + Field("port", "Port", "int", default=self.default_port), + Field("database", "Database index", "text", default="0"), + Field("username", "Username (empty if none)", "text", default=""), + Field("password", "Password (empty if none)", "text", default=""), + ] + + def generate(self, *, auth: bool, ports: PortAllocator, answers: dict[str, Any]) -> DatabaseSpec: + return DatabaseSpec( + id=self.new_id(), + engine=self.key, + name=f"valkey_{secrets.token_hex(4)}", + managed=True, + host=self.service_name("valkey", auth), + port=self.default_port, + host_port=ports.free(), + database="0", + username="", + password=generate_password(16) if auth else None, + ) + + def env_vars(self, spec: DatabaseSpec) -> dict[str, str]: + p = spec.env_prefix + out = {f"{p}_PORT": str(spec.host_port)} + if spec.auth: + out[f"{p}_PASS"] = spec.password or "" + return out + + def agent_database(self, spec: DatabaseSpec) -> str: + return spec.database or "0" +``` + +- [ ] **Step 3: `engines/mongo.py`** + +```python +from __future__ import annotations + +import secrets +from typing import Any + +from core.specs import DatabaseSpec +from core.utils import generate_password +from engines.base import DbEngine +from services.ports import PortAllocator + + +class MongoEngine(DbEngine): + key, display, default_port = "mongodb", "MongoDB", 27017 + template = "engines/mongodb.yml.j2" + auth_variants = True + + def generate(self, *, auth: bool, ports: PortAllocator, answers: dict[str, Any]) -> DatabaseSpec: + db_name = f"mongo_{secrets.token_hex(4)}" + return DatabaseSpec( + id=self.new_id(), + engine=self.key, + name=db_name, + managed=True, + host=self.service_name("mongo", auth), + port=self.default_port, + host_port=ports.free(), + database=db_name, + username="admin" if auth else "", + password=generate_password(16) if auth else None, + ) + + def env_vars(self, spec: DatabaseSpec) -> dict[str, str]: + p = spec.env_prefix + out = {f"{p}_PORT": str(spec.host_port), f"{p}_DB": spec.database or ""} + if spec.auth: + out[f"{p}_USER"] = spec.username or "" + out[f"{p}_PASS"] = spec.password or "" + return out +``` + +- [ ] **Step 4: `engines/sqlite.py`** + +```python +"""SQLite: a file mounted into the agent. No Compose service.""" + +from __future__ import annotations + +from typing import Any + +from core.fields import Field +from core.specs import DatabaseSpec +from engines.base import DbEngine +from services.ports import PortAllocator + +CONFIG_DIR = "/config" + + +class SqliteEngine(DbEngine): + key, display = "sqlite", "SQLite" + template = None + auth_variants = False + + def fields_existing(self) -> list[Field]: + return [Field("path", "Database Path (relative or absolute)", "text")] + + def fields_new(self) -> list[Field]: + return [Field("name", "Database Name", "text", default="local")] + + def generate(self, *, auth: bool, ports: PortAllocator, answers: dict[str, Any]) -> DatabaseSpec: + name = str(answers.get("name") or "local") + if not name.endswith(".sqlite"): + name += ".sqlite" + return DatabaseSpec( + id=self.new_id(), + engine=self.key, + name=name, + managed=False, + path=name, # relative: ./name mounted to /config/name + database=f"{CONFIG_DIR}/{name}", + ) + + def from_existing(self, answers: dict[str, Any]) -> DatabaseSpec: + raw = str(answers["path"]) + absolute = raw.startswith("/") + return DatabaseSpec( + id=self.new_id(), + engine=self.key, + name=answers.get("label") or "External DB", + managed=False, + path=raw, + database=raw if absolute else f"{CONFIG_DIR}/{raw}", + ) + + @staticmethod + def mount_for(spec: DatabaseSpec) -> tuple[str, str] | None: + """(host_path, container_path) if the file must be bind-mounted into the agent.""" + if spec.database and spec.database.startswith(f"{CONFIG_DIR}/"): + rel = spec.database[len(CONFIG_DIR) + 1 :] + return (f"./{rel}", spec.database) + return None + + def env_vars(self, spec: DatabaseSpec) -> dict[str, str]: + return {} + + def agent_entry(self, spec: DatabaseSpec) -> dict[str, Any]: + return {"name": spec.name, "database": spec.database, "type": self.key, "generated_id": spec.id} + + def describe(self, spec: DatabaseSpec) -> str: + return "Local File" +``` + +- [ ] **Step 5: `engines/docker_volume.py`** + +```python +"""Docker volume backup target. Requires the Docker socket on the agent. No Compose service.""" + +from __future__ import annotations + +from typing import Any + +from core.fields import Field +from core.specs import DatabaseSpec +from engines.base import DbEngine +from services.ports import PortAllocator + + +class DockerVolumeEngine(DbEngine): + key, display = "docker-volume", "Docker Volume" + template = None + has_modes = False + warning = "Requires the Docker socket. It will be mounted on the agent (/var/run/docker.sock)." + + def fields_existing(self) -> list[Field]: + return [ + Field("volume", "Volume Name (e.g. databases_sqlite-data)", "text"), + Field("container", "Container Name (optional, enables auto-restart after restore)", "text", default=""), + ] + + def fields_new(self) -> list[Field]: + return self.fields_existing() + + def generate(self, *, auth: bool, ports: PortAllocator, answers: dict[str, Any]) -> DatabaseSpec: + return self.from_existing(answers) + + def from_existing(self, answers: dict[str, Any]) -> DatabaseSpec: + return DatabaseSpec( + id=self.new_id(), + engine=self.key, + name=answers.get("label") or "Docker Volume", + managed=False, + volume=str(answers["volume"]).strip(), + container=(str(answers.get("container") or "").strip() or None), + ) + + def env_vars(self, spec: DatabaseSpec) -> dict[str, str]: + return {} + + def agent_entry(self, spec: DatabaseSpec) -> dict[str, Any]: + entry = {"name": spec.name, "type": self.key, "volume_name": spec.volume, "generated_id": spec.id} + if spec.container: + entry["container_name"] = spec.container + return entry + + def describe(self, spec: DatabaseSpec) -> str: + return f"volume: {spec.volume}" +``` + +- [ ] **Step 6: `engines/__init__.py`** + +```python +"""Engine registry. Explicit imports keep PyInstaller happy (no dynamic discovery).""" + +from __future__ import annotations + +from engines.docker_volume import DockerVolumeEngine +from engines.mongo import MongoEngine +from engines.redis import RedisEngine +from engines.registry import EngineRegistry +from engines.sql import ( + FirebirdEngine, + MariaDbEngine, + MssqlEngine, + MySqlEngine, + PostgresClusterEngine, + PostgresEngine, +) +from engines.sqlite import SqliteEngine +from engines.valkey import ValkeyEngine + +ALL = ( + PostgresEngine(), + PostgresClusterEngine(), + MySqlEngine(), + MariaDbEngine(), + SqliteEngine(), + FirebirdEngine(), + MongoEngine(), + RedisEngine(), + ValkeyEngine(), + MssqlEngine(), + DockerVolumeEngine(), +) + +registry = EngineRegistry(ALL) + +__all__ = ["ALL", "EngineRegistry", "registry"] +``` + +L'ordre = ordre d'affichage dans le select (identique au legacy). + +- [ ] **Step 7: Vérifier** + +Run: `uv run python -c " +from engines import registry +from services.ports import FixedPortAllocator +p = FixedPortAllocator() +print(registry.keys()) +for e in registry: + if e.template is None: continue + for auth in ((True, False) if e.auth_variants else (True,)): + s = e.generate(auth=auth, ports=p, answers={}) + ctx = e.template_ctx(s); assert ctx['name'] == s.host and set(e.env_vars(s)) >= {s.env_prefix + '_PORT'} + print(f'{e.key:20} auth={auth!s:5} {s.host:24} env={len(e.env_vars(s))} entry.db={e.agent_entry(s)[\"database\"]!r}') +sq = registry.get('sqlite'); s = sq.generate(auth=False, ports=p, answers={'name':'x'}); print(sq.agent_entry(s), sq.mount_for(s)) +dv = registry.get('docker-volume'); print(dv.agent_entry(dv.from_existing({'volume':'v','container':''})))"` +Expected: 11 clés dans l'ordre legacy ; une ligne par moteur/variante avec `entry.db` = `'0'` pour redis/valkey, `'master'` mssql, `/var/lib/firebird/data/mirror.fdb` firebird ; sqlite `{'name': 'x.sqlite', 'database': '/config/x.sqlite', 'type': 'sqlite', 'generated_id': ...} ('./x.sqlite', '/config/x.sqlite')` ; docker-volume sans `container_name`. + +- [ ] **Step 8: Commit** + +```bash +git add engines/ +git commit -m "feat(engines): add redis, valkey, mongodb, sqlite, docker-volume engines and registry" +``` + +--- + +### Task 5 : Templates Jinja2 + +**Files:** +- Create: `templates/agent.yml.j2`, `templates/dashboard.yml.j2` +- Create: `templates/engines/postgresql.yml.j2`, `mariadb.yml.j2`, `mssql.yml.j2`, `firebird.yml.j2`, `mongodb.yml.j2`, `redis.yml.j2`, `valkey.yml.j2` +- Create: `templates/engines.map.json` +- Move: `.github/assets/templates/agent.yml` → `templates/agent.yml`, `dashboard.yml` → `templates/dashboard.yml` +- Modify: `pyproject.toml` (`jinja2`), `.gitleaks.toml` + +**Interfaces:** +- Produces: contrat de contexte. + - `agent.yml.j2` : `host_gateway: bool`, `docker_socket: bool`, `mounts: list[{host, container}]`, `services: list[{name, volume, body}]`, `tz_var, edge_key_var, log_level_var, polling_var: str`. + - `dashboard.yml.j2` : `db_mode: "external"|"internal"|"custom"`, `project_name_var, host_port_var, tz_var, log_level_var, project_secret_var, project_url_var, pg_port_var, postgres_db_var, postgres_user_var, postgres_password_var: str`. + - `engines/*.yml.j2` : `name, volume, auth, port_var, db_var, user_var, password_var` (+ `root_password_var` firebird). + +- [ ] **Step 1: Ajouter Jinja2** + +Run: `uv add "jinja2>=3.1"` +Expected: `pyproject.toml` et `uv.lock` mis à jour. + +- [ ] **Step 2: Déplacer les templates legacy** + +Run: `mkdir -p templates/engines && git mv .github/assets/templates/agent.yml templates/agent.yml && git mv .github/assets/templates/dashboard.yml templates/dashboard.yml && rmdir .github/assets/templates 2>/dev/null; ls templates` + +- [ ] **Step 3: `templates/agent.yml.j2`** + +```jinja +services: + agent: + restart: unless-stopped + image: portabase/agent:latest + volumes: + - ./databases.json:/config/config.json +{%- for m in mounts %} + - {{ m.host }}:{{ m.container }} +{%- endfor %} +{%- if docker_socket %} + - /var/run/docker.sock:/var/run/docker.sock +{%- endif %} +{%- if host_gateway %} + extra_hosts: + - "localhost:host-gateway" +{%- endif %} + environment: + TZ: "{{ tz_var }}" + EDGE_KEY: "{{ edge_key_var }}" + LOG_LEVEL: "{{ log_level_var }}" + POLLING: "{{ polling_var }}" + networks: + - portabase +{% for s in services %} +{{ s.body }} +{%- endfor %} +{% if services %} +volumes: +{%- for s in services %} + {{ s.volume }}: +{%- endfor %} +{% endif %} +networks: + portabase: + name: portabase_network + external: true +``` + +- [ ] **Step 4: `templates/dashboard.yml.j2`** + +```jinja +name: {{ project_name_var }} +services: + portabase: + container_name: {{ project_name_var }}-app + image: portabase/portabase:latest + restart: unless-stopped + env_file: + - .env + ports: + - "{{ host_port_var }}:80" + environment: + - TZ={{ tz_var }} + - LOG_LEVEL={{ log_level_var }} + - PROJECT_SECRET={{ project_secret_var }} + - PROJECT_URL={{ project_url_var }} + volumes: + - portabase-data:/data +{%- if db_mode == "external" %} + depends_on: + db: + condition: service_healthy +{%- endif %} + healthcheck: + test: ["CMD-SHELL", "curl -f http://localhost/api/health"] + interval: 30s + timeout: 5s + retries: 3 + start_period: 60s +{%- if db_mode == "external" %} + db: + container_name: {{ project_name_var }}-pg + image: postgres:17-alpine + restart: unless-stopped + ports: + - "{{ pg_port_var }}:5432" + volumes: + - postgres-data:/var/lib/postgresql/data + environment: + - POSTGRES_DB={{ postgres_db_var }} + - POSTGRES_USER={{ postgres_user_var }} + - POSTGRES_PASSWORD={{ postgres_password_var }} + healthcheck: + test: ["CMD-SHELL", "pg_isready -U {{ postgres_user_var }} -d {{ postgres_db_var }}"] + interval: 10s + timeout: 5s + retries: 5 +{%- endif %} +volumes: +{%- if db_mode == "external" %} + postgres-data: +{%- endif %} + portabase-data: +``` + +- [ ] **Step 5: Templates moteurs** + +`templates/engines/postgresql.yml.j2` : + +```jinja + {{ name }}: + image: postgres:17-alpine + restart: unless-stopped + networks: + - portabase + ports: + - "{{ port_var }}:5432" + volumes: + - {{ volume }}:/var/lib/postgresql/data + environment: + - POSTGRES_DB={{ db_var }} + - POSTGRES_USER={{ user_var }} + - POSTGRES_PASSWORD={{ password_var }} + healthcheck: + test: ["CMD-SHELL", "pg_isready -U {{ user_var }} -d {{ db_var }}"] + interval: 10s + timeout: 5s + retries: 5 +``` + +`templates/engines/mariadb.yml.j2` : + +```jinja + {{ name }}: + image: mariadb:latest + restart: unless-stopped + networks: + - portabase + ports: + - "{{ port_var }}:3306" + environment: + - MYSQL_DATABASE={{ db_var }} + - MYSQL_USER={{ user_var }} + - MYSQL_PASSWORD={{ password_var }} + - MYSQL_RANDOM_ROOT_PASSWORD=yes + volumes: + - {{ volume }}:/var/lib/mysql + healthcheck: + test: ["CMD-SHELL", "mariadb-admin ping -h localhost -u {{ user_var }} -p{{ password_var }}"] + interval: 10s + timeout: 5s + retries: 5 +``` + +`templates/engines/mssql.yml.j2` : + +```jinja + {{ name }}: + image: mcr.microsoft.com/azure-sql-edge:latest + restart: unless-stopped + networks: + - portabase + ports: + - "{{ port_var }}:1433" + environment: + - ACCEPT_EULA=Y + - MSSQL_SA_PASSWORD={{ password_var }} + volumes: + - {{ volume }}:/var/opt/mssql + healthcheck: + test: ["CMD-SHELL", "cat /proc/net/tcp6 | grep -q '059901' || exit 1"] + interval: 10s + timeout: 5s + retries: 20 +``` + +`templates/engines/firebird.yml.j2` : + +```jinja + {{ name }}: + image: firebirdsql/firebird + restart: unless-stopped + networks: + - portabase + ports: + - "{{ port_var }}:3050" + volumes: + - {{ volume }}:/var/lib/firebird/data + environment: + - FIREBIRD_DATABASE={{ db_var }} + - FIREBIRD_USER={{ user_var }} + - FIREBIRD_PASSWORD={{ password_var }} + - FIREBIRD_ROOT_PASSWORD={{ root_password_var }} + - FIREBIRD_DATABASE_DEFAULT_CHARSET=UTF8 + healthcheck: + test: ["CMD-SHELL", "nc -z localhost 3050"] + interval: 10s + timeout: 5s + retries: 5 +``` + +`templates/engines/mongodb.yml.j2` : + +```jinja + {{ name }}: + image: mongo:latest + restart: unless-stopped + networks: + - portabase + ports: + - "{{ port_var }}:27017" + environment: +{%- if auth %} + - MONGO_INITDB_ROOT_USERNAME={{ user_var }} + - MONGO_INITDB_ROOT_PASSWORD={{ password_var }} +{%- endif %} + - MONGO_INITDB_DATABASE={{ db_var }} +{%- if auth %} + command: mongod --auth +{%- endif %} + volumes: + - {{ volume }}:/data/db + healthcheck: + test: ["CMD-SHELL", "mongosh --eval 'db.runCommand({ping:1})' --quiet"] + interval: 10s + timeout: 5s + retries: 5 +``` + +`templates/engines/redis.yml.j2` : + +```jinja + {{ name }}: + image: redis:latest + restart: unless-stopped + ports: + - "{{ port_var }}:6379" + volumes: + - {{ volume }}:/data +{%- if auth %} + environment: + - REDIS_PASSWORD={{ password_var }} + command: ["redis-server", "--requirepass", "{{ password_var }}", "--appendonly", "yes"] +{%- else %} + command: ["redis-server", "--appendonly", "yes"] +{%- endif %} + networks: + - portabase + - default + healthcheck: + test: ["CMD-SHELL", "redis-cli {% if auth %}-a {{ password_var }} {% endif %}ping | grep PONG"] + interval: 10s + timeout: 5s + retries: 5 +``` + +`templates/engines/valkey.yml.j2` : + +```jinja + {{ name }}: + image: valkey/valkey:latest + restart: unless-stopped +{%- if auth %} + command: --requirepass "{{ password_var }}" +{%- else %} + environment: + - ALLOW_EMPTY_PASSWORD=yes +{%- endif %} + ports: + - "{{ port_var }}:6379" + volumes: + - {{ volume }}:/data + networks: + - portabase + - default + healthcheck: + test: ["CMD-SHELL", "valkey-cli {% if auth %}-a {{ password_var }} {% endif %}ping | grep PONG"] + interval: 10s + timeout: 5s + retries: 5 +``` + +Différence assumée vs snippets legacy : `restart: unless-stopped` ajouté sur redis et valkey (spec §5.7). + +- [ ] **Step 6: `templates/engines.map.json`** + +```json +{ + "postgresql": "engines/postgresql.yml.j2", + "postgresql-cluster": "engines/postgresql.yml.j2", + "mysql": "engines/mariadb.yml.j2", + "mariadb": "engines/mariadb.yml.j2", + "mssql": "engines/mssql.yml.j2", + "firebird": "engines/firebird.yml.j2", + "mongodb": "engines/mongodb.yml.j2", + "redis": "engines/redis.yml.j2", + "valkey": "engines/valkey.yml.j2" +} +``` + +- [ ] **Step 7: `.gitleaks.toml`** + +Retirer la ligne `'''\.github/assets/templates/.*''',` de `paths`. + +- [ ] **Step 8: Rendu manuel de contrôle** + +Run: `uv run python -c " +import jinja2, yaml +env = jinja2.Environment(loader=jinja2.FileSystemLoader('templates'), undefined=jinja2.StrictUndefined, keep_trailing_newline=True, autoescape=False) +body = env.get_template('engines/redis.yml.j2').render(name='db-redis-auth-ab12', volume='db-redis-auth-ab12-data', auth=True, port_var='\${DB_REDIS_AUTH_AB12_PORT}', db_var='', user_var='', password_var='\${DB_REDIS_AUTH_AB12_PASS}') +out = env.get_template('agent.yml.j2').render(host_gateway=True, docker_socket=True, mounts=[{'host':'./x.sqlite','container':'/config/x.sqlite'}], services=[{'name':'db-redis-auth-ab12','volume':'db-redis-auth-ab12-data','body':body}], tz_var='\${TZ}', edge_key_var='\${EDGE_KEY}', log_level_var='\${LOG_LEVEL}', polling_var='\${POLLING}') +print(out); d = yaml.safe_load(out); print(sorted(d['services']), d['volumes'], d['services']['agent']['extra_hosts']) +for mode in ('external','internal','custom'): + o = env.get_template('dashboard.yml.j2').render(db_mode=mode, project_name_var='pb', host_port_var='8887', tz_var='\${TZ}', log_level_var='\${LOG_LEVEL}', project_secret_var='\${PROJECT_SECRET}', project_url_var='\${PROJECT_URL}', pg_port_var='\${PG_PORT}', postgres_db_var='\${POSTGRES_DB}', postgres_user_var='\${POSTGRES_USER}', postgres_password_var='\${POSTGRES_PASSWORD}') + print(mode, sorted(yaml.safe_load(o)['services']), sorted(yaml.safe_load(o)['volumes']))"` +Expected: compose agent imprimé avec socket, extra_hosts, mount sqlite, service redis ; `['agent', 'db-redis-auth-ab12'] {'db-redis-auth-ab12-data': None} ['localhost:host-gateway']` ; dashboard `external ['db', 'portabase'] ['portabase-data', 'postgres-data']`, `internal ['portabase'] ['portabase-data']`, `custom ['portabase'] ['portabase-data']`. + +- [ ] **Step 9: Commit** + +```bash +git add templates/ pyproject.toml uv.lock .gitleaks.toml +git commit -m "feat(templates): add Jinja2 compose templates at repo root, move legacy templates" +``` + +--- + +### Task 6 : `services/templates.py` — `Manifest`, `TemplateRepository` + +**Files:** +- Create: `services/templates.py` + +**Interfaces:** +- Consumes: `HttpClient`, `GlobalConfig.cache_dir`, `core.version`, `TemplateError`. +- Produces: + - `Manifest(schema, version, files: dict[str, FileEntry], engines: dict[str, str], generated_at, commit)` avec `from_json(data)`, `from_directory(dir, version)`. + - `TemplateRepository(http, cache_dir, version, base_url=TEMPLATE_BASE_URL, local_dir: Path | None = None)` : `resolve() -> Path` (dossier prêt, fetch si besoin), `get(name) -> jinja2.Template`, `engine_template(key) -> jinja2.Template`, `manifest -> Manifest`, propriété `source: str` (`local` / `cache` / `remote`). + - `TemplateRepository.from_environment(http, config) -> TemplateRepository` : lit `PORTABASE_TEMPLATES_DIR`, `PORTABASE_TEMPLATES_VERSION`, détection dev (`./templates` à côté de `main.py` si non frozen). + - `TEMPLATE_BASE_URL` importée depuis `core/config.py` (inchangée). + +- [ ] **Step 1: Écrire le module** + +```python +"""Versioned remote templates with manifest verification and a local cache. + +Resolution order: explicit local dir (dev) → cache hit → remote fetch. No 'latest' fallback: +a CLI version only ever renders with the templates published for that exact version. +""" + +from __future__ import annotations + +import hashlib +import json +import os +import sys +from dataclasses import dataclass +from pathlib import Path + +import jinja2 + +from core.config import TEMPLATE_BASE_URL, GlobalConfig +from core.errors import NetworkError, TemplateError +from core.version import UNKNOWN, current_version +from services.http import HttpClient + +MANIFEST_NAME = "manifest.json" +SUPPORTED_SCHEMA = 1 + + +@dataclass(frozen=True) +class FileEntry: + sha256: str + size: int + + +@dataclass(frozen=True) +class Manifest: + schema: int + version: str + files: dict[str, FileEntry] + engines: dict[str, str] + generated_at: str = "" + commit: str = "" + + @classmethod + def from_json(cls, data: dict) -> Manifest: + try: + schema = int(data["schema"]) + if schema != SUPPORTED_SCHEMA: + raise TemplateError( + f"Unsupported template manifest schema {schema} (this CLI supports {SUPPORTED_SCHEMA}).", + hint="Update the CLI: portabase update", + ) + files = { + name: FileEntry(sha256=str(e["sha256"]).lower(), size=int(e["size"])) + for name, e in data["files"].items() + } + return cls( + schema=schema, + version=str(data["version"]), + files=files, + engines=dict(data.get("engines", {})), + generated_at=str(data.get("generated_at", "")), + commit=str(data.get("commit", "")), + ) + except (KeyError, TypeError, ValueError) as e: + raise TemplateError("Template manifest is malformed.", cause=e) from e + + @classmethod + def from_directory(cls, directory: Path, version: str) -> Manifest: + """Manifest computed from a local directory (dev mode / render checks).""" + files = {} + for path in sorted(directory.rglob("*.j2")): + rel = path.relative_to(directory).as_posix() + files[rel] = FileEntry(sha256=_sha256(path), size=path.stat().st_size) + engines_map = directory / "engines.map.json" + engines = json.loads(engines_map.read_text(encoding="utf-8")) if engines_map.exists() else {} + return cls(schema=SUPPORTED_SCHEMA, version=version, files=files, engines=engines) + + +def _sha256(path: Path) -> str: + h = hashlib.sha256() + with open(path, "rb") as f: + for chunk in iter(lambda: f.read(1 << 20), b""): + h.update(chunk) + return h.hexdigest() + + +class TemplateRepository: + def __init__( + self, + http: HttpClient, + cache_dir: Path, + version: str, + *, + base_url: str = TEMPLATE_BASE_URL, + local_dir: Path | None = None, + ) -> None: + self.http = http + self.version = version + self.base_url = base_url.rstrip("/") + self.local_dir = local_dir + self.cache_dir = cache_dir / "templates" / version + self._manifest: Manifest | None = None + self._env: jinja2.Environment | None = None + self.source = "unresolved" + + # ---- construction ----------------------------------------------------- + + @classmethod + def from_environment(cls, http: HttpClient, config: GlobalConfig) -> TemplateRepository: + version = os.environ.get("PORTABASE_TEMPLATES_VERSION") or current_version() + local = os.environ.get("PORTABASE_TEMPLATES_DIR") + local_dir = Path(local) if local else None + if local_dir is None and not getattr(sys, "frozen", False): + candidate = Path(__file__).resolve().parent.parent / "templates" + if (candidate / "agent.yml.j2").exists(): + local_dir = candidate + return cls(http, config.cache_dir, version, local_dir=local_dir) + + # ---- resolution ------------------------------------------------------- + + @property + def manifest(self) -> Manifest: + if self._manifest is None: + self.resolve() + assert self._manifest is not None + return self._manifest + + def resolve(self) -> Path: + """Ensure a verified template directory exists locally and return it.""" + if self.local_dir is not None: + if not (self.local_dir / "agent.yml.j2").exists(): + raise TemplateError(f"Template directory {self.local_dir} has no agent.yml.j2.") + self._manifest = Manifest.from_directory(self.local_dir, self.version) + self.source = "local" + return self.local_dir + + if self.version == UNKNOWN: + raise TemplateError( + "Cannot resolve template version (CLI version unknown).", + hint="Set PORTABASE_TEMPLATES_DIR to a local templates folder or PORTABASE_TEMPLATES_VERSION.", + ) + + remote_manifest = self._fetch_manifest() + if remote_manifest is None: + cached = self._cached_manifest() + if cached is None: + raise TemplateError( + f"Templates for version {self.version} are unavailable and not cached.", + hint="Check your internet connection, or set PORTABASE_TEMPLATES_DIR.", + ) + self._manifest = cached + self.source = "cache" + self._verify_cache_complete(cached) + return self.cache_dir + + if remote_manifest.version != self.version: + raise TemplateError( + f"Template manifest is for version {remote_manifest.version}, expected {self.version}." + ) + self._sync(remote_manifest) + self._manifest = remote_manifest + self.source = "remote" + return self.cache_dir + + # ---- access ----------------------------------------------------------- + + def get(self, name: str) -> jinja2.Template: + directory = self.resolve() + if name not in self.manifest.files: + raise TemplateError(f"Template '{name}' is not part of version {self.version}.") + if self._env is None: + self._env = jinja2.Environment( + loader=jinja2.FileSystemLoader(str(directory)), + undefined=jinja2.StrictUndefined, + keep_trailing_newline=True, + autoescape=False, + ) + try: + return self._env.get_template(name) + except jinja2.TemplateError as e: + raise TemplateError(f"Template '{name}' failed to load: {e}", cause=e) from e + + def engine_template(self, engine_key: str) -> jinja2.Template: + name = self.manifest.engines.get(engine_key) + if name is None: + raise TemplateError(f"No template mapped for engine '{engine_key}' in version {self.version}.") + return self.get(name) + + # ---- internals -------------------------------------------------------- + + def _url(self, name: str) -> str: + return f"{self.base_url}/{self.version}/{name}" + + def _fetch_manifest(self) -> Manifest | None: + try: + return Manifest.from_json(self.http.get_json(self._url(MANIFEST_NAME))) + except NetworkError: + return None + + def _cached_manifest(self) -> Manifest | None: + path = self.cache_dir / MANIFEST_NAME + if not path.exists(): + return None + try: + return Manifest.from_json(json.loads(path.read_text(encoding="utf-8"))) + except (OSError, ValueError, TemplateError): + return None + + def _verify_cache_complete(self, manifest: Manifest) -> None: + for name, entry in manifest.files.items(): + path = self.cache_dir / name + if not path.exists() or _sha256(path) != entry.sha256: + raise TemplateError( + f"Cached template '{name}' is missing or corrupt and the network is unavailable.", + hint="Reconnect and retry; the cache will be refreshed.", + ) + + def _sync(self, manifest: Manifest) -> None: + self.cache_dir.mkdir(parents=True, exist_ok=True) + for name, entry in manifest.files.items(): + path = self.cache_dir / name + if path.exists() and path.stat().st_size == entry.size and _sha256(path) == entry.sha256: + continue + path.parent.mkdir(parents=True, exist_ok=True) + tmp = path.with_suffix(path.suffix + ".tmp") + try: + self.http.download(self._url(name), tmp) + except NetworkError as e: + raise TemplateError(f"Could not download template '{name}'.", cause=e) from e + if tmp.stat().st_size != entry.size or _sha256(tmp) != entry.sha256: + tmp.unlink(missing_ok=True) + raise TemplateError(f"Template '{name}' failed integrity check (sha256 mismatch).") + os.replace(tmp, path) + for stale in self.cache_dir.rglob("*.j2"): + if stale.relative_to(self.cache_dir).as_posix() not in manifest.files: + stale.unlink(missing_ok=True) + (self.cache_dir / MANIFEST_NAME).write_text( + json.dumps( + { + "schema": manifest.schema, + "version": manifest.version, + "generated_at": manifest.generated_at, + "commit": manifest.commit, + "files": {n: {"sha256": e.sha256, "size": e.size} for n, e in manifest.files.items()}, + "engines": manifest.engines, + }, + indent=2, + ), + encoding="utf-8", + ) +``` + +`TEMPLATE_BASE_URL` reste définie dans `core/config.py` (le legacy `core/network.py` l'importe de là) ; `services/templates.py` l'importe depuis `core.config`. Pas de circularité : `core` n'importe jamais `services`. + +- [ ] **Step 2: Vérifier en mode local (dev)** + +Run: `uv run python -c " +from pathlib import Path +from services.http import HttpClient +from services.templates import TemplateRepository +from core.config import GlobalConfig +r = TemplateRepository.from_environment(HttpClient(), GlobalConfig()) +print(r.source, r.resolve(), r.source, len(r.manifest.files), r.manifest.engines['mysql']) +print(r.engine_template('redis').render(name='n', volume='v', auth=False, port_var='1', db_var='', user_var='', password_var='')[:40].strip())"` +Expected: `unresolved /templates local 9 engines/mariadb.yml.j2` puis `n:` (début du service rendu). + +- [ ] **Step 3: Vérifier le mode remote contre un serveur local** + +```bash +# Terminal 1 — publie templates/ comme S3 sous la version 99.0.0 avec un manifest +mkdir -p /tmp/pb-s3/99.0.0 && cp -r templates/. /tmp/pb-s3/99.0.0/ && cd /tmp/pb-s3/99.0.0 && \ +uv --directory /home/soluce/Documents/PROJETS/Portabase/cli run python -c " +import json, hashlib, pathlib +files = {p.as_posix(): {'sha256': hashlib.sha256(p.read_bytes()).hexdigest(), 'size': p.stat().st_size} for p in sorted(pathlib.Path('.').rglob('*.j2'))} +json.dump({'schema':1,'version':'99.0.0','generated_at':'now','commit':'x','files':files,'engines':json.load(open('engines.map.json'))}, open('manifest.json','w'), indent=2)" && \ +cd /tmp/pb-s3 && python3 -m http.server 8765 +``` + +Terminal 2 : +```bash +uv run python -c " +import tempfile; from pathlib import Path +from services.http import HttpClient +from services.templates import TemplateRepository +cache = Path(tempfile.mkdtemp()) +r = TemplateRepository(HttpClient(), cache, '99.0.0', base_url='http://127.0.0.1:8765') +print(r.resolve(), r.source, sorted(p.name for p in (cache/'templates'/'99.0.0').iterdir())) +r2 = TemplateRepository(HttpClient(), cache, '99.0.0', base_url='http://127.0.0.1:8765'); r2.resolve(); print('second:', r2.source) +r3 = TemplateRepository(HttpClient(), cache, '99.0.0', base_url='http://127.0.0.1:1'); r3.resolve(); print('offline:', r3.source) +try: TemplateRepository(HttpClient(), Path(tempfile.mkdtemp()), '99.0.0', base_url='http://127.0.0.1:1').resolve() +except Exception as e: print('offline no cache:', type(e).__name__, e.code) +try: TemplateRepository(HttpClient(), cache, '98.0.0', base_url='http://127.0.0.1:8765').resolve() +except Exception as e: print('missing version:', type(e).__name__, e.code)" +``` +Expected: `... remote ['agent.yml.j2', 'dashboard.yml.j2', 'engines', 'manifest.json']`, `second: remote` (manifest re-fetché, fichiers en cache non re-téléchargés), `offline: cache`, `offline no cache: TemplateError E_TEMPLATE`, `missing version: TemplateError E_TEMPLATE`. Arrêter le serveur. + +- [ ] **Step 4: Commit** + +```bash +git add services/templates.py +git commit -m "feat(services): add TemplateRepository with manifest verification and cache" +``` + +--- + +### Task 7 : `scripts/render_check.py` + +**Files:** +- Create: `scripts/render_check.py` + +**Interfaces:** +- Consumes: `TemplateRepository` (mode local), `engines.registry`, `FixedPortAllocator`. +- Produces: script exécutable, exit 0 si tous les rendus sont du YAML valide (et `docker compose config` valide si Docker disponible), exit 1 sinon. Réutilisé par la CI (Task 8) et remplacé par un appel à `ComposeRenderer` au Plan 4. + +- [ ] **Step 1: Écrire le script** + +```python +#!/usr/bin/env python3 +"""Render every template with fixture contexts and validate the output. + +Usage: uv run python scripts/render_check.py [--templates DIR] [--no-compose] +Exit 0 on success. Prints one line per rendered case. +""" + +from __future__ import annotations + +import argparse +import os +import shutil +import subprocess +import sys +import tempfile +from pathlib import Path + +import yaml + +sys.path.insert(0, str(Path(__file__).resolve().parent.parent)) + +from core.config import GlobalConfig # noqa: E402 +from engines import registry # noqa: E402 +from services.http import HttpClient # noqa: E402 +from services.ports import FixedPortAllocator # noqa: E402 +from services.templates import TemplateRepository # noqa: E402 + +AGENT_GLOBALS = { + "tz_var": "${TZ}", + "edge_key_var": "${EDGE_KEY}", + "log_level_var": "${LOG_LEVEL}", + "polling_var": "${POLLING}", +} +AGENT_ENV = 'TZ="UTC"\nEDGE_KEY="x"\nLOG_LEVEL="info"\nPOLLING="5"\n' +DASHBOARD_VARS = { + "project_name_var": "pb", + "host_port_var": "${HOST_PORT}", + "tz_var": "${TZ}", + "log_level_var": "${LOG_LEVEL}", + "project_secret_var": "${PROJECT_SECRET}", + "project_url_var": "${PROJECT_URL}", + "pg_port_var": "${PG_PORT}", + "postgres_db_var": "${POSTGRES_DB}", + "postgres_user_var": "${POSTGRES_USER}", + "postgres_password_var": "${POSTGRES_PASSWORD}", +} +DASHBOARD_ENV = ( + 'HOST_PORT="8887"\nTZ="UTC"\nLOG_LEVEL="info"\nPROJECT_SECRET="s"\nPROJECT_URL="http://localhost"\n' + 'PG_PORT="5433"\nPOSTGRES_DB="pb"\nPOSTGRES_USER="pb"\nPOSTGRES_PASSWORD="p"\n' +) + + +class Failure(Exception): + pass + + +def validate(label: str, compose: str, env_text: str, use_compose: bool) -> None: + try: + doc = yaml.safe_load(compose) + except yaml.YAMLError as e: + raise Failure(f"{label}: invalid YAML: {e}\n{compose}") from e + if not isinstance(doc, dict) or "services" not in doc: + raise Failure(f"{label}: no services key\n{compose}") + if use_compose: + with tempfile.TemporaryDirectory() as tmp: + Path(tmp, "docker-compose.yml").write_text(compose, encoding="utf-8") + Path(tmp, ".env").write_text(env_text, encoding="utf-8") + Path(tmp, "databases.json").write_text('{"databases": []}', encoding="utf-8") + proc = subprocess.run( + ["docker", "compose", "-p", "rendercheck", "config", "--quiet"], + cwd=tmp, + capture_output=True, + text=True, + check=False, + ) + if proc.returncode != 0: + raise Failure(f"{label}: docker compose config failed:\n{proc.stderr}\n{compose}") + print(f"ok {label}") + + +def agent_cases(repo: TemplateRepository) -> list[tuple[str, str, str]]: + ports = FixedPortAllocator() + cases = [] + # 1. empty agent, all toggles off + cases.append(("agent/empty", render_agent(repo, [], False, False, []), AGENT_ENV)) + # 2. toggles on, sqlite mount + cases.append( + ( + "agent/toggles", + render_agent(repo, [], True, True, [{"host": "./x.sqlite", "container": "/config/x.sqlite"}]), + AGENT_ENV, + ) + ) + # 3. one case per engine/variant + all_services, all_env = [], AGENT_ENV + for engine in registry: + if engine.template is None: + continue + for auth in (True, False) if engine.auth_variants else (True,): + spec = engine.generate(auth=auth, ports=ports, answers={}) + env = engine.env_vars(spec) + env_text = AGENT_ENV + "".join(f'{k}="{v}"\n' for k, v in env.items()) + body = repo.engine_template(engine.key).render(**engine.template_ctx(spec)) + service = {"name": spec.host, "volume": f"{spec.host}-data", "body": body} + cases.append((f"agent/{engine.key}{'/auth' if auth else '/noauth' if engine.auth_variants else ''}", + render_agent(repo, [service], False, False, []), env_text)) + all_services.append(service) + all_env += "".join(f'{k}="{v}"\n' for k, v in env.items()) + # 4. everything at once + cases.append(("agent/all", render_agent(repo, all_services, True, True, []), all_env)) + return cases + + +def render_agent(repo, services, host_gateway, docker_socket, mounts) -> str: + return repo.get("agent.yml.j2").render( + services=services, host_gateway=host_gateway, docker_socket=docker_socket, mounts=mounts, **AGENT_GLOBALS + ) + + +def dashboard_cases(repo: TemplateRepository) -> list[tuple[str, str, str]]: + return [ + (f"dashboard/{mode}", repo.get("dashboard.yml.j2").render(db_mode=mode, **DASHBOARD_VARS), DASHBOARD_ENV) + for mode in ("external", "internal", "custom") + ] + + +def engines_check(repo: TemplateRepository) -> None: + mapped = repo.manifest.engines + for engine in registry: + if engine.template is None: + continue + if mapped.get(engine.key) != engine.template: + raise Failure(f"engines.map.json: {engine.key} -> {mapped.get(engine.key)} but code says {engine.template}") + if engine.template not in repo.manifest.files: + raise Failure(f"{engine.key}: template {engine.template} not found") + for key in mapped: + if key not in registry: + raise Failure(f"engines.map.json maps unknown engine '{key}'") + print(f"ok engines-check ({len(mapped)} mapped)") + + +def main() -> int: + parser = argparse.ArgumentParser() + parser.add_argument("--templates", default=os.environ.get("PORTABASE_TEMPLATES_DIR", "templates")) + parser.add_argument("--no-compose", action="store_true", help="Skip docker compose config validation") + args = parser.parse_args() + + use_compose = not args.no_compose and shutil.which("docker") is not None + if not use_compose: + print("note: docker not available, YAML validation only") + repo = TemplateRepository(HttpClient(), GlobalConfig().cache_dir, "local", local_dir=Path(args.templates)) + try: + engines_check(repo) + for label, compose, env_text in agent_cases(repo) + dashboard_cases(repo): + validate(label, compose, env_text, use_compose) + except Failure as e: + print(f"FAIL {e}", file=sys.stderr) + return 1 + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) +``` + +- [ ] **Step 2: Exécuter** + +Run: `uv run python scripts/render_check.py` +Expected: `ok engines-check (9 mapped)` puis une ligne `ok` par cas : `agent/empty`, `agent/toggles`, `agent/postgresql`, `agent/postgresql-cluster`, `agent/mysql`, `agent/mariadb`, `agent/firebird`, `agent/mongodb/auth`, `agent/mongodb/noauth`, `agent/redis/auth`, `agent/redis/noauth`, `agent/valkey/auth`, `agent/valkey/noauth`, `agent/mssql`, `agent/all`, `dashboard/external`, `dashboard/internal`, `dashboard/custom`. Exit 0. + +Si `docker compose config` échoue sur un cas : lire l'erreur, corriger le template (pas le script). + +- [ ] **Step 3: Ruff sur le script** + +Run: `uv run ruff check scripts/ && uv run ruff format scripts/` + +- [ ] **Step 4: Commit** + +```bash +git add scripts/render_check.py +git commit -m "ci: add render_check script validating every template with fixtures" +``` + +--- + +### Task 8 : CI — `render-check`, `engines-check`, manifest à l'upload, hotfix + +**Files:** +- Modify: `.github/workflows/ci.yml` +- Modify: `.github/workflows/templates-upload.yml` +- Create: `.github/workflows/templates-hotfix.yml` + +- [ ] **Step 1: Ajouter le job `render-check` à `ci.yml`** (après `test`) + +```yaml + render-check: + name: render-check + runs-on: ubuntu-latest + permissions: + contents: read + steps: + - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4 + - uses: astral-sh/setup-uv@caf0cab7a618c569241d31dcd442f54681755d39 # v3 + - name: Install + run: uv sync --frozen --all-groups + - name: Render and validate templates (YAML + docker compose config) + run: uv run python scripts/render_check.py --templates templates +``` + +Le job `engines-check` de la spec est couvert par la fonction `engines_check()` du même script (une seule exécution, deux vérifications). Pas de job séparé. + +- [ ] **Step 2: `templates-upload.yml` — source et manifest** + +Remplacer les deux étapes d'upload par : + +```yaml + - name: Generate manifest + env: + VERSION: ${{ inputs.version }} + run: | + set -euo pipefail + CLEAN_VERSION="${VERSION#v}" + cd templates + FILES=$(find . -name '*.j2' -type f | sort | while read -r f; do + rel="${f#./}" + printf '{"%s":{"sha256":"%s","size":%s}}\n' "$rel" "$(sha256sum "$f" | cut -d' ' -f1)" "$(stat -c%s "$f")" + done | jq -s 'add') + jq -n \ + --arg version "$CLEAN_VERSION" \ + --arg commit "$GITHUB_SHA" \ + --arg date "$(date -u +%FT%TZ)" \ + --argjson files "$FILES" \ + --argjson engines "$(cat engines.map.json)" \ + '{schema:1, version:$version, generated_at:$date, commit:$commit, files:$files, engines:$engines}' \ + > manifest.json + cat manifest.json + + - name: Upload versioned templates + env: + VERSION: ${{ inputs.version }} + run: | + CLEAN_VERSION="${VERSION#v}" + s3cmd $S3CMD_ARGS sync templates/ \ + "s3://${{ secrets.S3_BUCKET }}/cli/public/templates/${CLEAN_VERSION}/" --acl-public --delete-removed + + - name: Upload latest templates (stable only, legacy fallback) + if: ${{ !inputs.is_prerelease }} + run: | + s3cmd $S3CMD_ARGS sync templates/ \ + "s3://${{ secrets.S3_BUCKET }}/cli/public/templates/latest/" --acl-public +``` + +`latest/` reste alimenté pour les vieux binaires (fallback legacy). Le nouveau code ne le lit jamais. À retirer quand plus aucune version legacy n'est supportée. + +- [ ] **Step 3: `templates-hotfix.yml`** + +```yaml +name: Templates hotfix + +on: + workflow_dispatch: + inputs: + version: + description: "Existing CLI version to re-publish templates for (e.g. 26.09.0). Templates must stay compatible with that version's code." + required: true + type: string + +permissions: {} + +jobs: + hotfix: + uses: ./.github/workflows/templates-upload.yml + with: + version: ${{ inputs.version }} + is_prerelease: true # never touch latest/ from a hotfix + secrets: + S3_ENDPOINT: ${{ secrets.S3_ENDPOINT }} + S3_ACCESS_KEY: ${{ secrets.S3_ACCESS_KEY }} + S3_SECRET_KEY: ${{ secrets.S3_SECRET_KEY }} + S3_BUCKET: ${{ secrets.S3_BUCKET }} +``` + +- [ ] **Step 4: Valider les YAML** + +Run: `for f in .github/workflows/ci.yml .github/workflows/templates-upload.yml .github/workflows/templates-hotfix.yml; do uv run python -c "import yaml,sys; yaml.safe_load(open('$f')); print('ok $f')"; done` + +- [ ] **Step 5: Tester la génération du manifest en local** + +Run: `cd templates && FILES=$(find . -name '*.j2' -type f | sort | while read -r f; do rel="${f#./}"; printf '{"%s":{"sha256":"%s","size":%s}}\n' "$rel" "$(sha256sum "$f" | cut -d' ' -f1)" "$(stat -c%s "$f")"; done | jq -s 'add') && jq -n --arg version 0.0.0 --arg commit x --arg date now --argjson files "$FILES" --argjson engines "$(cat engines.map.json)" '{schema:1, version:$version, generated_at:$date, commit:$commit, files:$files, engines:$engines}' | uv run python -c "import json,sys; from services.templates import Manifest; m = Manifest.from_json(json.load(sys.stdin)); print(len(m.files), 'files,', len(m.engines), 'engines')"; cd ..` +Expected: `9 files, 9 engines`. + +- [ ] **Step 6: Commit** + +```bash +git add .github/workflows/ +git commit -m "ci: render-check job, manifest generation on template upload, hotfix workflow" +``` + +--- + +### Task 9 : PR et release candidate + +- [ ] **Step 1: Lint complet et PR** + +Run: `uv run ruff check . && uv run ruff format --check . && uv run python scripts/render_check.py --no-compose` +Puis : +```bash +git checkout -b refactor/templates-engines +git push -u origin refactor/templates-engines +``` +PR « feat: Jinja2 templates, engine registry, template repository ». Checks attendus verts : `lint`, `test`, `render-check`, `gitleaks`, `plumber`, `build-smoke`. + +- [ ] **Step 2: Release candidate** + +Après merge : Bump version `26.08.0rc2` (ou suivant), channel `rc`. Vérifier sur S3 que `cli/public/templates/26.08.0rc2/` contient `manifest.json`, `agent.yml.j2`, `engines/`, **et** `agent.yml` / `dashboard.yml` legacy. Installer le binaire rc et lancer `portabase agent test-rc` (code legacy) : doit fonctionner comme avant (fetch `agent.yml` sous la version exacte). + +--- + +## Self-review + +**Spec coverage :** +- §5.4 `TemplateRepository` : résolution version/env/dev ✔, cache ✔, manifest sha256+size ✔, suppression fichiers obsolètes ✔, pas de `latest` côté client ✔, Jinja2 `StrictUndefined` ✔, schéma inconnu → `TemplateError` ✔, version ≠ → `TemplateError` ✔. +- §5.6 templates : `agent.yml.j2` avec `mounts`, `docker_socket`, `host_gateway`, `services`, `volumes` ✔ ; moteurs avec `{% if auth %}` ✔ ; `dashboard.yml.j2` avec `db_mode` ✔. +- §6 moteurs : hiérarchie, hooks, registre imports explicites ✔ ; `agent_database` hook ✔ ; Redis/Valkey séparés ✔ ; `describe` pour `db list` (Plan 4). +- §6.1 options : `option_fields`, `non_default_options`, projection ✔ ; parsing `-o` et prompts → Plan 4 (flow). +- §9.1 `render-check` ✔, `engines-check` (fusionné dans le script) ✔. §9.3 manifest ✔, hotfix ✔. +- §10 D : shippable en rc, legacy intact ✔. + +**Placeholders :** aucun. + +**Cohérence :** `DbEngine.template_ctx` produit `name/volume/auth/*_var` = variables consommées par tous les `.j2` ✔ ; `FirebirdEngine.template_ctx` ajoute `root_password_var` consommé par `firebird.yml.j2` ✔ ; `render_check.render_agent` passe `AGENT_GLOBALS` = variables `*_var` de `agent.yml.j2` ✔ ; `Manifest.engines` clé → `TemplateRepository.engine_template` ✔ ; `FixedPortAllocator` défini Task 1, utilisé Task 7 ✔. + +**Écarts connus :** +- `DatabaseSpec.host_port` est `None` pour les specs chargées depuis une install legacy tant que Plan 4 ne lit pas `.env` ; sans effet ici. diff --git a/docs/superpowers/plans/2026-09-11-plan-4-render-commands.md b/docs/superpowers/plans/2026-09-11-plan-4-render-commands.md new file mode 100644 index 0000000..e93ca1c --- /dev/null +++ b/docs/superpowers/plans/2026-09-11-plan-4-render-commands.md @@ -0,0 +1,1935 @@ +# Plan 4 — Rendu déclaratif et commandes agent / dashboard / db / build (chantiers E + F) + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Remplacer le code legacy (`agent.py`, `db.py`, `dashboard.py`, chirurgie texte du compose) par un rendu complet depuis l'état (`.env` + `databases.json` + faits du compose), un flux d'ajout de base partagé, des commandes entièrement pilotables par flags, et la commande `build`. Fin de la refonte : plus aucun `LegacyCommand`, plus de `console` global, plus de fallback `latest`. + +**Architecture:** `AgentProject`/`DashboardProject` chargent l'état depuis le dossier ; `ComposeRenderer` produit `docker-compose.yml` (+ `databases.json`) via Jinja2 ; `RenderResult.write()` valide, sauvegarde un compose legacy en `.legacy.yml`, écrit atomiquement. `AddDatabaseFlow` collecte un `DatabaseSpec` (flags → prompts → défauts) et mute le projet ; `AgentCommand` et `DbAddCommand` l'utilisent tous deux. + +**Tech Stack:** Python 3.12, Typer, Jinja2, PyYAML, questionary/Rich via `ui/`. + +**Spec:** `docs/superpowers/specs/2026-09-11-cli-refactor-design.md` — sections 4.2, 4.3, 5.1–5.3, 5.5, 5.7, 6.1, 7.2 (Summary, DataTable, Diff), 10 (E, F). + +## Global Constraints + +- Prérequis : Plans 1–3 exécutés. +- Règle de dépendance descendante (`commands → services, engines, ui, core` ; `services → engines, core` ; `flows` dans `commands/`). +- `.env` ne contient que des variables consommées par les conteneurs. Aucune clé `PORTABASE_*`. +- `databases.json` conserve exactement le format legacy (projection par `DbEngine.agent_entry`). +- Détection `managed` : `.env` contient `{PREFIX}_PORT` pour le `host` de l'entrée. +- Le compose généré porte l'en-tête `# Generated by Portabase CLI . Do not edit — use docker-compose.override.yml.` ; un compose sans cet en-tête est sauvegardé en `docker-compose.legacy.yml` avant la première réécriture (une seule fois). +- Toute commande mutante : `templates.resolve()` **avant** de muter quoi que ce soit. +- Pas de tests unitaires. Vérifications exécutables par commande, en interactif et en `--non-interactive`, sur une install neuve et sur une install legacy générée avec le binaire 26.07.6. +- Ce plan supprime : `commands/common.py` (déjà), `core/network.py`, `core/docker.py`, `templates/compose.py`, `templates/__init__.py`, `templates/agent.yml`, `templates/dashboard.yml`, `LegacyCommand`, les fonctions legacy de `core/config.py`, `console`/`print_banner`/`HINTS`/`check_system`/`start_docker`/`validate_work_dir`/`get_free_port`/`get_random_hint` de `core/utils.py`, toutes les `per-file-ignores` ruff. + +--- + +## File Structure + +| Fichier | Action | Responsabilité | +|---|---|---| +| `engines/base.py` | modifier | `label_default` | +| `services/envfile.py` | créer | `EnvFile` | +| `services/compose_facts.py` | créer | `ComposeFacts` | +| `services/project.py` | créer | `AgentProject`, `DashboardProject`, `ProjectKind`, `detect_kind`, `spec_from_entry` | +| `services/renderer.py` | créer | `ComposeRenderer`, `RenderResult`, `WriteReport` | +| `services/docker.py` | modifier | `remove_volume` | +| `ui/components/summary.py`, `table.py`, `diff.py` | créer | composants | +| `ui/__init__.py` | modifier | `summary`, `table`, `diff` | +| `commands/flows/__init__.py`, `add_database.py` | créer | `AddDatabaseFlow` | +| `commands/agent.py` | réécrire | `AgentCommand` | +| `commands/dashboard.py` | réécrire | `DashboardCommand` | +| `commands/db.py` | réécrire | `DbCommands` (`add`, `remove`, `list`) | +| `commands/build.py` | créer | `BuildCommand` | +| `commands/decrypt.py` | réécrire | `DecryptCommand` | +| `core/crypto.py` | modifier | `DecryptionError(PortabaseError)` | +| `commands/base.py` | modifier | retirer `LegacyCommand` | +| `main.py` | modifier | câblage final | +| `core/config.py`, `core/utils.py` | modifier | retirer le legacy | +| `scripts/render_check.py` | modifier | utiliser `ComposeRenderer` | +| `.github/workflows/ci.yml` | modifier | `build-smoke` avec `agent --non-interactive` | +| `pyproject.toml` | modifier | retirer `per-file-ignores` | +| `README.md` | modifier | note migration | + +--- + +### Task 1 : `EnvFile` + +**Files:** +- Create: `services/envfile.py` + +**Interfaces:** +- Produces: `EnvFile(path)` : `load() -> EnvFile` (classmethod `EnvFile.load(path)`), `get(key, default=None)`, `set(key, value)`, `merge(mapping)`, `remove(key)`, `remove_prefix(prefix)`, `as_dict() -> dict[str, str]`, `save()`, `exists`. Préserve ordre, commentaires, lignes vides. + +- [ ] **Step 1: Écrire le module** + +```python +"""Dotenv file kept as a list of lines so comments and order survive rewrites. + +Only container runtime variables live here. Values are always written double-quoted. +""" + +from __future__ import annotations + +import os +import re +from collections.abc import Mapping +from dataclasses import dataclass, field +from pathlib import Path + +_LINE = re.compile(r"""^\s*(?:export\s+)?([A-Za-z_][A-Za-z0-9_]*)\s*=\s*(.*?)\s*$""") + + +def _unquote(raw: str) -> str: + if len(raw) >= 2 and raw[0] == raw[-1] and raw[0] in "\"'": + inner = raw[1:-1] + if raw[0] == '"': + return inner.replace('\\"', '"').replace("\\\\", "\\") + return inner + # unquoted: strip trailing comment + return raw.split(" #", 1)[0].rstrip() + + +def _quote(value: str) -> str: + return '"' + value.replace("\\", "\\\\").replace('"', '\\"') + '"' + + +@dataclass +class EnvFile: + path: Path + _lines: list[str] = field(default_factory=list) # raw lines, without newline + _index: dict[str, int] = field(default_factory=dict) # key -> line number + + @classmethod + def load(cls, path: Path) -> EnvFile: + env = cls(path) + if path.exists(): + text = path.read_text(encoding="utf-8") + env._lines = text.splitlines() + for i, line in enumerate(env._lines): + m = _LINE.match(line) + if m and not line.lstrip().startswith("#"): + env._index[m.group(1)] = i + return env + + @property + def exists(self) -> bool: + return self.path.exists() + + def get(self, key: str, default: str | None = None) -> str | None: + i = self._index.get(key) + if i is None: + return default + m = _LINE.match(self._lines[i]) + return _unquote(m.group(2)) if m else default + + def as_dict(self) -> dict[str, str]: + return {k: self.get(k) or "" for k in self._index} + + def set(self, key: str, value: str) -> None: + line = f"{key}={_quote(str(value))}" + i = self._index.get(key) + if i is None: + self._lines.append(line) + self._index[key] = len(self._lines) - 1 + else: + self._lines[i] = line + + def merge(self, mapping: Mapping[str, str]) -> None: + for k, v in mapping.items(): + self.set(k, v) + + def remove(self, key: str) -> None: + i = self._index.pop(key, None) + if i is None: + return + del self._lines[i] + self._index = {k: (n - 1 if n > i else n) for k, n in self._index.items()} + + def remove_prefix(self, prefix: str) -> None: + for key in [k for k in self._index if k.startswith(prefix + "_")]: + self.remove(key) + + def save(self) -> None: + self.path.parent.mkdir(parents=True, exist_ok=True) + tmp = self.path.with_suffix(".env.tmp") + tmp.write_text("\n".join(self._lines) + "\n", encoding="utf-8") + os.replace(tmp, self.path) +``` + +- [ ] **Step 2: Vérifier** + +Run: `uv run python -c " +import tempfile; from pathlib import Path +from services.envfile import EnvFile +p = Path(tempfile.mkdtemp())/'.env' +p.write_text('# header\nTZ=\"UTC\"\nEDGE_KEY=\"a=b\"\n\nDB_PG_A1_PORT=\"5433\"\nDB_PG_A1_PASS=\"p\\\"q\"\nCUSTOM=plain # note\n') +e = EnvFile.load(p); print(e.get('TZ'), e.get('EDGE_KEY'), e.get('DB_PG_A1_PASS'), e.get('CUSTOM')) +e.set('TZ','Europe/Paris'); e.merge({'NEW':'x'}); e.remove_prefix('DB_PG_A1'); e.save() +print(p.read_text())"` +Expected: `UTC a=b p"q plain` puis le fichier avec `# header`, `TZ="Europe/Paris"`, `EDGE_KEY`, ligne vide conservée, `CUSTOM` réécrit tel quel, `NEW="x"` en fin, plus aucune `DB_PG_A1_*`. + +- [ ] **Step 3: Commit** + +```bash +git add services/envfile.py +git commit -m "feat(services): add EnvFile preserving order and comments" +``` + +--- + +### Task 2 : `ComposeFacts`, `project.py`, `label_default` + +**Files:** +- Create: `services/compose_facts.py` +- Create: `services/project.py` +- Modify: `engines/base.py` (ajouter `label_default = "External DB"` ; `DockerVolumeEngine.label_default = "Docker Volume"` dans `engines/docker_volume.py`) + +**Interfaces:** +- Produces: + - `ComposeFacts(path)` : `exists`, `is_generated` (en-tête présent), `host_gateway -> bool`, `raw -> dict`. + - `ProjectKind = Literal["agent", "dashboard"]`, `detect_kind(path) -> ProjectKind` (`ConfigError` sinon). + - `spec_from_entry(entry: dict, env: EnvFile) -> DatabaseSpec`. + - `AgentProject(path, env, databases, host_gateway)` : `load(path)`, `create(path, env_vars: dict, host_gateway)`, `managed`, `needs_docker_socket`, `sqlite_mounts`, `add(spec, engine)`, `remove(spec, engine)`, `find(id_or_name) -> DatabaseSpec`, `save_state()` (écrit `.env` seulement ; `databases.json` est rendu). + - `DashboardProject(path, env)` : `load(path)`, `create(path, env_vars)`, `db_mode`, `save_state()`. + +- [ ] **Step 1: `services/compose_facts.py`** + +```python +"""Read-only structural facts from an existing docker-compose.yml. Never writes.""" + +from __future__ import annotations + +from pathlib import Path + +import yaml + +GENERATED_MARKER = "# Generated by Portabase CLI" + + +class ComposeFacts: + def __init__(self, path: Path) -> None: + self.path = path + self.raw: dict = {} + self.text = "" + if path.exists(): + try: + self.text = path.read_text(encoding="utf-8") + loaded = yaml.safe_load(self.text) + self.raw = loaded if isinstance(loaded, dict) else {} + except (OSError, yaml.YAMLError): + self.raw = {} + + @property + def exists(self) -> bool: + return self.path.exists() + + @property + def is_generated(self) -> bool: + return self.text.startswith(GENERATED_MARKER) + + def _service(self, name: str) -> dict: + services = self.raw.get("services") or {} + svc = services.get(name) if isinstance(services, dict) else None + return svc if isinstance(svc, dict) else {} + + @property + def host_gateway(self) -> bool: + extra = self._service("agent").get("extra_hosts") + if isinstance(extra, list): + return any("host-gateway" in str(x) for x in extra) + if isinstance(extra, dict): + return any("host-gateway" in str(v) for v in extra.values()) + return False +``` + +- [ ] **Step 2: `services/project.py`** + +```python +"""Project state loaded from .env + databases.json + compose facts. Nothing else is stored.""" + +from __future__ import annotations + +import json +from dataclasses import dataclass, field +from pathlib import Path +from typing import Any, Literal + +from core.errors import ConfigError, ValidationError +from core.specs import DatabaseSpec +from engines.base import DbEngine +from engines.sqlite import SqliteEngine +from services.compose_facts import ComposeFacts +from services.envfile import EnvFile + +ProjectKind = Literal["agent", "dashboard"] +DATABASES_FILE = "databases.json" +COMPOSE_FILE = "docker-compose.yml" +ENV_FILE = ".env" + + +def detect_kind(path: Path) -> ProjectKind: + if (path / DATABASES_FILE).exists(): + return "agent" + env = EnvFile.load(path / ENV_FILE) + if env.get("PROJECT_SECRET") is not None: + return "dashboard" + raise ConfigError( + f"{path} is not a Portabase agent or dashboard folder.", + hint="Expected databases.json (agent) or a .env with PROJECT_SECRET (dashboard).", + ) + + +def spec_from_entry(entry: dict[str, Any], env: EnvFile) -> DatabaseSpec: + engine = str(entry.get("type", "")) + host = entry.get("host") + managed, host_port, root_password = False, None, None + if host: + prefix = str(host).upper().replace("-", "_") + raw_port = env.get(f"{prefix}_PORT") + if raw_port and raw_port.isdigit(): + managed, host_port = True, int(raw_port) + root_password = env.get(f"{prefix}_ROOT_PASS") + return DatabaseSpec( + id=str(entry.get("generated_id") or DbEngine.new_id()), + engine=engine, + name=str(entry.get("name", "")), + managed=managed, + host=str(host) if host else None, + port=int(entry["port"]) if entry.get("port") not in (None, "") else None, + host_port=host_port, + database=str(entry["database"]) if entry.get("database") is not None else None, + username=str(entry["username"]) if entry.get("username") is not None else None, + password=str(entry["password"]) if entry.get("password") not in (None, "") else None, + root_password=root_password, + path=str(entry["database"]) if engine == "sqlite" and entry.get("database") else None, + volume=str(entry["volume_name"]) if entry.get("volume_name") else None, + container=str(entry["container_name"]) if entry.get("container_name") else None, + options=dict(entry.get("options") or {}), + ) + + +@dataclass +class AgentProject: + path: Path + env: EnvFile + databases: list[DatabaseSpec] = field(default_factory=list) + host_gateway: bool = False + + # ---- construction ----------------------------------------------------- + + @classmethod + def load(cls, path: Path) -> AgentProject: + path = path.resolve() + env_path, db_path = path / ENV_FILE, path / DATABASES_FILE + if not env_path.exists() or not db_path.exists(): + raise ConfigError( + f"Not a Portabase agent folder: {path}", + hint=f"Expected {ENV_FILE} and {DATABASES_FILE}.", + ) + env = EnvFile.load(env_path) + try: + data = json.loads(db_path.read_text(encoding="utf-8")) + except (OSError, ValueError) as e: + raise ConfigError(f"{db_path} is not valid JSON.", cause=e) from e + entries = data.get("databases", []) if isinstance(data, dict) else [] + databases = [spec_from_entry(e, env) for e in entries if isinstance(e, dict)] + project = cls(path, env, databases, ComposeFacts(path / COMPOSE_FILE).host_gateway) + project.validate() + return project + + @classmethod + def create(cls, path: Path, env_vars: dict[str, str], *, host_gateway: bool) -> AgentProject: + path.mkdir(parents=True, exist_ok=True) + env = EnvFile.load(path / ENV_FILE) + env.merge(env_vars) + return cls(path, env, [], host_gateway) + + # ---- derived facts ---------------------------------------------------- + + @property + def managed(self) -> list[DatabaseSpec]: + return [d for d in self.databases if d.managed] + + @property + def needs_docker_socket(self) -> bool: + return any(d.engine == "docker-volume" for d in self.databases) + + @property + def sqlite_mounts(self) -> list[tuple[str, str]]: + mounts = [] + for d in self.databases: + if d.engine == "sqlite": + m = SqliteEngine.mount_for(d) + if m and m not in mounts: + mounts.append(m) + return mounts + + def validate(self) -> None: + seen: set[str] = set() + for d in self.managed: + if d.host in seen: + raise ConfigError(f"Two managed databases share the service name '{d.host}'.") + seen.add(d.host or "") + + # ---- mutation --------------------------------------------------------- + + def add(self, spec: DatabaseSpec, engine: DbEngine) -> None: + if spec.managed: + self.env.merge(engine.env_vars(spec)) + self.databases.append(spec) + self.validate() + + def remove(self, spec: DatabaseSpec, engine: DbEngine) -> None: + self.databases = [d for d in self.databases if d.id != spec.id] + if spec.managed and spec.host: + self.env.remove_prefix(spec.env_prefix) + + def find(self, id_or_name: str) -> DatabaseSpec: + matches = [d for d in self.databases if d.id == id_or_name or d.id.startswith(id_or_name) or d.name == id_or_name] + if not matches: + raise ValidationError(f"No database matching '{id_or_name}'.", hint="See: portabase db list") + if len(matches) > 1: + raise ValidationError(f"'{id_or_name}' matches several databases; use the id.") + return matches[0] + + def save_state(self) -> None: + self.env.save() + + +@dataclass +class DashboardProject: + path: Path + env: EnvFile + + @classmethod + def load(cls, path: Path) -> DashboardProject: + path = path.resolve() + env = EnvFile.load(path / ENV_FILE) + if env.get("PROJECT_SECRET") is None: + raise ConfigError(f"Not a Portabase dashboard folder: {path}", hint="Expected a .env with PROJECT_SECRET.") + return cls(path, env) + + @classmethod + def create(cls, path: Path, env_vars: dict[str, str]) -> DashboardProject: + path.mkdir(parents=True, exist_ok=True) + env = EnvFile.load(path / ENV_FILE) + env.merge(env_vars) + return cls(path, env) + + @property + def db_mode(self) -> Literal["external", "internal", "custom"]: + host = self.env.get("POSTGRES_HOST") + if host is None: + return "internal" + return "external" if host == "db" else "custom" + + @property + def project_name(self) -> str: + return self.env.get("PROJECT_NAME") or self.path.name + + def save_state(self) -> None: + self.env.save() +``` + +- [ ] **Step 3: `label_default`** + +Dans `engines/base.py`, après `has_modes: bool = True` : `label_default: str = "External DB"`. Dans `engines/docker_volume.py`, après `has_modes = False` : `label_default = "Docker Volume"`. Remplacer dans `from_existing` de `base.py` et `sqlite.py` `or "External DB"` par `or self.label_default`, et dans `docker_volume.py` `or "Docker Volume"` par `or self.label_default`. + +- [ ] **Step 4: Vérifier sur une install legacy réelle** + +Générer une install avec le binaire 26.07.6 (télécharger depuis la release GitHub) ou avec `git stash`/checkout du tag : + +```bash +cd /tmp && rm -rf legacy-agent && git -C /home/soluce/Documents/PROJETS/Portabase/cli stash -u -q; git -C /home/soluce/Documents/PROJETS/Portabase/cli checkout -q 26.07.6 +uv --directory /home/soluce/Documents/PROJETS/Portabase/cli run python /home/soluce/Documents/PROJETS/Portabase/cli/main.py agent legacy-agent --key "$(printf '{"serverUrl":"http://x","agentId":"a","masterKeyB64":"k"}' | base64 -w0)" +``` +Au wizard : tz `UTC`, polling `5`, extra_hosts `y`, puis `database` → `new` → `postgresql` (ownership `n`, clean `clean`), puis `database` → `new` → `redis` → `with-auth`, puis `database` → `existing` → `sqlite` → `Display` / path `ext.sqlite`, puis `docker-volume` (`Vol`, `myvol`, container vide), puis `done`, ne pas démarrer. + +```bash +git -C /home/soluce/Documents/PROJETS/Portabase/cli checkout -q main; git -C /home/soluce/Documents/PROJETS/Portabase/cli stash pop -q +uv --directory /home/soluce/Documents/PROJETS/Portabase/cli run python -c " +from pathlib import Path +from services.project import AgentProject, detect_kind +p = AgentProject.load(Path('/tmp/legacy-agent')) +print(detect_kind(p.path), 'gateway:', p.host_gateway, 'socket:', p.needs_docker_socket, 'mounts:', p.sqlite_mounts) +for d in p.databases: print(f'{d.engine:14} managed={d.managed!s:5} host={d.host} host_port={d.host_port} db={d.database} opts={d.options}')" +``` +Expected: `agent gateway: True socket: True mounts: [('./ext.sqlite', '/config/ext.sqlite')]` ; postgresql `managed=True host=db-pg-xxxx host_port=` ; redis `managed=True host=db-redis-auth-xxxx` ; sqlite `managed=False` ; docker-volume `managed=False`. + +- [ ] **Step 5: Commit** + +```bash +git add services/compose_facts.py services/project.py engines/base.py engines/sqlite.py engines/docker_volume.py +git commit -m "feat(services): add ComposeFacts and project state loaded from .env, databases.json and compose" +``` + +--- + +### Task 3 : `ComposeRenderer` et `RenderResult` + +**Files:** +- Create: `services/renderer.py` +- Modify: `services/docker.py` (ajouter `remove_volume`) + +**Interfaces:** +- Consumes: `TemplateRepository`, `EngineRegistry`, `AgentProject`, `DashboardProject`, `SqliteEngine.mount_for`. +- Produces: + - `ComposeRenderer(templates, engines, cli_version)` : `render_agent(project, *, inline=False) -> RenderResult`, `render_dashboard(project, *, inline=False) -> RenderResult`. + - `RenderResult(compose: str, databases: list[dict] | None)` : `validate()` (`TemplateError`), `write(path) -> WriteReport`, `diff_against(path) -> str`. + - `WriteReport(backed_up: Path | None, wrote: list[Path])`. + - `DockerRunner.remove_volume(name) -> None`. + +- [ ] **Step 1: `services/renderer.py`** + +```python +"""State → docker-compose.yml (+ databases.json). The only writer of those files.""" + +from __future__ import annotations + +import difflib +import json +import os +import shutil +from dataclasses import dataclass, field +from pathlib import Path +from typing import Any + +import jinja2 +import yaml + +from core.errors import TemplateError +from core.specs import DatabaseSpec +from engines.registry import EngineRegistry +from services.compose_facts import GENERATED_MARKER, ComposeFacts +from services.envfile import EnvFile +from services.project import COMPOSE_FILE, DATABASES_FILE, AgentProject, DashboardProject +from services.templates import TemplateRepository + +LEGACY_BACKUP = "docker-compose.legacy.yml" + + +@dataclass +class WriteReport: + backed_up: Path | None = None + wrote: list[Path] = field(default_factory=list) + + +@dataclass +class RenderResult: + compose: str + databases: list[dict[str, Any]] | None = None + + def validate(self) -> None: + try: + doc = yaml.safe_load(self.compose) + except yaml.YAMLError as e: + raise TemplateError("Rendered compose is not valid YAML; templates are broken.", cause=e) from e + if not isinstance(doc, dict) or "services" not in doc: + raise TemplateError("Rendered compose has no 'services' section; templates are broken.") + + def write(self, path: Path) -> WriteReport: + self.validate() + report = WriteReport() + compose_path = path / COMPOSE_FILE + facts = ComposeFacts(compose_path) + if facts.exists and not facts.is_generated: + backup = path / LEGACY_BACKUP + if not backup.exists(): + shutil.copy2(compose_path, backup) + report.backed_up = backup + _atomic_write(compose_path, self.compose) + report.wrote.append(compose_path) + if self.databases is not None: + db_path = path / DATABASES_FILE + _atomic_write(db_path, json.dumps({"databases": self.databases}, indent=2) + "\n") + try: + os.chmod(db_path, 0o666) # agent container may run as another uid (legacy behaviour) + except OSError: + pass + report.wrote.append(db_path) + return report + + def diff_against(self, path: Path) -> str: + current = (path / COMPOSE_FILE).read_text(encoding="utf-8") if (path / COMPOSE_FILE).exists() else "" + return "".join( + difflib.unified_diff( + current.splitlines(keepends=True), + self.compose.splitlines(keepends=True), + fromfile=f"{COMPOSE_FILE} (current)", + tofile=f"{COMPOSE_FILE} (rendered)", + ) + ) + + +def _atomic_write(path: Path, content: str) -> None: + path.parent.mkdir(parents=True, exist_ok=True) + tmp = path.with_suffix(path.suffix + ".tmp") + tmp.write_text(content, encoding="utf-8") + os.replace(tmp, path) + + +class ComposeRenderer: + def __init__(self, templates: TemplateRepository, engines: EngineRegistry, cli_version: str) -> None: + self.templates = templates + self.engines = engines + self.cli_version = cli_version + + def header(self) -> str: + return f"{GENERATED_MARKER} {self.cli_version}. Do not edit — use docker-compose.override.yml.\n" + + # ---- agent ------------------------------------------------------------ + + def render_agent(self, project: AgentProject, *, inline: bool = False) -> RenderResult: + env = project.env + ctx = { + "host_gateway": project.host_gateway, + "docker_socket": project.needs_docker_socket, + "mounts": [{"host": h, "container": c} for h, c in project.sqlite_mounts], + "services": [self._service(spec, inline) for spec in project.managed], + "tz_var": _var(env, "TZ", inline), + "edge_key_var": _var(env, "EDGE_KEY", inline), + "log_level_var": _var(env, "LOG_LEVEL", inline), + "polling_var": _var(env, "POLLING", inline), + } + compose = self.header() + self._render("agent.yml.j2", ctx) + databases = [self.engines.get(d.engine).agent_entry(d) for d in project.databases] + return RenderResult(compose=compose, databases=databases) + + def _service(self, spec: DatabaseSpec, inline: bool) -> dict[str, str]: + engine = self.engines.get(spec.engine) + body = self._render_template(self.templates.engine_template(spec.engine), engine.template_ctx(spec, inline=inline)) + return {"name": spec.host or "", "volume": f"{spec.host}-data", "body": body} + + # ---- dashboard -------------------------------------------------------- + + def render_dashboard(self, project: DashboardProject, *, inline: bool = False) -> RenderResult: + env = project.env + ctx = { + "db_mode": project.db_mode, + "project_name_var": project.project_name, # literal, as the legacy CLI did + "host_port_var": _var(env, "HOST_PORT", inline), + "tz_var": _var(env, "TZ", inline), + "log_level_var": _var(env, "LOG_LEVEL", inline), + "project_secret_var": _var(env, "PROJECT_SECRET", inline), + "project_url_var": _var(env, "PROJECT_URL", inline), + "pg_port_var": _var(env, "PG_PORT", inline), + "postgres_db_var": _var(env, "POSTGRES_DB", inline), + "postgres_user_var": _var(env, "POSTGRES_USER", inline), + "postgres_password_var": _var(env, "POSTGRES_PASSWORD", inline), + } + return RenderResult(compose=self.header() + self._render("dashboard.yml.j2", ctx), databases=None) + + # ---- internals -------------------------------------------------------- + + def _render(self, name: str, ctx: dict[str, Any]) -> str: + return self._render_template(self.templates.get(name), ctx) + + @staticmethod + def _render_template(template: jinja2.Template, ctx: dict[str, Any]) -> str: + try: + return template.render(**ctx) + except jinja2.TemplateError as e: + raise TemplateError(f"Template rendering failed: {e}", cause=e) from e + + +def _var(env: EnvFile, key: str, inline: bool) -> str: + return (env.get(key) or "") if inline else f"${{{key}}}" +``` + +- [ ] **Step 2: `DockerRunner.remove_volume`** (dans `services/docker.py`, après `ensure_network`) + +```python + def remove_volume(self, name: str) -> bool: + """True if removed, False if it did not exist. Raises on other failures.""" + proc = subprocess.run([self.binary, "volume", "rm", name], capture_output=True, text=True, check=False) + if proc.returncode == 0: + return True + if "no such volume" in (proc.stderr or "").lower(): + return False + raise DockerError(f"Could not remove volume '{name}': {proc.stderr.strip()}") +``` + +- [ ] **Step 3: Vérifier le rendu sur l'install legacy et le `--diff`** + +Run: `uv run python -c " +from pathlib import Path +from services.project import AgentProject +from services.renderer import ComposeRenderer +from services.templates import TemplateRepository +from services.http import HttpClient +from core.config import GlobalConfig +from engines import registry +import yaml +repo = TemplateRepository.from_environment(HttpClient(), GlobalConfig()) +r = ComposeRenderer(repo, registry, '0.0.0-dev') +p = AgentProject.load(Path('/tmp/legacy-agent')) +res = r.render_agent(p); res.validate() +doc = yaml.safe_load(res.compose) +print(sorted(doc['services']), doc['services']['agent']['volumes'], doc['services']['agent'].get('extra_hosts')) +print(res.diff_against(p.path)[:1200]) +print(len(res.databases), [d['type'] for d in res.databases])"` +Expected: services = `agent` + le service postgres + le service redis (mêmes noms que le compose legacy) ; volumes agent = `databases.json`, `./ext.sqlite:/config/ext.sqlite`, socket ; `extra_hosts` présent ; diff limité à l'en-tête, l'ordre des lignes, `restart: unless-stopped` sur redis ; `4 ['postgresql', 'redis', 'sqlite', 'docker-volume']`. + +Comparer aussi `res.databases` à `/tmp/legacy-agent/databases.json` : mêmes clés et valeurs par entrée (à l'ordre des clés près). + +- [ ] **Step 4: Commit** + +```bash +git add services/renderer.py services/docker.py +git commit -m "feat(services): add ComposeRenderer with validation, atomic write and legacy backup" +``` + +--- + +### Task 4 : Composants `Summary`, `DataTable`, `Diff` + +**Files:** +- Create: `ui/components/summary.py`, `ui/components/table.py`, `ui/components/diff.py` +- Modify: `ui/__init__.py` + +**Interfaces:** +- Produces: `UI.summary(rows: list[tuple[str, str]], *, title: str | None = None)`, `UI.table(columns: list[str], rows: list[list[str]], *, title: str | None = None)`, `UI.diff(text: str)`. + +- [ ] **Step 1: `ui/components/summary.py`** + +```python +from __future__ import annotations + +import re + +from rich.panel import Panel +from rich.table import Table + +from ui.components.base import Component + +_SENSITIVE = re.compile(r"(password|secret|key|token)", re.I) +_URL_CREDS = re.compile(r"://([^:/@]+):([^@/]+)@") + + +def mask(label: str, value: str) -> str: + if _SENSITIVE.search(label): + return "••••••••" + return _URL_CREDS.sub(r"://\1:****@", value) + + +class Summary(Component): + def __call__(self, rows: list[tuple[str, str]], *, title: str | None = None) -> None: + table = Table(show_header=False, box=None, padding=(0, 2)) + table.add_column("Property", style="bold cyan") + table.add_column("Value", style="white") + for label, value in rows: + table.add_row(label, mask(label, str(value))) + self.console.print("") + self.console.print(Panel(table, title=f"[bold white]{title}[/bold white]" if title else None, border_style="bold blue", expand=False)) +``` + +- [ ] **Step 2: `ui/components/table.py`** + +```python +from __future__ import annotations + +from rich.table import Table + +from ui.components.base import Component + +_STYLES = ["cyan", "blue", "magenta", "green", "white", "dim"] + + +class DataTable(Component): + def __call__(self, columns: list[str], rows: list[list[str]], *, title: str | None = None) -> None: + table = Table(title=title) + for i, col in enumerate(columns): + table.add_column(col, style=_STYLES[i % len(_STYLES)]) + for row in rows: + table.add_row(*[str(c) for c in row]) + self.console.print(table) +``` + +- [ ] **Step 3: `ui/components/diff.py`** + +```python +from __future__ import annotations + +from rich.syntax import Syntax + +from ui.components.base import Component + + +class Diff(Component): + def __call__(self, text: str) -> None: + if not text.strip(): + self.console.print("[info]ℹ No changes.[/info]") + return + self.console.print(Syntax(text, "diff", theme="ansi_dark", word_wrap=False)) +``` + +- [ ] **Step 4: Façade** — ajouter à `ui/__init__.py` les imports et méthodes : + +```python +from ui.components.diff import Diff +from ui.components.summary import Summary +from ui.components.table import DataTable + + def summary(self, rows: list[tuple[str, str]], *, title: str | None = None) -> None: + Summary(self.console)(rows, title=title) + + def table(self, columns: list[str], rows: list[list[str]], *, title: str | None = None) -> None: + DataTable(self.console)(columns, rows, title=title) + + def diff(self, text: str) -> None: + Diff(self.console)(text) +``` + +- [ ] **Step 5: Vérifier** + +Run: `uv run python -c " +from ui import UI +ui = UI() +ui.summary([('Name','x'),('Password','hunter2'),('Connection URL','postgresql://u:p@h:5432/d')], title='PROPOSED') +ui.table(['A','B'], [['1','2']], title='T') +ui.diff('--- a\n+++ b\n@@ -1 +1 @@\n-old\n+new\n'); ui.diff('')"` +Expected: panneau avec `••••••••` et `:****@`, table, diff colorisé, `ℹ No changes.`. + +- [ ] **Step 6: Commit** + +```bash +git add ui/ +git commit -m "feat(ui): add Summary, DataTable and Diff components" +``` + +--- + +### Task 5 : `AddDatabaseFlow` + +**Files:** +- Create: `commands/flows/__init__.py` (vide) +- Create: `commands/flows/add_database.py` + +**Interfaces:** +- Consumes: `UI`, `EngineRegistry`, `PortAllocator`, `Form`, `Field`, `DatabaseSpec`, `AgentProject`. +- Produces: `AddDatabaseFlow(ui, engines, ports)` : `collect(values: dict) -> tuple[DatabaseSpec, DbEngine]`, `apply(project, spec, engine) -> None`, `parse_options(items: list[str]) -> dict[str, str]` (static). + +- [ ] **Step 1: Écrire le module** + +```python +"""Shared 'add a database' wizard. Flags fill `values`; anything missing is prompted or errors.""" + +from __future__ import annotations + +from typing import Any + +from core.errors import ValidationError +from core.fields import Field +from core.specs import DatabaseSpec +from engines.base import DbEngine +from engines.registry import EngineRegistry +from services.ports import PortAllocator +from services.project import AgentProject +from ui import UI + +FLOW_KEYS = {"engine", "mode", "auth", "label", "options"} + + +class AddDatabaseFlow: + def __init__(self, ui: UI, engines: EngineRegistry, ports: PortAllocator) -> None: + self.ui = ui + self.engines = engines + self.ports = ports + + # ---- public ----------------------------------------------------------- + + @staticmethod + def parse_options(items: list[str] | None) -> dict[str, str]: + out: dict[str, str] = {} + for item in items or []: + if "=" not in item: + raise ValidationError(f"Invalid option '{item}'.", hint="Use -o KEY=VALUE") + key, value = item.split("=", 1) + out[key.strip()] = value.strip() + return out + + def collect(self, values: dict[str, Any]) -> tuple[DatabaseSpec, DbEngine]: + form = self.ui.form() + engine = self.engines.get( + form.choice("Select Database Engine", self.engines.choices(), value=values.get("engine"), name="engine") + ) + if engine.warning: + self.ui.warning(engine.warning) + + mode = "new" + if engine.has_modes: + mode = form.choice("Configuration Mode", ["new", "existing"], value=values.get("mode"), default="new", name="mode") + + auth = True + if mode == "new" and engine.auth_variants: + raw = values.get("auth") + if raw is None: + auth = form.choice("Variant", ["with-auth", "no-auth"], name="auth") == "with-auth" + else: + auth = bool(raw) + + fields = list(engine.fields_new() if mode == "new" else engine.fields_existing()) + if mode == "existing" or not engine.has_modes: + fields.insert(0, Field("label", "Display Name", "text", default=engine.label_default)) + + self._reject_irrelevant(values, fields, engine, mode) + + if mode == "existing": + self.ui.info(f"{engine.display} — existing database") + answers = form.collect(fields, values) + answers["options"] = self._collect_options(form, engine, values.get("options") or {}) + + if mode == "new": + spec = engine.generate(auth=auth, ports=self.ports, answers=answers) + else: + spec = engine.from_existing(answers) + return spec.with_options(answers["options"]), engine + + def apply(self, project: AgentProject, spec: DatabaseSpec, engine: DbEngine) -> None: + project.add(spec, engine) + + # ---- internals -------------------------------------------------------- + + def _collect_options(self, form, engine: DbEngine, provided: dict[str, str]) -> dict[str, Any]: + option_fields = engine.option_fields() + known = {f.name for f in option_fields} + unknown = set(provided) - known + if unknown: + raise ValidationError( + f"Unknown option(s) for {engine.key}: {', '.join(sorted(unknown))}.", + hint=("Valid options: " + ", ".join(sorted(known))) if known else f"{engine.key} has no options.", + ) + if not option_fields: + return {} + return form.collect(option_fields, provided) + + @staticmethod + def _reject_irrelevant(values: dict[str, Any], fields: list[Field], engine: DbEngine, mode: str) -> None: + relevant = {f.name for f in fields} | FLOW_KEYS + extra = sorted(k for k, v in values.items() if v is not None and k not in relevant) + if extra: + raise ValidationError( + f"Option(s) not applicable to {engine.key} in '{mode}' mode: {', '.join('--' + k.replace('_', '-') for k in extra)}.", + hint="Applicable: " + ", ".join(f"--{f.name.replace('_', '-')}" for f in fields) if fields else "No extra input needed.", + ) +``` + +- [ ] **Step 2: Vérifier en non-interactif** + +Run: `uv run python -c " +from ui import UI +from engines import registry +from services.ports import FixedPortAllocator +from commands.flows.add_database import AddDatabaseFlow +from core.errors import ValidationError +f = AddDatabaseFlow(UI(non_interactive=True), registry, FixedPortAllocator()) +s, e = f.collect({'engine':'postgresql','mode':'new','options': f.parse_options(['clean_mode=none'])}); print(e.key, s.managed, s.options) +s, e = f.collect({'engine':'redis','mode':'new','auth':False}); print(s.host[:9], s.auth) +s, e = f.collect({'engine':'sqlite','mode':'existing','path':'x.sqlite'}); print(s.name, s.database) +s, e = f.collect({'engine':'docker-volume','volume':'v'}); print(s.name, s.volume, s.container) +for bad in ({'engine':'postgresql','mode':'existing'}, {'engine':'redis','mode':'new','host':'h'}, {'engine':'mysql','mode':'new','options':{'clean_mode':'x'}}, {'engine':'nope'}): + try: f.collect(bad) + except ValidationError as err: print('ERR', err.message)"` +Expected: +``` +postgresql True {'clean_mode': 'none', 'keep_ownership': False} +db-redis- False +External DB /config/x.sqlite +Docker Volume v None +ERR Missing --host +ERR Option(s) not applicable to redis in 'new' mode: --host. +ERR Unknown option(s) for mysql: clean_mode. +ERR Unknown engine 'nope'. +``` + +- [ ] **Step 3: Commit** + +```bash +git add commands/flows/ +git commit -m "feat(commands): add AddDatabaseFlow shared by agent and db add" +``` + +--- + +### Task 6 : `commands/db.py` — `add`, `remove`, `list` + +**Files:** +- Modify: `commands/db.py` (réécriture complète) + +**Interfaces:** +- Produces: `DbCommands(ui, telemetry, engines, ports, templates, renderer, docker)` groupe `db` avec `DbAddCommand`, `DbRemoveCommand`, `DbListCommand`. + +- [ ] **Step 1: Réécrire le module** + +```python +"""db add / remove / list.""" + +from __future__ import annotations + +from pathlib import Path +from typing import Annotated + +import typer + +from commands.base import Command, CommandGroup +from commands.flows.add_database import AddDatabaseFlow +from core.errors import ValidationError +from engines.registry import EngineRegistry +from services.docker import DockerRunner +from services.ports import PortAllocator +from services.project import AgentProject +from services.renderer import ComposeRenderer, WriteReport +from services.telemetry import Telemetry +from services.templates import TemplateRepository +from ui import UI + +NameArg = Annotated[Path, typer.Argument(help="Agent folder")] + + +def report_write(ui: UI, report: WriteReport) -> None: + if report.backed_up: + ui.warning(f"Legacy compose backed up to {report.backed_up.name}. Manual edits belong in docker-compose.override.yml.") + + +class _DbCommand(Command): + panel = "Configuration" + no_args_is_help = True + + def __init__(self, ui, telemetry, engines: EngineRegistry, ports: PortAllocator, templates: TemplateRepository, renderer: ComposeRenderer, docker: DockerRunner) -> None: + super().__init__(ui, telemetry) + self.engines, self.ports, self.templates, self.renderer, self.docker = engines, ports, templates, renderer, docker + + def render_and_write(self, project: AgentProject) -> None: + with self.ui.status("Rendering configuration..."): + result = self.renderer.render_agent(project) + project.save_state() + report = result.write(project.path) + report_write(self.ui, report) + + +class DbAddCommand(_DbCommand): + name, help = "add", "Add a database to an agent." + + def run( + self, + name: NameArg, + engine: Annotated[str | None, typer.Option("--engine", "-e", help="Database engine")] = None, + mode: Annotated[str | None, typer.Option("--mode", help="new (container) or existing")] = None, + auth: Annotated[bool | None, typer.Option("--auth/--no-auth", help="Auth variant for mongodb/redis/valkey")] = None, + label: Annotated[str | None, typer.Option("--label", help="Display name")] = None, + host: Annotated[str | None, typer.Option("--host")] = None, + port: Annotated[int | None, typer.Option("--port")] = None, + database: Annotated[str | None, typer.Option("--database")] = None, + user: Annotated[str | None, typer.Option("--user")] = None, + password: Annotated[str | None, typer.Option("--password", help="Prefer --password-stdin")] = None, + password_stdin: Annotated[bool, typer.Option("--password-stdin", help="Read password from stdin")] = False, + path: Annotated[str | None, typer.Option("--path", help="SQLite file path (existing)")] = None, + db_name: Annotated[str | None, typer.Option("--name", help="SQLite file name (new)")] = None, + volume: Annotated[str | None, typer.Option("--volume", help="Docker volume name")] = None, + container: Annotated[str | None, typer.Option("--container", help="Container to restart after restore")] = None, + option: Annotated[list[str] | None, typer.Option("--option", "-o", help="Engine option KEY=VALUE (repeatable)")] = None, + ) -> None: + if password_stdin: + import sys + + password = sys.stdin.readline().rstrip("\n") + elif password is not None: + self.ui.warning("--password is visible in shell history; prefer --password-stdin.") + + project_path = self.require_project_dir(name) + self.templates.resolve() + project = AgentProject.load(project_path) + + flow = AddDatabaseFlow(self.ui, self.engines, self.ports) + values = { + "engine": engine, "mode": mode, "auth": auth, "label": label, "host": host, "port": port, + "database": database, "username": user, "password": password, "path": path, "name": db_name, + "volume": volume, "container": container, "options": flow.parse_options(option), + } + spec, eng = flow.collect(values) + flow.apply(project, spec, eng) + self.render_and_write(project) + + self.ui.success(f"Added {eng.display} database '{spec.name}' ({eng.describe(spec)}).") + self.ui.info(f"Restart the agent to apply changes: portabase restart {project_path.name}") + + +class DbRemoveCommand(_DbCommand): + name, help = "remove", "Remove a database from an agent." + + def run( + self, + name: NameArg, + target: Annotated[str | None, typer.Option("--id", "--name", "-i", help="Database id (or prefix) or display name")] = None, + purge_volume: Annotated[bool, typer.Option("--purge-volume", help="Also delete the Docker volume of a managed database")] = False, + yes: Annotated[bool, typer.Option("--yes", "-y", help="Skip confirmation")] = False, + ) -> None: + project_path = self.require_project_dir(name) + self.templates.resolve() + project = AgentProject.load(project_path) + if not project.databases: + self.ui.warning("No databases to remove.") + return + + if target is None: + choices = [f"{d.name} ({d.engine}) [{d.id[:8]}]" for d in project.databases] + picked = self.ui.form().choice("Which database to remove?", choices, name="id") + spec = project.databases[choices.index(picked)] + else: + spec = project.find(target) + engine = self.engines.get(spec.engine) + + if not yes: + extra = " and its Docker volume" if (purge_volume and spec.managed) else "" + self.confirm_or_abort(f"Remove '{spec.name}' ({engine.describe(spec)}){extra}?", default=False) + + project.remove(spec, engine) + self.render_and_write(project) + self.ui.success(f"Removed {spec.name}") + + if spec.managed: + volume_name = f"{self.docker.project_name(project_path)}_{spec.host}-data" + if purge_volume: + self.require_docker(self.docker) + removed = self.docker.remove_volume(volume_name) + self.ui.success(f"Deleted volume {volume_name}" if removed else f"Volume {volume_name} did not exist") + else: + self.ui.info(f"Data volume kept: {volume_name}. Delete it with: docker volume rm {volume_name}") + self.ui.info(f"Restart the agent to apply changes: portabase restart {project_path.name}") + + +class DbListCommand(_DbCommand): + name, help = "list", "List the databases of an agent." + + def run(self, name: NameArg) -> None: + project = AgentProject.load(self.require_project_dir(name)) + if not project.databases: + self.ui.warning("No databases configured.") + return + rows = [] + for d in project.databases: + engine = self.engines.get(d.engine) + opts = ", ".join(f"{k}={v}" for k, v in engine.non_default_options(d).items()) + rows.append([d.name, d.database or "", d.engine, engine.describe(d), d.username or "" if d.engine not in ("sqlite", "docker-volume") else "N/A", opts, d.id[:8] + "..."]) + self.ui.table(["Display Name", "Database", "Type", "Host:Port", "User", "Options", "ID"], rows, title=f"Databases for {project.path.name}") + + +class DbCommands(CommandGroup): + name, help, panel = "db", "Manage the databases of an agent.", "Configuration" + + def __init__(self, ui: UI, telemetry: Telemetry, engines: EngineRegistry, ports: PortAllocator, templates: TemplateRepository, renderer: ComposeRenderer, docker: DockerRunner) -> None: + super().__init__(ui, telemetry) + self._deps = (ui, telemetry, engines, ports, templates, renderer, docker) + + @property + def commands(self) -> list[Command]: + return [DbAddCommand(*self._deps), DbRemoveCommand(*self._deps), DbListCommand(*self._deps)] +``` + +Note : `sys` importé localement dans `run` pour `--password-stdin` ; déplacer en tête de module si ruff le demande. + +- [ ] **Step 2: Commit** (vérification à Task 9, une fois `main.py` câblé) + +```bash +git add commands/db.py +git commit -m "feat(commands): rewrite db add/remove/list on the declarative renderer" +``` + +--- + +### Task 7 : `commands/agent.py` et `commands/dashboard.py` + +**Files:** +- Modify: `commands/agent.py` (réécriture complète) +- Modify: `commands/dashboard.py` (réécriture complète) + +**Interfaces:** +- Produces: `AgentCommand(ui, telemetry, docker, templates, renderer, engines, ports)` ; `DashboardCommand(ui, telemetry, docker, templates, renderer, ports)`. + +- [ ] **Step 1: `commands/agent.py`** + +```python +"""portabase agent NAME — create an agent folder. Databases are added by db add (or the interactive loop).""" + +from __future__ import annotations + +from pathlib import Path +from typing import Annotated + +import typer + +from commands.base import Command +from commands.db import report_write +from commands.flows.add_database import AddDatabaseFlow +from core.errors import ValidationError +from core.utils import validate_edge_key +from engines.registry import EngineRegistry +from services.docker import DockerRunner +from services.ports import PortAllocator +from services.project import AgentProject +from services.renderer import ComposeRenderer +from services.telemetry import Telemetry +from services.templates import TemplateRepository +from ui import UI + +NETWORK = "portabase_network" + + +def _edge_key(value: str) -> str: + if not validate_edge_key(value): + raise ValidationError("Invalid Edge Key.", hint="Expected Base64 or JSON with serverUrl, agentId, masterKeyB64.") + return value + + +class AgentCommand(Command): + name, help, panel = "agent", "Create a new Portabase Agent instance.", "Creation" + no_args_is_help = True + + def __init__(self, ui: UI, telemetry: Telemetry, docker: DockerRunner, templates: TemplateRepository, renderer: ComposeRenderer, engines: EngineRegistry, ports: PortAllocator) -> None: + super().__init__(ui, telemetry) + self.docker, self.templates, self.renderer, self.engines, self.ports = docker, templates, renderer, engines, ports + + def run( + self, + name: Annotated[str, typer.Argument(help="Agent name (creates a folder)")], + key: Annotated[str | None, typer.Option("--key", "-k", help="Edge Key")] = None, + tz: Annotated[str | None, typer.Option("--tz", help="Timezone")] = None, + polling: Annotated[int | None, typer.Option("--polling", help="Polling frequency in seconds")] = None, + host_gateway: Annotated[bool | None, typer.Option("--host-gateway/--no-host-gateway", help="Map localhost to host-gateway")] = None, + start: Annotated[bool, typer.Option("--start", "-s", help="Start immediately")] = False, + force: Annotated[bool, typer.Option("--force", "-f", help="Overwrite an existing folder")] = False, + ) -> None: + self.ui.banner() + self.require_docker(self.docker) + self.docker.ensure_network(NETWORK) + self.templates.resolve() + + path = Path(name).resolve() + if path.exists() and not force: + self.ui.warning(f"Directory '{name}' already exists.") + self.confirm_or_abort("Overwrite?", default=False) + + form = self.ui.form() + env_vars = { + "EDGE_KEY": form.text("Edge Key", value=key, validator=_edge_key, name="key"), + "TZ": form.text("Timezone", value=tz, default="UTC", name="tz"), + "POLLING": str(form.integer("Polling frequency (seconds)", value=polling, default=5, name="polling")), + "LOG_LEVEL": "info", + } + gateway = form.confirm("Add extra_hosts mapping (localhost -> host-gateway)?", value=host_gateway, default=False, name="host_gateway") + + project = AgentProject.create(path, env_vars, host_gateway=gateway) + self._write(project) + self.ui.success(f"Agent '{name}' created in {path}") + + if not self.ui.non_interactive: + self.ui.section("Database Setup") + flow = AddDatabaseFlow(self.ui, self.engines, self.ports) + while self.ui.confirm("Add a database?", default=True): + spec, engine = flow.collect({}) + flow.apply(project, spec, engine) + self._write(project) + self.ui.success(f"Added {engine.display} '{spec.name}' ({engine.describe(spec)})") + else: + self.ui.hint(f"Add databases with: portabase db add {name} --engine postgresql --mode new") + + if start or (not self.ui.non_interactive and self.ui.confirm("Start agent now?", default=False)): + with self.ui.status("Starting agent..."): + self.docker.compose(path, ["up", "-d"]) + self.ui.success("Agent started.") + else: + self.ui.info(f"Run: portabase start {name}") + + def _write(self, project: AgentProject) -> None: + with self.ui.status("Rendering configuration..."): + result = self.renderer.render_agent(project) + project.save_state() + report = result.write(project.path) + report_write(self.ui, report) +``` + +- [ ] **Step 2: `commands/dashboard.py`** + +```python +"""portabase dashboard NAME — create a dashboard folder.""" + +from __future__ import annotations + +import secrets +from pathlib import Path +from typing import Annotated +from urllib.parse import quote + +import typer + +from commands.base import Command +from commands.db import report_write +from core.utils import generate_password, slugify_project_name +from services.docker import DockerRunner +from services.ports import PortAllocator +from services.project import DashboardProject +from services.renderer import ComposeRenderer +from services.telemetry import Telemetry +from services.templates import TemplateRepository +from ui import UI + +DB_MODES = ("external", "internal", "custom") +MODE_LABELS = { + "external": "Dedicated Docker Container (Recommended)", + "internal": "Embedded Database (In-container)", + "custom": "Custom/Existing Database", +} + + +class DashboardCommand(Command): + name, help, panel = "dashboard", "Create a new Portabase Dashboard instance.", "Creation" + no_args_is_help = True + + def __init__(self, ui: UI, telemetry: Telemetry, docker: DockerRunner, templates: TemplateRepository, renderer: ComposeRenderer, ports: PortAllocator) -> None: + super().__init__(ui, telemetry) + self.docker, self.templates, self.renderer, self.ports = docker, templates, renderer, ports + + def run( + self, + name: Annotated[str, typer.Argument(help="Dashboard name (creates a folder)")], + port: Annotated[int | None, typer.Option("--port", help="Web port")] = None, + db_mode: Annotated[str | None, typer.Option("--db-mode", help="external | internal | custom")] = None, + db_host: Annotated[str | None, typer.Option("--db-host")] = None, + db_port: Annotated[int | None, typer.Option("--db-port")] = None, + db_name: Annotated[str | None, typer.Option("--db-name")] = None, + db_user: Annotated[str | None, typer.Option("--db-user")] = None, + db_password_stdin: Annotated[bool, typer.Option("--db-password-stdin", help="Read the custom DB password from stdin")] = False, + tz: Annotated[str | None, typer.Option("--tz")] = None, + start: Annotated[bool, typer.Option("--start", "-s")] = False, + force: Annotated[bool, typer.Option("--force", "-f")] = False, + yes: Annotated[bool, typer.Option("--yes", "-y", help="Skip the configuration confirmation")] = False, + ) -> None: + self.ui.banner() + self.require_docker(self.docker) + self.templates.resolve() + + path = Path(name).resolve() + if path.exists() and not force: + self.ui.warning(f"Directory '{name}' already exists.") + self.confirm_or_abort("Overwrite?", default=False) + + form = self.ui.form() + web_port = form.integer("Web Port", value=port, default=8887, name="port") + mode = form.choice("Database Setup", list(DB_MODES), value=db_mode, default="external", name="db_mode") + project_name = slugify_project_name(path.name) + + env_vars = { + "HOST_PORT": str(web_port), + "PROJECT_SECRET": secrets.token_hex(32), + "PROJECT_URL": f"http://localhost:{web_port}", + "PROJECT_NAME": project_name, + "TZ": form.text("Timezone", value=tz, default="Europe/Paris", name="tz"), + "LOG_LEVEL": "info", + } + rows = [("Dashboard Name", name), ("Path", str(path)), ("Access URL", env_vars["PROJECT_URL"]), ("Database Setup", MODE_LABELS[mode])] + + if mode == "external": + pg_pass, pg_port = generate_password(16), self.ports.free() + env_vars.update(self._pg_env("portabase", "portabase", pg_pass, "db", 5432, pg_port)) + rows.append(("Internal Port", str(pg_port))) + elif mode == "custom": + self.ui.info("External Database Configuration") + host = form.text("Host", value=db_host, default="localhost", name="db_host") + dport = form.integer("Port", value=db_port, default=5432, name="db_port") + dbname = form.text("Database Name", value=db_name, default="portabase", name="db_name") + user = form.text("Username", value=db_user, name="db_user") + if db_password_stdin: + import sys + + password = sys.stdin.readline().rstrip("\n") + else: + password = form.secret("Password", name="db_password") + env_vars.update(self._pg_env(dbname, user, password, host, dport, dport)) + rows += [("DB Host", host), ("DB Name", dbname), ("Connection URL", env_vars["DATABASE_URL"])] + + rows.append(("Files to Create", "docker-compose.yml, .env")) + self.ui.summary(rows, title="PROPOSED CONFIGURATION") + if not yes: + self.confirm_or_abort("Apply this configuration and generate files?", default=True) + + project = DashboardProject.create(path, env_vars) + with self.ui.status("Rendering configuration..."): + result = self.renderer.render_dashboard(project) + project.save_state() + report = result.write(path) + report_write(self.ui, report) + self.ui.success(f"Dashboard '{name}' created in {path}") + + if start or (not self.ui.non_interactive and self.ui.confirm("Start dashboard now?", default=False)): + with self.ui.status("Starting..."): + self.docker.compose(path, ["up", "-d"]) + self.ui.success(f"Live at: {env_vars['PROJECT_URL']}") + else: + self.ui.info(f"Run: portabase start {name}") + + @staticmethod + def _pg_env(db: str, user: str, password: str, host: str, port: int, host_port: int) -> dict[str, str]: + return { + "POSTGRES_DB": db, + "POSTGRES_USER": user, + "POSTGRES_PASSWORD": password, + "POSTGRES_HOST": host, + "DATABASE_URL": f"postgresql://{quote(user, safe='')}:{quote(password, safe='')}@{host}:{port}/{db}?schema=public", + "PG_PORT": str(host_port), + } +``` + +`--yes` en non-interactif : `confirm_or_abort(default=True)` renvoie `True` sans prompt, donc `--yes` n'est nécessaire que pour sauter l'affichage ; conservé pour la lisibilité des scripts. + +- [ ] **Step 3: Commit** + +```bash +git add commands/agent.py commands/dashboard.py +git commit -m "feat(commands): rewrite agent and dashboard on the declarative renderer" +``` + +--- + +### Task 8 : `commands/build.py` + +**Files:** +- Create: `commands/build.py` + +**Interfaces:** +- Produces: `BuildCommand(ui, telemetry, templates, renderer)`. + +- [ ] **Step 1: Écrire le module** + +```python +"""portabase build PATH — re-render compose from state. Also the legacy migration entry point.""" + +from __future__ import annotations + +from pathlib import Path +from typing import Annotated + +import typer + +from commands.base import Command +from commands.db import report_write +from core.errors import ValidationError +from services.project import COMPOSE_FILE, DATABASES_FILE, ENV_FILE, AgentProject, DashboardProject, detect_kind +from services.renderer import ComposeRenderer, RenderResult +from services.telemetry import Telemetry +from services.templates import TemplateRepository +from ui import UI + + +class BuildCommand(Command): + name, help, panel = "build", "Re-render docker-compose.yml from the component's configuration.", "Configuration" + no_args_is_help = True + + def __init__(self, ui: UI, telemetry: Telemetry, templates: TemplateRepository, renderer: ComposeRenderer) -> None: + super().__init__(ui, telemetry) + self.templates, self.renderer = templates, renderer + + def run( + self, + path: Annotated[Path, typer.Argument(help="Component folder")], + diff: Annotated[bool, typer.Option("--diff", help="Show the diff, write nothing")] = False, + stdout: Annotated[bool, typer.Option("--stdout", help="Print the compose, write nothing")] = False, + inline_env: Annotated[bool, typer.Option("--inline-env", help="Substitute values instead of ${VAR} references")] = False, + output: Annotated[Path | None, typer.Option("--output", "-o", help="Write files to another directory")] = None, + ) -> None: + if sum([diff, stdout, output is not None]) > 1: + raise ValidationError("Use only one of --diff, --stdout, --output.") + path = self.require_project_dir(path) + self.templates.resolve() + kind = detect_kind(path) + + if kind == "agent": + project = AgentProject.load(path) + result: RenderResult = self.renderer.render_agent(project, inline=inline_env) + else: + project = DashboardProject.load(path) + result = self.renderer.render_dashboard(project, inline=inline_env) + result.validate() + + if inline_env and not stdout: + self.ui.warning("--inline-env writes secrets in clear text into the compose file.") + + if stdout: + self.ui.console.print(result.compose, end="", markup=False, highlight=False) + return + if diff: + self.ui.diff(result.diff_against(path)) + return + + target = (output or path).resolve() + if output is not None: + target.mkdir(parents=True, exist_ok=True) + (target / ENV_FILE).write_text((path / ENV_FILE).read_text(encoding="utf-8"), encoding="utf-8") + report = result.write(target) + report_write(self.ui, report) + self.ui.success(f"Rendered {', '.join(p.name for p in report.wrote)} in {target}") + if kind == "agent" and output is None: + self.ui.info(f"Restart to apply: portabase restart {path.name}") +``` + +`--stdout` imprime via `console.print(markup=False)` pour qu'aucun `[x]` du compose ne soit interprété comme balise Rich. Pour un pipe propre, `main.py` doit **ne pas** afficher la notification de mise à jour quand `--stdout` est présent (déjà géré : non-interactif ou stdin non-TTY ; ajouter `"--stdout" in sys.argv` à `_notify_update` par sécurité, Task 9). + +- [ ] **Step 2: Commit** + +```bash +git add commands/build.py +git commit -m "feat(commands): add build command (re-render, --diff, --stdout, --inline-env, --output)" +``` + +--- + +### Task 8b : `commands/decrypt.py` en classe + +**Files:** +- Modify: `core/crypto.py` (`DecryptionError` hérite de `PortabaseError`) +- Modify: `commands/decrypt.py` (réécriture) + +**Interfaces:** +- Produces: `DecryptCommand(ui, telemetry)` ; `core.crypto.DecryptionError(PortabaseError)` avec `code = "E_CRYPTO"`, `exit_code = 8`. + +- [ ] **Step 1: `core/crypto.py`** + +Remplacer la définition de `DecryptionError` par : + +```python +from core.errors import PortabaseError + + +class DecryptionError(PortabaseError): + """Raised when a ``.enc`` file cannot be decrypted.""" + + code = "E_CRYPTO" + exit_code = 8 +``` + +Le reste du module (fonctions pures de déchiffrement) est inchangé. + +- [ ] **Step 2: `commands/decrypt.py`** + +```python +"""portabase decrypt INPUT [OUTPUT] — decrypt .enc backups (file or folder).""" + +from __future__ import annotations + +from pathlib import Path +from typing import Annotated + +import typer + +from commands.base import Command +from core.crypto import ( + ENC_SUFFIX, + DecryptionError, + decrypt_enc_file, + default_output_for, + load_master_key, +) +from core.errors import ConfigError, ValidationError + + +def _looks_like_dir(path: Path) -> bool: + if path.exists(): + return path.is_dir() + return str(path).endswith(("/", "\\")) or path.suffix == "" + + +class DecryptCommand(Command): + name, help, panel = "decrypt", "Decrypt Portabase .enc backup files (single file or folder).", "Configuration" + no_args_is_help = True + + def run( + self, + input_path: Annotated[Path, typer.Argument(help="A .enc file, or a folder containing .enc files.")], + output_path: Annotated[Path | None, typer.Argument(help="Output file or folder (must match the input type). Defaults to the input directory.")] = None, + key: Annotated[Path | None, typer.Option("--key", "-k", help="Master key file. Defaults to ./master_key.bin")] = None, + ) -> None: + input_path = input_path.resolve() + if not input_path.exists(): + raise ConfigError(f"Input path not found: {input_path}") + master_key = load_master_key(key.resolve() if key else None) # raises DecryptionError + if input_path.is_dir(): + self._folder(input_path, output_path, master_key) + else: + self._single(input_path, output_path, master_key) + + def _single(self, enc_path: Path, output_path: Path | None, master_key: bytes) -> None: + if enc_path.suffix != ENC_SUFFIX: + self.ui.warning(f"{enc_path.name} does not end with {ENC_SUFFIX}; decrypting anyway.") + if output_path is None: + out = enc_path.parent / default_output_for(enc_path) + elif _looks_like_dir(output_path): + out = output_path.resolve() / default_output_for(enc_path) + else: + out = output_path.resolve() + try: + decrypt_enc_file(enc_path, out, master_key) + except OSError as e: + raise DecryptionError(f"I/O error on {enc_path.name}: {e}", cause=e) from e + self.ui.success(f"Decrypted {enc_path.name} → {out}") + + def _folder(self, in_dir: Path, output_path: Path | None, master_key: bytes) -> None: + enc_files = sorted(p for p in in_dir.iterdir() if p.is_file() and p.suffix == ENC_SUFFIX) + if not enc_files: + self.ui.warning(f"No {ENC_SUFFIX} files found in {in_dir}.") + return + if output_path is None: + out_dir = in_dir + elif _looks_like_dir(output_path): + out_dir = output_path.resolve() + else: + raise ValidationError("Input is a folder, so the output must be a folder too.") + out_dir.mkdir(parents=True, exist_ok=True) + + failures: list[tuple[str, str]] = [] + with self.ui.status(f"Decrypting {len(enc_files)} file(s)..."): + for enc_path in enc_files: + out = out_dir / default_output_for(enc_path) + try: + decrypt_enc_file(enc_path, out, master_key) + except (DecryptionError, OSError) as e: + failures.append((enc_path.name, str(e))) + continue + succeeded = len(enc_files) - len(failures) + self.ui.info(f"Done: {succeeded} succeeded, {len(failures)} failed of {len(enc_files)} file(s).") + if failures: + for name, reason in failures: + self.ui.print(f" [danger]•[/danger] {name}: {reason}") + raise DecryptionError(f"{len(failures)} file(s) failed to decrypt.") +``` + +Différence assumée : les succès individuels ne sont plus imprimés un par un sous le spinner (un `console.print` sous `status` est autorisé mais bruyant) ; le résumé final reste. `main.py` remplace le `LegacyCommand` decrypt par `DecryptCommand(ui, telemetry)` (Task 9). + +- [ ] **Step 3: Vérifier** + +Run: `uv run python main.py decrypt /tmp/nope; echo "exit=$?"` → `E_CONFIG`, exit 3. `uv run python main.py decrypt . ; echo "exit=$?"` sans `master_key.bin` → `E_CRYPTO`, exit 8. Avec un vrai `.enc` et sa clé si disponible : déchiffrement identique à 26.08.12. + +- [ ] **Step 4: Commit** + +```bash +git add core/crypto.py commands/decrypt.py +git commit -m "refactor(commands): rewrite decrypt as a Command class" +``` + +--- + +### Task 9 : Câblage final, suppression du legacy + +**Files:** +- Modify: `main.py` +- Modify: `commands/base.py` (retirer `LegacyCommand`) +- Delete: `core/network.py`, `core/docker.py`, `templates/compose.py`, `templates/__init__.py`, `templates/agent.yml`, `templates/dashboard.yml` +- Modify: `core/config.py` (retirer les fonctions legacy, garder `GlobalConfig`, `TEMPLATE_BASE_URL`, `GLOBAL_CONFIG_DIR/FILE`) +- Modify: `core/utils.py` (garder uniquement `generate_password`, `slugify_project_name`, `validate_edge_key`, et l'import `current_version` re-exporté peut disparaître) +- Modify: `pyproject.toml` (retirer `per-file-ignores`) +- Modify: `.github/workflows/templates-upload.yml` (retirer l'étape `latest`) +- Modify: `scripts/render_check.py` + +- [ ] **Step 1: `main.py` — remplacer les `LegacyCommand` et `legacy_db`** + +Imports à remplacer : + +```python +from commands.agent import AgentCommand +from commands.build import BuildCommand +from commands.dashboard import DashboardCommand +from commands.db import DbCommands +from commands.decrypt import DecryptCommand +from engines import registry as engine_registry +from services.ports import PortAllocator +from services.renderer import ComposeRenderer +from services.templates import TemplateRepository +``` + +(supprimer `from commands import agent as legacy_agent`, `dashboard as legacy_dashboard`, `db as legacy_db`, `decrypt as legacy_decrypt`, `from commands.base import LegacyCommand`.) + +Dans `build_app`, après `updater = Updater(http, version)` : + +```python + templates = TemplateRepository.from_environment(http, config) + ports = PortAllocator() + renderer = ComposeRenderer(templates, engine_registry, version) +``` + +Liste `commands` : + +```python + commands = [ + AgentCommand(ui, telemetry, docker, templates, renderer, engine_registry, ports), + DashboardCommand(ui, telemetry, docker, templates, renderer, ports), + StartCommand(ui, telemetry, docker), + StopCommand(ui, telemetry, docker), + RestartCommand(ui, telemetry, docker), + LogsCommand(ui, telemetry, docker), + UninstallCommand(ui, telemetry, docker), + BuildCommand(ui, telemetry, templates, renderer), + DecryptCommand(ui, telemetry), + UpdateCommand(ui, telemetry, checker, updater), + ] + for cmd in commands: + cmd.register(app) + DbCommands(ui, telemetry, engine_registry, ports, templates, renderer, docker).register(app) + ConfigCommands(ui, telemetry, config).register(app) +``` + +Dans `_notify_update`, condition étendue : `or "--stdout" in sys.argv`. + +Retirer du catcher les branches `click.exceptions.Exit` et `click.exceptions.Abort` ? **Non** : `--help` lève toujours `Exit(0)` et `version_callback` lève `typer.Exit()`. Garder `Exit` ; retirer `Abort` (plus de `typer.confirm`). + +- [ ] **Step 2: Nettoyage** + +```bash +git rm core/network.py core/docker.py templates/compose.py templates/__init__.py templates/agent.yml templates/dashboard.yml +``` + +`commands/base.py` : supprimer la classe `LegacyCommand` et l'import `Callable` s'il devient inutilisé. + +`core/config.py` : ne garder que les constantes (`TEMPLATE_BASE_URL`, `GLOBAL_CONFIG_DIR`, `GLOBAL_CONFIG_FILE`), les imports nécessaires et `GlobalConfig`. Supprimer `write_file`, `write_env_file`, `load_global_config`, `save_global_config`, `get_config_value`, `set_config_value`, `load_db_config`, `save_db_config`, `add_db_to_json`. + +`core/utils.py` : ne garder que `generate_password`, `slugify_project_name`, `validate_edge_key` et leurs imports (`base64`, `binascii`, `json`, `re`, `secrets`, `string`). Supprimer `questionary_style`, `custom_theme`, `HINTS`, `get_random_hint`, `console`, `BANNER`, `print_banner`, `get_free_port`, `start_docker`, `check_system`, `validate_work_dir`, le re-export `current_version`. + +Vérifier qu'aucune référence ne subsiste : +Run: `grep -rn "core.network\|core.docker\|templates.compose\|LegacyCommand\|get_random_hint\|print_banner\|check_system\|validate_work_dir\|get_free_port\|load_db_config\|add_db_to_json\|write_env_file\|get_config_value" --include=*.py . | grep -v ".venv"` +Expected: aucune sortie. + +`pyproject.toml` : supprimer entièrement `[tool.ruff.lint.per-file-ignores]` ; retirer `"templates"` de `known-first-party`. + +`.github/workflows/templates-upload.yml` : supprimer l'étape `Upload latest templates (stable only, legacy fallback)` — **seulement si** plus aucune version legacy n'est supportée. Sinon la garder ; par défaut la garder et ouvrir une issue « retirer latest/ ». Décision utilisateur. + +- [ ] **Step 3: `scripts/render_check.py` via `ComposeRenderer`** + +Remplacer `agent_cases`, `render_agent`, `dashboard_cases` par une construction de projets en mémoire : + +```python +from core.specs import DatabaseSpec # noqa: E402 +from services.envfile import EnvFile # noqa: E402 +from services.project import AgentProject, DashboardProject # noqa: E402 +from services.renderer import ComposeRenderer # noqa: E402 + + +def agent_project(tmp: Path, specs: list, engines_for, host_gateway=False, sqlite=False) -> AgentProject: + env = EnvFile(tmp / ".env") + env.merge({"TZ": "UTC", "EDGE_KEY": "x", "LOG_LEVEL": "info", "POLLING": "5"}) + project = AgentProject(tmp, env, [], host_gateway) + for spec, engine in zip(specs, engines_for): + project.add(spec, engine) + if sqlite: + sq = registry.get("sqlite") + project.add(sq.generate(auth=False, ports=FixedPortAllocator(), answers={"name": "x"}), sq) + return project + + +def agent_cases(renderer: ComposeRenderer) -> list[tuple[str, str, str]]: + ports = FixedPortAllocator() + tmp = Path(tempfile.mkdtemp()) + cases = [] + empty = agent_project(tmp, [], []) + cases.append(("agent/empty", renderer.render_agent(empty).compose, env_text(empty))) + toggles = agent_project(tmp, [], [], host_gateway=True, sqlite=True) + dv = registry.get("docker-volume") + toggles.add(dv.from_existing({"volume": "v"}), dv) + cases.append(("agent/toggles", renderer.render_agent(toggles).compose, env_text(toggles))) + all_specs, all_engines = [], [] + for engine in registry: + if engine.template is None: + continue + for auth in (True, False) if engine.auth_variants else (True,): + spec = engine.generate(auth=auth, ports=ports, answers={}) + one = agent_project(tmp, [spec], [engine]) + label = f"agent/{engine.key}" + ("/auth" if auth else "/noauth" if engine.auth_variants else "") + cases.append((label, renderer.render_agent(one).compose, env_text(one))) + all_specs.append(spec); all_engines.append(engine) + everything = agent_project(tmp, all_specs, all_engines, host_gateway=True, sqlite=True) + cases.append(("agent/all", renderer.render_agent(everything).compose, env_text(everything))) + return cases + + +def env_text(project) -> str: + return "".join(f'{k}="{v}"\n' for k, v in project.env.as_dict().items()) + + +def dashboard_cases(renderer: ComposeRenderer) -> list[tuple[str, str, str]]: + tmp = Path(tempfile.mkdtemp()) + base = {"HOST_PORT": "8887", "PROJECT_SECRET": "s", "PROJECT_URL": "http://localhost:8887", "PROJECT_NAME": "pb", "TZ": "UTC", "LOG_LEVEL": "info"} + pg = {"POSTGRES_DB": "pb", "POSTGRES_USER": "pb", "POSTGRES_PASSWORD": "p", "PG_PORT": "5433", "DATABASE_URL": "postgresql://pb:p@db:5432/pb"} + variants = {"external": {**base, **pg, "POSTGRES_HOST": "db"}, "internal": base, "custom": {**base, **pg, "POSTGRES_HOST": "remote"}} + cases = [] + for mode, vars_ in variants.items(): + env = EnvFile(tmp / f".env.{mode}"); env.merge(vars_) + project = DashboardProject(tmp, env) + assert project.db_mode == mode, (project.db_mode, mode) + cases.append((f"dashboard/{mode}", renderer.render_dashboard(project).compose, env_text(project))) + return cases +``` + +et dans `main()` : + +```python + repo = TemplateRepository(HttpClient(), GlobalConfig().cache_dir, "local", local_dir=Path(args.templates)) + renderer = ComposeRenderer(repo, registry, "render-check") + try: + engines_check(repo) + for label, compose, env_text_ in agent_cases(renderer) + dashboard_cases(renderer): + validate(label, compose, env_text_, use_compose) +``` + +Supprimer `AGENT_GLOBALS`, `AGENT_ENV`, `DASHBOARD_VARS`, `DASHBOARD_ENV`, `render_agent` devenus inutiles. Ajouter `import tempfile` déjà présent. + +Run: `uv run python scripts/render_check.py` +Expected: mêmes cas qu'au Plan 3, tous `ok`, dont `agent/toggles` avec socket + mount + extra_hosts. + +- [ ] **Step 4: Lint** + +Run: `uv run ruff check . && uv run ruff format --check .` +Expected: passe **sans aucune** exception par fichier. + +- [ ] **Step 5: Commit** + +```bash +git add -A +git commit -m "refactor: remove legacy commands and helpers; wire agent, dashboard, db, build on the renderer" +``` + +--- + +### Task 10 : Vérification de bout en bout + +Docker requis. `K` = edge key de test : `K=$(printf '{"serverUrl":"http://x","agentId":"a","masterKeyB64":"k"}' | base64 -w0)`. + +- [ ] **Step 1: Agent non-interactif + db add** + +```bash +cd /tmp && rm -rf ni-agent +M="uv --directory /home/soluce/Documents/PROJETS/Portabase/cli run python /home/soluce/Documents/PROJETS/Portabase/cli/main.py" +$M --non-interactive agent ni-agent --key "$K" --tz Europe/Paris --polling 7 --host-gateway; echo "exit=$?" +$M --non-interactive db add ni-agent --engine postgresql --mode new -o clean_mode=drop_schemas; echo "exit=$?" +$M --non-interactive db add ni-agent --engine redis --mode new --no-auth; echo "exit=$?" +printf 'secret\n' | $M --non-interactive db add ni-agent --engine mysql --mode existing --label Prod --host db.example --port 3306 --database app --user app --password-stdin; echo "exit=$?" +$M --non-interactive db add ni-agent --engine sqlite --mode new --name local; echo "exit=$?" +$M --non-interactive db add ni-agent --engine docker-volume --volume some_vol; echo "exit=$?" +$M db list ni-agent +cat ni-agent/.env; head -3 ni-agent/docker-compose.yml; python3 -c "import json; print([ (d['type'], d.get('options')) for d in json.load(open('ni-agent/databases.json'))['databases']])" +(cd ni-agent && docker compose config --quiet && echo "compose config OK") +``` +Expected: 6 × `exit=0` ; table à 6 lignes ; `.env` avec `TZ="Europe/Paris"`, `POLLING="7"`, `DB_PG_*` (4 vars), `DB_REDIS_*_PORT` seul ; en-tête `# Generated by Portabase CLI` ; options `{'clean_mode': 'drop_schemas'}` sur postgres, `None` ailleurs ; `compose config OK`. + +- [ ] **Step 2: Erreurs non-interactives** + +```bash +$M --non-interactive db add ni-agent --engine postgresql --mode existing; echo "exit=$?" +$M --non-interactive db add ni-agent --engine redis --mode new --host x; echo "exit=$?" +$M --non-interactive db add ni-agent --engine postgresql --mode new -o nope=1; echo "exit=$?" +$M --non-interactive agent ni-agent --key "$K"; echo "exit=$?" +``` +Expected: `Missing --host` exit 2 ; `not applicable` exit 2 ; `Unknown option(s)` exit 2 ; `Directory 'ni-agent' already exists` → confirm par défaut False → `Cancelled.` exit 130. + +- [ ] **Step 3: db remove, build** + +```bash +ID=$(python3 -c "import json; print([d for d in json.load(open('ni-agent/databases.json'))['databases'] if d['type']=='redis'][0]['generated_id'])") +$M db remove ni-agent --id "$ID" --yes; echo "exit=$?" +grep -c "db-redis" ni-agent/docker-compose.yml ni-agent/.env +$M build ni-agent --diff +$M build ni-agent --stdout --inline-env | head -20 +$M build ni-agent --output /tmp/ni-export && ls /tmp/ni-export +``` +Expected: `Removed`, `0` occurrences de `db-redis` dans les deux fichiers, diff « No changes. », compose inline avec valeurs littérales (pas de `${`), export contenant `docker-compose.yml`, `.env`, `databases.json`. + +- [ ] **Step 4: Lifecycle réel** + +```bash +$M start ni-agent && sleep 5 && $M logs ni-agent --no-follow | tail -5 && $M stop ni-agent && $M uninstall ni-agent --force +``` +Expected: services `agent` et `db-pg-*` démarrent (`docker compose ps` pendant le `sleep` si besoin), puis arrêt et suppression. + +- [ ] **Step 5: Install legacy** + +Réutiliser `/tmp/legacy-agent` (Task 2 step 4 ; le recréer sinon). + +```bash +cp -r /tmp/legacy-agent /tmp/legacy-copy +$M build /tmp/legacy-copy --diff +$M db add /tmp/legacy-copy --engine valkey --mode new --auth --non-interactive; echo "exit=$?" +ls /tmp/legacy-copy; head -1 /tmp/legacy-copy/docker-compose.yml +(cd /tmp/legacy-copy && docker compose config --quiet && echo "compose config OK") +diff <(python3 -c "import json; print(sorted((d['type'], d.get('host')) for d in json.load(open('/tmp/legacy-agent/databases.json'))['databases']))") <(python3 -c "import json; print(sorted((d['type'], d.get('host')) for d in json.load(open('/tmp/legacy-copy/databases.json'))['databases'] if d['type']!='valkey'))") && echo "entries preserved" +$M start /tmp/legacy-copy && $M stop /tmp/legacy-copy && $M uninstall /tmp/legacy-copy --force +``` +Expected: diff montre en-tête + `restart:` sur redis ; `db add` exit 0 avec le warning `Legacy compose backed up to docker-compose.legacy.yml` ; `docker-compose.legacy.yml` présent ; en-tête `# Generated` ; `compose config OK` ; `entries preserved` ; les anciens services démarrent avec leurs volumes existants (noms de service inchangés). + +- [ ] **Step 6: Dashboard** + +```bash +cd /tmp && rm -rf ni-dash +$M --non-interactive dashboard ni-dash --port 8899 --db-mode external --yes; echo "exit=$?" +(cd ni-dash && docker compose config --quiet && echo OK && grep -c "db:" docker-compose.yml) +$M --non-interactive dashboard ni-dash2 --port 8898 --db-mode internal --yes && (cd ni-dash2 && grep -c "postgres" docker-compose.yml) +printf 'pw\n' | $M --non-interactive dashboard ni-dash3 --port 8897 --db-mode custom --db-host pg.example --db-user u --db-password-stdin --yes && grep DATABASE_URL ni-dash3/.env +$M build ni-dash3 --diff +``` +Expected: external `OK 1` ; internal `0` ; custom `.env` avec `DATABASE_URL="postgresql://u:pw@pg.example:5432/portabase?schema=public"` ; diff vide. + +- [ ] **Step 7: Interactif** + +`$M agent int-agent` sans flags : bannière, prompts Edge Key / Timezone / Polling / extra_hosts, création, boucle « Add a database? » → ajouter un postgres (prompts options avec aide affichée) puis un redis (variante), répondre non, ne pas démarrer. Puis `$M db remove int-agent` avec sélection interactive. Ctrl-C au milieu d'un prompt → `Cancelled.` exit 130 sans traceback, fichiers cohérents (`docker compose config` passe). + +- [ ] **Step 8: CI `build-smoke`** + +Dans `ci.yml`, étape `Smoke` du job `build-smoke` : + +```yaml + - name: Smoke + env: + PORTABASE_TEMPLATES_DIR: ${{ github.workspace }}/templates + run: | + ./dist/portabase_smoke --version + cd "$(mktemp -d)" + K=$(printf '{"serverUrl":"http://x","agentId":"a","masterKeyB64":"k"}' | base64 -w0) + "$GITHUB_WORKSPACE/dist/portabase_smoke" --non-interactive agent smoke --key "$K" + "$GITHUB_WORKSPACE/dist/portabase_smoke" --non-interactive db add smoke --engine postgresql --mode new + "$GITHUB_WORKSPACE/dist/portabase_smoke" --non-interactive build smoke --diff + cd smoke && docker compose config --quiet +``` + +Le runner GitHub a Docker ; `agent` appelle `require_docker` + `ensure_network` → fonctionne. Si le daemon n'est pas disponible sur un runner donné, l'étape échoue explicitement (`E_DOCKER`) — c'est voulu. + +- [ ] **Step 9: README** + +Ajouter une section « Upgrading from 26.07 or earlier » : le premier `db add`/`build` re-génère `docker-compose.yml` (sauvegarde `docker-compose.legacy.yml`), les personnalisations vont dans `docker-compose.override.yml`, `portabase build --diff` montre les changements avant. + +- [ ] **Step 10: Commit, PR, rc** + +```bash +git add -A +git commit -m "ci: smoke-test agent creation and rendering with the built binary; document migration" +git checkout -b refactor/render-commands && git push -u origin refactor/render-commands +``` +PR « refactor: declarative rendering, flag-driven commands, build » → merge → Bump `26.09.0rc1` → tester le binaire rc sur une vraie install legacy → Bump `26.09.0` stable. + +--- + +## Self-review + +**Spec coverage :** +- §4.2 inventaire commandes : `agent` (flags, boucle interactive, aucune DB en non-interactif) ✔ ; `dashboard` (flags, modes, `--yes`) ✔ ; `db add` (tous flags + `-o`, rejet des flags non pertinents) ✔ ; `db remove` (`--id/--name`, `--purge-volume`, volume conservé par défaut) ✔ ; `db list` (options non-défaut) ✔ ; `build` (`--diff`, `--stdout`, `--inline-env`, `--output`) ✔ ; détection `kind` ✔ ; `back` supprimé ✔. +- §4.3 `AddDatabaseFlow.collect/apply`, séquence, usage par les deux commandes, rendu après chaque ajout ✔. +- §5.1 `DatabaseSpec.from_json` → `spec_from_entry` ; `AgentProject` (managed, socket, mounts, validate doublons) ✔ ; `DashboardProject.db_mode` ✔ ; `host` managé sans `_PORT` → traité externe (warning non émis : ajouter `ui.warning` dans `DbListCommand` si souhaité — non bloquant). +- §5.2 `EnvFile` ✔ (ordre, commentaires, quotes, merge, remove_prefix, atomique). +- §5.3 `ComposeFacts.host_gateway` liste/dict, jamais d'erreur ✔. +- §5.5 renderer, contexte, `*_var`, validation avant écriture, en-tête, `templates.resolve()` avant mutation ✔. +- §5.7 legacy : backup `.legacy.yml` une fois, `start/stop/logs` sans migration, `build --diff` ✔. +- §6.1 options : `-o`, validation clés, prompts avec `help`, projection non-défaut ✔. +- §7.2 `Summary` (masque), `DataTable`, `Diff` ✔. +- §10 F : `build-smoke` étendu, rc obligatoire ✔. + +**Placeholders :** aucun. + +**Cohérence :** `report_write` défini dans `commands/db.py`, importé par `agent`, `dashboard`, `build` ✔ ; `DbEngine.describe/label_default/non_default_options` utilisés ✔ ; `SqliteEngine.mount_for` (Plan 3) utilisé par `AgentProject.sqlite_mounts` ✔ ; `RenderResult.write/diff_against/validate` utilisés par les 4 commandes ✔ ; `DockerRunner.remove_volume` ajouté et utilisé ✔ ; `UI.summary/table/diff` ajoutés et utilisés ✔ ; `TemplateRepository.engine_template` (Plan 3) utilisé par `_service` ✔. + +**Écarts connus :** +- `templates-upload.yml` `latest/` : décision utilisateur (Task 9 step 2). +- Import local de `sys` dans deux `run` : ruff peut préférer un import de module ; déplacer. diff --git a/docs/superpowers/specs/2026-09-11-cli-refactor-design.md b/docs/superpowers/specs/2026-09-11-cli-refactor-design.md new file mode 100644 index 0000000..cb62728 --- /dev/null +++ b/docs/superpowers/specs/2026-09-11-cli-refactor-design.md @@ -0,0 +1,592 @@ +# Refonte du CLI Portabase — Design + +Date : 2026-09-11 +Statut : validé en brainstorming, en attente de relecture avant plan d'implémentation. + +## 1. Objectifs et périmètre + +Refonte structurelle du CLI sans changement de dépendances majeures (Typer, Rich, questionary, requests, PyYAML conservés ; Jinja2 ajouté). + +Objectifs : + +- Supprimer la duplication entre `commands/agent.py` et `commands/db.py` (~500 lignes du wizard "ajouter une base" copiées). +- Remplacer la génération de `docker-compose.yml` par chirurgie texte (`.replace`, regex, ancres) par un rendu Jinja2 complet et déterministe. +- POO sur toute l'application : commandes, services, moteurs, composants UI en classes. Les utilitaires purs (`slugify`, `generate_password`, `validate_edge_key`) restent des fonctions. +- Chaque commande configurable intégralement par paramètres, sans mode interactif. +- Bibliothèque de composants `ui/` (approche shadcn : tokens, composants stateless, façade unique). +- Catcher d'erreurs unique avec hiérarchie d'exceptions et codes de sortie stables. +- Couche télémétrie prête pour OpenTelemetry, opt-in, sans dépendance immédiate. +- Plus d'auto-update : notification seulement, mise à jour manuelle. +- CI de PR (lint, Gitleaks, Plumber, validation des templates, build smoke), suppression du script `./release`. + +Hors périmètre de cette spec : + +- Tests automatisés (spec suivante ; la structure en tient compte : injection de dépendances, services sans I/O terminal, job `test` vide en CI). +- Sortie machine `--json`. +- Exporter OTel réel (le contrat est posé, l'implémentation viendra avec un endpoint). +- Fichier de spec déclaratif `build -f spec.yml`. +- Support Windows (inchangé : code présent, hors matrice de build). + +## 2. Décisions structurantes + +| Sujet | Décision | +|---|---| +| Layout | Plat, conservé (`main.py` racine, `--paths=.`). Nouveaux dossiers `services/`, `engines/`, `ui/`. | +| État d'une install | Aucun fichier d'état ajouté. Source de vérité = `.env` (variables runtime des conteneurs uniquement) + `databases.json` (contrat agent, inchangé) + lecture structurelle du compose existant pour le seul fait non dérivable (`host_gateway`). | +| Compose | Artefact dérivé, propriété du CLI, re-rendu intégralement à chaque commande mutante. Personnalisations utilisateur via `docker-compose.override.yml` (mécanisme Compose natif). | +| Templates | 100 % remote (S3), versionnés par version CLI exacte, manifest avec sha256, cache disque. Suppression du fallback `latest`. Source dans `templates/` à la racine du dépôt. | +| Création multi-DB | En non-interactif, `portabase agent` crée un agent sans base ; les bases s'ajoutent par `portabase db add` (un appel par base). En interactif, `agent` enchaîne sur une boucle « Add a database? » qui réutilise le même flux que `db add`. Pas de DSL `--db engine:opts`. | +| Options moteur | Flag générique répétable `-o/--option KEY=VALUE`, validé contre `DbEngine.option_fields()`. Pas de flag Typer par option. | +| Moteurs DB | Classes Python (`engines/`), registre à imports explicites. Pas de manifeste data-driven. | +| Input UI | questionary uniquement. `rich.prompt` et `typer.prompt` bannis (ruff). | +| Non-interactif | Flag `--non-interactive`, env `PORTABASE_NON_INTERACTIVE`, ou `stdin` non-TTY. Géré par `ui.Form`, pas par les commandes. | +| Erreurs | `PortabaseError` + sous-classes, codes de sortie distincts, un seul `try` dans `main.py`. `except:` nus interdits. | +| Télémétrie | Interface `Telemetry`, `NoopTelemetry` par défaut, opt-in via config globale, jamais de prompt. | +| Updater | Notification après la commande (cache 24 h, silencieux si hors ligne), `portabase update` manuel avec vérification de checksum. | +| Release | `bump.yml` (`workflow_dispatch`) remplace `./release`. Workflows de release sur tag inchangés. | +| Mot de passe | `generate_password` retire `$` et `` ` `` des symboles (cassent `--requirepass "${PASSWORD}"` via shell). Ne s'applique qu'aux nouvelles bases. | + +## 3. Structure des fichiers + +``` +cli/ +├── main.py # build_app(), catcher d'erreurs, codes de sortie +├── pyproject.toml # + jinja2 ; pyinstaller/ruff/pytest en groupe dev +│ +├── commands/ +│ ├── base.py # Command ABC, CommandGroup +│ ├── agent.py # AgentCommand +│ ├── dashboard.py # DashboardCommand +│ ├── build.py # BuildCommand +│ ├── lifecycle.py # Start/Stop/Restart/Logs/Uninstall (ex-common.py) +│ ├── db.py # DbCommands : add / remove / list +│ ├── config.py # ConfigCommands : get / set +│ ├── update.py # UpdateCommand +│ └── flows/ +│ └── add_database.py # AddDatabaseFlow : collecte + application, partagé par agent et db add +│ +├── services/ +│ ├── project.py # AgentProject, DashboardProject, DatabaseSpec, detect_kind() +│ ├── envfile.py # EnvFile +│ ├── compose_facts.py # ComposeFacts (lecture structurelle, jamais d'écriture) +│ ├── renderer.py # ComposeRenderer, RenderResult +│ ├── templates.py # TemplateRepository, Manifest +│ ├── docker.py # DockerRunner +│ ├── ports.py # PortAllocator +│ ├── http.py # HttpClient +│ ├── updater.py # UpdateChecker, Updater +│ └── telemetry.py # Telemetry ABC, NoopTelemetry, ConsoleTelemetry, TelemetryFactory +│ +├── engines/ +│ ├── __init__.py # registry = EngineRegistry([...]) — imports explicites +│ ├── base.py # DbEngine ABC, Field +│ ├── registry.py # EngineRegistry +│ ├── sql.py # StandardSqlEngine + Postgres/PostgresCluster/MySQL/MariaDB/MSSQL/Firebird +│ ├── redis.py # RedisEngine +│ ├── valkey.py # ValkeyEngine +│ ├── mongo.py # MongoEngine +│ ├── sqlite.py # SqliteEngine +│ └── docker_volume.py # DockerVolumeEngine +│ +├── ui/ +│ ├── __init__.py # façade UI +│ ├── theme.py # PALETTE → RICH_THEME + QUESTIONARY_STYLE +│ ├── form.py # Form (flag → prompt → défaut → erreur) +│ └── components/ +│ ├── base.py # Component(console) +│ ├── banner.py message.py section.py summary.py table.py +│ ├── status.py hints.py diff.py prompt.py +│ +├── core/ +│ ├── errors.py # PortabaseError + sous-classes +│ ├── config.py # GlobalConfig (~/.portabase/config.json) +│ ├── version.py # current_version() +│ └── utils.py # slugify, generate_password, validate_edge_key — fonctions pures +│ +├── templates/ # source des templates remote (assets, pas un package Python) +│ ├── agent.yml.j2 +│ ├── dashboard.yml.j2 +│ ├── engines.map.json # clé moteur → template (pour engines-check et manifest) +│ └── engines/ +│ ├── postgresql.yml.j2 mysql.yml.j2 mariadb.yml.j2 mssql.yml.j2 +│ ├── firebird.yml.j2 mongodb.yml.j2 redis.yml.j2 valkey.yml.j2 +│ +├── scripts/ +│ └── render_check.py # rend tous les templates avec fixtures, valide YAML + compose config +│ +├── .github/workflows/ +│ ├── ci.yml # PR : lint, render-check, engines-check, gitleaks, plumber, build-smoke, test +│ ├── bump.yml # workflow_dispatch : bump version + tag +│ ├── templates-hotfix.yml # workflow_dispatch : re-upload templates vers une version existante +│ ├── release.yml, release-candidate.yml, python.yml, github.yml # inchangés (hors durcissement) +│ └── templates-upload.yml # + génération manifest.json, source templates/ +│ +├── .gitleaks.toml +└── supprimés : release, templates/compose.py, templates/__init__.py, commands/common.py, + core/network.py, core/docker.py, .github/assets/templates/ +``` + +Règle de dépendance, descendante uniquement : + +- `commands` → `services`, `engines`, `ui`, `core` +- `services` → `engines`, `core` (jamais `ui` : un service lève, n'affiche rien) +- `engines` → `core` +- `ui` → `core` + +## 4. Commandes + +### 4.1 `Command` + +```python +class Command(ABC): + name: str + help: str + panel: str = "General" + + def __init__(self, ui: UI, telemetry: Telemetry): ... + def register(self, app: typer.Typer) -> None: + app.command(self.name, help=self.help, rich_help_panel=self.panel)(self.run) + + @abstractmethod + def run(self, *args, **kwargs) -> None: ... +``` + +`base.py` wrappe `run` dans `telemetry.span(f"command.{name}")`. Les dépendances (`DockerRunner`, `TemplateRepository`, `EngineRegistry`, `PortAllocator`) sont injectées par constructeur dans `main.build_app()`. + +Signatures Typer en `Annotated[...]`. Chaque option qui correspond à une question du wizard a une valeur par défaut `None` : présente → utilisée, absente → prompt (interactif) ou défaut/erreur (non-interactif). Une seule méthode `_collect()` par commande, aucun `if non_interactive` dans la logique métier. + +### 4.2 Inventaire + +| Commande | Options notables | Effet | +|---|---|---| +| `agent NAME` | `--key`, `--tz`, `--polling`, `--host-gateway/--no-host-gateway`, `--start`, `--force`, `--non-interactive` | crée le dossier, `.env`, `databases.json` vide, rend le compose. En interactif, enchaîne sur une boucle « Add a database? » (`AddDatabaseFlow`, rendu après chaque ajout). En non-interactif, ne crée aucune base. | +| `dashboard NAME` | `--port`, `--db-mode external\|internal\|custom`, `--db-host/--db-port/--db-name/--db-user/--db-password-stdin`, `--start`, `--force` | crée `.env`, rend le compose. | +| `db add NAME` | `--engine`, `--mode new\|existing`, `--auth/--no-auth`, `--name`, `--host`, `--port`, `--database`, `--user`, `--password`, `--password-stdin`, `--path`, `--volume`, `--container`, `--label`, `-o/--option KEY=VALUE` (répétable) | collecte via `AddDatabaseFlow` selon le moteur, mute `.env` + `databases.json`, re-rend. Flag ou option fourni mais non pertinent pour le moteur/mode → `ValidationError`. | +| `db remove NAME` | `--id` ou `--name`, `--purge-volume` | retire l'entrée, retire les variables `.env` du service, re-rend. Le volume Docker n'est supprimé que sur `--purge-volume`. | +| `db list NAME` | — | lecture seule. | +| `build PATH` | `--diff`, `--stdout`, `--inline-env`, `--output DIR` | re-rend depuis l'état. Sans option : écrit en place (= migration legacy). `--inline-env` substitue les valeurs au lieu de `${VAR}` avec avertissement secrets en clair. | +| `start/stop/restart/logs/uninstall PATH` | inchangées (`uninstall --force`) | n'utilisent pas le renderer, fonctionnent sur toute install. | +| `config get/set` | inchangées + clés `telemetry`, `telemetry_endpoint`, `channel` | config globale. | +| `update` | — | mise à jour manuelle avec vérification checksum. | +| `decrypt INPUT [OUTPUT]` | `--key` | déchiffre des sauvegardes `.enc` (ajouté en 26.08.12) ; `DecryptCommand`, `DecryptionError(PortabaseError)` code `E_CRYPTO` exit 8. `core/crypto.py` reste un module de fonctions pures. | + +Options globales : `--verbose`, `--debug`, `--no-color`, `--non-interactive`. Détection `kind` d'un dossier : `databases.json` présent → agent ; `PROJECT_SECRET` dans `.env` → dashboard. + +Le choix `back` dans les selects disparaît : interactif = Ctrl-C (`UserAbort`) ou entrée "cancel" en fin de liste. + +### 4.3 `AddDatabaseFlow` (`commands/flows/add_database.py`) + +Le wizard d'ajout de base est un objet réutilisable, pas une commande. C'est la duplication actuelle entre `agent.py` et `db.py` qui disparaît. + +```python +class AddDatabaseFlow: + def __init__(self, ui: UI, engines: EngineRegistry, ports: PortAllocator): ... + + def collect(self, values: dict) -> DatabaseSpec: + """values = flags parsés (engine, mode, auth, host…, options). + Champ manquant → prompt (interactif) / défaut / ValidationError (non-interactif).""" + + def apply(self, project: AgentProject, spec: DatabaseSpec) -> None: + """Mute project.env (variables du service si managed) et project.databases, en mémoire.""" +``` + +Séquence de `collect` : moteur (`--engine` ou select) → affiche `engine.warning` s'il existe → mode (`--mode` ou select ; sqlite et docker-volume n'ont pas de mode `existing`/`new` au sens service : sqlite distingue fichier créé vs chemin existant, docker-volume n'a qu'un mode) → variante auth si `engine.auth_variants` → `Form.collect(engine.fields_new() | fields_existing(), values)` → `Form.collect(engine.option_fields(), values["options"])` → `engine.generate(...)` ou construction depuis les réponses. + +Utilisation : + +- `DbAddCommand.run` : `AgentProject.load` → `templates.ensure()` → `flow.collect(flags)` → `flow.apply` → `renderer.render_agent` → `write`. +- `AgentCommand.run` (interactif seulement) : après le premier rendu, `while ui.confirm("Add a database?", default=True)` : `flow.collect({})` → `flow.apply` → rendu + écriture. Rendu après chaque ajout : un Ctrl-C au milieu laisse un état cohérent sur disque. + +`flows/` vit dans `commands/` parce qu'il prompte via `ui` ; il ne fait aucune I/O fichier (c'est `RenderResult.write` qui écrit). + +## 5. État, templates, rendu + +### 5.1 Modèle de données (`services/project.py`) + +Objets en mémoire construits depuis le disque, jamais persistés tels quels. + +```python +@dataclass(frozen=True) +class DatabaseSpec: + id: str; engine: str; name: str; managed: bool + host: str | None; port: int | None; database: str | None + username: str | None; password: str | None + path: str | None # sqlite + volume: str | None; container: str | None # docker-volume + options: dict + + @classmethod + def from_json(cls, raw: dict, env: EnvFile) -> "DatabaseSpec": ... + def to_json(self) -> dict: ... # format databases.json actuel, inchangé + @property + def env_prefix(self) -> str: ... # "db-pg-a1f2" → "DB_PG_A1F2" + + +@dataclass +class AgentProject: + path: Path; env: EnvFile; databases: list[DatabaseSpec]; host_gateway: bool + + @property + def needs_docker_socket(self) -> bool # une entrée docker-volume + @property + def managed(self) -> list[DatabaseSpec] + @property + def sqlite_mounts(self) -> list[tuple[str, str]] # database commence par /config/ → ./x:/config/x + + +@dataclass +class DashboardProject: + path: Path; env: EnvFile + @property + def db_mode(self) -> Literal["external", "custom", "internal"] + # POSTGRES_HOST absent → internal ; == "db" → external ; sinon custom +``` + +Détection `managed` : `.env` contient `{PREFIX}_PORT` pour ce `host` (toutes les bases `new` l'écrivent, aucune `existing`). Si l'agent tolère les clés inconnues dans `databases.json`, une clé explicite `managed: true` sera ajoutée et la détection deviendra le fallback — à vérifier côté agent. + +Cas limites : + +- `host` managé sans `{PREFIX}_PORT` dans `.env` → `ui.warning`, la base est traitée comme externe. +- Deux entrées avec le même `host` → `ConfigError` avant tout rendu. +- `.env` ou `databases.json` absent → `ConfigError("Not a Portabase agent folder")`. + +### 5.2 `EnvFile` (`services/envfile.py`) + +Remplace `write_env_file`. Parse `KEY="v"`, `KEY='v'`, `KEY=v`, `export KEY=`, commentaires, lignes vides. Conserve l'ordre et les commentaires (liste de lignes typées). `merge()` met à jour en place et ajoute en fin ; `remove(prefix)` retire les `PREFIX_*`. Écriture toujours quotée `"…"`, `"` et `\` échappés. Sauvegarde atomique (tmp + `os.replace`). + +`.env` ne contient que des variables consommées par les conteneurs. Aucune métadonnée CLI. + +### 5.3 `ComposeFacts` (`services/compose_facts.py`) + +`yaml.safe_load` du compose existant, lecture seule, jamais réécrit. Expose `host_gateway` (présence de `extra_hosts` sur `services.agent`, forme liste ou dict tolérée). Compose absent ou invalide → valeurs par défaut + `ui.warning`, jamais d'erreur. + +### 5.4 `TemplateRepository` (`services/templates.py`) + +- URL : `{TEMPLATE_BASE_URL}/{version}/manifest.json` puis fichiers listés. +- Résolution de version : `current_version()` ; sinon `PORTABASE_TEMPLATES_VERSION` ; sinon `PORTABASE_TEMPLATES_DIR` (court-circuite S3) ; sinon `TemplateError`. En dev non-frozen, `./templates` à côté de `main.py` est utilisé automatiquement s'il existe. +- Cache `~/.portabase/cache/templates//`. Séquence `ensure()` : GET manifest (10 s) → pour chaque fichier, sha256 identique en cache → skip, sinon GET + vérification sha256 et taille → écriture. Fichiers en cache absents du manifest supprimés. Manifest injoignable avec cache complet → warning et cache ; sans cache → `TemplateError` avec hint. +- Jinja2 : `Environment(undefined=StrictUndefined, keep_trailing_newline=True, autoescape=False)`. `{{ }}` ne collisionne pas avec `${}` Compose. + +Manifest : + +```json +{ + "schema": 1, + "version": "26.09.0", + "generated_at": "2026-09-11T14:02:17Z", + "commit": "858d4926…", + "files": { + "agent.yml.j2": { "sha256": "…", "size": 612 }, + "engines/postgresql.yml.j2": { "sha256": "…", "size": 498 } + }, + "engines": { + "postgresql": "engines/postgresql.yml.j2", + "postgresql-cluster": "engines/postgresql.yml.j2" + } +} +``` + +`schema` inconnu → `TemplateError`. `version` ≠ version demandée → `TemplateError`. `engines` sert à `engines-check` en CI et à `get_engine(key)`. + +### 5.5 `ComposeRenderer` (`services/renderer.py`) + +```python +class ComposeRenderer: + def __init__(self, templates: TemplateRepository, engines: EngineRegistry): ... + def render_agent(self, project: AgentProject, inline: bool = False) -> RenderResult: ... + def render_dashboard(self, project: DashboardProject, inline: bool = False) -> RenderResult: ... +``` + +Contexte `agent.yml.j2` : `host_gateway`, `docker_socket`, `mounts` (sqlite), `services` (liste de `{name, volume, body}` où `body` est le rendu du template moteur). Le renderer passe aux templates moteurs des variables **déjà formées** (`port_var = "${DB_PG_A1F2_PORT}"` ou valeur littérale si `inline`) : la logique de nommage reste en Python, les templates restent lisibles. + +`RenderResult` : `compose: str`, `databases: list[dict]`. `write(path)` valide d'abord (`yaml.safe_load` du compose → sinon `TemplateError`, un template remote cassé ne corrompt jamais une install), puis écrit `docker-compose.yml` et `databases.json` atomiquement. Le compose porte un en-tête `# Generated by Portabase CLI . Do not edit — use docker-compose.override.yml.` + +Ordre dans une commande mutante : collecte → `templates.ensure()` → mutation `.env`/`databases.json` en mémoire → rendu → validation → écriture. Le manifest est vérifié avant toute mutation. + +### 5.6 Templates + +`agent.yml.j2` : + +```jinja +services: + agent: + restart: unless-stopped + image: portabase/agent:latest + volumes: + - ./databases.json:/config/config.json +{%- for m in mounts %} + - {{ m.host }}:{{ m.container }} +{%- endfor %} +{%- if docker_socket %} + - /var/run/docker.sock:/var/run/docker.sock +{%- endif %} +{%- if host_gateway %} + extra_hosts: + - "localhost:host-gateway" +{%- endif %} + environment: + TZ: "${TZ}" + EDGE_KEY: "${EDGE_KEY}" + LOG_LEVEL: "${LOG_LEVEL}" + POLLING: "${POLLING}" + networks: + - portabase +{% for s in services %} +{{ s.body }} +{%- endfor %} +{% if services %} +volumes: +{%- for s in services %} + {{ s.volume }}: +{%- endfor %} +{% endif %} +networks: + portabase: + name: portabase_network + external: true +``` + +Templates moteurs : un par moteur, variante auth par `{% if auth %}` (10 snippets actuels → 8 templates ; `postgresql-cluster` réutilise `postgresql.yml.j2`). `dashboard.yml.j2` : `{% if db_mode == "external" %}` autour du service `db`, de `depends_on` et du volume — remplace les trois `re.sub` de `dashboard.py`. + +### 5.7 Installs legacy + +Aucun marqueur de version nécessaire. `AgentProject.load()` fonctionne sur toute install (`.env` + `databases.json` existent déjà). Au premier `RenderResult.write()` sur un compose sans l'en-tête `# Generated by Portabase CLI`, le fichier est copié en `docker-compose.legacy.yml` et un avertissement est affiché. `portabase build PATH --diff` permet de voir le diff avant. Les commandes `start/stop/logs` ne déclenchent rien. + +Différences attendues au premier rendu d'une install ancienne : `restart: unless-stopped` ajouté sur redis/valkey (absent des snippets actuels) ; à mentionner dans le changelog rc. + +## 6. Moteurs DB (`engines/`) + +```python +@dataclass(frozen=True) +class Field: + name: str; prompt: str + kind: Literal["text", "int", "secret", "bool", "choice"] + default: Any = None; choices: tuple[str, ...] = (); help: str | None = None + validator: Callable[[Any], Any] | None = None + + +class DbEngine(ABC): + key: str; display: str; default_port: int + template: str | None # None = aucun service Compose (sqlite, docker-volume, existing) + auth_variants: bool = False + warning: str | None = None + + def fields_existing(self) -> list[Field]: ... # défaut : host, port, database, username, password + def fields_new(self) -> list[Field]: ... # défaut : [] (tout généré) + def option_fields(self) -> list[Field]: ... # défaut : [] + def generate(self, service: str, auth: bool, ports: PortAllocator) -> DatabaseSpec: ... + def env_vars(self, spec: DatabaseSpec) -> dict[str, str]: ... + def template_ctx(self, spec: DatabaseSpec, inline: bool) -> dict: ... + def agent_entry(self, spec: DatabaseSpec) -> dict: ... # projection databases.json + def agent_database(self, spec: DatabaseSpec) -> str: ... # défaut : spec.database ; hook pour "0", chemin… +``` + +Hiérarchie : `StandardSqlEngine` (postgresql, postgresql-cluster, mysql, mariadb, mssql, firebird), `RedisEngine`, `ValkeyEngine`, `MongoEngine`, `SqliteEngine`, `DockerVolumeEngine`. Redis et Valkey sont deux classes indépendantes dans deux fichiers, sans base commune (images, commandes et healthchecks divergent ; ce qu'elles partagent — `agent_database = "0"`, `auth_variants` — passe par les hooks de `DbEngine`). Les sous-classes ne surchargent que leurs particularités : + +| Moteur | Particularité | +|---|---| +| postgresql | `option_fields` : `keep_ownership` (bool, défaut False), `clean_mode` (choice clean/none/drop_schemas/drop_database, défaut clean) | +| postgresql-cluster | `warning` superuser ; pas d'options | +| firebird | `agent_entry.name = "mirror.fdb"` ; var `_ROOT_PASS` | +| mssql | `agent_entry.username = "sa"` | +| redis | `agent_database = "0"` ; `auth_variants = True` ; no-auth → `_PORT` seul | +| valkey | idem redis, classe et template distincts | +| mongodb | `auth_variants = True` | +| sqlite | `fields_new` : nom de fichier ; `fields_existing` : chemin ; pas de template ; mount si chemin relatif | +| docker-volume | `fields` : volume, container (optionnel), label ; `warning` socket ; pas de template | + +`EngineRegistry` : dict `key → instance`, imports explicites (compatible PyInstaller). `get(key)` inconnu → `ValidationError` avec la liste des clés. + +### 6.1 Options moteur + +Certains moteurs exposent des options que l'agent lit dans `databases.json` (`options` : aujourd'hui `keep_ownership` et `clean_mode` pour PostgreSQL). Le système doit accepter de nouvelles options sans toucher à la signature Typer. + +- Déclaration : `DbEngine.option_fields() -> list[Field]`. Un `Field` comme les autres : nom, prompt, type, défaut, choix, validateur. +- Saisie non-interactive : flag générique répétable `-o KEY=VALUE` / `--option KEY=VALUE` sur `db add`. Parsé en `dict[str, str]`, converti selon `Field.kind` (`bool` : `true/false/1/0/yes/no`, `int`, `choice` validé contre `choices`). Clé inconnue pour ce moteur → `ValidationError` listant les options valides. +- Saisie interactive : `Form.collect(engine.option_fields(), values["options"])`, un prompt par option non fournie, avec le texte d'aide actuel (par exemple l'explication de `--no-owner` / `pg_restore --clean`) porté par `Field.help`. +- Stockage : `DatabaseSpec.options: dict` (valeurs typées). +- Projection : `agent_entry()` n'écrit dans `options` que les valeurs différentes du défaut. Comportement actuel conservé : `keep_ownership` absent si False, `clean_mode` absent si `clean`. Aucune clé `options` si vide. +- Affichage : `db list` montre les options non-défaut ; `Summary` les inclut lors de l'ajout. + +Ajouter une option = une ligne dans `option_fields()` du moteur concerné. + +## 7. `ui/` + +### 7.1 Principes + +- Tokens uniques (`ui/theme.py`) : `PALETTE` → `RICH_THEME` et `QUESTIONARY_STYLE`. +- Composants stateless, un par fichier, `Component(console)`. +- Façade `UI` : seule chose importée par `commands/`. Rich et questionary ne sont jamais importés hors de `ui/`. +- Le markup Rich est autorisé dans les arguments texte (`ui.success("Added [bold]x[/bold]")`). +- `NO_COLOR` / `--no-color` → `Console(no_color=True)`, style questionary vide. +- Jamais de prompt à l'intérieur d'un `ui.status()` (structurellement garanti : les services ne promptent pas). + +### 7.2 Composants + +| Composant | Remplace | +|---|---| +| `Banner` | `print_banner` | +| `Message` (`success/info/warning/error`) | ~60 `console.print("[success]✔ …")` | +| `Section` | `Panel("[bold]Database Setup[/bold]")` | +| `Summary` (masque auto des clés `password/secret/key`) | `Table(show_header=False)` de dashboard | +| `DataTable` | `Table` de `db list` | +| `Status` (context manager, hint injecté) | `console.status(msg + hint)` | +| `Hint` | `get_random_hint` | +| `Diff` | nouveau, pour `build --diff` | +| `Prompt` (`text/integer/secret/confirm/select/path`) | `rich.prompt.*`, `questionary.*` épars | + +Règle anti-dérive : un composant n'existe que s'il a un appelant. L'inventaire ci-dessus est un plafond. + +### 7.3 `Form` + +```python +class Form: + def ask(self, field: Field, value: Any | None) -> Any: + # 1. valeur du flag → validée + # 2. non-interactif : défaut, sinon ValidationError("Missing --") + # 3. interactif : prompt selon field.kind (dispatch dict), None (Ctrl-C) → UserAbort + # validation en boucle jusqu'à valeur acceptée + def collect(self, fields: list[Field], values: dict) -> dict: ... + def text(...), integer(...), confirm(...), choice(...) # raccourcis +``` + +`non_interactive` résolu une fois dans `main.py`. `ui.confirm()` en non-interactif renvoie le défaut ; les confirmations destructives ont `default=False` et un flag `--force`. + +## 8. Erreurs, télémétrie, updater + +### 8.1 Hiérarchie (`core/errors.py`) + +| Classe | `code` | exit | +|---|---|---| +| `PortabaseError` | `E_GENERIC` | 1 | +| `UserAbort` | `E_ABORT` | 130 | +| `ValidationError` | `E_VALIDATION` | 2 | +| `ConfigError` | `E_CONFIG` | 3 | +| `DockerError` | `E_DOCKER` | 4 | +| `TemplateError` | `E_TEMPLATE` | 5 | +| `NetworkError` | `E_NETWORK` | 6 | +| `UpdateError` | `E_UPDATE` | 7 | +| `DecryptionError` (`core/crypto.py`) | `E_CRYPTO` | 8 | + +Constructeur : `(message, *, hint=None, cause=None)`. Les exceptions tierces (`requests`, `subprocess`, `yaml`, `jinja2`) sont wrappées à la frontière du service. `typer.Exit` n'est plus levé hors de `main.py`. Ruff : `E722`, `BLE001`, `S110`, `TID251`. + +### 8.2 Catcher (`main.py`) + +`app(standalone_mode=False)` dans un seul `try` : `UserAbort` → "Cancelled." exit 130 ; `PortabaseError` → `ui.error(e)` (message, hint, code ; `--verbose` ajoute cause et traceback), `telemetry.error(e)`, exit `e.exit_code` ; `click.UsageError` → mappé en `ValidationError` ; `KeyboardInterrupt` → exit 130 ; `Exception` → "Unexpected error", télémétrie `unexpected=True`, exit 1. `finally: telemetry.flush()`. + +### 8.3 Télémétrie (`services/telemetry.py`) + +```python +class Telemetry(ABC): + def session(self, **attrs) -> ContextManager # span racine par invocation + def span(self, name: str, **attrs) -> ContextManager + def event(self, name: str, **attrs) -> None + def error(self, exc: Exception, unexpected: bool = False) -> None + def flush(self) -> None +``` + +Implémentations : `NoopTelemetry` (défaut), `ConsoleTelemetry` (`--debug`, stderr), `OtelTelemetry` (futur, import lazy, construit seulement si `telemetry=true` et `telemetry_endpoint` défini). Spans : `Command.run`, `TemplateRepository.ensure`, `ComposeRenderer.render`, `DockerRunner.compose`. Attributs : commande, moteur, mode, durée, code de sortie, `error.code`, version CLI, OS. Jamais : nom d'agent, chemin, clé, credentials, contenu de fichier. + +Opt-in : `portabase config set telemetry true` ou `PORTABASE_TELEMETRY=1`. Une ligne d'information à la première exécution, aucun prompt. + +### 8.4 Updater (`services/updater.py`) + +`UpdateChecker.notify(ui)` appelé après la commande, cache 24 h (`~/.portabase/cache/release.json`), silencieux si hors ligne, `--stdout` ou non-interactif. `Updater.apply()` vérifie le sha256 via `checksums.txt` de la release avant remplacement du binaire. Canal `beta` conservé. + +## 9. CI, sécurité, release + +### 9.1 `ci.yml` (`pull_request`, `push: main`) + +| Job | Contenu | +|---|---| +| `lint` | `ruff check`, `ruff format --check` | +| `render-check` | `scripts/render_check.py` : rend `agent.yml.j2` (0 base, chaque moteur auth/no-auth, socket, host_gateway, mounts sqlite) et `dashboard.yml.j2` × 3 modes via le vrai `ComposeRenderer` (`PORTABASE_TEMPLATES_DIR=./templates`), puis `yaml.safe_load` et `docker compose config` avec `.env` fixture | +| `engines-check` | chaque `DbEngine.template` existe dans `templates/`, chaque template a un moteur, `engines.map.json` cohérent | +| `gitleaks` | action pinnée ; `.gitleaks.toml` allowlist `templates/**` et `HINTS` | +| `plumber` | action drop-in, `verify-attestation: true` | +| `build-smoke` | PyInstaller linux/amd64, `./dist/portabase --version`, `agent smoke --key --non-interactive` avec templates locaux | +| `test` | `pytest` — vide, réservé à la spec tests | + +### 9.2 Durcissement + +- Toutes les actions pinnées par SHA avec commentaire de version ; Dependabot `github-actions` et `uv` hebdomadaires. +- `permissions: {}` au top de chaque workflow, permissions explicites par job. `packages: write` retiré (inutilisé). +- `templates-upload.yml` : plus de `~/.s3cfg` par heredoc ; credentials par variables d'environnement. +- `actions/attest-build-provenance` sur les binaires. + +### 9.3 Release + +`bump.yml` (`workflow_dispatch`, inputs `version`, `channel: stable|rc`) : validation regex, `stable` uniquement depuis `main`, `sed` `pyproject.toml` + `CITATION.cff`, commit `chore(release): X`, tag, push. Les workflows sur tag restent inchangés. `./release` supprimé. Pas de release-please (historique non conventional). Si `main` exige une PR, le workflow ouvre une PR au lieu de pousser — à régler selon la protection de branche. + +`templates-hotfix.yml` (`workflow_dispatch`, input `version`) : re-sync `templates/` vers `templates//` et régénère le manifest. Réservé aux corrections compatibles avec le code de cette version. + +`templates-upload.yml` : source `templates/`, génération de `manifest.json` (sha256, taille, version, commit, date, mapping moteurs depuis `engines.map.json`) avant `s3cmd sync`. + +### 9.4 `pyproject.toml` + +```toml +dependencies = ["typer", "rich", "questionary", "requests", "pyyaml", "jinja2"] +[dependency-groups] +dev = ["pyinstaller", "ruff", "pytest"] +``` + +## 10. Ordre des chantiers + +Graphe de dépendances, pas un calendrier. + +``` +[A] Hygiène CI ─────────────────────────────────────────────┐ indépendant + ci.yml, pin SHA, permissions, bump.yml, pyproject │ + │ +[B] Fondations │ + core/errors, ui/, services/{envfile,docker,http,ports}, │ + main.py catcher, Command ABC │ + │ │ + ├──► [C] Lifecycle en POO (start/stop/…/config/update) + │ (ancien code agent/db/dashboard via LegacyCommand) + │ + └──► [D] Templates .j2 + TemplateRepository + engines/ + render-check + │ + ▼ + [E] Rendu : project, compose_facts, renderer, build + │ + ▼ + [F] agent / dashboard / db réécrits, ancien code supprimé + │ + ▼ + [G] OTel réel, --json, spec tests +``` + +Contraintes : + +- B avant tout code métier. +- D avant E (le renderer se construit contre des templates réels). +- E avant F. +- C et F ne touchent pas les mêmes fichiers ; C peut aller avant ou après D/E. +- A avant F de préférence : `render-check` et `build-smoke` sont le seul filet avant la spec tests. + +Points de livraison : + +| Après | État | Canal | +|---|---|---| +| A | fonctionnellement identique, CI verte | stable | +| B + C | lifecycle en POO, ui/ et erreurs neuves ; `agent`/`db`/`dashboard` = ancien code via `LegacyCommand` | stable | +| D | templates `.j2` uploadés sous la nouvelle version ; l'ancien code lit `agent.yml`, coexistence sur S3 | rc | +| E + F | bascule complète | rc obligatoire, puis stable | + +Risques et parades : + +| Étape | Risque | Parade | +|---|---|---| +| A | mauvais SHA casse un workflow | tag rc jetable | +| B | sur-conception de `ui/` | un composant = un appelant | +| D | template `.j2` diverge d'un snippet actuel | diff manuel des rendus contre l'ancien CLI, une fois | +| E | `ComposeFacts` lit mal un vieux compose | tolérance, warning, jamais de crash | +| F | install legacy cassée après `db add` | `.legacy.yml`, `build --diff`, changelog rc | +| F | mots de passe existants avec `$` | ne pas régénérer ; correction pour les nouvelles bases seulement | + +## 11. Questions ouvertes + +- L'agent tolère-t-il des clés inconnues dans `databases.json` ? Si oui : clé `managed: true` explicite. +- Protection de la branche `main` : `bump.yml` pousse directement ou ouvre une PR ? +- Garder la clé `engines` dans le manifest (double source de vérité avec le code) ou s'en tenir au registre Python ? diff --git a/engines/__init__.py b/engines/__init__.py new file mode 100644 index 0000000..4422409 --- /dev/null +++ b/engines/__init__.py @@ -0,0 +1,65 @@ +from __future__ import annotations + +from collections.abc import Iterable, Iterator + +from core.errors import ValidationError +from engines.base import DbEngine +from engines.docker_volume import DockerVolumeEngine +from engines.firebird import FirebirdEngine +from engines.mariadb import MariaDbEngine +from engines.mongodb import MongoEngine +from engines.mssql import MssqlEngine +from engines.mysql import MySqlEngine +from engines.postgresql import PostgresClusterEngine, PostgresEngine +from engines.redis import RedisEngine +from engines.sqlite import SqliteEngine +from engines.valkey import ValkeyEngine + + +class EngineRegistry: + def __init__(self, engines: Iterable[DbEngine]) -> None: + self._by_key: dict[str, DbEngine] = {} + for engine in engines: + if engine.key in self._by_key: + raise ValueError(f"Duplicate engine key: {engine.key}") + self._by_key[engine.key] = engine + + def get(self, key: str) -> DbEngine: + try: + return self._by_key[key] + except KeyError: + raise ValidationError( + f"Unknown engine '{key}'.", + hint="Available: " + ", ".join(self.keys()), + ) from None + + def keys(self) -> list[str]: + return list(self._by_key) + + def choices(self) -> list[str]: + return self.keys() + + def __iter__(self) -> Iterator[DbEngine]: + return iter(self._by_key.values()) + + def __contains__(self, key: str) -> bool: + return key in self._by_key + + +ALL = ( + PostgresEngine(), + PostgresClusterEngine(), + MySqlEngine(), + MariaDbEngine(), + SqliteEngine(), + FirebirdEngine(), + MongoEngine(), + RedisEngine(), + ValkeyEngine(), + MssqlEngine(), + DockerVolumeEngine(), +) + +registry = EngineRegistry(ALL) + +__all__ = ["ALL", "EngineRegistry", "registry"] diff --git a/engines/base.py b/engines/base.py new file mode 100644 index 0000000..5107c10 --- /dev/null +++ b/engines/base.py @@ -0,0 +1,175 @@ +from __future__ import annotations + +import secrets +import uuid +from abc import ABC, abstractmethod +from typing import Any + +from core.fields import Field +from core.specs import DatabaseSpec +from core.utils import generate_password +from services.ports import PortAllocator + +STANDARD_EXISTING_FIELDS = ( + Field("host", "Host", "text", default="localhost"), + Field("port", "Port", "int"), + Field("database", "Database Name", "text"), + Field("username", "Username", "text"), + Field("password", "Password", "secret"), +) + + +class DbEngine(ABC): + key: str + display: str + default_port: int | None = None + template: str | None = None + auth_variants: bool = False + warning: str | None = None + has_modes: bool = True + label_default: str = "External DB" + + abstract: bool = False + required: tuple[str, ...] = ("key", "display") + + def __init_subclass__(cls, **kwargs: Any) -> None: + super().__init_subclass__(**kwargs) + cls.abstract = "abstract" in cls.__dict__ and cls.__dict__["abstract"] + if cls.abstract: + return + missing = [name for name in cls.required if not getattr(cls, name, None)] + if missing: + raise TypeError( + f"{cls.__module__}.{cls.__qualname__} is missing required engine " + f"attribute(s): {', '.join(missing)}. Set them as class attributes, " + "or set `abstract = True` if this class is only a base for others." + ) + if cls.template is not None and not cls.template.startswith("engines/"): + raise TypeError( + f"{cls.__qualname__}.template must be a path under 'engines/', " + f"got {cls.template!r} (e.g. 'engines/{cls.key}.yml.j2')." + ) + + def fields_existing(self) -> list[Field]: + return [ + Field("port", "Port", "int", default=self.default_port) + if f.name == "port" + else f + for f in STANDARD_EXISTING_FIELDS + ] + + def fields_new(self) -> list[Field]: + return [] + + def option_fields(self) -> list[Field]: + return [] + + @abstractmethod + def generate( + self, *, auth: bool, ports: PortAllocator, answers: dict[str, Any] + ) -> DatabaseSpec: ... + + def from_existing(self, answers: dict[str, Any]) -> DatabaseSpec: + return DatabaseSpec( + id=self.new_id(), + engine=self.key, + name=answers.get("label") or self.label_default, + managed=False, + host=answers["host"], + port=int(answers["port"]), + database=answers["database"], + username=answers["username"], + password=answers["password"], + ) + + def env_vars(self, spec: DatabaseSpec) -> dict[str, str]: + p = spec.env_prefix + return { + f"{p}_PORT": str(spec.host_port), + f"{p}_DB": spec.database or "", + f"{p}_USER": spec.username or "", + f"{p}_PASS": spec.password or "", + } + + def template_ctx( + self, spec: DatabaseSpec, *, inline: bool = False + ) -> dict[str, Any]: + return { + "name": spec.host, + "volume": f"{spec.host}-data", + "auth": spec.auth, + "port_var": self.var(spec, "PORT", spec.host_port, inline), + "db_var": self.var(spec, "DB", spec.database, inline), + "user_var": self.var(spec, "USER", spec.username, inline), + "password_var": self.var(spec, "PASS", spec.password, inline), + } + + def agent_entry(self, spec: DatabaseSpec) -> dict[str, Any]: + entry: dict[str, Any] = { + "name": spec.name, + "database": self.agent_database(spec), + "type": self.key, + "username": spec.username or "", + "password": spec.password or "", + "port": spec.port, + "host": spec.host, + "generated_id": spec.id, + } + options = self.non_default_options(spec) + if options: + entry["options"] = options + return entry + + def agent_database(self, spec: DatabaseSpec) -> str: + return spec.database or "" + + def describe(self, spec: DatabaseSpec) -> str: + return f"{spec.host}:{spec.port}" + + def non_default_options(self, spec: DatabaseSpec) -> dict[str, Any]: + defaults = {f.name: f.default for f in self.option_fields()} + return { + k: v for k, v in spec.options.items() if k in defaults and v != defaults[k] + } + + @staticmethod + def new_id() -> str: + return str(uuid.uuid4()) + + @staticmethod + def service_name(slug: str, auth: bool = False) -> str: + suffix = "auth-" if auth else "" + return f"db-{slug}-{suffix}{secrets.token_hex(2)}" + + @staticmethod + def var(spec: DatabaseSpec, suffix: str, value: Any, inline: bool) -> str: + if inline: + return str(value if value is not None else "") + return f"${{{spec.env_prefix}_{suffix}}}" + + +class StandardSqlEngine(DbEngine): + abstract = True + required = (*DbEngine.required, "template", "default_port", "slug", "db_prefix") + + slug: str + db_prefix: str + default_user = "admin" + + def generate( + self, *, auth: bool, ports: PortAllocator, answers: dict[str, Any] + ) -> DatabaseSpec: + db_name = f"{self.db_prefix}_{secrets.token_hex(4)}" + return DatabaseSpec( + id=self.new_id(), + engine=self.key, + name=db_name, + managed=True, + host=self.service_name(self.slug), + port=self.default_port, + host_port=ports.free(), + database=db_name, + username=self.default_user, + password=generate_password(16), + options=dict(answers.get("options", {})), + ) diff --git a/engines/docker_volume.py b/engines/docker_volume.py new file mode 100644 index 0000000..8e37c97 --- /dev/null +++ b/engines/docker_volume.py @@ -0,0 +1,65 @@ +from __future__ import annotations + +from typing import Any + +from core.fields import Field +from core.specs import DatabaseSpec +from engines.base import DbEngine +from services.ports import PortAllocator + + +class DockerVolumeEngine(DbEngine): + key, display = "docker-volume", "Docker Volume" + template = None + has_modes = False + label_default = "Docker Volume" + warning = ( + "Requires the Docker socket. It will be mounted on the agent " + "(/var/run/docker.sock)." + ) + + def fields_existing(self) -> list[Field]: + return [ + Field("volume", "Volume Name (e.g. databases_sqlite-data)", "text"), + Field( + "container", + "Container Name (optional, enables auto-restart after restore)", + "text", + default="", + ), + ] + + def fields_new(self) -> list[Field]: + return self.fields_existing() + + def generate( + self, *, auth: bool, ports: PortAllocator, answers: dict[str, Any] + ) -> DatabaseSpec: + return self.from_existing(answers) + + def from_existing(self, answers: dict[str, Any]) -> DatabaseSpec: + return DatabaseSpec( + id=self.new_id(), + engine=self.key, + name=answers.get("label") or self.label_default, + managed=False, + volume=str(answers["volume"]).strip(), + container=(str(answers.get("container") or "").strip() or None), + ) + + def env_vars(self, spec: DatabaseSpec) -> dict[str, str]: + return {} + + def agent_entry(self, spec: DatabaseSpec) -> dict[str, Any]: + entry = { + "name": spec.name, + "type": self.key, + "volume_name": spec.volume, + "generated_id": spec.id, + } + if spec.container: + entry["container_name"] = spec.container + return entry + + def describe(self, spec: DatabaseSpec) -> str: + return f"volume: {spec.volume}" diff --git a/engines/firebird.py b/engines/firebird.py new file mode 100644 index 0000000..be17def --- /dev/null +++ b/engines/firebird.py @@ -0,0 +1,52 @@ +from __future__ import annotations + +from typing import Any + +from core.specs import DatabaseSpec +from core.utils import generate_password +from engines.base import StandardSqlEngine +from services.ports import PortAllocator + + +class FirebirdEngine(StandardSqlEngine): + key, display, default_port = "firebird", "Firebird", 3050 + template, slug, db_prefix = "engines/firebird.yml.j2", "firebird", "fb" + DATA_DIR = "/var/lib/firebird/data" + + def generate( + self, *, auth: bool, ports: PortAllocator, answers: dict[str, Any] + ) -> DatabaseSpec: + db_file = "mirror.fdb" + return DatabaseSpec( + id=self.new_id(), + engine=self.key, + name=db_file, + managed=True, + host=self.service_name(self.slug), + port=self.default_port, + host_port=ports.free(), + database=f"{self.DATA_DIR}/{db_file}", + username="alice", + password=generate_password(16), + root_password=generate_password(16), + ) + + @staticmethod + def _file_name(spec: DatabaseSpec) -> str: + return (spec.database or "").rsplit("/", 1)[-1] + + def env_vars(self, spec: DatabaseSpec) -> dict[str, str]: + base = super().env_vars(spec) + base[f"{spec.env_prefix}_DB"] = self._file_name(spec) + base[f"{spec.env_prefix}_ROOT_PASS"] = spec.root_password or "" + return base + + def template_ctx( + self, spec: DatabaseSpec, *, inline: bool = False + ) -> dict[str, Any]: + ctx = super().template_ctx(spec, inline=inline) + ctx["db_var"] = self.var(spec, "DB", self._file_name(spec), inline) + ctx["root_password_var"] = self.var( + spec, "ROOT_PASS", spec.root_password, inline + ) + return ctx diff --git a/engines/mariadb.py b/engines/mariadb.py new file mode 100644 index 0000000..ea19c40 --- /dev/null +++ b/engines/mariadb.py @@ -0,0 +1,8 @@ +from __future__ import annotations + +from engines.base import StandardSqlEngine + + +class MariaDbEngine(StandardSqlEngine): + key, display, default_port = "mariadb", "MariaDB", 3306 + template, slug, db_prefix = "engines/mariadb.yml.j2", "mariadb", "mysql" diff --git a/engines/mongodb.py b/engines/mongodb.py new file mode 100644 index 0000000..fca6a4b --- /dev/null +++ b/engines/mongodb.py @@ -0,0 +1,40 @@ +from __future__ import annotations + +import secrets +from typing import Any + +from core.specs import DatabaseSpec +from core.utils import generate_password +from engines.base import DbEngine +from services.ports import PortAllocator + + +class MongoEngine(DbEngine): + key, display, default_port = "mongodb", "MongoDB", 27017 + template = "engines/mongodb.yml.j2" + auth_variants = True + + def generate( + self, *, auth: bool, ports: PortAllocator, answers: dict[str, Any] + ) -> DatabaseSpec: + db_name = f"mongo_{secrets.token_hex(4)}" + return DatabaseSpec( + id=self.new_id(), + engine=self.key, + name=db_name, + managed=True, + host=self.service_name("mongo", auth), + port=self.default_port, + host_port=ports.free(), + database=db_name, + username="admin" if auth else "", + password=generate_password(16) if auth else None, + ) + + def env_vars(self, spec: DatabaseSpec) -> dict[str, str]: + p = spec.env_prefix + out = {f"{p}_PORT": str(spec.host_port), f"{p}_DB": spec.database or ""} + if spec.auth: + out[f"{p}_USER"] = spec.username or "" + out[f"{p}_PASS"] = spec.password or "" + return out diff --git a/engines/mssql.py b/engines/mssql.py new file mode 100644 index 0000000..2d110bc --- /dev/null +++ b/engines/mssql.py @@ -0,0 +1,33 @@ +from __future__ import annotations + +from typing import Any + +from core.specs import DatabaseSpec +from core.utils import generate_password +from engines.base import StandardSqlEngine +from services.ports import PortAllocator + + +class MssqlEngine(StandardSqlEngine): + key, display, default_port = "mssql", "Microsoft SQL Server", 1433 + template, slug, db_prefix = "engines/mssql.yml.j2", "mssql", "master" + + def generate( + self, *, auth: bool, ports: PortAllocator, answers: dict[str, Any] + ) -> DatabaseSpec: + return DatabaseSpec( + id=self.new_id(), + engine=self.key, + name="MSSQL", + managed=True, + host=self.service_name(self.slug), + port=self.default_port, + host_port=ports.free(), + database="master", + username="sa", + password=generate_password(16), + ) + + def env_vars(self, spec: DatabaseSpec) -> dict[str, str]: + p = spec.env_prefix + return {f"{p}_PORT": str(spec.host_port), f"{p}_PASS": spec.password or ""} diff --git a/engines/mysql.py b/engines/mysql.py new file mode 100644 index 0000000..8cea92d --- /dev/null +++ b/engines/mysql.py @@ -0,0 +1,8 @@ +from __future__ import annotations + +from engines.mariadb import MariaDbEngine + + +class MySqlEngine(MariaDbEngine): + key, display = "mysql", "MySQL" + template = "engines/mysql.yml.j2" diff --git a/engines/postgresql.py b/engines/postgresql.py new file mode 100644 index 0000000..79b74dd --- /dev/null +++ b/engines/postgresql.py @@ -0,0 +1,49 @@ +from __future__ import annotations + +from core.fields import Field +from engines.base import StandardSqlEngine + + +class PostgresEngine(StandardSqlEngine): + key, display, default_port = "postgresql", "PostgreSQL", 5432 + template, slug, db_prefix = "engines/postgresql.yml.j2", "pg", "pg" + + def option_fields(self) -> list[Field]: + return [ + Field( + "keep_ownership", + "Keep ownership?", + "bool", + default=False, + help=( + "When enabled, omits --no-owner and --no-privileges from the dump. " + "Ownership and role assignments are preserved. By default these " + "flags are applied to keep restores portable across users and " + "environments." + ), + ), + Field( + "clean_mode", + "Clean mode", + "choice", + default="clean", + choices=("clean", "none", "drop_schemas", "drop_database"), + help=( + "How the target database is cleaned before a restore. clean: " + "pg_restore --clean --if-exists. none: no pre-clean. drop_schemas: " + "drop every non-system schema CASCADE (works on managed Postgres). " + "drop_database: DROP DATABASE + CREATE DATABASE — requires CREATEDB " + "or superuser; most managed providers do not allow it." + ), + ), + ] + + +class PostgresClusterEngine(StandardSqlEngine): + key, display, default_port = "postgresql-cluster", "PostgreSQL Cluster", 5432 + template, slug, db_prefix = "engines/postgresql-cluster.yml.j2", "pg", "pg" + warning = ( + "Postgres Cluster requires a superuser. Cluster backup/restore uses " + "pg_dumpall, which dumps all databases and global objects (roles, " + "tablespaces). The provided user must be a Postgres superuser." + ) diff --git a/engines/redis.py b/engines/redis.py new file mode 100644 index 0000000..0a872a8 --- /dev/null +++ b/engines/redis.py @@ -0,0 +1,51 @@ +from __future__ import annotations + +import secrets +from typing import Any + +from core.fields import Field +from core.specs import DatabaseSpec +from core.utils import generate_password +from engines.base import DbEngine +from services.ports import PortAllocator + + +class RedisEngine(DbEngine): + key, display, default_port = "redis", "Redis", 6379 + template = "engines/redis.yml.j2" + auth_variants = True + + def fields_existing(self) -> list[Field]: + return [ + Field("host", "Host", "text", default="localhost"), + Field("port", "Port", "int", default=self.default_port), + Field("database", "Database index", "text", default="0"), + Field("username", "Username (empty if none)", "text", default=""), + Field("password", "Password (empty if none)", "text", default=""), + ] + + def generate( + self, *, auth: bool, ports: PortAllocator, answers: dict[str, Any] + ) -> DatabaseSpec: + return DatabaseSpec( + id=self.new_id(), + engine=self.key, + name=f"redis_{secrets.token_hex(4)}", + managed=True, + host=self.service_name("redis", auth), + port=self.default_port, + host_port=ports.free(), + database="0", + username="", + password=generate_password(16) if auth else None, + ) + + def env_vars(self, spec: DatabaseSpec) -> dict[str, str]: + p = spec.env_prefix + out = {f"{p}_PORT": str(spec.host_port)} + if spec.auth: + out[f"{p}_PASS"] = spec.password or "" + return out + + def agent_database(self, spec: DatabaseSpec) -> str: + return spec.database or "0" diff --git a/engines/sqlite.py b/engines/sqlite.py new file mode 100644 index 0000000..232d76d --- /dev/null +++ b/engines/sqlite.py @@ -0,0 +1,70 @@ +from __future__ import annotations + +from typing import Any + +from core.fields import Field +from core.specs import DatabaseSpec +from engines.base import DbEngine +from services.ports import PortAllocator + +CONFIG_DIR = "/config" + + +class SqliteEngine(DbEngine): + key, display = "sqlite", "SQLite" + template = None + auth_variants = False + + def fields_existing(self) -> list[Field]: + return [Field("path", "Database Path (relative or absolute)", "text")] + + def fields_new(self) -> list[Field]: + return [Field("name", "Database Name", "text", default="local")] + + def generate( + self, *, auth: bool, ports: PortAllocator, answers: dict[str, Any] + ) -> DatabaseSpec: + name = str(answers.get("name") or "local") + if not name.endswith(".sqlite"): + name += ".sqlite" + return DatabaseSpec( + id=self.new_id(), + engine=self.key, + name=name, + managed=False, + path=name, + database=f"{CONFIG_DIR}/{name}", + ) + + def from_existing(self, answers: dict[str, Any]) -> DatabaseSpec: + raw = str(answers["path"]) + absolute = raw.startswith("/") + return DatabaseSpec( + id=self.new_id(), + engine=self.key, + name=answers.get("label") or self.label_default, + managed=False, + path=raw, + database=raw if absolute else f"{CONFIG_DIR}/{raw}", + ) + + @staticmethod + def mount_for(spec: DatabaseSpec) -> tuple[str, str] | None: + if spec.database and spec.database.startswith(f"{CONFIG_DIR}/"): + rel = spec.database[len(CONFIG_DIR) + 1 :] + return (f"./{rel}", spec.database) + return None + + def env_vars(self, spec: DatabaseSpec) -> dict[str, str]: + return {} + + def agent_entry(self, spec: DatabaseSpec) -> dict[str, Any]: + return { + "name": spec.name, + "database": spec.database, + "type": self.key, + "generated_id": spec.id, + } + + def describe(self, spec: DatabaseSpec) -> str: + return "Local File" diff --git a/engines/valkey.py b/engines/valkey.py new file mode 100644 index 0000000..c9a95c6 --- /dev/null +++ b/engines/valkey.py @@ -0,0 +1,51 @@ +from __future__ import annotations + +import secrets +from typing import Any + +from core.fields import Field +from core.specs import DatabaseSpec +from core.utils import generate_password +from engines.base import DbEngine +from services.ports import PortAllocator + + +class ValkeyEngine(DbEngine): + key, display, default_port = "valkey", "Valkey", 6379 + template = "engines/valkey.yml.j2" + auth_variants = True + + def fields_existing(self) -> list[Field]: + return [ + Field("host", "Host", "text", default="localhost"), + Field("port", "Port", "int", default=self.default_port), + Field("database", "Database index", "text", default="0"), + Field("username", "Username (empty if none)", "text", default=""), + Field("password", "Password (empty if none)", "text", default=""), + ] + + def generate( + self, *, auth: bool, ports: PortAllocator, answers: dict[str, Any] + ) -> DatabaseSpec: + return DatabaseSpec( + id=self.new_id(), + engine=self.key, + name=f"valkey_{secrets.token_hex(4)}", + managed=True, + host=self.service_name("valkey", auth), + port=self.default_port, + host_port=ports.free(), + database="0", + username="", + password=generate_password(16) if auth else None, + ) + + def env_vars(self, spec: DatabaseSpec) -> dict[str, str]: + p = spec.env_prefix + out = {f"{p}_PORT": str(spec.host_port)} + if spec.auth: + out[f"{p}_PASS"] = spec.password or "" + return out + + def agent_database(self, spec: DatabaseSpec) -> str: + return spec.database or "0" diff --git a/main.py b/main.py index 320a703..fe5da03 100644 --- a/main.py +++ b/main.py @@ -1,91 +1,215 @@ -from typing import Optional +import os +import platform +import sys +from dataclasses import dataclass +from typing import Annotated +import click import typer -from commands import agent, common, config, dashboard, db, decrypt -from core.updater import check_for_updates, update_cli -from core.utils import console, current_version - -app = typer.Typer( - no_args_is_help=True, - add_completion=False, +from commands.agent import AgentCommand +from commands.build import BuildCommand +from commands.config import ConfigCommands +from commands.dashboard import DashboardCommand +from commands.db import DbCommands +from commands.decrypt import DecryptCommand +from commands.lifecycle import ( + LogsCommand, + RestartCommand, + StartCommand, + StopCommand, + UninstallCommand, ) +from commands.update import UpdateCommand +from core.config import GlobalConfig +from core.errors import PortabaseError, UserAbort, ValidationError +from core.version import current_version +from engines import registry as engine_registry +from services.docker import DockerRunner +from services.http import HttpClient +from services.ports import PortAllocator +from services.renderer import ComposeRenderer +from services.telemetry import NoopTelemetry, Telemetry +from services.templates import TemplateRepository +from services.updater import UpdateChecker, Updater, is_frozen +from ui import UI + + +@dataclass +class Settings: + non_interactive: bool = False + verbose: bool = False + no_color: bool = False + + @classmethod + def from_env(cls, argv: list[str]) -> "Settings": + env_flag = os.environ.get("PORTABASE_NON_INTERACTIVE", "").lower() + settings = cls( + non_interactive=env_flag in ("1", "true", "yes") or not sys.stdin.isatty(), + no_color=bool(os.environ.get("NO_COLOR")) or "--no-color" in argv, + ) + if settings.no_color: + # Typer renders --help with its own Rich console, which only honors + # the NO_COLOR convention; --help is handled before any callback runs. + os.environ["NO_COLOR"] = "1" + return settings + + +def build_app( + ui: UI, telemetry: Telemetry, config: GlobalConfig, settings: Settings +) -> tuple[typer.Typer, UpdateChecker]: + app = typer.Typer( + no_args_is_help=True, add_completion=False, rich_markup_mode="rich" + ) + http = HttpClient() + docker = DockerRunner() + version = current_version() + checker = UpdateChecker(http, config, version) + updater = Updater(http, version) + templates = TemplateRepository.bundled() + ports = PortAllocator() + renderer = ComposeRenderer(templates, engine_registry, version) + + def version_callback(value: bool) -> None: + if value: + ui.print(f"Portabase CLI version: {version}") + latest = checker.available(force=True) + if latest: + ui.warning(f"A new version is available: [bold]{latest}[/bold]") + raise typer.Exit() + + @app.callback( + help="Portabase CLI to manage agents, dashboards and databases.", + invoke_without_command=True, + ) + def root( + ctx: typer.Context, + _version: Annotated[ + bool | None, + typer.Option( + "--version", + help="Show the version and exit.", + callback=version_callback, + is_eager=True, + ), + ] = None, + verbose: Annotated[ + bool, typer.Option("--verbose", help="Show error causes and tracebacks.") + ] = False, + no_color: Annotated[ + bool, typer.Option("--no-color", help="Disable colors.") + ] = False, + non_interactive: Annotated[ + bool, + typer.Option( + "--non-interactive", + envvar="PORTABASE_NON_INTERACTIVE", + help="Never prompt; fail on missing input.", + ), + ] = False, + ) -> None: + settings.verbose = verbose + settings.no_color = settings.no_color or no_color + settings.non_interactive = settings.non_interactive or non_interactive + ui.configure( + verbose=settings.verbose, + no_color=settings.no_color, + non_interactive=settings.non_interactive, + ) + if ctx.invoked_subcommand is None: + ui.out(ctx.get_help() + "\n") + raise typer.Exit() + + commands = [ + AgentCommand( + ui, telemetry, docker, templates, renderer, engine_registry, ports + ), + DashboardCommand(ui, telemetry, docker, templates, renderer, ports), + StartCommand(ui, telemetry, docker), + StopCommand(ui, telemetry, docker), + RestartCommand(ui, telemetry, docker), + LogsCommand(ui, telemetry, docker), + UninstallCommand(ui, telemetry, docker), + BuildCommand(ui, telemetry, templates, renderer), + DecryptCommand(ui, telemetry), + UpdateCommand(ui, telemetry, checker, updater), + ] + for cmd in commands: + cmd.register(app) + + DbCommands( + ui, telemetry, engine_registry, ports, templates, renderer, docker + ).register(app) + ConfigCommands(ui, telemetry, config).register(app) + return app, checker + + +def _notify_update( + ui: UI, checker: UpdateChecker, settings: Settings, invoked: str | None +) -> None: + if not is_frozen() or settings.non_interactive or invoked in ("update", None): + return + if "--stdout" in sys.argv: + return + latest = checker.available() + if latest: + ui.print("") + ui.warning( + f"A new version of Portabase CLI is available: [bold]{latest}[/bold] " + f"(current: {checker.current})" + ) + ui.info("Run [bold]portabase update[/bold] to update.") + + +def main() -> None: + settings = Settings.from_env(sys.argv[1:]) + config = GlobalConfig() + ui = UI(non_interactive=settings.non_interactive, no_color=settings.no_color) + telemetry = NoopTelemetry() + app, checker = build_app(ui, telemetry, config, settings) + invoked = next((a for a in sys.argv[1:] if not a.startswith("-")), None) + exit_code = 0 + + try: + with telemetry.session(cli_version=current_version(), os=platform.system()): + result = app(standalone_mode=False) + if isinstance(result, int): + exit_code = result + except UserAbort as e: + ui.warning(e.message) + telemetry.event("abort") + exit_code = e.exit_code + except PortabaseError as e: + ui.error(e) + telemetry.error(e) + exit_code = e.exit_code + except click.exceptions.NoArgsIsHelpError: + exit_code = 0 + except click.exceptions.Exit as e: + exit_code = e.exit_code + except click.UsageError as e: + err = ValidationError( + e.format_message(), hint="Run 'portabase --help' for usage." + ) + ui.error(err) + telemetry.error(err) + exit_code = err.exit_code + except KeyboardInterrupt: + ui.print("") + ui.warning("Canceled.") + exit_code = 130 + except Exception as e: # noqa: BLE001 — last resort: a bug, not an expected error + wrapped = PortabaseError("Unexpected error: " + str(e), cause=e) + ui.error(wrapped, unexpected=True) + telemetry.error(e, unexpected=True) + exit_code = 1 + finally: + telemetry.flush() + if exit_code == 0: + _notify_update(ui, checker, settings, invoked) + raise SystemExit(exit_code) -def version_callback(value: bool): - if value: - console.print(f"Portabase CLI version: {current_version()}") - check_for_updates(force=True) - raise typer.Exit() - - -@app.callback() -def main( - ctx: typer.Context, - _: Optional[bool] = typer.Option( - None, - "--version", - help="Show the version and exit.", - callback=version_callback, - is_eager=True, - ), -): - """ - Portabase CLI to manage agents, dashboards and databases. - """ - if ctx.invoked_subcommand != "update": - check_for_updates() - - -@app.command(help="Update the CLI to the latest version.", rich_help_panel="System") -def update(): - update_cli() - - -app.command( - help="Create a new Portabase Agent instance.", - rich_help_panel="Creation", - no_args_is_help=True, -)(agent.agent) -app.command( - help="Create a new Portabase Dashboard instance.", - rich_help_panel="Creation", - no_args_is_help=True, -)(dashboard.dashboard) -app.command( - help="Start a Portabase component.", - rich_help_panel="Lifecycle", - no_args_is_help=True, -)(common.start) -app.command( - help="Stop a Portabase component.", - rich_help_panel="Lifecycle", - no_args_is_help=True, -)(common.stop) -app.command( - help="Restart a Portabase component.", - rich_help_panel="Lifecycle", - no_args_is_help=True, -)(common.restart) -app.command( - help="View logs of a Portabase component.", - rich_help_panel="Lifecycle", - no_args_is_help=True, -)(common.logs) -app.command( - help="Uninstall and delete a Portabase component.", - rich_help_panel="Lifecycle", - no_args_is_help=True, -)(common.uninstall) - -app.command( - help="Decrypt Portabase .enc backup files (single file or folder).", - rich_help_panel="Configuration", - no_args_is_help=True, -)(decrypt.decrypt) - -app.add_typer(db.app, name="db", rich_help_panel="Configuration") -app.add_typer(config.app, name="config", rich_help_panel="Configuration") if __name__ == "__main__": - app() + main() diff --git a/portabase.spec b/portabase.spec new file mode 100644 index 0000000..2d67e20 --- /dev/null +++ b/portabase.spec @@ -0,0 +1,49 @@ +# PyInstaller build definition. Declarative, so the workflows do not carry +# build flags and `uv run pyinstaller portabase.spec` reproduces CI locally. +# +# The binary name comes from PORTABASE_BINARY_NAME (default: portabase); the +# release matrix sets it to portabase__. +import os + +from PyInstaller.utils.hooks import collect_all, collect_data_files + +NAME = os.environ.get("PORTABASE_BINARY_NAME", "portabase") + +datas = [ + ("pyproject.toml", "."), + ("templates", "templates"), +] +binaries = [] +hiddenimports = [] + +for package in ("rich", "requests"): + pkg_datas, pkg_binaries, pkg_hidden = collect_all(package) + datas += pkg_datas + binaries += pkg_binaries + hiddenimports += pkg_hidden + +datas += collect_data_files("certifi") + +a = Analysis( + ["main.py"], + pathex=["."], + binaries=binaries, + datas=datas, + hiddenimports=hiddenimports, + noarchive=False, +) + +pyz = PYZ(a.pure) + +exe = EXE( + pyz, + a.scripts, + a.binaries, + a.datas, + [], + name=NAME, + debug=False, + strip=False, + upx=False, + console=True, +) diff --git a/pyproject.toml b/pyproject.toml index 8657581..5ae4b14 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -5,12 +5,66 @@ description = "The official command line interface (CLI) for managing and deploy readme = "README.md" requires-python = ">=3.12" dependencies = [ - "rich>=14.2.0", "typer>=0.20.0", - "pyinstaller>=6.17.0", - "requests>=2.32.5", - "pyyaml >=6.0.0", + "rich>=14.2.0", "questionary>=2.1.0", + "requests>=2.32.5", "pyyaml>=6.0.3", - "cryptography>=44.0.0" + "cryptography>=44.0.0", + "jinja2>=3.1", +] + +[dependency-groups] +dev = [ + "pyinstaller>=6.17.0", + "ruff>=0.16.0", + "pytest>=8.3", + "mypy>=1.15", +] + +[tool.ruff] +target-version = "py312" +line-length = 88 +extend-exclude = [".venv", "build", "dist", "docs", "*.md", "*.spec"] + +[tool.ruff.lint] +select = [ + "E", "F", "W", + "I", + "UP", + "B", + "BLE", + "S110", + "E722", + "TID251", + "SIM", + "TRY201", + "PLW1510", +] +ignore = [ + "B008", + "E501", ] + +[tool.ruff.lint.per-file-ignores] +"ui/**" = ["TID251"] + +[tool.ruff.lint.flake8-tidy-imports.banned-api] +"rich.prompt".msg = "Use ui.form() / ui.confirm() instead." +"rich.console".msg = "Only ui/ may build a Console. Use the UI facade." +"typer.prompt".msg = "Use ui.form() instead." +"typer.confirm".msg = "Use ui.confirm() instead." + +[tool.ruff.lint.isort] +known-first-party = ["commands", "core", "engines", "services", "ui"] + +[tool.pytest.ini_options] +testpaths = ["tests"] +addopts = "-q" + +[tool.mypy] +python_version = "3.12" +files = ["commands", "core", "engines", "services", "ui", "scripts", "main.py"] +ignore_missing_imports = true +warn_unused_ignores = true +warn_redundant_casts = true diff --git a/release b/release deleted file mode 100755 index 09d0903..0000000 --- a/release +++ /dev/null @@ -1,96 +0,0 @@ -#!/bin/bash - -set -e - -if [ -z "$1" ]; then - echo "Usage: ./release " - echo "Example: ./release v1.0.0" - exit 1 -fi - -VERSION=$1 -CURRENT_BRANCH=$(git rev-parse --abbrev-ref HEAD) - -if [ "$CURRENT_BRANCH" = "main" ]; then - if [[ ! "$VERSION" =~ ^v?[0-9]+\.[0-9]+\.[0-9]+$ ]]; then - echo "Error: On 'main' branch, only release tags (vX.Y.Z) are allowed." - exit 1 - fi -else - if [[ ! "$VERSION" =~ ^v?[0-9]+\.[0-9]+\.[0-9]+([-.]?(rc|alpha|beta|a|b)[0-9]*(\.[0-9]+)?)?$ ]]; then - echo "Error: On branch '$CURRENT_BRANCH', only pre-release tags matching vX.Y.Z[-.](rc|a|b|...)W are allowed." - echo "Examples: 1.0.0-rc.1, 26.01.1a, 26.01.1b, 26.01.1rc1" - exit 1 - fi -fi - -CLEAN_VERSION=${VERSION#v} -CURRENT_DATE=$(date +%Y-%m-%d) - -echo "Preparing release $VERSION..." - - -# package.json -if [ -f package.json ]; then - echo "Updating package.json..." - if sed --version >/dev/null 2>&1; then - sed -i "s/\"version\": \".*\"/\"version\": \"$CLEAN_VERSION\"/" package.json - else - sed -i '' "s/\"version\": \".*\"/\"version\": \"$CLEAN_VERSION\"/" package.json - fi -fi - -# pyproject.toml -if [ -f pyproject.toml ]; then - echo "Updating pyproject.toml..." - if sed --version >/dev/null 2>&1; then - sed -i "s/^version = \".*\"/version = \"$CLEAN_VERSION\"/" pyproject.toml - else - sed -i '' "s/^version = \".*\"/version = \"$CLEAN_VERSION\"/" pyproject.toml - fi -fi - -# Cargo.toml -if [ -f Cargo.toml ]; then - echo "Updating Cargo.toml..." - if sed --version >/dev/null 2>&1; then - sed -i "s/^version = \".*\"/version = \"$CLEAN_VERSION\"/" Cargo.toml - else - sed -i '' "s/^version = \".*\"/version = \"$CLEAN_VERSION\"/" Cargo.toml - fi -fi - -# CITATION.cff -if [ -f CITATION.cff ]; then - echo "Updating CITATION.cff..." - if sed --version >/dev/null 2>&1; then - sed -i "s/^version: .*/version: $CLEAN_VERSION/" CITATION.cff - sed -i "s/^date-released: .*/date-released: \"$CURRENT_DATE\"/" CITATION.cff - else - sed -i '' "s/^version: .*/version: $CLEAN_VERSION/" CITATION.cff - sed -i '' "s/^date-released: .*/date-released: \"$CURRENT_DATE\"/" CITATION.cff - fi -fi - -git add . - -if ! git diff-index --quiet HEAD --; then - echo "Committing changes..." - git commit -m "chore(release): $VERSION" -else - echo "No changes to commit. Proceeding to tag..." -fi - -if git rev-parse "$VERSION" >/dev/null 2>&1; then - echo "Tag $VERSION already exists. Aborting." - exit 1 -fi - -echo "Creating tag $VERSION..." -git tag -a "$VERSION" -m "Release $VERSION" - -echo "Pushing changes and tags to remote..." -git push -git push origin "$VERSION" - -echo "Successfully released $VERSION!" diff --git a/scripts/render_check.py b/scripts/render_check.py new file mode 100644 index 0000000..68c7952 --- /dev/null +++ b/scripts/render_check.py @@ -0,0 +1,215 @@ +#!/usr/bin/env python3 + +from __future__ import annotations + +import argparse +import os +import shutil +import subprocess +import sys +import tempfile +from pathlib import Path + +import yaml + +sys.path.insert(0, str(Path(__file__).resolve().parent.parent)) + +from core.specs import DatabaseSpec # noqa: E402 +from engines import registry # noqa: E402 +from engines.base import DbEngine # noqa: E402 +from services.envfile import EnvFile # noqa: E402 +from services.ports import FixedPortAllocator # noqa: E402 +from services.project import AgentProject, DashboardProject # noqa: E402 +from services.renderer import ComposeRenderer # noqa: E402 +from services.templates import TemplateRepository # noqa: E402 + +AGENT_ENV = {"TZ": "UTC", "EDGE_KEY": "x", "LOG_LEVEL": "info", "POLLING": "5"} +DASHBOARD_BASE = { + "HOST_PORT": "8887", + "PROJECT_SECRET": "s", + "PROJECT_URL": "http://localhost:8887", + "PROJECT_NAME": "pb", + "TZ": "UTC", + "LOG_LEVEL": "info", +} +DASHBOARD_PG = { + "POSTGRES_DB": "pb", + "POSTGRES_USER": "pb", + "POSTGRES_PASSWORD": "p", + "PG_PORT": "5433", + "DATABASE_URL": "postgresql://pb:p@db:5432/pb", +} + + +class Failure(Exception): + pass + + +def env_text(project: AgentProject | DashboardProject) -> str: + return "".join(f'{k}="{v}"\n' for k, v in project.env.as_dict().items()) + + +def validate(label: str, compose: str, env: str, use_compose: bool) -> None: + try: + doc = yaml.safe_load(compose) + except yaml.YAMLError as e: + raise Failure(f"{label}: invalid YAML: {e}\n{compose}") from e + if not isinstance(doc, dict) or "services" not in doc: + raise Failure(f"{label}: no services key\n{compose}") + if use_compose: + with tempfile.TemporaryDirectory() as tmp: + Path(tmp, "docker-compose.yml").write_text(compose, encoding="utf-8") + Path(tmp, ".env").write_text(env, encoding="utf-8") + Path(tmp, "databases.json").write_text( + '{"databases": []}', encoding="utf-8" + ) + proc = subprocess.run( + ["docker", "compose", "-p", "rendercheck", "config", "--quiet"], + cwd=tmp, + capture_output=True, + text=True, + check=False, + ) + if proc.returncode != 0: + raise Failure( + f"{label}: docker compose config failed:\n{proc.stderr}\n{compose}" + ) + print(f"ok {label}") + + +def agent_project( + tmp: Path, + pairs: list[tuple[DatabaseSpec, DbEngine]], + *, + host_gateway: bool = False, + sqlite: bool = False, + docker_volume: bool = False, +) -> AgentProject: + env = EnvFile(tmp / ".env") + env.merge(AGENT_ENV) + project = AgentProject(tmp, env, [], host_gateway) + for spec, engine in pairs: + project.add(spec, engine) + if sqlite: + sq = registry.get("sqlite") + project.add( + sq.generate(auth=False, ports=FixedPortAllocator(), answers={"name": "x"}), + sq, + ) + if docker_volume: + dv = registry.get("docker-volume") + project.add(dv.from_existing({"volume": "v"}), dv) + return project + + +def agent_cases(renderer: ComposeRenderer) -> list[tuple[str, str, str]]: + ports = FixedPortAllocator() + tmp = Path(tempfile.mkdtemp()) + cases: list[tuple[str, str, str]] = [] + + empty = agent_project(tmp, []) + cases.append(("agent/empty", renderer.render_agent(empty).compose, env_text(empty))) + + toggles = agent_project(tmp, [], host_gateway=True, sqlite=True, docker_volume=True) + cases.append( + ("agent/toggles", renderer.render_agent(toggles).compose, env_text(toggles)) + ) + + everything: list[tuple[DatabaseSpec, DbEngine]] = [] + for engine in registry: + if engine.template is None: + continue + for auth in (True, False) if engine.auth_variants else (True,): + spec = engine.generate(auth=auth, ports=ports, answers={}) + one = agent_project(tmp, [(spec, engine)]) + variant = ("/auth" if auth else "/noauth") if engine.auth_variants else "" + cases.append( + ( + f"agent/{engine.key}{variant}", + renderer.render_agent(one).compose, + env_text(one), + ) + ) + everything.append((spec, engine)) + + combined = agent_project(tmp, everything, host_gateway=True, sqlite=True) + cases.append( + ("agent/all", renderer.render_agent(combined).compose, env_text(combined)) + ) + return cases + + +def dashboard_cases(renderer: ComposeRenderer) -> list[tuple[str, str, str]]: + tmp = Path(tempfile.mkdtemp()) + variants = { + "external": {**DASHBOARD_BASE, **DASHBOARD_PG, "POSTGRES_HOST": "db"}, + "internal": DASHBOARD_BASE, + "custom": {**DASHBOARD_BASE, **DASHBOARD_PG, "POSTGRES_HOST": "remote"}, + } + cases = [] + for mode, values in variants.items(): + env = EnvFile(tmp / f".env.{mode}") + env.merge(values) + project = DashboardProject(tmp, env) + if project.db_mode != mode: + raise Failure(f"db_mode mismatch: {project.db_mode} != {mode}") + cases.append( + ( + f"dashboard/{mode}", + renderer.render_dashboard(project).compose, + env_text(project), + ) + ) + return cases + + +def engines_check(repo: TemplateRepository) -> None: + shipped = set(repo.names()) + used = set() + for engine in registry: + if engine.template is None: + continue + if engine.template not in shipped: + raise Failure( + f"{engine.key}: template {engine.template} not found in {repo.root}" + ) + used.add(engine.template) + orphans = { + name for name in shipped if name.startswith("engines/") and name not in used + } + if orphans: + raise Failure( + f"template(s) not used by any engine: {', '.join(sorted(orphans))}" + ) + print(f"ok engines-check ({len(used)} templates, {len(registry.keys())} engines)") + + +def main() -> int: + parser = argparse.ArgumentParser() + parser.add_argument( + "--templates", default=os.environ.get("PORTABASE_TEMPLATES_DIR", "templates") + ) + parser.add_argument( + "--no-compose", + action="store_true", + help="Skip docker compose config validation", + ) + args = parser.parse_args() + + use_compose = not args.no_compose and shutil.which("docker") is not None + if not use_compose: + print("note: docker not available, YAML validation only") + repo = TemplateRepository(Path(args.templates)) + renderer = ComposeRenderer(repo, registry, "render-check") + try: + engines_check(repo) + for label, compose, env in agent_cases(renderer) + dashboard_cases(renderer): + validate(label, compose, env, use_compose) + except Failure as e: + print(f"FAIL {e}", file=sys.stderr) + return 1 + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/services/__init__.py b/services/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/services/compose_facts.py b/services/compose_facts.py new file mode 100644 index 0000000..f54bd7f --- /dev/null +++ b/services/compose_facts.py @@ -0,0 +1,43 @@ +from __future__ import annotations + +from pathlib import Path + +import yaml + +GENERATED_MARKER = "# Generated by Portabase CLI" + + +class ComposeFacts: + def __init__(self, path: Path) -> None: + self.path = path + self.raw: dict = {} + self.text = "" + if path.exists(): + try: + self.text = path.read_text(encoding="utf-8") + loaded = yaml.safe_load(self.text) + self.raw = loaded if isinstance(loaded, dict) else {} + except (OSError, yaml.YAMLError): + self.raw = {} + + @property + def exists(self) -> bool: + return self.path.exists() + + @property + def is_generated(self) -> bool: + return self.text.startswith(GENERATED_MARKER) + + def _service(self, name: str) -> dict: + services = self.raw.get("services") or {} + svc = services.get(name) if isinstance(services, dict) else None + return svc if isinstance(svc, dict) else {} + + @property + def host_gateway(self) -> bool: + extra = self._service("agent").get("extra_hosts") + if isinstance(extra, list): + return any("host-gateway" in str(x) for x in extra) + if isinstance(extra, dict): + return any("host-gateway" in str(v) for v in extra.values()) + return False diff --git a/services/docker.py b/services/docker.py new file mode 100644 index 0000000..8dd3491 --- /dev/null +++ b/services/docker.py @@ -0,0 +1,126 @@ +from __future__ import annotations + +import platform +import shutil +import subprocess +import time +from pathlib import Path + +from core.errors import DockerError +from core.utils import slugify_project_name + +_START_COMMANDS = { + "Linux": ["sudo", "systemctl", "start", "docker"], + "Darwin": ["open", "--background", "-a", "Docker"], + "Windows": ["cmd", "/c", "start", "docker"], +} + + +class DockerRunner: + def __init__(self, docker_bin: str | None = None) -> None: + self._bin = docker_bin + + @property + def binary(self) -> str: + if self._bin is None: + found = shutil.which("docker") + if found is None: + raise DockerError( + "Docker not found (binary missing).", + hint="Install Docker: https://docs.docker.com/get-docker/", + ) + self._bin = found + return self._bin + + def available(self) -> bool: + return shutil.which("docker") is not None + + def daemon_running(self) -> bool: + try: + subprocess.run( + [self.binary, "info"], + stdout=subprocess.DEVNULL, + stderr=subprocess.DEVNULL, + check=True, + ) + return True + except (subprocess.CalledProcessError, OSError): + return False + + def start_daemon(self, *, wait_seconds: int = 20) -> bool: + cmd = _START_COMMANDS.get(platform.system()) + if cmd is None: + return False + try: + subprocess.run(cmd, check=True) + except (subprocess.CalledProcessError, OSError) as e: + raise DockerError(f"Failed to start Docker: {e}", cause=e) from e + deadline = time.monotonic() + wait_seconds + while time.monotonic() < deadline: + if self.daemon_running(): + return True + time.sleep(2) + return False + + def ensure_network(self, name: str) -> None: + inspect = subprocess.run( + [self.binary, "network", "inspect", name], + stdout=subprocess.DEVNULL, + stderr=subprocess.DEVNULL, + check=False, + ) + if inspect.returncode == 0: + return + try: + subprocess.run( + [self.binary, "network", "create", name], + stdout=subprocess.DEVNULL, + check=True, + ) + except subprocess.CalledProcessError as e: + raise DockerError( + f"Could not create Docker network '{name}'.", cause=e + ) from e + + def remove_volume(self, name: str) -> bool: + proc = subprocess.run( + [self.binary, "volume", "rm", name], + capture_output=True, + text=True, + check=False, + ) + if proc.returncode == 0: + return True + if "no such volume" in (proc.stderr or "").lower(): + return False + raise DockerError(f"Could not remove volume '{name}': {proc.stderr.strip()}") + + @staticmethod + def project_name(cwd: Path) -> str: + return slugify_project_name(cwd.resolve().name) + + def compose( + self, + cwd: Path, + args: list[str], + *, + check: bool = True, + capture: bool = False, + ) -> subprocess.CompletedProcess: + cmd = [self.binary, "compose", "-p", self.project_name(cwd), *args] + try: + return subprocess.run( + cmd, + cwd=cwd, + check=check, + capture_output=capture, + text=capture, + ) + except subprocess.CalledProcessError as e: + raise DockerError( + f"docker compose {' '.join(args)} failed (exit {e.returncode}).", + hint=f"Run it manually in {cwd} to see the full output.", + cause=e, + ) from e + except OSError as e: + raise DockerError(f"Could not run docker: {e}", cause=e) from e diff --git a/services/envfile.py b/services/envfile.py new file mode 100644 index 0000000..2cc9f3e --- /dev/null +++ b/services/envfile.py @@ -0,0 +1,85 @@ +from __future__ import annotations + +import os +import re +from collections.abc import Mapping +from dataclasses import dataclass, field +from pathlib import Path + +_LINE = re.compile(r"""^\s*(?:export\s+)?([A-Za-z_][A-Za-z0-9_]*)\s*=\s*(.*?)\s*$""") + + +def _unquote(raw: str) -> str: + if len(raw) >= 2 and raw[0] == raw[-1] and raw[0] in "\"'": + inner = raw[1:-1] + if raw[0] == '"': + return inner.replace('\\"', '"').replace("\\\\", "\\") + return inner + return raw.split(" #", 1)[0].rstrip() + + +def _quote(value: str) -> str: + return '"' + value.replace("\\", "\\\\").replace('"', '\\"') + '"' + + +@dataclass +class EnvFile: + path: Path + _lines: list[str] = field(default_factory=list) + _index: dict[str, int] = field(default_factory=dict) + + @classmethod + def load(cls, path: Path) -> EnvFile: + env = cls(path) + if path.exists(): + text = path.read_text(encoding="utf-8") + env._lines = text.splitlines() + for i, line in enumerate(env._lines): + m = _LINE.match(line) + if m and not line.lstrip().startswith("#"): + env._index[m.group(1)] = i + return env + + @property + def exists(self) -> bool: + return self.path.exists() + + def get(self, key: str, default: str | None = None) -> str | None: + i = self._index.get(key) + if i is None: + return default + m = _LINE.match(self._lines[i]) + return _unquote(m.group(2)) if m else default + + def as_dict(self) -> dict[str, str]: + return {k: self.get(k) or "" for k in self._index} + + def set(self, key: str, value: str) -> None: + line = f"{key}={_quote(str(value))}" + i = self._index.get(key) + if i is None: + self._lines.append(line) + self._index[key] = len(self._lines) - 1 + else: + self._lines[i] = line + + def merge(self, mapping: Mapping[str, str]) -> None: + for k, v in mapping.items(): + self.set(k, v) + + def remove(self, key: str) -> None: + i = self._index.pop(key, None) + if i is None: + return + del self._lines[i] + self._index = {k: (n - 1 if n > i else n) for k, n in self._index.items()} + + def remove_prefix(self, prefix: str) -> None: + for key in [k for k in self._index if k.startswith(prefix + "_")]: + self.remove(key) + + def save(self) -> None: + self.path.parent.mkdir(parents=True, exist_ok=True) + tmp = self.path.with_name(self.path.name + ".tmp") + tmp.write_text("\n".join(self._lines) + "\n", encoding="utf-8") + os.replace(tmp, self.path) diff --git a/services/http.py b/services/http.py new file mode 100644 index 0000000..03661e4 --- /dev/null +++ b/services/http.py @@ -0,0 +1,79 @@ +from __future__ import annotations + +from collections.abc import Callable +from pathlib import Path +from typing import Any + +import requests + +from core.errors import NetworkError + +_HINT = "Check your internet connection or proxy settings." + + +class HttpClient: + def __init__( + self, timeout: float = 10.0, user_agent: str = "portabase-cli" + ) -> None: + self.timeout = timeout + self.session = requests.Session() + self.session.headers["User-Agent"] = user_agent + + def get_json(self, url: str) -> Any: + try: + r = self.session.get(url, timeout=self.timeout) + r.raise_for_status() + return r.json() + except requests.RequestException as e: + raise NetworkError(f"GET {url} failed: {e}", hint=_HINT, cause=e) from e + except ValueError as e: + raise NetworkError(f"GET {url}: response is not JSON", cause=e) from e + + def get_text(self, url: str) -> str: + try: + r = self.session.get(url, timeout=self.timeout) + r.raise_for_status() + return r.text + except requests.RequestException as e: + raise NetworkError(f"GET {url} failed: {e}", hint=_HINT, cause=e) from e + + def status(self, url: str) -> int: + try: + return self.session.get(url, timeout=self.timeout, stream=True).status_code + except requests.RequestException as e: + raise NetworkError(f"GET {url} failed: {e}", hint=_HINT, cause=e) from e + + def download( + self, + url: str, + dest: Path, + on_progress: Callable[[int], None] | None = None, + *, + timeout: float = 30.0, + ) -> int: + written = 0 + try: + with self.session.get(url, stream=True, timeout=timeout) as r: + r.raise_for_status() + with open(dest, "wb") as f: + for chunk in r.iter_content(chunk_size=64 * 1024): + if not chunk: + continue + f.write(chunk) + written += len(chunk) + if on_progress: + on_progress(len(chunk)) + except requests.RequestException as e: + dest.unlink(missing_ok=True) + raise NetworkError( + f"Download of {url} failed: {e}", hint=_HINT, cause=e + ) from e + return written + + def content_length(self, url: str) -> int | None: + try: + r = self.session.head(url, timeout=self.timeout, allow_redirects=True) + value = r.headers.get("content-length") + return int(value) if value else None + except (requests.RequestException, ValueError): + return None diff --git a/services/ports.py b/services/ports.py new file mode 100644 index 0000000..7bff3fa --- /dev/null +++ b/services/ports.py @@ -0,0 +1,29 @@ +from __future__ import annotations + +import socket + + +class PortAllocator: + def __init__(self) -> None: + self._given: set[int] = set() + + def free(self) -> int: + for _ in range(50): + with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s: + s.bind(("", 0)) + port = s.getsockname()[1] + if port not in self._given: + self._given.add(port) + return port + raise RuntimeError("Could not allocate a free port") + + +class FixedPortAllocator(PortAllocator): + def __init__(self, start: int = 40000) -> None: + super().__init__() + self._next = start + + def free(self) -> int: + port = self._next + self._next += 1 + return port diff --git a/services/project.py b/services/project.py new file mode 100644 index 0000000..9436beb --- /dev/null +++ b/services/project.py @@ -0,0 +1,198 @@ +from __future__ import annotations + +import json +from dataclasses import dataclass, field +from pathlib import Path +from typing import Any, Literal + +from core.errors import ConfigError, ValidationError +from core.specs import DatabaseSpec +from engines.base import DbEngine +from engines.sqlite import SqliteEngine +from services.compose_facts import ComposeFacts +from services.envfile import EnvFile + +ProjectKind = Literal["agent", "dashboard"] +DATABASES_FILE = "databases.json" +COMPOSE_FILE = "docker-compose.yml" +ENV_FILE = ".env" + + +def detect_kind(path: Path) -> ProjectKind: + if (path / DATABASES_FILE).exists(): + return "agent" + env = EnvFile.load(path / ENV_FILE) + if env.get("PROJECT_SECRET") is not None: + return "dashboard" + raise ConfigError( + f"{path} is not a Portabase agent or dashboard folder.", + hint="Expected databases.json (agent) or a .env with PROJECT_SECRET (dashboard).", + ) + + +def _opt_str(entry: dict[str, Any], key: str) -> str | None: + value = entry.get(key) + return str(value) if value not in (None, "") else None + + +def spec_from_entry(entry: dict[str, Any], env: EnvFile) -> DatabaseSpec: + engine = str(entry.get("type", "")) + host = entry.get("host") + managed, host_port, root_password = False, None, None + if host: + prefix = str(host).upper().replace("-", "_") + raw_port = env.get(f"{prefix}_PORT") + if raw_port and raw_port.isdigit(): + managed, host_port = True, int(raw_port) + root_password = env.get(f"{prefix}_ROOT_PASS") + port = entry.get("port") + return DatabaseSpec( + id=str(entry.get("generated_id") or DbEngine.new_id()), + engine=engine, + name=str(entry.get("name", "")), + managed=managed, + host=str(host) if host else None, + port=int(port) if port not in (None, "") else None, + host_port=host_port, + database=str(entry["database"]) if entry.get("database") is not None else None, + username=str(entry["username"]) if entry.get("username") is not None else None, + password=_opt_str(entry, "password"), + root_password=root_password, + path=_opt_str(entry, "database") if engine == "sqlite" else None, + volume=_opt_str(entry, "volume_name"), + container=_opt_str(entry, "container_name"), + options=dict(entry.get("options") or {}), + ) + + +@dataclass +class AgentProject: + path: Path + env: EnvFile + databases: list[DatabaseSpec] = field(default_factory=list) + host_gateway: bool = False + + @classmethod + def load(cls, path: Path) -> AgentProject: + path = path.resolve() + env_path, db_path = path / ENV_FILE, path / DATABASES_FILE + if not env_path.exists() or not db_path.exists(): + raise ConfigError( + f"Not a Portabase agent folder: {path}", + hint=f"Expected {ENV_FILE} and {DATABASES_FILE}.", + ) + env = EnvFile.load(env_path) + try: + data = json.loads(db_path.read_text(encoding="utf-8")) + except (OSError, ValueError) as e: + raise ConfigError(f"{db_path} is not valid JSON.", cause=e) from e + entries = data.get("databases", []) if isinstance(data, dict) else [] + databases = [spec_from_entry(e, env) for e in entries if isinstance(e, dict)] + facts = ComposeFacts(path / COMPOSE_FILE) + project = cls(path, env, databases, facts.host_gateway) + project.validate() + return project + + @classmethod + def create( + cls, path: Path, env_vars: dict[str, str], *, host_gateway: bool + ) -> AgentProject: + path.mkdir(parents=True, exist_ok=True) + env = EnvFile.load(path / ENV_FILE) + env.merge(env_vars) + return cls(path, env, [], host_gateway) + + @property + def managed(self) -> list[DatabaseSpec]: + return [d for d in self.databases if d.managed] + + @property + def needs_docker_socket(self) -> bool: + return any(d.engine == "docker-volume" for d in self.databases) + + @property + def sqlite_mounts(self) -> list[tuple[str, str]]: + mounts: list[tuple[str, str]] = [] + for d in self.databases: + if d.engine == "sqlite": + m = SqliteEngine.mount_for(d) + if m and m not in mounts: + mounts.append(m) + return mounts + + def validate(self) -> None: + seen: set[str] = set() + for d in self.managed: + if d.host in seen: + raise ConfigError( + f"Two managed databases share the service name '{d.host}'." + ) + seen.add(d.host or "") + + def add(self, spec: DatabaseSpec, engine: DbEngine) -> None: + if spec.managed: + self.env.merge(engine.env_vars(spec)) + self.databases.append(spec) + self.validate() + + def remove(self, spec: DatabaseSpec, engine: DbEngine) -> None: + self.databases = [d for d in self.databases if d.id != spec.id] + if spec.managed and spec.host: + self.env.remove_prefix(spec.env_prefix) + + def find(self, id_or_name: str) -> DatabaseSpec: + matches = [ + d + for d in self.databases + if d.id == id_or_name or d.id.startswith(id_or_name) or d.name == id_or_name + ] + if not matches: + raise ValidationError( + f"No database matching '{id_or_name}'.", hint="See: portabase db list" + ) + if len(matches) > 1: + raise ValidationError( + f"'{id_or_name}' matches several databases; use the id." + ) + return matches[0] + + def save_state(self) -> None: + self.env.save() + + +@dataclass +class DashboardProject: + path: Path + env: EnvFile + + @classmethod + def load(cls, path: Path) -> DashboardProject: + path = path.resolve() + env = EnvFile.load(path / ENV_FILE) + if env.get("PROJECT_SECRET") is None: + raise ConfigError( + f"Not a Portabase dashboard folder: {path}", + hint="Expected a .env with PROJECT_SECRET.", + ) + return cls(path, env) + + @classmethod + def create(cls, path: Path, env_vars: dict[str, str]) -> DashboardProject: + path.mkdir(parents=True, exist_ok=True) + env = EnvFile.load(path / ENV_FILE) + env.merge(env_vars) + return cls(path, env) + + @property + def db_mode(self) -> Literal["external", "internal", "custom"]: + host = self.env.get("POSTGRES_HOST") + if host is None: + return "internal" + return "external" if host == "db" else "custom" + + @property + def project_name(self) -> str: + return self.env.get("PROJECT_NAME") or self.path.name + + def save_state(self) -> None: + self.env.save() diff --git a/services/renderer.py b/services/renderer.py new file mode 100644 index 0000000..57d748b --- /dev/null +++ b/services/renderer.py @@ -0,0 +1,167 @@ +from __future__ import annotations + +import contextlib +import difflib +import json +import os +import shutil +from dataclasses import dataclass, field +from pathlib import Path +from typing import Any + +import jinja2 +import yaml + +from core.errors import TemplateError +from core.specs import DatabaseSpec +from engines import EngineRegistry +from services.compose_facts import GENERATED_MARKER, ComposeFacts +from services.envfile import EnvFile +from services.project import ( + COMPOSE_FILE, + DATABASES_FILE, + AgentProject, + DashboardProject, +) +from services.templates import TemplateRepository + +LEGACY_BACKUP = "docker-compose.legacy.yml" + + +@dataclass +class WriteReport: + backed_up: Path | None = None + wrote: list[Path] = field(default_factory=list) + + +@dataclass +class RenderResult: + compose: str + databases: list[dict[str, Any]] | None = None + + def validate(self) -> None: + try: + doc = yaml.safe_load(self.compose) + except yaml.YAMLError as e: + raise TemplateError( + "Rendered compose is not valid YAML; templates are broken.", cause=e + ) from e + if not isinstance(doc, dict) or "services" not in doc: + raise TemplateError( + "Rendered compose has no 'services' section; templates are broken." + ) + + def write(self, path: Path) -> WriteReport: + self.validate() + report = WriteReport() + compose_path = path / COMPOSE_FILE + facts = ComposeFacts(compose_path) + if facts.exists and not facts.is_generated: + backup = path / LEGACY_BACKUP + if not backup.exists(): + shutil.copy2(compose_path, backup) + report.backed_up = backup + _atomic_write(compose_path, self.compose) + report.wrote.append(compose_path) + if self.databases is not None: + db_path = path / DATABASES_FILE + _atomic_write( + db_path, json.dumps({"databases": self.databases}, indent=2) + "\n" + ) + with contextlib.suppress(OSError): + os.chmod(db_path, 0o666) + report.wrote.append(db_path) + return report + + def diff_against(self, path: Path) -> str: + compose_path = path / COMPOSE_FILE + current = ( + compose_path.read_text(encoding="utf-8") if compose_path.exists() else "" + ) + return "".join( + difflib.unified_diff( + current.splitlines(keepends=True), + self.compose.splitlines(keepends=True), + fromfile=f"{COMPOSE_FILE} (current)", + tofile=f"{COMPOSE_FILE} (rendered)", + ) + ) + + +def _atomic_write(path: Path, content: str) -> None: + path.parent.mkdir(parents=True, exist_ok=True) + tmp = path.with_name(path.name + ".tmp") + tmp.write_text(content, encoding="utf-8") + os.replace(tmp, path) + + +def _var(env: EnvFile, key: str, inline: bool) -> str: + return (env.get(key) or "") if inline else f"${{{key}}}" + + +class ComposeRenderer: + def __init__( + self, templates: TemplateRepository, engines: EngineRegistry, cli_version: str + ) -> None: + self.templates = templates + self.engines = engines + self.cli_version = cli_version + + def header(self) -> str: + return f"{GENERATED_MARKER} {self.cli_version}. Do not edit.\n" + + def render_agent( + self, project: AgentProject, *, inline: bool = False + ) -> RenderResult: + env = project.env + ctx = { + "host_gateway": project.host_gateway, + "docker_socket": project.needs_docker_socket, + "mounts": [{"host": h, "container": c} for h, c in project.sqlite_mounts], + "services": [self._service(spec, inline) for spec in project.managed], + "tz_var": _var(env, "TZ", inline), + "edge_key_var": _var(env, "EDGE_KEY", inline), + "log_level_var": _var(env, "LOG_LEVEL", inline), + "polling_var": _var(env, "POLLING", inline), + } + compose = self.header() + self._render("agent.yml.j2", ctx) + databases = [ + self.engines.get(d.engine).agent_entry(d) for d in project.databases + ] + return RenderResult(compose=compose, databases=databases) + + def _service(self, spec: DatabaseSpec, inline: bool) -> dict[str, str]: + engine = self.engines.get(spec.engine) + template = self.templates.get(engine.template or "") + body = self._render_template(template, engine.template_ctx(spec, inline=inline)) + return {"name": spec.host or "", "volume": f"{spec.host}-data", "body": body} + + def render_dashboard( + self, project: DashboardProject, *, inline: bool = False + ) -> RenderResult: + env = project.env + ctx = { + "db_mode": project.db_mode, + "project_name_var": project.project_name, + "host_port_var": _var(env, "HOST_PORT", inline), + "tz_var": _var(env, "TZ", inline), + "log_level_var": _var(env, "LOG_LEVEL", inline), + "project_secret_var": _var(env, "PROJECT_SECRET", inline), + "project_url_var": _var(env, "PROJECT_URL", inline), + "pg_port_var": _var(env, "PG_PORT", inline), + "postgres_db_var": _var(env, "POSTGRES_DB", inline), + "postgres_user_var": _var(env, "POSTGRES_USER", inline), + "postgres_password_var": _var(env, "POSTGRES_PASSWORD", inline), + } + compose = self.header() + self._render("dashboard.yml.j2", ctx) + return RenderResult(compose=compose, databases=None) + + def _render(self, name: str, ctx: dict[str, Any]) -> str: + return self._render_template(self.templates.get(name), ctx) + + @staticmethod + def _render_template(template: jinja2.Template, ctx: dict[str, Any]) -> str: + try: + return template.render(**ctx) + except jinja2.TemplateError as e: + raise TemplateError(f"Template rendering failed: {e}", cause=e) from e diff --git a/services/telemetry.py b/services/telemetry.py new file mode 100644 index 0000000..de6184c --- /dev/null +++ b/services/telemetry.py @@ -0,0 +1,39 @@ +from __future__ import annotations + +from abc import ABC, abstractmethod +from collections.abc import Iterator +from contextlib import contextmanager +from typing import Any + + +class Telemetry(ABC): + @abstractmethod + def session(self, **attrs: Any): ... + + @abstractmethod + def span(self, name: str, **attrs: Any): ... + + @abstractmethod + def event(self, name: str, **attrs: Any) -> None: ... + + @abstractmethod + def error(self, exc: BaseException, *, unexpected: bool = False) -> None: ... + + def flush(self) -> None: + return None + + +class NoopTelemetry(Telemetry): + @contextmanager + def session(self, **attrs: Any) -> Iterator[None]: + yield + + @contextmanager + def span(self, name: str, **attrs: Any) -> Iterator[None]: + yield + + def event(self, name: str, **attrs: Any) -> None: + return None + + def error(self, exc: BaseException, *, unexpected: bool = False) -> None: + return None diff --git a/services/templates.py b/services/templates.py new file mode 100644 index 0000000..dbd5b6c --- /dev/null +++ b/services/templates.py @@ -0,0 +1,63 @@ +from __future__ import annotations + +import os +import sys +from pathlib import Path + +import jinja2 + +from core.errors import TemplateError + +TEMPLATES_DIR = "templates" +ROOT_TEMPLATE = "agent.yml.j2" + + +class TemplateRepository: + def __init__(self, root: Path) -> None: + self.root = root + self._env: jinja2.Environment | None = None + + @classmethod + def bundled(cls) -> TemplateRepository: + override = os.environ.get("PORTABASE_TEMPLATES_DIR") + if override: + return cls(Path(override)) + bundle = getattr(sys, "_MEIPASS", None) + base = Path(bundle) if bundle else Path(__file__).resolve().parent.parent + return cls(base / TEMPLATES_DIR) + + def resolve(self) -> Path: + if not (self.root / ROOT_TEMPLATE).exists(): + raise TemplateError( + f"No templates found in {self.root}.", + hint=( + "The CLI ships its own templates; either the build is broken " + "or PORTABASE_TEMPLATES_DIR points at the wrong folder." + ), + ) + return self.root + + def names(self) -> list[str]: + return sorted( + p.relative_to(self.root).as_posix() for p in self.root.rglob("*.j2") + ) + + def get(self, name: str) -> jinja2.Template: + root = self.resolve() + if self._env is None: + self._env = jinja2.Environment( + loader=jinja2.FileSystemLoader(str(root)), + undefined=jinja2.StrictUndefined, + keep_trailing_newline=True, + autoescape=False, # noqa: S701 — renders YAML, not HTML + ) + try: + return self._env.get_template(name) + except jinja2.TemplateNotFound as e: + raise TemplateError( + f"Template '{name}' is missing from {root}.", cause=e + ) from e + except jinja2.TemplateError as e: + raise TemplateError( + f"Template '{name}' failed to load: {e}", cause=e + ) from e diff --git a/services/updater.py b/services/updater.py new file mode 100644 index 0000000..708ec6b --- /dev/null +++ b/services/updater.py @@ -0,0 +1,222 @@ +from __future__ import annotations + +import hashlib +import json +import os +import platform +import shutil +import subprocess +import sys +import tempfile +import time +from collections.abc import Callable +from dataclasses import dataclass +from pathlib import Path + +from core.config import GlobalConfig +from core.errors import NetworkError, UpdateError +from core.version import UNKNOWN, is_prerelease, parse_version +from services.http import HttpClient + +GITHUB_REPO = "Portabase/cli" +RELEASES_URL = f"https://api.github.com/repos/{GITHUB_REPO}/releases" +CACHE_TTL = 24 * 3600 + + +@dataclass(frozen=True) +class Release: + tag: str + assets: dict[str, str] + prerelease: bool + + @classmethod + def from_api(cls, data: dict) -> Release: + return cls( + tag=str(data.get("tag_name", "")).lstrip("v"), + assets={ + a["name"]: a["browser_download_url"] for a in data.get("assets", []) + }, + prerelease=bool(data.get("prerelease", False)), + ) + + +def platform_asset_name() -> str: + system = platform.system().lower() + system = "macos" if system == "darwin" else system + machine = platform.machine().lower() + arch = "arm64" if machine in ("arm64", "aarch64") else "amd64" + name = f"portabase_{system}_{arch}" + return name + ".exe" if system == "windows" else name + + +def is_frozen() -> bool: + return bool(getattr(sys, "frozen", False)) + + +class UpdateChecker: + def __init__(self, http: HttpClient, config: GlobalConfig, current: str) -> None: + self.http = http + self.config = config + self.current = current + self.cache_file = config.cache_dir / "release.json" + + @property + def include_prerelease(self) -> bool: + channel = self.config.update_channel + if channel: + return channel == "beta" + return is_prerelease(self.current) + + def fetch_latest(self) -> Release | None: + if self.include_prerelease: + releases = self.http.get_json(RELEASES_URL) + return Release.from_api(releases[0]) if releases else None + return Release.from_api(self.http.get_json(f"{RELEASES_URL}/latest")) + + def latest(self, *, force: bool = False) -> Release | None: + if not force: + cached = self._read_cache() + if cached is not None: + return cached + try: + release = self.fetch_latest() + except NetworkError: + return None + if release is not None: + self._write_cache(release) + return release + + def available(self, *, force: bool = False) -> str | None: + if self.current == UNKNOWN: + return None + release = self.latest(force=force) + if release is None: + return None + if parse_version(release.tag) > parse_version(self.current): + return release.tag + return None + + def _read_cache(self) -> Release | None: + try: + with open(self.cache_file, encoding="utf-8") as f: + data = json.load(f) + if time.time() - float(data.get("checked_at", 0)) > CACHE_TTL: + return None + if data.get("channel_pre") != self.include_prerelease: + return None + return Release( + tag=data["tag"], + assets=data.get("assets", {}), + prerelease=bool(data.get("prerelease")), + ) + except (OSError, ValueError, KeyError): + return None + + def _write_cache(self, release: Release) -> None: + try: + self.cache_file.parent.mkdir(parents=True, exist_ok=True) + with open(self.cache_file, "w", encoding="utf-8") as f: + json.dump( + { + "checked_at": time.time(), + "channel_pre": self.include_prerelease, + "tag": release.tag, + "assets": release.assets, + "prerelease": release.prerelease, + }, + f, + ) + except OSError: + pass + + +class Updater: + CHECKSUMS_ASSET = "checksums.txt" + + def __init__(self, http: HttpClient, current: str) -> None: + self.http = http + self.current = current + + def target_path(self) -> Path: + if is_frozen(): + return Path(sys.executable).resolve() + if platform.system().lower() == "windows": + return Path(os.environ.get("APPDATA", "")) / "Portabase" / "portabase.exe" + default = Path("/usr/local/bin/portabase") + if default.exists(): + return default + return Path.home() / ".local" / "bin" / "portabase" + + def download( + self, release: Release, on_progress: Callable[[int], None] | None = None + ) -> Path: + name = platform_asset_name() + url = release.assets.get(name) + if url is None: + available = ", ".join(sorted(release.assets)) + raise UpdateError( + f"No binary for this platform ({name}) in release {release.tag}.", + hint=f"Available: {available}" if available else None, + ) + fd, tmp = tempfile.mkstemp(prefix="portabase_update_") + os.close(fd) + tmp_path = Path(tmp) + try: + self.http.download(url, tmp_path, on_progress, timeout=60) + self._verify(release, name, tmp_path) + except BaseException: + tmp_path.unlink(missing_ok=True) + raise + return tmp_path + + def expected_size(self, release: Release) -> int | None: + url = release.assets.get(platform_asset_name()) + return self.http.content_length(url) if url else None + + def _verify(self, release: Release, name: str, path: Path) -> None: + url = release.assets.get(self.CHECKSUMS_ASSET) + if url is None: + raise UpdateError( + f"Release {release.tag} has no {self.CHECKSUMS_ASSET}; " + "refusing to install." + ) + expected = None + for line in self.http.get_text(url).splitlines(): + parts = line.split() + if len(parts) == 2 and parts[1].lstrip("*") == name: + expected = parts[0].lower() + if expected is None: + raise UpdateError( + f"{name} not listed in {self.CHECKSUMS_ASSET}; refusing to install." + ) + digest = hashlib.sha256() + with open(path, "rb") as f: + for chunk in iter(lambda: f.read(1 << 20), b""): + digest.update(chunk) + if digest.hexdigest() != expected: + raise UpdateError( + "Checksum mismatch for downloaded binary; refusing to install." + ) + + def install(self, tmp: Path, target: Path) -> None: + system = platform.system().lower() + if system != "windows": + tmp.chmod(0o755) + target.parent.mkdir(parents=True, exist_ok=True) + backup = target.with_name(target.name + ".old") + writable = os.access(target.parent, os.W_OK) and ( + not target.exists() or os.access(target, os.W_OK) + ) + try: + if writable or system == "windows": + if target.exists(): + backup.unlink(missing_ok=True) + target.rename(backup) + shutil.move(str(tmp), str(target)) + else: + if target.exists(): + subprocess.run(["sudo", "mv", str(target), str(backup)], check=True) + subprocess.run(["sudo", "mv", str(tmp), str(target)], check=True) + subprocess.run(["sudo", "chmod", "+x", str(target)], check=True) + except (OSError, subprocess.CalledProcessError) as e: + raise UpdateError(f"Could not install to {target}: {e}", cause=e) from e diff --git a/templates/agent.yml.j2 b/templates/agent.yml.j2 new file mode 100644 index 0000000..d865063 --- /dev/null +++ b/templates/agent.yml.j2 @@ -0,0 +1,36 @@ +services: + agent: + restart: unless-stopped + image: portabase/agent:latest + volumes: + - ./databases.json:/config/config.json +{%- for m in mounts %} + - {{ m.host }}:{{ m.container }} +{%- endfor %} +{%- if docker_socket %} + - /var/run/docker.sock:/var/run/docker.sock +{%- endif %} +{%- if host_gateway %} + extra_hosts: + - "localhost:host-gateway" +{%- endif %} + environment: + TZ: "{{ tz_var }}" + EDGE_KEY: "{{ edge_key_var }}" + LOG_LEVEL: "{{ log_level_var }}" + POLLING: "{{ polling_var }}" + networks: + - portabase +{% for s in services %} +{{ s.body }} +{%- endfor %} +{% if services %} +volumes: +{%- for s in services %} + {{ s.volume }}: +{%- endfor %} +{% endif %} +networks: + portabase: + name: portabase_network + external: true diff --git a/templates/compose.py b/templates/compose.py deleted file mode 100644 index d2bc011..0000000 --- a/templates/compose.py +++ /dev/null @@ -1,203 +0,0 @@ -AGENT_POSTGRES_SNIPPET = """ - ${SERVICE_NAME}: - image: postgres:17-alpine - restart: unless-stopped - networks: - - portabase - ports: - - "${PORT}:5432" - volumes: - - ${VOL_NAME}:/var/lib/postgresql/data - environment: - - POSTGRES_DB=${DB_NAME} - - POSTGRES_USER=${USER} - - POSTGRES_PASSWORD=${PASSWORD} - healthcheck: - test: ["CMD-SHELL", "pg_isready -U ${USER} -d ${DB_NAME}"] - interval: 10s - timeout: 5s - retries: 5 -""" - -AGENT_MARIADB_SNIPPET = """ - ${SERVICE_NAME}: - image: mariadb:latest - restart: unless-stopped - networks: - - portabase - ports: - - "${PORT}:3306" - environment: - - MYSQL_DATABASE=${DB_NAME} - - MYSQL_USER=${USER} - - MYSQL_PASSWORD=${PASSWORD} - - MYSQL_RANDOM_ROOT_PASSWORD=yes - volumes: - - ${VOL_NAME}:/var/lib/mysql - healthcheck: - test: ["CMD-SHELL", "mariadb-admin ping -h localhost -u ${USER} -p${PASSWORD}"] - interval: 10s - timeout: 5s - retries: 5 -""" - -AGENT_MONGODB_AUTH_SNIPPET = """ - ${SERVICE_NAME}: - image: mongo:latest - restart: unless-stopped - networks: - - portabase - ports: - - "${PORT}:27017" - environment: - - MONGO_INITDB_ROOT_USERNAME=${USER} - - MONGO_INITDB_ROOT_PASSWORD=${PASSWORD} - - MONGO_INITDB_DATABASE=${DB_NAME} - command: mongod --auth - volumes: - - ${VOL_NAME}:/data/db - healthcheck: - test: ["CMD-SHELL", "mongosh --eval 'db.runCommand({ping:1})' --quiet"] - interval: 10s - timeout: 5s - retries: 5 -""" - -AGENT_MONGODB_SNIPPET = """ - ${SERVICE_NAME}: - image: mongo:latest - restart: unless-stopped - networks: - - portabase - ports: - - "${PORT}:27017" - environment: - - MONGO_INITDB_DATABASE=${DB_NAME} - volumes: - - ${VOL_NAME}:/data/db - healthcheck: - test: ["CMD-SHELL", "mongosh --eval 'db.runCommand({ping:1})' --quiet"] - interval: 10s - timeout: 5s - retries: 5 -""" - -AGENT_FIREBIRD_SNIPPET = """ - ${SERVICE_NAME}: - image: firebirdsql/firebird - restart: unless-stopped - networks: - - portabase - ports: - - "${PORT}:3050" - volumes: - - ${VOL_NAME}:/var/lib/firebird/data - environment: - - FIREBIRD_DATABASE=${DB_NAME} - - FIREBIRD_USER=${USER} - - FIREBIRD_PASSWORD=${PASSWORD} - - FIREBIRD_ROOT_PASSWORD=${ROOT_PASSWORD} - - FIREBIRD_DATABASE_DEFAULT_CHARSET=UTF8 - healthcheck: - test: ["CMD-SHELL", "nc -z localhost 3050"] - interval: 10s - timeout: 5s - retries: 5 -""" - - -AGENT_REDIS_SNIPPET = """ - ${SERVICE_NAME}: - image: redis:latest - ports: - - "${PORT}:6379" - volumes: - - ${VOL_NAME}:/data - command: [ "redis-server", "--appendonly", "yes" ] - networks: - - portabase - - default - healthcheck: - test: ["CMD-SHELL", "redis-cli ping | grep PONG"] - interval: 10s - timeout: 5s - retries: 5 -""" - -AGENT_REDIS_AUTH_SNIPPET = """ - ${SERVICE_NAME}: - image: redis:latest - ports: - - "${PORT}:6379" - volumes: - - ${VOL_NAME}:/data - environment: - - REDIS_PASSWORD=${PASSWORD} - command: [ "redis-server", "--requirepass", "${PASSWORD}", "--appendonly", "yes" ] - networks: - - portabase - - default - healthcheck: - test: ["CMD-SHELL", "redis-cli -a ${PASSWORD} ping | grep PONG"] - interval: 10s - timeout: 5s - retries: 5 -""" - - -AGENT_VALKEY_SNIPPET = """ - ${SERVICE_NAME}: - image: valkey/valkey:latest - environment: - - ALLOW_EMPTY_PASSWORD=yes - ports: - - "${PORT}:6379" - volumes: - - ${VOL_NAME}:/data - networks: - - portabase - - default - healthcheck: - test: ["CMD-SHELL", "valkey-cli ping | grep PONG"] - interval: 10s - timeout: 5s - retries: 5 -""" - -AGENT_VALKEY_AUTH_SNIPPET = """ - ${SERVICE_NAME}: - image: valkey/valkey:latest - command: --requirepass "${PASSWORD}" - ports: - - "${PORT}:6379" - volumes: - - ${VOL_NAME}:/data - networks: - - portabase - - default - healthcheck: - test: ["CMD-SHELL", "valkey-cli -a ${PASSWORD} ping | grep PONG"] - interval: 10s - timeout: 5s - retries: 5 -""" - -AGENT_MSSQL_SNIPPET = """ - ${SERVICE_NAME}: - image: mcr.microsoft.com/azure-sql-edge:latest - restart: unless-stopped - networks: - - portabase - ports: - - "${PORT}:1433" - environment: - - ACCEPT_EULA=Y - - MSSQL_SA_PASSWORD=${PASSWORD} - volumes: - - ${VOL_NAME}:/var/opt/mssql - healthcheck: - test: ["CMD-SHELL", "cat /proc/net/tcp6 | grep -q '059901' || exit 1"] - interval: 10s - timeout: 5s - retries: 20 -""" diff --git a/templates/dashboard.yml.j2 b/templates/dashboard.yml.j2 new file mode 100644 index 0000000..aa20be5 --- /dev/null +++ b/templates/dashboard.yml.j2 @@ -0,0 +1,52 @@ +name: {{ project_name_var }} +services: + portabase: + container_name: {{ project_name_var }}-app + image: portabase/portabase:latest + restart: unless-stopped + env_file: + - .env + ports: + - "{{ host_port_var }}:80" + environment: + - TZ={{ tz_var }} + - LOG_LEVEL={{ log_level_var }} + - PROJECT_SECRET={{ project_secret_var }} + - PROJECT_URL={{ project_url_var }} + volumes: + - portabase-data:/data +{%- if db_mode == "external" %} + depends_on: + db: + condition: service_healthy +{%- endif %} + healthcheck: + test: ["CMD-SHELL", "curl -f http://localhost/api/health"] + interval: 30s + timeout: 5s + retries: 3 + start_period: 60s +{%- if db_mode == "external" %} + db: + container_name: {{ project_name_var }}-pg + image: postgres:17-alpine + restart: unless-stopped + ports: + - "{{ pg_port_var }}:5432" + volumes: + - postgres-data:/var/lib/postgresql/data + environment: + - POSTGRES_DB={{ postgres_db_var }} + - POSTGRES_USER={{ postgres_user_var }} + - POSTGRES_PASSWORD={{ postgres_password_var }} + healthcheck: + test: ["CMD-SHELL", "pg_isready -U {{ postgres_user_var }} -d {{ postgres_db_var }}"] + interval: 10s + timeout: 5s + retries: 5 +{%- endif %} +volumes: +{%- if db_mode == "external" %} + postgres-data: +{%- endif %} + portabase-data: diff --git a/templates/engines/firebird.yml.j2 b/templates/engines/firebird.yml.j2 new file mode 100644 index 0000000..f11f488 --- /dev/null +++ b/templates/engines/firebird.yml.j2 @@ -0,0 +1,20 @@ + {{ name }}: + image: firebirdsql/firebird + restart: unless-stopped + networks: + - portabase + ports: + - "{{ port_var }}:3050" + volumes: + - {{ volume }}:/var/lib/firebird/data + environment: + - FIREBIRD_DATABASE={{ db_var }} + - FIREBIRD_USER={{ user_var }} + - FIREBIRD_PASSWORD={{ password_var }} + - FIREBIRD_ROOT_PASSWORD={{ root_password_var }} + - FIREBIRD_DATABASE_DEFAULT_CHARSET=UTF8 + healthcheck: + test: ["CMD-SHELL", "nc -z localhost 3050"] + interval: 10s + timeout: 5s + retries: 5 diff --git a/templates/engines/mariadb.yml.j2 b/templates/engines/mariadb.yml.j2 new file mode 100644 index 0000000..31c6c22 --- /dev/null +++ b/templates/engines/mariadb.yml.j2 @@ -0,0 +1,19 @@ + {{ name }}: + image: mariadb:latest + restart: unless-stopped + networks: + - portabase + ports: + - "{{ port_var }}:3306" + environment: + - MYSQL_DATABASE={{ db_var }} + - MYSQL_USER={{ user_var }} + - MYSQL_PASSWORD={{ password_var }} + - MYSQL_RANDOM_ROOT_PASSWORD=yes + volumes: + - {{ volume }}:/var/lib/mysql + healthcheck: + test: ["CMD-SHELL", "mariadb-admin ping -h localhost -u {{ user_var }} -p{{ password_var }}"] + interval: 10s + timeout: 5s + retries: 5 diff --git a/templates/engines/mongodb.yml.j2 b/templates/engines/mongodb.yml.j2 new file mode 100644 index 0000000..c1e0e6e --- /dev/null +++ b/templates/engines/mongodb.yml.j2 @@ -0,0 +1,23 @@ + {{ name }}: + image: mongo:latest + restart: unless-stopped + networks: + - portabase + ports: + - "{{ port_var }}:27017" + environment: +{%- if auth %} + - MONGO_INITDB_ROOT_USERNAME={{ user_var }} + - MONGO_INITDB_ROOT_PASSWORD={{ password_var }} +{%- endif %} + - MONGO_INITDB_DATABASE={{ db_var }} +{%- if auth %} + command: mongod --auth +{%- endif %} + volumes: + - {{ volume }}:/data/db + healthcheck: + test: ["CMD-SHELL", "mongosh --eval 'db.runCommand({ping:1})' --quiet"] + interval: 10s + timeout: 5s + retries: 5 diff --git a/templates/engines/mssql.yml.j2 b/templates/engines/mssql.yml.j2 new file mode 100644 index 0000000..2855ea7 --- /dev/null +++ b/templates/engines/mssql.yml.j2 @@ -0,0 +1,17 @@ + {{ name }}: + image: mcr.microsoft.com/azure-sql-edge:latest + restart: unless-stopped + networks: + - portabase + ports: + - "{{ port_var }}:1433" + environment: + - ACCEPT_EULA=Y + - MSSQL_SA_PASSWORD={{ password_var }} + volumes: + - {{ volume }}:/var/opt/mssql + healthcheck: + test: ["CMD-SHELL", "cat /proc/net/tcp6 | grep -q '059901' || exit 1"] + interval: 10s + timeout: 5s + retries: 20 diff --git a/templates/engines/mysql.yml.j2 b/templates/engines/mysql.yml.j2 new file mode 100644 index 0000000..31c6c22 --- /dev/null +++ b/templates/engines/mysql.yml.j2 @@ -0,0 +1,19 @@ + {{ name }}: + image: mariadb:latest + restart: unless-stopped + networks: + - portabase + ports: + - "{{ port_var }}:3306" + environment: + - MYSQL_DATABASE={{ db_var }} + - MYSQL_USER={{ user_var }} + - MYSQL_PASSWORD={{ password_var }} + - MYSQL_RANDOM_ROOT_PASSWORD=yes + volumes: + - {{ volume }}:/var/lib/mysql + healthcheck: + test: ["CMD-SHELL", "mariadb-admin ping -h localhost -u {{ user_var }} -p{{ password_var }}"] + interval: 10s + timeout: 5s + retries: 5 diff --git a/templates/engines/postgresql-cluster.yml.j2 b/templates/engines/postgresql-cluster.yml.j2 new file mode 100644 index 0000000..6dcf870 --- /dev/null +++ b/templates/engines/postgresql-cluster.yml.j2 @@ -0,0 +1,18 @@ + {{ name }}: + image: postgres:17-alpine + restart: unless-stopped + networks: + - portabase + ports: + - "{{ port_var }}:5432" + volumes: + - {{ volume }}:/var/lib/postgresql/data + environment: + - POSTGRES_DB={{ db_var }} + - POSTGRES_USER={{ user_var }} + - POSTGRES_PASSWORD={{ password_var }} + healthcheck: + test: ["CMD-SHELL", "pg_isready -U {{ user_var }} -d {{ db_var }}"] + interval: 10s + timeout: 5s + retries: 5 diff --git a/templates/engines/postgresql.yml.j2 b/templates/engines/postgresql.yml.j2 new file mode 100644 index 0000000..6dcf870 --- /dev/null +++ b/templates/engines/postgresql.yml.j2 @@ -0,0 +1,18 @@ + {{ name }}: + image: postgres:17-alpine + restart: unless-stopped + networks: + - portabase + ports: + - "{{ port_var }}:5432" + volumes: + - {{ volume }}:/var/lib/postgresql/data + environment: + - POSTGRES_DB={{ db_var }} + - POSTGRES_USER={{ user_var }} + - POSTGRES_PASSWORD={{ password_var }} + healthcheck: + test: ["CMD-SHELL", "pg_isready -U {{ user_var }} -d {{ db_var }}"] + interval: 10s + timeout: 5s + retries: 5 diff --git a/templates/engines/redis.yml.j2 b/templates/engines/redis.yml.j2 new file mode 100644 index 0000000..52a18ec --- /dev/null +++ b/templates/engines/redis.yml.j2 @@ -0,0 +1,22 @@ + {{ name }}: + image: redis:latest + restart: unless-stopped + ports: + - "{{ port_var }}:6379" + volumes: + - {{ volume }}:/data +{%- if auth %} + environment: + - REDIS_PASSWORD={{ password_var }} + command: ["redis-server", "--requirepass", "{{ password_var }}", "--appendonly", "yes"] +{%- else %} + command: ["redis-server", "--appendonly", "yes"] +{%- endif %} + networks: + - portabase + - default + healthcheck: + test: ["CMD-SHELL", "redis-cli {% if auth %}-a {{ password_var }} {% endif %}ping | grep PONG"] + interval: 10s + timeout: 5s + retries: 5 diff --git a/templates/engines/valkey.yml.j2 b/templates/engines/valkey.yml.j2 new file mode 100644 index 0000000..885d123 --- /dev/null +++ b/templates/engines/valkey.yml.j2 @@ -0,0 +1,21 @@ + {{ name }}: + image: valkey/valkey:latest + restart: unless-stopped +{%- if auth %} + command: --requirepass "{{ password_var }}" +{%- else %} + environment: + - ALLOW_EMPTY_PASSWORD=yes +{%- endif %} + ports: + - "{{ port_var }}:6379" + volumes: + - {{ volume }}:/data + networks: + - portabase + - default + healthcheck: + test: ["CMD-SHELL", "valkey-cli {% if auth %}-a {{ password_var }} {% endif %}ping | grep PONG"] + interval: 10s + timeout: 5s + retries: 5 diff --git a/ui/__init__.py b/ui/__init__.py new file mode 100644 index 0000000..d91ffcd --- /dev/null +++ b/ui/__init__.py @@ -0,0 +1,105 @@ +from __future__ import annotations + +import sys + +from rich.console import Console + +from core.errors import PortabaseError +from ui.components.banner import Banner +from ui.components.diff import Diff +from ui.components.hints import Hint +from ui.components.message import Message +from ui.components.progress import Progress +from ui.components.prompt import Prompt +from ui.components.section import Section +from ui.components.status import Status +from ui.components.summary import Summary +from ui.components.table import DataTable +from ui.form import Form +from ui.theme import QUESTIONARY_STYLE, QUESTIONARY_STYLE_PLAIN, RICH_THEME + + +class UI: + def __init__( + self, + console: Console | None = None, + *, + non_interactive: bool = False, + verbose: bool = False, + no_color: bool = False, + ) -> None: + self.non_interactive = non_interactive + self.verbose = verbose + self.no_color = no_color + self.console = console or self._make_console() + + def configure( + self, + *, + non_interactive: bool | None = None, + verbose: bool | None = None, + no_color: bool | None = None, + ) -> None: + if non_interactive is not None: + self.non_interactive = non_interactive + if verbose is not None: + self.verbose = verbose + if no_color is not None and no_color != self.no_color: + self.no_color = no_color + self.console = self._make_console() + + def _make_console(self) -> Console: + return Console(theme=RICH_THEME, no_color=self.no_color) + + def print(self, renderable, **kwargs) -> None: + self.console.print(renderable, **kwargs) + + def out(self, text: str) -> None: + sys.stdout.write(text) + + def banner(self) -> None: + Banner(self.console)() + + def success(self, text: str) -> None: + Message(self.console).success(text) + + def info(self, text: str) -> None: + Message(self.console).info(text) + + def warning(self, text: str) -> None: + Message(self.console).warning(text) + + def error(self, exc: PortabaseError, *, unexpected: bool = False) -> None: + Message(self.console).error(exc, verbose=self.verbose, unexpected=unexpected) + + def hint(self, text: str | None = None) -> None: + Hint(self.console)(text) + + def section(self, title: str) -> None: + Section(self.console)(title) + + def status(self, text: str): + return Status(self.console)(text) + + def progress(self) -> Progress: + return Progress(self.console) + + def summary(self, rows: list[tuple[str, str]], *, title: str | None = None) -> None: + Summary(self.console)(rows, title=title) + + def table( + self, columns: list[str], rows: list[list[str]], *, title: str | None = None + ) -> None: + DataTable(self.console)(columns, rows, title=title) + + def diff(self, text: str) -> None: + Diff(self.console)(text) + + def form(self) -> Form: + style = QUESTIONARY_STYLE_PLAIN if self.no_color else QUESTIONARY_STYLE + return Form(Prompt(self.console, style), self.non_interactive) + + def confirm( + self, question: str, *, default: bool = False, value: bool | None = None + ) -> bool: + return self.form().confirm(question, value=value, default=default) diff --git a/ui/components/__init__.py b/ui/components/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/ui/components/banner.py b/ui/components/banner.py new file mode 100644 index 0000000..c2e7b79 --- /dev/null +++ b/ui/components/banner.py @@ -0,0 +1,18 @@ +from __future__ import annotations + +from rich.align import Align + +from ui.components.base import Component +from ui.components.hints import Hint + +BANNER = """ +[brand]█▀█ █▀█ █▀█ ▀█▀ ▄▀█ █▄▄ ▄▀█ █▀ █▀▀[/brand] +[brand]█▀▀ █▄█ █▀▄ █ █▀█ █▄█ █▀█ ▄█ ██▄[/brand] +[hint]Deploy your infrastructure anywhere.[/hint] +""" + + +class Banner(Component): + def __call__(self) -> None: + self.console.print(Align.center(BANNER)) + self.console.print(Align.center(Hint(self.console).random() + "\n")) diff --git a/ui/components/base.py b/ui/components/base.py new file mode 100644 index 0000000..bf3495e --- /dev/null +++ b/ui/components/base.py @@ -0,0 +1,8 @@ +from __future__ import annotations + +from rich.console import Console + + +class Component: + def __init__(self, console: Console) -> None: + self.console = console diff --git a/ui/components/diff.py b/ui/components/diff.py new file mode 100644 index 0000000..fa4ec68 --- /dev/null +++ b/ui/components/diff.py @@ -0,0 +1,13 @@ +from __future__ import annotations + +from rich.syntax import Syntax + +from ui.components.base import Component + + +class Diff(Component): + def __call__(self, text: str) -> None: + if not text.strip(): + self.console.print("[info]ℹ No changes.[/info]") + return + self.console.print(Syntax(text, "diff", theme="ansi_dark", word_wrap=False)) diff --git a/ui/components/hints.py b/ui/components/hints.py new file mode 100644 index 0000000..2b6e834 --- /dev/null +++ b/ui/components/hints.py @@ -0,0 +1,39 @@ +from __future__ import annotations + +import random + +from ui.components.base import Component + +HINTS = [ + "The Edge Key contains the connection details for dashboard and agent communication.", + "Portabase uses Docker Compose to isolate your databases.", + "List every configured database with 'portabase db list '.", + "Running 'portabase stop' will gracefully shut down your containers.", + "The agent polls GitHub for configuration updates.", + "Logs can be viewed in real time with 'portabase logs '.", + "Custom environment variables can be added to the generated .env file.", + "Need to update? Use 'portabase update' to get the latest version.", + "You can add several databases to a single agent during setup.", + "Portabase Dashboard provides a web interface to manage your infrastructure.", + "Docker not running? The CLI offers to start it for you.", + "All configurations are stored locally in the component's folder.", + "The 'portabase restart' command is useful after manual .env modifications.", + "Portabase is open source. Visit our GitHub to contribute.", + "Use the --start flag with 'agent' or 'dashboard' to skip the final prompt.", + "Internal databases are automatically backed up when using volumes.", + "The dashboard requires a PostgreSQL database to store its own data.", + "Switch the update channel to 'beta' with 'portabase config set update_channel beta'.", + "The Portabase network keeps communication between your containers private.", + "Lost your Edge Key? You can find it in the dashboard.", + "The 'portabase uninstall' command safely removes containers and their data.", + "Use 'portabase --version' to check your current installation details.", + "The 'databases.json' file keeps track of all managed database instances.", +] + + +class Hint(Component): + def random(self) -> str: + return f"[hint]{random.choice(HINTS)}[/hint]" + + def __call__(self, text: str | None = None) -> None: + self.console.print(f"[hint]{text}[/hint]" if text else self.random()) diff --git a/ui/components/message.py b/ui/components/message.py new file mode 100644 index 0000000..ea58795 --- /dev/null +++ b/ui/components/message.py @@ -0,0 +1,35 @@ +from __future__ import annotations + +import traceback + +from core.errors import PortabaseError +from ui.components.base import Component + + +class Message(Component): + def success(self, text: str) -> None: + self.console.print(f"[success]✔ {text}[/success]") + + def info(self, text: str) -> None: + self.console.print(f"[info]ℹ {text}[/info]") + + def warning(self, text: str) -> None: + self.console.print(f"[warning]⚠ {text}[/warning]") + + def error( + self, exc: PortabaseError, *, verbose: bool = False, unexpected: bool = False + ) -> None: + label = "Unexpected error" if unexpected else "Error" + self.console.print(f"[danger]✖ {label}:[/danger] {exc.message}") + if exc.hint: + self.console.print(f" [hint]↳ {exc.hint}[/hint]") + if verbose or unexpected: + self.console.print(f" [hint]code: {exc.code}[/hint]") + if verbose and exc.cause is not None: + self.console.print( + f" [hint]cause: {type(exc.cause).__name__}: {exc.cause}[/hint]" + ) + if verbose: + self.console.print( + "".join(traceback.format_exception(exc)), highlight=False, markup=False + ) diff --git a/ui/components/progress.py b/ui/components/progress.py new file mode 100644 index 0000000..3082328 --- /dev/null +++ b/ui/components/progress.py @@ -0,0 +1,34 @@ +from __future__ import annotations + +from collections.abc import Callable, Iterator +from contextlib import contextmanager + +from rich.progress import ( + BarColumn, + DownloadColumn, + SpinnerColumn, + TextColumn, + TransferSpeedColumn, +) +from rich.progress import Progress as RichProgress + +from ui.components.base import Component +from ui.components.hints import Hint + + +class Progress(Component): + @contextmanager + def download(self, description: str, total: int) -> Iterator[Callable[[int], None]]: + with RichProgress( + SpinnerColumn(), + TextColumn( + "[progress.description]{task.description}\n" + + Hint(self.console).random() + ), + BarColumn(), + DownloadColumn(), + TransferSpeedColumn(), + console=self.console, + ) as progress: + task = progress.add_task(description, total=total or None) + yield lambda n: progress.update(task, advance=n) diff --git a/ui/components/prompt.py b/ui/components/prompt.py new file mode 100644 index 0000000..28a926f --- /dev/null +++ b/ui/components/prompt.py @@ -0,0 +1,45 @@ +from __future__ import annotations + +from collections.abc import Sequence + +import questionary +from questionary import Style +from rich.console import Console + +from ui.components.base import Component + + +class Prompt(Component): + def __init__(self, console: Console, style: Style) -> None: + super().__init__(console) + self.style = style + + def text(self, message: str, *, default: str | None = None) -> str | None: + return questionary.text(message, default=default or "", style=self.style).ask() + + def integer(self, message: str, *, default: int | None = None) -> int | None: + answer = questionary.text( + message, + default="" if default is None else str(default), + validate=lambda v: ( + v.strip().lstrip("-").isdigit() or "Enter a whole number" + ), + style=self.style, + ).ask() + return None if answer is None else int(answer) + + def secret(self, message: str) -> str | None: + return questionary.password(message, style=self.style).ask() + + def confirm(self, message: str, *, default: bool = False) -> bool | None: + return questionary.confirm(message, default=default, style=self.style).ask() + + def select( + self, message: str, choices: Sequence[str], *, default: str | None = None + ) -> str | None: + return questionary.select( + message, choices=list(choices), default=default, style=self.style + ).ask() + + def path(self, message: str, *, default: str | None = None) -> str | None: + return questionary.path(message, default=default or "", style=self.style).ask() diff --git a/ui/components/section.py b/ui/components/section.py new file mode 100644 index 0000000..97be076 --- /dev/null +++ b/ui/components/section.py @@ -0,0 +1,11 @@ +from __future__ import annotations + +from rich.panel import Panel + +from ui.components.base import Component + + +class Section(Component): + def __call__(self, title: str) -> None: + self.console.print("") + self.console.print(Panel(f"[bold]{title}[/bold]", style="cyan", expand=False)) diff --git a/ui/components/status.py b/ui/components/status.py new file mode 100644 index 0000000..4e53488 --- /dev/null +++ b/ui/components/status.py @@ -0,0 +1,12 @@ +from __future__ import annotations + +from contextlib import AbstractContextManager + +from ui.components.base import Component +from ui.components.hints import Hint + + +class Status(Component): + def __call__(self, text: str, *, spinner: str = "dots") -> AbstractContextManager: + message = f"[bold magenta]{text}[/bold magenta]\n{Hint(self.console).random()}" + return self.console.status(message, spinner=spinner) diff --git a/ui/components/summary.py b/ui/components/summary.py new file mode 100644 index 0000000..def6e4c --- /dev/null +++ b/ui/components/summary.py @@ -0,0 +1,37 @@ +from __future__ import annotations + +import re + +from rich.panel import Panel +from rich.table import Table + +from ui.components.base import Component + +_SENSITIVE = re.compile(r"(password|secret|key|token)", re.I) +_URL_CREDS = re.compile(r"://([^:/@]+):([^@/]+)@") + + +def mask(label: str, value: str) -> str: + if _SENSITIVE.search(label): + return "••••••••" + return _URL_CREDS.sub(r"://\1:****@", value) + + +class Summary(Component): + def __call__( + self, rows: list[tuple[str, str]], *, title: str | None = None + ) -> None: + table = Table(show_header=False, box=None, padding=(0, 2)) + table.add_column("Property", style="bold cyan") + table.add_column("Value", style="white") + for label, value in rows: + table.add_row(label, mask(label, str(value))) + self.console.print("") + self.console.print( + Panel( + table, + title=f"[bold white]{title}[/bold white]" if title else None, + border_style="bold blue", + expand=False, + ) + ) diff --git a/ui/components/table.py b/ui/components/table.py new file mode 100644 index 0000000..193fc91 --- /dev/null +++ b/ui/components/table.py @@ -0,0 +1,19 @@ +from __future__ import annotations + +from rich.table import Table + +from ui.components.base import Component + +_STYLES = ["cyan", "blue", "magenta", "green", "white", "dim"] + + +class DataTable(Component): + def __call__( + self, columns: list[str], rows: list[list[str]], *, title: str | None = None + ) -> None: + table = Table(title=title) + for i, col in enumerate(columns): + table.add_column(col, style=_STYLES[i % len(_STYLES)]) + for row in rows: + table.add_row(*[str(c) for c in row]) + self.console.print(table) diff --git a/ui/form.py b/ui/form.py new file mode 100644 index 0000000..3f2003a --- /dev/null +++ b/ui/form.py @@ -0,0 +1,119 @@ +from __future__ import annotations + +from collections.abc import Callable, Sequence +from typing import Any + +from core.errors import UserAbort, ValidationError +from core.fields import Field +from ui.components.prompt import Prompt + +_TRUE = {"1", "true", "yes", "y", "on"} +_FALSE = {"0", "false", "no", "n", "off"} + + +class Form: + def __init__(self, prompt: Prompt, non_interactive: bool) -> None: + self.prompt = prompt + self.non_interactive = non_interactive + self._askers: dict[str, Callable[[Field], Any]] = { + "text": lambda f: self.prompt.text(f.prompt, default=f.default), + "int": lambda f: self.prompt.integer(f.prompt, default=f.default), + "secret": lambda f: self.prompt.secret(f.prompt), + "bool": lambda f: self.prompt.confirm(f.prompt, default=bool(f.default)), + "choice": lambda f: self.prompt.select( + f.prompt, f.choices, default=f.default + ), + "path": lambda f: self.prompt.path(f.prompt, default=f.default), + } + + def ask(self, field: Field, value: Any | None = None) -> Any: + if value is not None: + return self._coerce_and_validate(field, value) + if self.non_interactive: + if field.default is not None: + return self._coerce_and_validate(field, field.default) + raise ValidationError( + f"Missing {field.flag}", + hint=f"Required in non-interactive mode: {field.prompt}", + ) + return self._ask_until_valid(field) + + def collect( + self, fields: Sequence[Field], values: dict[str, Any] + ) -> dict[str, Any]: + return {f.name: self.ask(f, values.get(f.name)) for f in fields} + + def text( + self, prompt: str, *, value=None, default=None, validator=None, name="value" + ) -> str: + field = Field(name, prompt, "text", default=default, validator=validator) + return self.ask(field, value) + + def integer( + self, prompt: str, *, value=None, default=None, validator=None, name="value" + ) -> int: + field = Field(name, prompt, "int", default=default, validator=validator) + return self.ask(field, value) + + def secret(self, prompt: str, *, value=None, validator=None, name="value") -> str: + return self.ask(Field(name, prompt, "secret", validator=validator), value) + + def confirm( + self, prompt: str, *, value=None, default: bool = False, name="value" + ) -> bool: + return self.ask(Field(name, prompt, "bool", default=default), value) + + def choice( + self, + prompt: str, + choices: Sequence[str], + *, + value=None, + default=None, + name="value", + ) -> str: + field = Field(name, prompt, "choice", default=default, choices=tuple(choices)) + return self.ask(field, value) + + def _ask_until_valid(self, field: Field) -> Any: + if field.help: + self.prompt.console.print(f"[info]ℹ {field.help}[/info]") + while True: + answer = self._askers[field.kind](field) + if answer is None: + raise UserAbort() + try: + return self._coerce_and_validate(field, answer) + except ValidationError as e: + self.prompt.console.print(f"[danger]✖ {e.message}[/danger]") + + def _coerce_and_validate(self, field: Field, value: Any) -> Any: + value = self._coerce(field, value) + if field.kind == "choice" and value not in field.choices: + raise ValidationError( + f"Invalid value for {field.flag}: {value!r}", + hint="Choices: " + ", ".join(field.choices), + ) + if field.validator is not None: + value = field.validator(value) + return value + + @staticmethod + def _coerce(field: Field, value: Any) -> Any: + if field.kind == "int" and not isinstance(value, int): + try: + return int(str(value).strip()) + except ValueError as e: + raise ValidationError( + f"{field.flag} must be a whole number, got {value!r}" + ) from e + if field.kind == "bool" and not isinstance(value, bool): + s = str(value).strip().lower() + if s in _TRUE: + return True + if s in _FALSE: + return False + raise ValidationError(f"{field.flag} must be true or false, got {value!r}") + if field.kind in ("text", "secret", "path", "choice"): + return str(value) + return value diff --git a/ui/theme.py b/ui/theme.py new file mode 100644 index 0000000..816576f --- /dev/null +++ b/ui/theme.py @@ -0,0 +1,41 @@ +from __future__ import annotations + +from questionary import Style +from rich.theme import Theme + +PALETTE = { + "brand": "#ff6600", + "accent": "#5f00d7", + "info": "cyan", + "warning": "magenta", + "danger": "red", + "success": "green", + "muted": "grey50", +} + +RICH_THEME = Theme( + { + "info": f"dim {PALETTE['info']}", + "warning": PALETTE["warning"], + "danger": f"bold {PALETTE['danger']}", + "success": f"bold {PALETTE['success']}", + "title": f"bold white on {PALETTE['accent']}", + "key": f"bold {PALETTE['brand']}", + "value": "white", + "hint": f"italic {PALETTE['muted']}", + "brand": f"bold {PALETTE['brand']}", + } +) + +QUESTIONARY_STYLE = Style( + [ + ("qmark", f"fg:{PALETTE['brand']} bold"), + ("question", "bold"), + ("pointer", f"fg:{PALETTE['brand']} bold"), + ("highlighted", f"fg:black bg:{PALETTE['brand']} bold"), + ("selected", f"fg:{PALETTE['brand']} bold"), + ("answer", f"fg:{PALETTE['brand']}"), + ] +) + +QUESTIONARY_STYLE_PLAIN = Style([]) diff --git a/uv.lock b/uv.lock new file mode 100644 index 0000000..19cda1b --- /dev/null +++ b/uv.lock @@ -0,0 +1,961 @@ +version = 1 +revision = 3 +requires-python = ">=3.12" +resolution-markers = [ + "python_full_version >= '3.15'", + "python_full_version < '3.15'", +] + +[[package]] +name = "altgraph" +version = "0.17.5" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/7e/f8/97fdf103f38fed6792a1601dbc16cc8aac56e7459a9fff08c812d8ae177a/altgraph-0.17.5.tar.gz", hash = "sha256:c87b395dd12fabde9c99573a9749d67da8d29ef9de0125c7f536699b4a9bc9e7", size = 48428, upload-time = "2025-11-21T20:35:50.583Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/a9/ba/000a1996d4308bc65120167c21241a3b205464a2e0b58deda26ae8ac21d1/altgraph-0.17.5-py2.py3-none-any.whl", hash = "sha256:f3a22400bce1b0c701683820ac4f3b159cd301acab067c51c653e06961600597", size = 21228, upload-time = "2025-11-21T20:35:49.444Z" }, +] + +[[package]] +name = "annotated-doc" +version = "0.0.4" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/57/ba/046ceea27344560984e26a590f90bc7f4a75b06701f653222458922b558c/annotated_doc-0.0.4.tar.gz", hash = "sha256:fbcda96e87e9c92ad167c2e53839e57503ecfda18804ea28102353485033faa4", size = 7288, upload-time = "2025-11-10T22:07:42.062Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/1e/d3/26bf1008eb3d2daa8ef4cacc7f3bfdc11818d111f7e2d0201bc6e3b49d45/annotated_doc-0.0.4-py3-none-any.whl", hash = "sha256:571ac1dc6991c450b25a9c2d84a3705e2ae7a53467b5d111c24fa8baabbed320", size = 5303, upload-time = "2025-11-10T22:07:40.673Z" }, +] + +[[package]] +name = "ast-serialize" +version = "0.11.1" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/4e/a2/04a9383e7512c91c54b2f34b3ff86dc7d2610506f588c2bda36e952a68f7/ast_serialize-0.11.1.tar.gz", hash = "sha256:cc5db2983805f6be786488aac8c5998d5b71965488d1b18c44d435a2205a5cb4", size = 953785, upload-time = "2026-09-09T16:05:27.851Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/e2/cb/3577c66278e4ea4ff6968aee04327447335ecfc58c76123f68cf7f76d75e/ast_serialize-0.11.1-cp314-cp314-pyemscripten_2026_0_wasm32.whl", hash = "sha256:f7ca1557553fdab313999ad69156de154ff87152cae963698be2e765164bb6c0", size = 897021, upload-time = "2026-09-09T16:03:59.316Z" }, + { url = "https://files.pythonhosted.org/packages/47/db/9b4eed53ab0bb63653f56ef71062fa91693c8c26b0b648c58d98365c990c/ast_serialize-0.11.1-cp314-cp314t-macosx_10_12_x86_64.whl", hash = "sha256:8b54e44763a851c336ca137c60a2434511f0d155e0b924ef374bfbc8d8db8927", size = 1235148, upload-time = "2026-09-09T16:04:01.201Z" }, + { url = "https://files.pythonhosted.org/packages/fd/97/1ebe323015c3cf530f08e88df9929bfd9f5cf8165842bd4b37a7c958565c/ast_serialize-0.11.1-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:753afabcc4abf295f515ea33185cf997a1990b88a29d3a3336a9acf62ea85950", size = 1216082, upload-time = "2026-09-09T16:04:02.87Z" }, + { url = "https://files.pythonhosted.org/packages/ac/72/0d3a368edc2d1e0e188c0d3f23821f4e20a6a2291b23d9df482a6b11277f/ast_serialize-0.11.1-cp314-cp314t-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:98b8aba6bb682b9e8859c628382b4881c0600b28056d63c3168c227c449f3284", size = 1282848, upload-time = "2026-09-09T16:04:04.316Z" }, + { url = "https://files.pythonhosted.org/packages/9c/fb/2a546a57d7caa708d739c1747bbdf45cc3aa9dac9407a11d9d67e8a0e255/ast_serialize-0.11.1-cp314-cp314t-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:651a7890558896c3e08089e51ab3b3a43710eab15b72ad861c400bc9a51923ba", size = 1285337, upload-time = "2026-09-09T16:04:05.899Z" }, + { url = "https://files.pythonhosted.org/packages/bf/16/4ff4179584f8b8bec348787bf721cb7ce3d4c707c7878778065249b59b9a/ast_serialize-0.11.1-cp314-cp314t-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:ff8ddc1453bfe7934292409c7b8e9f68b0a6cff4e4c12535e4b4a66fd83aca0c", size = 1554906, upload-time = "2026-09-09T16:04:07.42Z" }, + { url = "https://files.pythonhosted.org/packages/df/e8/e54a54676ca8965e1ae3cfe065dc26c75158e0b3a3ce35af283da106fbf9/ast_serialize-0.11.1-cp314-cp314t-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:daec16b026c5741950a00c708acdf698c4f17babbe4829dd8232324822e9514f", size = 1301472, upload-time = "2026-09-09T16:04:08.782Z" }, + { url = "https://files.pythonhosted.org/packages/e0/ab/618835a2479868a0d4ac807dd04cf2f89d24afe22c4171538d9779c2772e/ast_serialize-0.11.1-cp314-cp314t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:9961abcb93bf03652ab09c00862a2396abacd57551c769ad34490a2ae508a61d", size = 1301293, upload-time = "2026-09-09T16:04:10.17Z" }, + { url = "https://files.pythonhosted.org/packages/61/cb/8acc29a1b279ccdfb30dcbdf099e1d8d34c4ee77be9be751c48061433129/ast_serialize-0.11.1-cp314-cp314t-manylinux_2_31_riscv64.whl", hash = "sha256:8ca9e48edf246f09fd9bdb64fbd9cd8d18cd2a69ffc24d44891466e17595094f", size = 1307807, upload-time = "2026-09-09T16:04:11.989Z" }, + { url = "https://files.pythonhosted.org/packages/da/74/bada875132f452cb5c1179e3156df43cee6df6cb51f773220688be3d4bb4/ast_serialize-0.11.1-cp314-cp314t-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:6a57e0e025e9dcee48e466b3cf909178e298d238148664e796fc0f60ded5e52e", size = 1356265, upload-time = "2026-09-09T16:04:13.429Z" }, + { url = "https://files.pythonhosted.org/packages/4f/ae/720a8042fc8c1d170f4cc7ed46295ef078f8198ed3476f58210a1e675e0c/ast_serialize-0.11.1-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:110386eebccf200446d5f4dcc215f80866a5d3d3d054d3ced6ec1bf386cd3f02", size = 1459957, upload-time = "2026-09-09T16:04:14.878Z" }, + { url = "https://files.pythonhosted.org/packages/7d/2c/ce2ff1ffd4d70376073393c154033728c8535c17211b7710fee7b0f04474/ast_serialize-0.11.1-cp314-cp314t-musllinux_1_2_armv7l.whl", hash = "sha256:ef3950747f3a588f692cf4d6f2937eedd1a2d5e199f863af29e8ec7da78a4f36", size = 1562369, upload-time = "2026-09-09T16:04:16.563Z" }, + { url = "https://files.pythonhosted.org/packages/fd/3e/d7a5dd5f609bd0bac45617ce9425e4bd1e369b998c8678edfd70c985408c/ast_serialize-0.11.1-cp314-cp314t-musllinux_1_2_i686.whl", hash = "sha256:0bc94b81878a3f999fe4cdb606bf00ef7dff6b4f66d0e089b2cf7bfc00d59beb", size = 1556466, upload-time = "2026-09-09T16:04:18.396Z" }, + { url = "https://files.pythonhosted.org/packages/99/cd/a45179802a8637f2be252163ec4e4948534fa8d88798dd391c9cc54e096e/ast_serialize-0.11.1-cp314-cp314t-musllinux_1_2_ppc64le.whl", hash = "sha256:341dfc9e6b4e47e5c37e1b7257a77d39f407cf49a371ee90225ab624c2fe8b36", size = 1687190, upload-time = "2026-09-09T16:04:19.965Z" }, + { url = "https://files.pythonhosted.org/packages/01/eb/ef206d764ec0554368501b8b5268f5468819dc0588815acb0575465643df/ast_serialize-0.11.1-cp314-cp314t-musllinux_1_2_riscv64.whl", hash = "sha256:7b524cb1b08e15db0f116075c39ab51674b6d3c8103ec1a59c8ca464be83e28a", size = 1481188, upload-time = "2026-09-09T16:04:21.368Z" }, + { url = "https://files.pythonhosted.org/packages/8b/37/9a724c523af7c8b97a659bf4dc28a8e60f97d0cf4d19be277db4067ca734/ast_serialize-0.11.1-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:5c2358b341aca08bd2e1005713bc480bc06eee9f345b67fcb826829aa7b8d0d3", size = 1500731, upload-time = "2026-09-09T16:04:22.905Z" }, + { url = "https://files.pythonhosted.org/packages/48/10/890d5fcaf568088de7989e8cdb4ca525f485bd6cb25eea49d45f017500af/ast_serialize-0.11.1-cp314-cp314t-win32.whl", hash = "sha256:b9e61143a5904f46daa0cde74ba60d3b7c0e3e000752ee138e2bc68e126a5803", size = 1119009, upload-time = "2026-09-09T16:04:24.469Z" }, + { url = "https://files.pythonhosted.org/packages/9d/61/2bd32bd16b5e2ee07badc8c32a73c545dc5a9f087ed19c07eee1243d5a1b/ast_serialize-0.11.1-cp314-cp314t-win_amd64.whl", hash = "sha256:31df8649761d5cb6eb0de2209bd4a167c7477c0516147c70c0b2f5bddc7839bb", size = 1155665, upload-time = "2026-09-09T16:04:26.638Z" }, + { url = "https://files.pythonhosted.org/packages/5e/b6/84bb5e1dc418e660524b873c5bac9be2bf13d38676c96feef4cd4e7d03f3/ast_serialize-0.11.1-cp314-cp314t-win_arm64.whl", hash = "sha256:2f8f33e416a4c7aad12ad757e895c676e953b4027b80b90718dccea83d45111a", size = 1131854, upload-time = "2026-09-09T16:04:28.099Z" }, + { url = "https://files.pythonhosted.org/packages/8c/dd/16de2c0d23a6b298c735d4e4b56d86fa70476eef98e2c66ceaa65503b62f/ast_serialize-0.11.1-cp315-abi3.abi3t-macosx_10_12_x86_64.whl", hash = "sha256:a015ede631eeb098a23de8ec0cfe11a1dd02159ab7a26bdc7c6bde2befbbc7b4", size = 1235495, upload-time = "2026-09-09T16:04:29.654Z" }, + { url = "https://files.pythonhosted.org/packages/27/f7/302d2251e6298bbeabc8a1815c9147127031517b305001a02f34fd46b6b5/ast_serialize-0.11.1-cp315-abi3.abi3t-macosx_11_0_arm64.whl", hash = "sha256:57b7d52d1f5c92905cd648bda9efc83cac33097de1e5bb68de8feca9b5f7e87a", size = 1215642, upload-time = "2026-09-09T16:04:31.186Z" }, + { url = "https://files.pythonhosted.org/packages/93/06/93b6527646613502f364cebfb23783fa6701cdbe90761ee26144f1fa20b9/ast_serialize-0.11.1-cp315-abi3.abi3t-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:e47b9b028efbc0486263a49f782ad0ae3e879855fe5a2897b59feefb78e1c40c", size = 1283526, upload-time = "2026-09-09T16:04:32.597Z" }, + { url = "https://files.pythonhosted.org/packages/2f/ac/ffb216262c9a582d039d846081b8c3017dc368cbed5a2520deca5cb7f8cb/ast_serialize-0.11.1-cp315-abi3.abi3t-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:00adb748034c7b1938ec56925f9f6230ffcd9c56d47355fe32485df29b90b977", size = 1287526, upload-time = "2026-09-09T16:04:34.214Z" }, + { url = "https://files.pythonhosted.org/packages/a2/06/19a4c4837c4d5e73b982938d6da3c9ae10513b61527575b3e1b8396f9298/ast_serialize-0.11.1-cp315-abi3.abi3t-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:82359d00ea808c260c327e43955c0190bb7baef152a01ae0b5c74d05387b9552", size = 1558354, upload-time = "2026-09-09T16:04:35.669Z" }, + { url = "https://files.pythonhosted.org/packages/e5/29/2373199907b2d97feed176115308dfb3dd0446566b047009828f266d2d66/ast_serialize-0.11.1-cp315-abi3.abi3t-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:f35bb9ddeed4008c7d3e272dd1999885d712e1ce623b0a65ba8a8ed1d51c35cf", size = 1302731, upload-time = "2026-09-09T16:04:37.423Z" }, + { url = "https://files.pythonhosted.org/packages/33/55/ab6805a1457dbe565c84f8d36e2aa4207ef22408adfd01d225823cd07484/ast_serialize-0.11.1-cp315-abi3.abi3t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:74093ed682ba58456f0d4450da7fb62b826fddd97824d44fc4a81ca6e0bf9e23", size = 1301594, upload-time = "2026-09-09T16:04:38.861Z" }, + { url = "https://files.pythonhosted.org/packages/52/e2/4f54eab2201bb420d39bf61423abcb5d4daffd28e2270e1c5eee2bf3c0f1/ast_serialize-0.11.1-cp315-abi3.abi3t-manylinux_2_31_riscv64.whl", hash = "sha256:4dd7218870c203eff4533cfc04a39f29077d6789b20f5b746e7648fe5762a548", size = 1309355, upload-time = "2026-09-09T16:04:40.543Z" }, + { url = "https://files.pythonhosted.org/packages/10/07/755dd98664e2374080b72ccb5fea63d9f11b4bec2409a05b2a4ebc97618d/ast_serialize-0.11.1-cp315-abi3.abi3t-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:1e8d20700766171a8a17f89cf53d7bd9712c509460d28a51f7f98340577b78aa", size = 1356645, upload-time = "2026-09-09T16:04:42.299Z" }, + { url = "https://files.pythonhosted.org/packages/34/a0/fadbde3e108064236e2679728ce6d8247aefc81bac5fd82505b59cf69172/ast_serialize-0.11.1-cp315-abi3.abi3t-musllinux_1_2_aarch64.whl", hash = "sha256:20b3371c0403099cd55d59f4bcedb6d3994d9db3fa41711c648760a89ef2a575", size = 1459696, upload-time = "2026-09-09T16:04:44.069Z" }, + { url = "https://files.pythonhosted.org/packages/02/fd/b4f95249bd895368ea8290668e27d1f0a5c4ad888bd72a34c7beb2a500d6/ast_serialize-0.11.1-cp315-abi3.abi3t-musllinux_1_2_armv7l.whl", hash = "sha256:cc1a78b913f8665dda145999b1b4805ad2ef442ecbe3b819512cf7f95e70498a", size = 1562517, upload-time = "2026-09-09T16:04:45.461Z" }, + { url = "https://files.pythonhosted.org/packages/68/98/6afac594380410710d9d863b9ef41e9c6ca89bd901bda1464ee9e99180a8/ast_serialize-0.11.1-cp315-abi3.abi3t-musllinux_1_2_i686.whl", hash = "sha256:38f7141881e783bccc362d201d7fd7437934216b669b9fef39febc2f63fd2d9b", size = 1556951, upload-time = "2026-09-09T16:04:47.12Z" }, + { url = "https://files.pythonhosted.org/packages/ae/6f/5a81b5cbf1731f4722bc90adf4b4ace7dda69ca29837844147063aeafb22/ast_serialize-0.11.1-cp315-abi3.abi3t-musllinux_1_2_ppc64le.whl", hash = "sha256:13903a3f212ab9e5d05a6c3f2337288f4f8f4c18e6ab3817417ae7a3d46dee14", size = 1691039, upload-time = "2026-09-09T16:04:48.691Z" }, + { url = "https://files.pythonhosted.org/packages/99/eb/d9b903206cbd6d5143838367d385fa88ec49eaae2cc12f284327c6e0af05/ast_serialize-0.11.1-cp315-abi3.abi3t-musllinux_1_2_riscv64.whl", hash = "sha256:e5a9c53e64732118ae8235d0e57eb90e2333e90ff749a1a7dca6f8bfdfa5200e", size = 1483297, upload-time = "2026-09-09T16:04:50.136Z" }, + { url = "https://files.pythonhosted.org/packages/ed/48/ee13b4079ec67e9b333a87e5bb8eee643ab1b49f7173b3fca71158c6cdf6/ast_serialize-0.11.1-cp315-abi3.abi3t-musllinux_1_2_x86_64.whl", hash = "sha256:aecb606f67f21fd1c0ffba1826470d2ee400e41b23881ad9a1b2157306865b1a", size = 1501686, upload-time = "2026-09-09T16:04:51.734Z" }, + { url = "https://files.pythonhosted.org/packages/15/67/2175b79e1ee6b042e4bf8ed6871e60cafea7e9cf3fc69e87d7a9d520ae77/ast_serialize-0.11.1-cp315-abi3.abi3t-win32.whl", hash = "sha256:d1ef9c478d8c8ca83499704d13e5ac32c840ed7ca7884aaaf716166aca5a2806", size = 1119522, upload-time = "2026-09-09T16:04:53.248Z" }, + { url = "https://files.pythonhosted.org/packages/f6/c3/8c96aa5f121e9bdda0346b2df3919185eb1ec9a2cce2d20e311b4575ca0b/ast_serialize-0.11.1-cp315-abi3.abi3t-win_amd64.whl", hash = "sha256:27eef0739e0110f5db1ff5dfa38b3dadfcb6f0c31975a2b412f9f2bd72139cea", size = 1157260, upload-time = "2026-09-09T16:04:54.944Z" }, + { url = "https://files.pythonhosted.org/packages/10/bc/8aa663209e335eca73c9e1006197222b6b8dafa6665e513cdd975aa65496/ast_serialize-0.11.1-cp315-abi3.abi3t-win_arm64.whl", hash = "sha256:f9454960bbf185d33c669ccd5c9b76c96709a4542a95a65e27342c4d10dd8e20", size = 1132060, upload-time = "2026-09-09T16:04:56.501Z" }, + { url = "https://files.pythonhosted.org/packages/aa/52/185edf155ff422744acfb2869a8adff84b7dc308027852feb97287b8688b/ast_serialize-0.11.1-cp315-cp315-pyemscripten_2026_5_wasm32.whl", hash = "sha256:2db9df3bca1431da25946ee8f2c50df01212e34e6029532bc90b92d05fa8d7a0", size = 897219, upload-time = "2026-09-09T16:04:58.217Z" }, + { url = "https://files.pythonhosted.org/packages/7f/a7/e738887429de70350d9e25f84a95efbdd086b79579a811509dee8b02d7d5/ast_serialize-0.11.1-cp39-abi3-macosx_10_12_x86_64.whl", hash = "sha256:7b9a6089f1337838492b217707e9a55d0b9b407fe9169a960fa12282abf2234c", size = 1240585, upload-time = "2026-09-09T16:04:59.616Z" }, + { url = "https://files.pythonhosted.org/packages/a1/f9/46b64fa883f3c8b8cf51629978f16d8a2da3d5c5c02f64add40864907703/ast_serialize-0.11.1-cp39-abi3-macosx_11_0_arm64.whl", hash = "sha256:68929da7fb1c7375f69641baac9519d5c41ee441cd09a591ea5dfc83107ffe6f", size = 1228038, upload-time = "2026-09-09T16:05:00.948Z" }, + { url = "https://files.pythonhosted.org/packages/ad/89/fe75c8d0f104a4be11cd0e23acecf129484c9ba06c7130033752d63a78c5/ast_serialize-0.11.1-cp39-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:8945557ed3015173dbf8d44184da9621add9162720acbc0ac4832b5a091b4e8a", size = 1292388, upload-time = "2026-09-09T16:05:02.498Z" }, + { url = "https://files.pythonhosted.org/packages/cb/56/3f21c881900d8254a35195e1e962355b4d570ccd9cc193b0ba08abb81952/ast_serialize-0.11.1-cp39-abi3-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:b9cbed53bb8992b72e587dd47d0ca71703f5f715420f48ed57587c06c2fcd213", size = 1294413, upload-time = "2026-09-09T16:05:04.212Z" }, + { url = "https://files.pythonhosted.org/packages/f3/b2/8d77be3dad1139158c59e370391f4990ea73f7f102983392272856bd3a63/ast_serialize-0.11.1-cp39-abi3-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:65c836d1ab7af65e64a1dd73de82572566e838dd27529b1124eac0ac86f35e76", size = 1565921, upload-time = "2026-09-09T16:05:05.753Z" }, + { url = "https://files.pythonhosted.org/packages/94/2e/0675b5796c897f957fbdac6af0ebeeaa9a63cee866c1c54f1e5136ef5bf6/ast_serialize-0.11.1-cp39-abi3-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:4ab6df9898600ff45149b86c12bb5aa3359cc71843ad99ad0c924d11392469ab", size = 1312160, upload-time = "2026-09-09T16:05:07.269Z" }, + { url = "https://files.pythonhosted.org/packages/93/c2/1749fa4efae0dc98aad3c3d29f0178e2609208c3ba34700b4551082e40d5/ast_serialize-0.11.1-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:3eebd3dd25a81839ac7d25b5f3fb3527f6141b35eca29f63d2613fe6254307fd", size = 1312405, upload-time = "2026-09-09T16:05:08.753Z" }, + { url = "https://files.pythonhosted.org/packages/00/14/2afc9f9d7db551f805b4d5e496946dc3dc2a703d27294ccc200b6ff07a7a/ast_serialize-0.11.1-cp39-abi3-manylinux_2_31_riscv64.whl", hash = "sha256:311333c5f58f55fc04121be2c5adffa42a19e1038a43a2cef862fdf58c45bf4f", size = 1319458, upload-time = "2026-09-09T16:05:10.552Z" }, + { url = "https://files.pythonhosted.org/packages/c7/4a/8b2866d9d6d5d0b037d7bf2b74db6e75a66695c503248632116f24009a8d/ast_serialize-0.11.1-cp39-abi3-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:aa8d0a9f6d02e7626cdf540555bc2f5313eabe803dcca04854fc628ddf4b1058", size = 1365055, upload-time = "2026-09-09T16:05:12.069Z" }, + { url = "https://files.pythonhosted.org/packages/47/17/73119504574f9b46610ffb718501b254c8adb727d4e43c5babc3c62d4529/ast_serialize-0.11.1-cp39-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:e209a797fdf8680d58d2e969a9ddb25058ea682f6cff59dbabc98d8aef4e0db1", size = 1467771, upload-time = "2026-09-09T16:05:13.458Z" }, + { url = "https://files.pythonhosted.org/packages/56/f5/776bfb7a856ba0bb9bd373b76a90230fb77b1a10c987810b621ed244d5ba/ast_serialize-0.11.1-cp39-abi3-musllinux_1_2_armv7l.whl", hash = "sha256:b391becd31e9889da438d388aac0362e3dfd222affbe45d920580c351486c787", size = 1571459, upload-time = "2026-09-09T16:05:15.15Z" }, + { url = "https://files.pythonhosted.org/packages/c7/43/86a655170ee36a8fdce65a922b7e34040bb319eb4ec251161032343c1fc3/ast_serialize-0.11.1-cp39-abi3-musllinux_1_2_i686.whl", hash = "sha256:021afb3482d27d1dace9c8999724a13083e96c7ece785e53e418bf5df3b50e0a", size = 1568959, upload-time = "2026-09-09T16:05:16.677Z" }, + { url = "https://files.pythonhosted.org/packages/ed/b1/015a3a156dc9bd46e9c4e4f24a939eb0b87179b1c53c9495ac6f088f13b4/ast_serialize-0.11.1-cp39-abi3-musllinux_1_2_ppc64le.whl", hash = "sha256:c1dc60251d93beff32d4147ecd4ccc518b0f022a900093f6f3021c2012416f0f", size = 1698173, upload-time = "2026-09-09T16:05:18.248Z" }, + { url = "https://files.pythonhosted.org/packages/0e/ea/a988b70980aca3a8a3fb731901dc911fe73a456cebbc225b767bbe5490ff/ast_serialize-0.11.1-cp39-abi3-musllinux_1_2_riscv64.whl", hash = "sha256:b77d6a2267227d68d8221cfca177f8c6d9f68019f3250401d0355d2638db9fb9", size = 1493298, upload-time = "2026-09-09T16:05:19.806Z" }, + { url = "https://files.pythonhosted.org/packages/de/15/e049b2701ddab9087d7f62448f4c2c1b1463e676baa471345cf3e4a31e1b/ast_serialize-0.11.1-cp39-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:e68ac7d7fd1a5d42a3a6d726d7cc8a7e0c9987934f11c6a5162aa2cf68e81e24", size = 1510213, upload-time = "2026-09-09T16:05:21.402Z" }, + { url = "https://files.pythonhosted.org/packages/46/a2/23d555eb842d1397dce54fac31851e6f151e03a0ae74db389d9b39718c33/ast_serialize-0.11.1-cp39-abi3-win32.whl", hash = "sha256:ed449d786b9032a7b85fcd6c2f7f54c4cd0e1e1ad254d396805ff922096bf08d", size = 1125239, upload-time = "2026-09-09T16:05:22.796Z" }, + { url = "https://files.pythonhosted.org/packages/1f/4a/251f3fd1b8a5549edaedf8f3ba0b9fb5060e194d3c0ed208b593fccaeff1/ast_serialize-0.11.1-cp39-abi3-win_amd64.whl", hash = "sha256:6b43f5a9b9a8dd20ba3124914e63aa7d7427de761ac849081cda403c2112fda0", size = 1164260, upload-time = "2026-09-09T16:05:24.714Z" }, + { url = "https://files.pythonhosted.org/packages/5e/b5/08e2f643feb9e4d72d42a484949ad51238b18262a01c7036196c201cc330/ast_serialize-0.11.1-cp39-abi3-win_arm64.whl", hash = "sha256:a579ea6f473aab3958a64734194b22fbaec108ec45d994c28d3bd721e1e1da32", size = 1137533, upload-time = "2026-09-09T16:05:26.095Z" }, +] + +[[package]] +name = "certifi" +version = "2026.4.22" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/25/ee/6caf7a40c36a1220410afe15a1cc64993a1f864871f698c0f93acb72842a/certifi-2026.4.22.tar.gz", hash = "sha256:8d455352a37b71bf76a79caa83a3d6c25afee4a385d632127b6afb3963f1c580", size = 137077, upload-time = "2026-04-22T11:26:11.191Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/22/30/7cd8fdcdfbc5b869528b079bfb76dcdf6056b1a2097a662e5e8c04f42965/certifi-2026.4.22-py3-none-any.whl", hash = "sha256:3cb2210c8f88ba2318d29b0388d1023c8492ff72ecdde4ebdaddbb13a31b1c4a", size = 135707, upload-time = "2026-04-22T11:26:09.372Z" }, +] + +[[package]] +name = "cffi" +version = "2.1.1" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "pycparser", marker = "implementation_name != 'PyPy'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/9e/ef/008a1939e372c06329a3fce4279c02f328488f3526744906eeec3da7ad5f/cffi-2.1.1.tar.gz", hash = "sha256:dd31f52ea1086513bb9df30f8fcee9b8918323ae067a3d5b78bc826a000712be", size = 530807, upload-time = "2026-08-03T21:21:18.939Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/10/69/43965eccfdead3b9220015fd1320e117be8c6ed01a62ffab76eeb752f5d5/cffi-2.1.1-cp312-cp312-macosx_10_15_x86_64.whl", hash = "sha256:c8c69575568085ba0b1b10c0249d779a214aea6f6522e949a0fc9fb0fcb449d0", size = 184821, upload-time = "2026-08-03T21:19:44.887Z" }, + { url = "https://files.pythonhosted.org/packages/54/7d/16e5a096677b5e313ca80cd5e5170efa3ea44624a82bb111925522da64b1/cffi-2.1.1-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:f81b3b8f3d4e343550fa4baa0e479bba9f2d29ce9c2e9b51d1ce1718d7442fcf", size = 184719, upload-time = "2026-08-03T21:19:46.129Z" }, + { url = "https://files.pythonhosted.org/packages/56/e6/8941622732edec876dd17d0453dce07317ae96db34f2ec1436c9d3785986/cffi-2.1.1-cp312-cp312-manylinux1_i686.manylinux2014_i686.manylinux_2_17_i686.manylinux_2_5_i686.whl", hash = "sha256:811bd1e21d32de12efca32393a0ab3f5133b54fce9bd44b8bd77ab07da14bf6a", size = 214799, upload-time = "2026-08-03T21:19:47.218Z" }, + { url = "https://files.pythonhosted.org/packages/44/de/f98430906df1545ffde0d543dd124a7a439bc2cd32b36b9c53f805df7333/cffi-2.1.1-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:68e62fe11f30d5ca8289242866f0a5291402d8529ca2178ab8afc5c9694ae890", size = 222389, upload-time = "2026-08-03T21:19:48.331Z" }, + { url = "https://files.pythonhosted.org/packages/6a/5b/717f1526b9957b34456313c31645c5b82b8fb5c3fe9e4752999be7128bfc/cffi-2.1.1-cp312-cp312-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl", hash = "sha256:4a7c934f7360e8cd64fe9efadcbd10c7c6364f531e432b9a4bf5ccbc9e0e8b50", size = 210249, upload-time = "2026-08-03T21:19:49.543Z" }, + { url = "https://files.pythonhosted.org/packages/64/b3/f8aa4f3e34986c7e4ec45072d1b1b9dd295b6b18007b45518d79726dd725/cffi-2.1.1-cp312-cp312-manylinux2014_s390x.manylinux_2_17_s390x.whl", hash = "sha256:3143d81e29e1e20a9ce10901ec369012947876596f75a222235965f2b7ae832e", size = 208775, upload-time = "2026-08-03T21:19:50.918Z" }, + { url = "https://files.pythonhosted.org/packages/b1/db/dceb9dd5b231e1da801793f8acc9f3c52a7e1afe40bb1aae37e02b0faad5/cffi-2.1.1-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:c1453022f490d2459a11819d83ad1d586e9ff65a12ac3e705ffebd46d3685dcf", size = 221822, upload-time = "2026-08-03T21:19:52.054Z" }, + { url = "https://files.pythonhosted.org/packages/a0/d2/6cd24ae3be000a634109c247d1475d62e5616d0dc78c82770942ec384248/cffi-2.1.1-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:208f941bb9d18e768138677f0a6d2ce01f590df56043dda1df1535ac57c88517", size = 225232, upload-time = "2026-08-03T21:19:53.109Z" }, + { url = "https://files.pythonhosted.org/packages/cb/52/3fa190537004dd7f0ab860a6dc7c0175b8667f68d1e618a46f5498d30250/cffi-2.1.1-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:210019b6c7cf07f081b4c54635c8cf744377001350e29cc0f81c4377b4797735", size = 223597, upload-time = "2026-08-03T21:19:54.515Z" }, + { url = "https://files.pythonhosted.org/packages/80/fb/0bb75b7039588c074b37ae99f40d9bfddf990ecb2fbc346ebccd2e56b9be/cffi-2.1.1-cp312-cp312-win32.whl", hash = "sha256:046bfc24911b37851ee1b51aab8bffe713d89c68c6a057b09484ce9fd5f69b4e", size = 175292, upload-time = "2026-08-03T21:19:55.566Z" }, + { url = "https://files.pythonhosted.org/packages/d9/79/615cc094e2fb508cade7de88d3b4f6c4ec2bab695c97bce9153dc65aadf5/cffi-2.1.1-cp312-cp312-win_amd64.whl", hash = "sha256:f53e442b08449d42821fa4a4fba000095af9f62742a500f978a9f557ec44339a", size = 185919, upload-time = "2026-08-03T21:19:56.89Z" }, + { url = "https://files.pythonhosted.org/packages/70/c6/d0ea84713fe46b243a436a18fcd47d639732747e21635c8a27191b06dc30/cffi-2.1.1-cp312-cp312-win_arm64.whl", hash = "sha256:7bde5e4cc5c10140859842b9d383af292b22639a4dffb725314baf45968cef80", size = 180093, upload-time = "2026-08-03T21:19:58.155Z" }, + { url = "https://files.pythonhosted.org/packages/9d/f4/035513d4117049066b4779dc3b7c0c0fdad175fa13731c9f4003f1cd1478/cffi-2.1.1-cp313-cp313-ios_13_0_arm64_iphoneos.whl", hash = "sha256:b5bdfd1c873d4e093aabc0ca84c4ca6dbc4f752afb5c86f146d9742580c9da2e", size = 194248, upload-time = "2026-08-03T21:19:59.399Z" }, + { url = "https://files.pythonhosted.org/packages/76/af/2aeb4dbb5fc41a04161ae9ff1518de7cec08e164f44a8ce6a4cf7fd2cd1d/cffi-2.1.1-cp313-cp313-ios_13_0_arm64_iphonesimulator.whl", hash = "sha256:31348097ff5bbe827ccc41795d4dd099d9f0625e7def00ee653c137a490c2a6c", size = 196908, upload-time = "2026-08-03T21:20:00.746Z" }, + { url = "https://files.pythonhosted.org/packages/a7/46/2e5fdde8555706dd98139a910ca11be02809f3f605ce956f655d0214e100/cffi-2.1.1-cp313-cp313-macosx_10_15_x86_64.whl", hash = "sha256:9d2055050ea716bd38b7f7f1579c275386646b4894c155a3e2f3cd62ed41b7c6", size = 184805, upload-time = "2026-08-03T21:20:02.02Z" }, + { url = "https://files.pythonhosted.org/packages/55/41/4c7042f317b9217502988f0873af87e16ad606dc20f84e546e3e6ce9764c/cffi-2.1.1-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:19ee6127ee34de7d83ce3d371ebc5ed91addbdcc39f9ab15ce4eb35a4e534971", size = 184764, upload-time = "2026-08-03T21:20:03.141Z" }, + { url = "https://files.pythonhosted.org/packages/43/1f/1c3d90d91811c8f86ced9ed637956c54bfe5b79ca98fe976d7f8c8979f6b/cffi-2.1.1-cp313-cp313-manylinux1_i686.manylinux2014_i686.manylinux_2_17_i686.manylinux_2_5_i686.whl", hash = "sha256:6a8dddef476fab96d066d578fc88526767b836ab5ab21754e1d5bf3879c31c7c", size = 214722, upload-time = "2026-08-03T21:20:04.377Z" }, + { url = "https://files.pythonhosted.org/packages/37/6f/3b5ce4c3b2192d250f04908f2bfd91ef34552ec8f7716a5d4abdb8d67bb2/cffi-2.1.1-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:f16c709686a78c727bbbf059f92b0bf41c6fc60deec706d2dc19f529175a6125", size = 222369, upload-time = "2026-08-03T21:20:05.544Z" }, + { url = "https://files.pythonhosted.org/packages/02/10/4b3c75dde3d9663c9e02ba05c2668b954f671d4bbe346413ca8c696b295a/cffi-2.1.1-cp313-cp313-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl", hash = "sha256:fcd22650c908d7b7da162bbfaab594a1227a15d1643a98c68b122ac642fa2264", size = 210175, upload-time = "2026-08-03T21:20:06.75Z" }, + { url = "https://files.pythonhosted.org/packages/df/62/14f74b9543e605d17701dc797b815958b8bb70b7624ce1b832ddad48ed6c/cffi-2.1.1-cp313-cp313-manylinux2014_s390x.manylinux_2_17_s390x.whl", hash = "sha256:aa9511c62d14da7aacc9b4bf51f3f697a621e83b2d6919008243c3aad168eea3", size = 208670, upload-time = "2026-08-03T21:20:08.04Z" }, + { url = "https://files.pythonhosted.org/packages/95/95/86342356ff5953b3fb06f7ef7c5bee212d45e770abc7218d451b9148313c/cffi-2.1.1-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:a931079504ecc49efed7744c476a5c343a92fabf66dec2db95edb1b2fdc770e2", size = 221824, upload-time = "2026-08-03T21:20:09.274Z" }, + { url = "https://files.pythonhosted.org/packages/eb/ff/7b3429ff53aafe931ed8a5fc69f481bbef7ba6de87ddcbb63d08f483f613/cffi-2.1.1-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:a2d7755bef5a12ed488f4ef1f1b69ee9191d7396083b755a5d2295f6edb4768b", size = 225148, upload-time = "2026-08-03T21:20:10.7Z" }, + { url = "https://files.pythonhosted.org/packages/34/34/a95870b9221e09cf4f2ce3178b1a210abdfe63a1bd357da940418d7b8d15/cffi-2.1.1-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:e0bcb7e0f677f543555d2adff3bf19c05f66cdb4796e5ff602442ab2fe3c4ef7", size = 223564, upload-time = "2026-08-03T21:20:12.165Z" }, + { url = "https://files.pythonhosted.org/packages/70/ea/839b50531021a647fb5e929f72cf97bc1ff702b5472166164b5b6e76b851/cffi-2.1.1-cp313-cp313-win32.whl", hash = "sha256:334644fbac4eff73d985a17a91226df55d0f394160c4cfb880e084c8f7161cac", size = 175263, upload-time = "2026-08-03T21:20:13.559Z" }, + { url = "https://files.pythonhosted.org/packages/60/a6/8b149b2c3f2e11aaa1618ef64500b45f50f22c57a977a4dff1aff1f91042/cffi-2.1.1-cp313-cp313-win_amd64.whl", hash = "sha256:1aa5645c30469b09530c4ebca77ebf8f17618293c58f8549cb1a543a50236e7d", size = 185688, upload-time = "2026-08-03T21:20:14.69Z" }, + { url = "https://files.pythonhosted.org/packages/01/9a/11f687cb39d6a3504060d5242f04f48c735afb4d3d533958a20594890cb2/cffi-2.1.1-cp313-cp313-win_arm64.whl", hash = "sha256:63bbfd5ded17c4840ac07cd8f1c21ba9d9708141f840b324f422f41b207e3973", size = 180078, upload-time = "2026-08-03T21:20:15.917Z" }, + { url = "https://files.pythonhosted.org/packages/d3/7b/d6bbf82b8b96e7391438898c42f5bd96dd02030fd5b64937d248220003e2/cffi-2.1.1-cp314-cp314-ios_13_0_arm64_iphoneos.whl", hash = "sha256:7dbb61fe3a7699468030f71bbe5f8a0e326a151daa91beb11a6fc1f980c55e1c", size = 194064, upload-time = "2026-08-03T21:20:17.148Z" }, + { url = "https://files.pythonhosted.org/packages/94/e6/bcc91b283be94735e268487a054004f0aa19947b6348fa367db53230abc8/cffi-2.1.1-cp314-cp314-ios_13_0_arm64_iphonesimulator.whl", hash = "sha256:f24fb43132a4c6b4cb4eb029492919b2db645be6808d738f244fd146c03c32cb", size = 196720, upload-time = "2026-08-03T21:20:18.268Z" }, + { url = "https://files.pythonhosted.org/packages/d9/99/c4b0c17cacdc9c3b8f280026286a9826d6a208c0f047591a3c3ce99b91fd/cffi-2.1.1-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:d28630f5854ab07ab1fd4aba756de52326c82e6be15d414b12793f1975048b54", size = 184964, upload-time = "2026-08-03T21:20:19.708Z" }, + { url = "https://files.pythonhosted.org/packages/b3/a9/9db617d05d7367c1ad0ab00b3aa6e6f9281edd689b4ee9ea0e5a84e89c97/cffi-2.1.1-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:661c298b4821edebead0c91edd2b00374d67ad7c5a1f7a91d4442633b79d6a72", size = 184962, upload-time = "2026-08-03T21:20:20.833Z" }, + { url = "https://files.pythonhosted.org/packages/67/b8/b42132ca113dc567d37684437b46ca1dafc885902b02a110a02d5b511857/cffi-2.1.1-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:58acb8ab8e295e6c5ea12f888cbb13cf21511ef2a3303a23f4325c29d17fe5c1", size = 222328, upload-time = "2026-08-03T21:20:22.118Z" }, + { url = "https://files.pythonhosted.org/packages/80/10/c5c0cbf0a657aecf59ef511409734230bf556f05a0d6c9eed7aa5c0a0166/cffi-2.1.1-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl", hash = "sha256:456a61fa52d579ebf9df2e9552ead5129855dbaff6c1e5a9b1bc408809bdc062", size = 209985, upload-time = "2026-08-03T21:20:23.401Z" }, + { url = "https://files.pythonhosted.org/packages/d5/6c/bfa0b87b03b9238148beca990292843c9396ba069b54496596594173de7b/cffi-2.1.1-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.whl", hash = "sha256:a4f00aa42f75d6e4595e8866e748cc1705adc0cddfeb2ca86d0d03993d63ba03", size = 208530, upload-time = "2026-08-03T21:20:24.628Z" }, + { url = "https://files.pythonhosted.org/packages/e9/02/4e7d553a7ac4b4238b38b3c1b80d486e9d4436f8d2acbf87a0997fe3f402/cffi-2.1.1-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:b0431303acaea1089ad4b3e9ce4e6518193def1118d4073ca848635ee4ea2e96", size = 221525, upload-time = "2026-08-03T21:20:25.758Z" }, + { url = "https://files.pythonhosted.org/packages/82/1d/a4aaf9babd75acb4d5f223bff71533bee748dd770a382619a798960ee9ba/cffi-2.1.1-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:64faea20f4e2613363a1a9b9c7dd73058f3ecd00133a511e72ad7c511658f527", size = 225053, upload-time = "2026-08-03T21:20:26.985Z" }, + { url = "https://files.pythonhosted.org/packages/81/10/5dc0e7bdd18e22107054288283380fc97a06ae3f1656a106908d666a3c88/cffi-2.1.1-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:5c58fe613dc5e5336357eff555824a314d8e43282600435c8d1cb6a7a2fedd13", size = 223213, upload-time = "2026-08-03T21:20:28.277Z" }, + { url = "https://files.pythonhosted.org/packages/0b/e9/d0061c364cde06ee43168a0d076ac1da512cbc380d44767b844ba34fe2b6/cffi-2.1.1-cp314-cp314-win32.whl", hash = "sha256:1a18a57b58cfb21fc28d72e876acf10eaed67a1ed96226f92af4df681d571c4c", size = 177682, upload-time = "2026-08-03T21:20:44.288Z" }, + { url = "https://files.pythonhosted.org/packages/a7/06/1c3e01e3ba14c39f6d10bfbac52753b7e22259e38088e5cfe1d704918690/cffi-2.1.1-cp314-cp314-win_amd64.whl", hash = "sha256:3222ba5d678f80a030e6afbcc33dc1ae5cb45facabb61cee2c7016b8432fde48", size = 187949, upload-time = "2026-08-03T21:20:45.623Z" }, + { url = "https://files.pythonhosted.org/packages/87/5b/da4e39efe18eeb89cf580ea9cfc66b6a7c3eadb808fc0cc1d3a295cb5a5d/cffi-2.1.1-cp314-cp314-win_arm64.whl", hash = "sha256:ab36d55f9ed2d067327667c2fea18dda018eb628dd6347aa01dda6cf1f5d3836", size = 182947, upload-time = "2026-08-03T21:20:46.955Z" }, + { url = "https://files.pythonhosted.org/packages/23/59/40338bf421c5accea1d45158170c87006ef1cd371b05c077e76476949728/cffi-2.1.1-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:7750c6449dff7864bb9bb27ddfb0267756189201a3afc911d82b3caacd70dfc3", size = 188504, upload-time = "2026-08-03T21:20:29.495Z" }, + { url = "https://files.pythonhosted.org/packages/7d/47/5ecf1023850036e674c77ec4de86182d309ae344e39e7cba984b7df5d647/cffi-2.1.1-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:0beceaabe56af686895136a2de78db54ecd8e4046b236b8fd6d6cb61389e9bf2", size = 188259, upload-time = "2026-08-03T21:20:31.291Z" }, + { url = "https://files.pythonhosted.org/packages/2a/9c/92934c3bea9f785b23eba304538c0b4d37a2a96d2431eb3a1bc87a11aa19/cffi-2.1.1-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:49cbc70e6542d4ccccb936558d1064a8012541e78f821f955cff24e357776c94", size = 223864, upload-time = "2026-08-03T21:20:32.571Z" }, + { url = "https://files.pythonhosted.org/packages/4d/45/ba4c93527bc38616a8bd36488acb69a2212d60486794f0c1f318949bbb76/cffi-2.1.1-cp314-cp314t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl", hash = "sha256:e2d65b31f36619cda3999b78b2aa9632e76b78448e7a56fc4240824200e7c4fc", size = 211538, upload-time = "2026-08-03T21:20:33.808Z" }, + { url = "https://files.pythonhosted.org/packages/80/e9/b6ef565e452acb932fb0cb5443f44a78efbd1233e566f02b5a83855e9115/cffi-2.1.1-cp314-cp314t-manylinux2014_s390x.manylinux_2_17_s390x.whl", hash = "sha256:28907ab9bfb6aa13184cfc17c6b8e1023c5ab6fd7076d8c20a35e59fe04f8f29", size = 210688, upload-time = "2026-08-03T21:20:34.974Z" }, + { url = "https://files.pythonhosted.org/packages/9a/95/eff5f0cee78d2eabc7eebffec40d3fc1876b5f3c95582e018bb4b99601f2/cffi-2.1.1-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:51b31d1c98274844cfd7838ce00bfc27c7423a4dc00fc0772fc3331c2cc90676", size = 223803, upload-time = "2026-08-03T21:20:36.564Z" }, + { url = "https://files.pythonhosted.org/packages/fa/01/579d39fb8bef00a335a23d83757b44feb24cd6345a2c451b64cb67b9c362/cffi-2.1.1-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:5e7cecbaadb83884793e05828cee59b210b24583b9c7425d0ba6a754fe22eb4e", size = 226763, upload-time = "2026-08-03T21:20:37.816Z" }, + { url = "https://files.pythonhosted.org/packages/8d/b0/0b44f47c60b01b57b6e2bbd92343f13a85a1d93bc46ccf6e47e244acd99c/cffi-2.1.1-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:25792eac27877609e7bb06d42ff88278a6624fff2ba9bbb523c09616b117e80f", size = 225688, upload-time = "2026-08-03T21:20:38.959Z" }, + { url = "https://files.pythonhosted.org/packages/eb/d2/3b7176cb570a1d3e27faf67b72f591af508036e0d8b2be2ef9af9e8c84bb/cffi-2.1.1-cp314-cp314t-win32.whl", hash = "sha256:8ef53b2de9bcb9197d31854256575d59dbac0cba72ac627bb291ef5eceb74be4", size = 182868, upload-time = "2026-08-03T21:20:40.388Z" }, + { url = "https://files.pythonhosted.org/packages/56/78/31f00c1bcd97c9bbf55f1bfdf5bc809a5de8887473e90bb9960dca825e80/cffi-2.1.1-cp314-cp314t-win_amd64.whl", hash = "sha256:616f097f2fe415bc92a247f02e11f634e1f9e9a83d327e3c915c15089c87869e", size = 194104, upload-time = "2026-08-03T21:20:41.725Z" }, + { url = "https://files.pythonhosted.org/packages/7b/1b/58496f2ed0a35de575250c02a43ab3cc2c04d494a88fed31c1cabc0fd176/cffi-2.1.1-cp314-cp314t-win_arm64.whl", hash = "sha256:ad2c86c495b899d862ea0f4b42891b8713a3bd45dd4105c7fd51c2a72f39f3a5", size = 186402, upload-time = "2026-08-03T21:20:43.042Z" }, + { url = "https://files.pythonhosted.org/packages/c1/8f/9ebe220eab48a093d1a5a5e339ab0dc7316eef3bb04d63c42f0251b61f50/cffi-2.1.1-cp315-cp315-ios_13_0_arm64_iphoneos.whl", hash = "sha256:dddad92b554513a31f272570678ba307fb9f618f05e3d4a5eacafff9eae03e1d", size = 194043, upload-time = "2026-08-03T21:20:48.179Z" }, + { url = "https://files.pythonhosted.org/packages/ff/69/844bad3ece306c4782c2ecb93597035b6690d48704b803914c199da1e8b3/cffi-2.1.1-cp315-cp315-ios_13_0_arm64_iphonesimulator.whl", hash = "sha256:da0e573f9f97159390c89d9f1a9e41908b66d408cc5b58d08cf3847d844c531b", size = 196737, upload-time = "2026-08-03T21:20:49.457Z" }, + { url = "https://files.pythonhosted.org/packages/1b/8a/af668013284634733f02d683458a0728739c7d6ddb5e14cb0c20832266fe/cffi-2.1.1-cp315-cp315-macosx_10_15_x86_64.whl", hash = "sha256:fb92203a88b3d3053034db775110081c49d28be6551923805e039924093761e4", size = 184933, upload-time = "2026-08-03T21:20:50.639Z" }, + { url = "https://files.pythonhosted.org/packages/0c/75/2f5207ff6d1a613133b23a5203cc0c2a628313b5eb3974d7956ae3c57950/cffi-2.1.1-cp315-cp315-macosx_11_0_arm64.whl", hash = "sha256:2ae64be792b8966f2c69538199728b290e34726562896df1e5dc8ffd8d8188e8", size = 185002, upload-time = "2026-08-03T21:20:52.173Z" }, + { url = "https://files.pythonhosted.org/packages/e2/31/9e1313b0a6e30e91b3b3d3fff51ae99c857c07738e3afcce1f7334e1b7ab/cffi-2.1.1-cp315-cp315-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:507a24c282e0f42f8ed737cf048572cbf580468da5555764a8331735e9c736b6", size = 222271, upload-time = "2026-08-03T21:20:53.462Z" }, + { url = "https://files.pythonhosted.org/packages/50/e3/f6234a833e6e08c7007003074723c406559eecf9b48dfc97471e5a8eb7a0/cffi-2.1.1-cp315-cp315-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl", hash = "sha256:246fa40ce8645a614ff682e0b70f37134e460eaf93a775e0cbe3cca585a67a80", size = 209919, upload-time = "2026-08-03T21:20:54.783Z" }, + { url = "https://files.pythonhosted.org/packages/0d/fc/5f74e293fced6edb51af3a46c4ccf6c23c9943774ecb375ddbd522c76add/cffi-2.1.1-cp315-cp315-manylinux2014_s390x.manylinux_2_17_s390x.whl", hash = "sha256:471cee653ae88de62096552e6d24ccb4a5adb8c8c9f10b5054d0122c15bf2779", size = 208529, upload-time = "2026-08-03T21:20:56.066Z" }, + { url = "https://files.pythonhosted.org/packages/44/16/29e6d01b388bef055ecd6ca8244b3f4d336bd09e92d5d892187b9601084e/cffi-2.1.1-cp315-cp315-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:aeae0e330c9f6acd681f647d46cefd30c29f93e3392882e792e82080c9691399", size = 221630, upload-time = "2026-08-03T21:20:57.336Z" }, + { url = "https://files.pythonhosted.org/packages/a4/18/fa7f1f6857d5eb88a4ca99ffcbfb7c387a287ccc154c64a73e86314745d7/cffi-2.1.1-cp315-cp315-musllinux_1_2_aarch64.whl", hash = "sha256:42a494cee34437f05546455144f2b5d9ac09b1face62bcfce597d2e521066688", size = 225134, upload-time = "2026-08-03T21:20:58.675Z" }, + { url = "https://files.pythonhosted.org/packages/e0/9f/e8e3dfa04a1b4c241f8c91faacad872b4d4efd051d49764ad4e2fd4b9fea/cffi-2.1.1-cp315-cp315-musllinux_1_2_x86_64.whl", hash = "sha256:cc572dace3f60ef98d7b12ff411d20f5362feb31a0439eab0085bbfd349982d7", size = 223197, upload-time = "2026-08-03T21:20:59.968Z" }, + { url = "https://files.pythonhosted.org/packages/f8/7e/8debeb04f1ab9fe2a6963964cd6f1aaf7192627b83926586a6a4e089c9fa/cffi-2.1.1-cp315-cp315-win32.whl", hash = "sha256:4f42141fc14250de6dde5ee7ea4432be017252d91f19c5ad043c084cea629cac", size = 177683, upload-time = "2026-08-03T21:21:14.901Z" }, + { url = "https://files.pythonhosted.org/packages/e0/31/5158704cc474ab65c1647932e88be78dc0873f47130e253be38bcaf13d01/cffi-2.1.1-cp315-cp315-win_amd64.whl", hash = "sha256:e6e8cff14d6fb0be70a09c0bdc58096f501952d04624ebf867e0e56da2df8960", size = 187897, upload-time = "2026-08-03T21:21:16.108Z" }, + { url = "https://files.pythonhosted.org/packages/cc/4b/b3a2da8570c704ffc0f9762cdc3ec0f02c8573798e0b5cf7f11c82bbb70f/cffi-2.1.1-cp315-cp315-win_arm64.whl", hash = "sha256:27350daa11d4f10c540e6e89dada4c54feb7256ad03e9a4dc075ebad7ba360d1", size = 182935, upload-time = "2026-08-03T21:21:17.271Z" }, + { url = "https://files.pythonhosted.org/packages/d0/ef/5443574510a1207e6f6bc38ba6e1f1de36cb48fef07b2728bb896a21f430/cffi-2.1.1-cp315-cp315t-macosx_10_15_x86_64.whl", hash = "sha256:c26608d2222fb1e94487e4a387d85f13eb55d5ed725cb25a0c589ac4ee60e7bc", size = 188464, upload-time = "2026-08-03T21:21:01.163Z" }, + { url = "https://files.pythonhosted.org/packages/7e/ae/a56fa8c4686ad50e148fcbc8d3ae0d03915ff5c30d795058988c24118cef/cffi-2.1.1-cp315-cp315t-macosx_11_0_arm64.whl", hash = "sha256:4be96343e422f2dfcd12ab5c9f5aebe03f82f737c6bffeca6830b3875cb44aab", size = 188262, upload-time = "2026-08-03T21:21:02.382Z" }, + { url = "https://files.pythonhosted.org/packages/53/b2/6187f46f2912276a3ae284076109cc5c8680482f11f766ccf26db4a86427/cffi-2.1.1-cp315-cp315t-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:937c0052c05a31ca1daf18de3158eed4dbfcb9cc107adbea227728d647be701e", size = 223779, upload-time = "2026-08-03T21:21:03.553Z" }, + { url = "https://files.pythonhosted.org/packages/8a/f6/c3ad28bd19f77047a03084424fbd4cbe997303267c14423737324be0385d/cffi-2.1.1-cp315-cp315t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl", hash = "sha256:df423d40ee8654634421812bc3b196da3f9bd7d32929da813f8394c4348a5358", size = 211520, upload-time = "2026-08-03T21:21:04.863Z" }, + { url = "https://files.pythonhosted.org/packages/a0/cd/ccac9013a5bd9fd764de118674ab9c805b5ca10c19270d90ee273f8b2240/cffi-2.1.1-cp315-cp315t-manylinux2014_s390x.manylinux_2_17_s390x.whl", hash = "sha256:a730a083190634c65cca36ba5f489531576ebd79bcd5c8e172130f6453127231", size = 210673, upload-time = "2026-08-03T21:21:06.223Z" }, + { url = "https://files.pythonhosted.org/packages/52/86/2976131c639aead931c5bee5aba67e4b09fbeb8018b6f282f70803f923a7/cffi-2.1.1-cp315-cp315t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:363e05fa78e15116c3c32c210ee36884fd6b9afa6d440e47112c3bd511d64cb6", size = 223835, upload-time = "2026-08-03T21:21:07.539Z" }, + { url = "https://files.pythonhosted.org/packages/ac/0c/33a7aeab2f9c76918c52e084beb39c570db3588133412929e8ec06fab90b/cffi-2.1.1-cp315-cp315t-musllinux_1_2_aarch64.whl", hash = "sha256:770de9db11e84213beec501cfcaa013b019820ca881e03344dea5844f7876d94", size = 226705, upload-time = "2026-08-03T21:21:08.774Z" }, + { url = "https://files.pythonhosted.org/packages/e3/26/2cde30fdde421130bfc18f70395731a6e6b2053c6a1978a5258ff04e72fa/cffi-2.1.1-cp315-cp315t-musllinux_1_2_x86_64.whl", hash = "sha256:7da0c5eff80f0197f3b3d1232ec5a682a9325f4ae9016a78f5f5ca35f9ced1f5", size = 225539, upload-time = "2026-08-03T21:21:09.911Z" }, + { url = "https://files.pythonhosted.org/packages/6d/cd/a361394c94b2129d604bb846f624a8e88255a3ee33129c434a00d715e64f/cffi-2.1.1-cp315-cp315t-win32.whl", hash = "sha256:06c72bb76605a4b0cd0aad6930b69d4baf7dd5d806cfc409b824191099700e66", size = 182707, upload-time = "2026-08-03T21:21:11.226Z" }, + { url = "https://files.pythonhosted.org/packages/9b/b5/ba2b299993c26577d529b6ae29841f9e15b9fcf004d65f423f4fcf94ade9/cffi-2.1.1-cp315-cp315t-win_amd64.whl", hash = "sha256:d9c275eaacd24aa73f94ffd6de08fc3f932424d8b6c376f4bed7cde376fe7bc3", size = 193772, upload-time = "2026-08-03T21:21:12.39Z" }, + { url = "https://files.pythonhosted.org/packages/aa/29/35e016098c814cd93de9cd320c66b5bfba14dc6ecedd3cb518fa7c408c69/cffi-2.1.1-cp315-cp315t-win_arm64.whl", hash = "sha256:d18e5ac0f2f03f4f518d3e23db0f0cad7faa1da8620e9c09461d443bbf6e6692", size = 186360, upload-time = "2026-08-03T21:21:13.636Z" }, +] + +[[package]] +name = "charset-normalizer" +version = "3.4.7" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/e7/a1/67fe25fac3c7642725500a3f6cfe5821ad557c3abb11c9d20d12c7008d3e/charset_normalizer-3.4.7.tar.gz", hash = "sha256:ae89db9e5f98a11a4bf50407d4363e7b09b31e55bc117b4f7d80aab97ba009e5", size = 144271, upload-time = "2026-04-02T09:28:39.342Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/0c/eb/4fc8d0a7110eb5fc9cc161723a34a8a6c200ce3b4fbf681bc86feee22308/charset_normalizer-3.4.7-cp312-cp312-macosx_10_13_universal2.whl", hash = "sha256:eca9705049ad3c7345d574e3510665cb2cf844c2f2dcfe675332677f081cbd46", size = 311328, upload-time = "2026-04-02T09:26:24.331Z" }, + { url = "https://files.pythonhosted.org/packages/f8/e3/0fadc706008ac9d7b9b5be6dc767c05f9d3e5df51744ce4cc9605de7b9f4/charset_normalizer-3.4.7-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:6178f72c5508bfc5fd446a5905e698c6212932f25bcdd4b47a757a50605a90e2", size = 208061, upload-time = "2026-04-02T09:26:25.568Z" }, + { url = "https://files.pythonhosted.org/packages/42/f0/3dd1045c47f4a4604df85ec18ad093912ae1344ac706993aff91d38773a2/charset_normalizer-3.4.7-cp312-cp312-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:e1421b502d83040e6d7fb2fb18dff63957f720da3d77b2fbd3187ceb63755d7b", size = 229031, upload-time = "2026-04-02T09:26:26.865Z" }, + { url = "https://files.pythonhosted.org/packages/dc/67/675a46eb016118a2fbde5a277a5d15f4f69d5f3f5f338e5ee2f8948fcf43/charset_normalizer-3.4.7-cp312-cp312-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:edac0f1ab77644605be2cbba52e6b7f630731fc42b34cb0f634be1a6eface56a", size = 225239, upload-time = "2026-04-02T09:26:28.044Z" }, + { url = "https://files.pythonhosted.org/packages/4b/f8/d0118a2f5f23b02cd166fa385c60f9b0d4f9194f574e2b31cef350ad7223/charset_normalizer-3.4.7-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:5649fd1c7bade02f320a462fdefd0b4bd3ce036065836d4f42e0de958038e116", size = 216589, upload-time = "2026-04-02T09:26:29.239Z" }, + { url = "https://files.pythonhosted.org/packages/b1/f1/6d2b0b261b6c4ceef0fcb0d17a01cc5bc53586c2d4796fa04b5c540bc13d/charset_normalizer-3.4.7-cp312-cp312-manylinux_2_31_armv7l.whl", hash = "sha256:203104ed3e428044fd943bc4bf45fa73c0730391f9621e37fe39ecf477b128cb", size = 202733, upload-time = "2026-04-02T09:26:30.5Z" }, + { url = "https://files.pythonhosted.org/packages/6f/c0/7b1f943f7e87cc3db9626ba17807d042c38645f0a1d4415c7a14afb5591f/charset_normalizer-3.4.7-cp312-cp312-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:298930cec56029e05497a76988377cbd7457ba864beeea92ad7e844fe74cd1f1", size = 212652, upload-time = "2026-04-02T09:26:31.709Z" }, + { url = "https://files.pythonhosted.org/packages/38/dd/5a9ab159fe45c6e72079398f277b7d2b523e7f716acc489726115a910097/charset_normalizer-3.4.7-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:708838739abf24b2ceb208d0e22403dd018faeef86ddac04319a62ae884c4f15", size = 211229, upload-time = "2026-04-02T09:26:33.282Z" }, + { url = "https://files.pythonhosted.org/packages/d5/ff/531a1cad5ca855d1c1a8b69cb71abfd6d85c0291580146fda7c82857caa1/charset_normalizer-3.4.7-cp312-cp312-musllinux_1_2_armv7l.whl", hash = "sha256:0f7eb884681e3938906ed0434f20c63046eacd0111c4ba96f27b76084cd679f5", size = 203552, upload-time = "2026-04-02T09:26:34.845Z" }, + { url = "https://files.pythonhosted.org/packages/c1/4c/a5fb52d528a8ca41f7598cb619409ece30a169fbdf9cdce592e53b46c3a6/charset_normalizer-3.4.7-cp312-cp312-musllinux_1_2_ppc64le.whl", hash = "sha256:4dc1e73c36828f982bfe79fadf5919923f8a6f4df2860804db9a98c48824ce8d", size = 230806, upload-time = "2026-04-02T09:26:36.152Z" }, + { url = "https://files.pythonhosted.org/packages/59/7a/071feed8124111a32b316b33ae4de83d36923039ef8cf48120266844285b/charset_normalizer-3.4.7-cp312-cp312-musllinux_1_2_riscv64.whl", hash = "sha256:aed52fea0513bac0ccde438c188c8a471c4e0f457c2dd20cdbf6ea7a450046c7", size = 212316, upload-time = "2026-04-02T09:26:37.672Z" }, + { url = "https://files.pythonhosted.org/packages/fd/35/f7dba3994312d7ba508e041eaac39a36b120f32d4c8662b8814dab876431/charset_normalizer-3.4.7-cp312-cp312-musllinux_1_2_s390x.whl", hash = "sha256:fea24543955a6a729c45a73fe90e08c743f0b3334bbf3201e6c4bc1b0c7fa464", size = 227274, upload-time = "2026-04-02T09:26:38.93Z" }, + { url = "https://files.pythonhosted.org/packages/8a/2d/a572df5c9204ab7688ec1edc895a73ebded3b023bb07364710b05dd1c9be/charset_normalizer-3.4.7-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:bb6d88045545b26da47aa879dd4a89a71d1dce0f0e549b1abcb31dfe4a8eac49", size = 218468, upload-time = "2026-04-02T09:26:40.17Z" }, + { url = "https://files.pythonhosted.org/packages/86/eb/890922a8b03a568ca2f336c36585a4713c55d4d67bf0f0c78924be6315ca/charset_normalizer-3.4.7-cp312-cp312-win32.whl", hash = "sha256:2257141f39fe65a3fdf38aeccae4b953e5f3b3324f4ff0daf9f15b8518666a2c", size = 148460, upload-time = "2026-04-02T09:26:41.416Z" }, + { url = "https://files.pythonhosted.org/packages/35/d9/0e7dffa06c5ab081f75b1b786f0aefc88365825dfcd0ac544bdb7b2b6853/charset_normalizer-3.4.7-cp312-cp312-win_amd64.whl", hash = "sha256:5ed6ab538499c8644b8a3e18debabcd7ce684f3fa91cf867521a7a0279cab2d6", size = 159330, upload-time = "2026-04-02T09:26:42.554Z" }, + { url = "https://files.pythonhosted.org/packages/9e/5d/481bcc2a7c88ea6b0878c299547843b2521ccbc40980cb406267088bc701/charset_normalizer-3.4.7-cp312-cp312-win_arm64.whl", hash = "sha256:56be790f86bfb2c98fb742ce566dfb4816e5a83384616ab59c49e0604d49c51d", size = 147828, upload-time = "2026-04-02T09:26:44.075Z" }, + { url = "https://files.pythonhosted.org/packages/c1/3b/66777e39d3ae1ddc77ee606be4ec6d8cbd4c801f65e5a1b6f2b11b8346dd/charset_normalizer-3.4.7-cp313-cp313-macosx_10_13_universal2.whl", hash = "sha256:f496c9c3cc02230093d8330875c4c3cdfc3b73612a5fd921c65d39cbcef08063", size = 309627, upload-time = "2026-04-02T09:26:45.198Z" }, + { url = "https://files.pythonhosted.org/packages/2e/4e/b7f84e617b4854ade48a1b7915c8ccfadeba444d2a18c291f696e37f0d3b/charset_normalizer-3.4.7-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:0ea948db76d31190bf08bd371623927ee1339d5f2a0b4b1b4a4439a65298703c", size = 207008, upload-time = "2026-04-02T09:26:46.824Z" }, + { url = "https://files.pythonhosted.org/packages/c4/bb/ec73c0257c9e11b268f018f068f5d00aa0ef8c8b09f7753ebd5f2880e248/charset_normalizer-3.4.7-cp313-cp313-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:a277ab8928b9f299723bc1a2dabb1265911b1a76341f90a510368ca44ad9ab66", size = 228303, upload-time = "2026-04-02T09:26:48.397Z" }, + { url = "https://files.pythonhosted.org/packages/85/fb/32d1f5033484494619f701e719429c69b766bfc4dbc61aa9e9c8c166528b/charset_normalizer-3.4.7-cp313-cp313-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:3bec022aec2c514d9cf199522a802bd007cd588ab17ab2525f20f9c34d067c18", size = 224282, upload-time = "2026-04-02T09:26:49.684Z" }, + { url = "https://files.pythonhosted.org/packages/fa/07/330e3a0dda4c404d6da83b327270906e9654a24f6c546dc886a0eb0ffb23/charset_normalizer-3.4.7-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:e044c39e41b92c845bc815e5ae4230804e8e7bc29e399b0437d64222d92809dd", size = 215595, upload-time = "2026-04-02T09:26:50.915Z" }, + { url = "https://files.pythonhosted.org/packages/e3/7c/fc890655786e423f02556e0216d4b8c6bcb6bdfa890160dc66bf52dee468/charset_normalizer-3.4.7-cp313-cp313-manylinux_2_31_armv7l.whl", hash = "sha256:f495a1652cf3fbab2eb0639776dad966c2fb874d79d87ca07f9d5f059b8bd215", size = 201986, upload-time = "2026-04-02T09:26:52.197Z" }, + { url = "https://files.pythonhosted.org/packages/d8/97/bfb18b3db2aed3b90cf54dc292ad79fdd5ad65c4eae454099475cbeadd0d/charset_normalizer-3.4.7-cp313-cp313-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:e712b419df8ba5e42b226c510472b37bd57b38e897d3eca5e8cfd410a29fa859", size = 211711, upload-time = "2026-04-02T09:26:53.49Z" }, + { url = "https://files.pythonhosted.org/packages/6f/a5/a581c13798546a7fd557c82614a5c65a13df2157e9ad6373166d2a3e645d/charset_normalizer-3.4.7-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:7804338df6fcc08105c7745f1502ba68d900f45fd770d5bdd5288ddccb8a42d8", size = 210036, upload-time = "2026-04-02T09:26:54.975Z" }, + { url = "https://files.pythonhosted.org/packages/8c/bf/b3ab5bcb478e4193d517644b0fb2bf5497fbceeaa7a1bc0f4d5b50953861/charset_normalizer-3.4.7-cp313-cp313-musllinux_1_2_armv7l.whl", hash = "sha256:481551899c856c704d58119b5025793fa6730adda3571971af568f66d2424bb5", size = 202998, upload-time = "2026-04-02T09:26:56.303Z" }, + { url = "https://files.pythonhosted.org/packages/e7/4e/23efd79b65d314fa320ec6017b4b5834d5c12a58ba4610aa353af2e2f577/charset_normalizer-3.4.7-cp313-cp313-musllinux_1_2_ppc64le.whl", hash = "sha256:f59099f9b66f0d7145115e6f80dd8b1d847176df89b234a5a6b3f00437aa0832", size = 230056, upload-time = "2026-04-02T09:26:57.554Z" }, + { url = "https://files.pythonhosted.org/packages/b9/9f/1e1941bc3f0e01df116e68dc37a55c4d249df5e6fa77f008841aef68264f/charset_normalizer-3.4.7-cp313-cp313-musllinux_1_2_riscv64.whl", hash = "sha256:f59ad4c0e8f6bba240a9bb85504faa1ab438237199d4cce5f622761507b8f6a6", size = 211537, upload-time = "2026-04-02T09:26:58.843Z" }, + { url = "https://files.pythonhosted.org/packages/80/0f/088cbb3020d44428964a6c97fe1edfb1b9550396bf6d278330281e8b709c/charset_normalizer-3.4.7-cp313-cp313-musllinux_1_2_s390x.whl", hash = "sha256:3dedcc22d73ec993f42055eff4fcfed9318d1eeb9a6606c55892a26964964e48", size = 226176, upload-time = "2026-04-02T09:27:00.437Z" }, + { url = "https://files.pythonhosted.org/packages/6a/9f/130394f9bbe06f4f63e22641d32fc9b202b7e251c9aef4db044324dac493/charset_normalizer-3.4.7-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:64f02c6841d7d83f832cd97ccf8eb8a906d06eb95d5276069175c696b024b60a", size = 217723, upload-time = "2026-04-02T09:27:02.021Z" }, + { url = "https://files.pythonhosted.org/packages/73/55/c469897448a06e49f8fa03f6caae97074fde823f432a98f979cc42b90e69/charset_normalizer-3.4.7-cp313-cp313-win32.whl", hash = "sha256:4042d5c8f957e15221d423ba781e85d553722fc4113f523f2feb7b188cc34c5e", size = 148085, upload-time = "2026-04-02T09:27:03.192Z" }, + { url = "https://files.pythonhosted.org/packages/5d/78/1b74c5bbb3f99b77a1715c91b3e0b5bdb6fe302d95ace4f5b1bec37b0167/charset_normalizer-3.4.7-cp313-cp313-win_amd64.whl", hash = "sha256:3946fa46a0cf3e4c8cb1cc52f56bb536310d34f25f01ca9b6c16afa767dab110", size = 158819, upload-time = "2026-04-02T09:27:04.454Z" }, + { url = "https://files.pythonhosted.org/packages/68/86/46bd42279d323deb8687c4a5a811fd548cb7d1de10cf6535d099877a9a9f/charset_normalizer-3.4.7-cp313-cp313-win_arm64.whl", hash = "sha256:80d04837f55fc81da168b98de4f4b797ef007fc8a79ab71c6ec9bc4dd662b15b", size = 147915, upload-time = "2026-04-02T09:27:05.971Z" }, + { url = "https://files.pythonhosted.org/packages/97/c8/c67cb8c70e19ef1960b97b22ed2a1567711de46c4ddf19799923adc836c2/charset_normalizer-3.4.7-cp314-cp314-macosx_10_15_universal2.whl", hash = "sha256:c36c333c39be2dbca264d7803333c896ab8fa7d4d6f0ab7edb7dfd7aea6e98c0", size = 309234, upload-time = "2026-04-02T09:27:07.194Z" }, + { url = "https://files.pythonhosted.org/packages/99/85/c091fdee33f20de70d6c8b522743b6f831a2f1cd3ff86de4c6a827c48a76/charset_normalizer-3.4.7-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:1c2aed2e5e41f24ea8ef1590b8e848a79b56f3a5564a65ceec43c9d692dc7d8a", size = 208042, upload-time = "2026-04-02T09:27:08.749Z" }, + { url = "https://files.pythonhosted.org/packages/87/1c/ab2ce611b984d2fd5d86a5a8a19c1ae26acac6bad967da4967562c75114d/charset_normalizer-3.4.7-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:54523e136b8948060c0fa0bc7b1b50c32c186f2fceee897a495406bb6e311d2b", size = 228706, upload-time = "2026-04-02T09:27:09.951Z" }, + { url = "https://files.pythonhosted.org/packages/a8/29/2b1d2cb00bf085f59d29eb773ce58ec2d325430f8c216804a0a5cd83cbca/charset_normalizer-3.4.7-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:715479b9a2802ecac752a3b0efa2b0b60285cf962ee38414211abdfccc233b41", size = 224727, upload-time = "2026-04-02T09:27:11.175Z" }, + { url = "https://files.pythonhosted.org/packages/47/5c/032c2d5a07fe4d4855fea851209cca2b6f03ebeb6d4e3afdb3358386a684/charset_normalizer-3.4.7-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:bd6c2a1c7573c64738d716488d2cdd3c00e340e4835707d8fdb8dc1a66ef164e", size = 215882, upload-time = "2026-04-02T09:27:12.446Z" }, + { url = "https://files.pythonhosted.org/packages/2c/c2/356065d5a8b78ed04499cae5f339f091946a6a74f91e03476c33f0ab7100/charset_normalizer-3.4.7-cp314-cp314-manylinux_2_31_armv7l.whl", hash = "sha256:c45e9440fb78f8ddabcf714b68f936737a121355bf59f3907f4e17721b9d1aae", size = 200860, upload-time = "2026-04-02T09:27:13.721Z" }, + { url = "https://files.pythonhosted.org/packages/0c/cd/a32a84217ced5039f53b29f460962abb2d4420def55afabe45b1c3c7483d/charset_normalizer-3.4.7-cp314-cp314-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:3534e7dcbdcf757da6b85a0bbf5b6868786d5982dd959b065e65481644817a18", size = 211564, upload-time = "2026-04-02T09:27:15.272Z" }, + { url = "https://files.pythonhosted.org/packages/44/86/58e6f13ce26cc3b8f4a36b94a0f22ae2f00a72534520f4ae6857c4b81f89/charset_normalizer-3.4.7-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:e8ac484bf18ce6975760921bb6148041faa8fef0547200386ea0b52b5d27bf7b", size = 211276, upload-time = "2026-04-02T09:27:16.834Z" }, + { url = "https://files.pythonhosted.org/packages/8f/fe/d17c32dc72e17e155e06883efa84514ca375f8a528ba2546bee73fc4df81/charset_normalizer-3.4.7-cp314-cp314-musllinux_1_2_armv7l.whl", hash = "sha256:a5fe03b42827c13cdccd08e6c0247b6a6d4b5e3cdc53fd1749f5896adcdc2356", size = 201238, upload-time = "2026-04-02T09:27:18.229Z" }, + { url = "https://files.pythonhosted.org/packages/6a/29/f33daa50b06525a237451cdb6c69da366c381a3dadcd833fa5676bc468b3/charset_normalizer-3.4.7-cp314-cp314-musllinux_1_2_ppc64le.whl", hash = "sha256:2d6eb928e13016cea4f1f21d1e10c1cebd5a421bc57ddf5b1142ae3f86824fab", size = 230189, upload-time = "2026-04-02T09:27:19.445Z" }, + { url = "https://files.pythonhosted.org/packages/b6/6e/52c84015394a6a0bdcd435210a7e944c5f94ea1055f5cc5d56c5fe368e7b/charset_normalizer-3.4.7-cp314-cp314-musllinux_1_2_riscv64.whl", hash = "sha256:e74327fb75de8986940def6e8dee4f127cc9752bee7355bb323cc5b2659b6d46", size = 211352, upload-time = "2026-04-02T09:27:20.79Z" }, + { url = "https://files.pythonhosted.org/packages/8c/d7/4353be581b373033fb9198bf1da3cf8f09c1082561e8e922aa7b39bf9fe8/charset_normalizer-3.4.7-cp314-cp314-musllinux_1_2_s390x.whl", hash = "sha256:d6038d37043bced98a66e68d3aa2b6a35505dc01328cd65217cefe82f25def44", size = 227024, upload-time = "2026-04-02T09:27:22.063Z" }, + { url = "https://files.pythonhosted.org/packages/30/45/99d18aa925bd1740098ccd3060e238e21115fffbfdcb8f3ece837d0ace6c/charset_normalizer-3.4.7-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:7579e913a5339fb8fa133f6bbcfd8e6749696206cf05acdbdca71a1b436d8e72", size = 217869, upload-time = "2026-04-02T09:27:23.486Z" }, + { url = "https://files.pythonhosted.org/packages/5c/05/5ee478aa53f4bb7996482153d4bfe1b89e0f087f0ab6b294fcf92d595873/charset_normalizer-3.4.7-cp314-cp314-win32.whl", hash = "sha256:5b77459df20e08151cd6f8b9ef8ef1f961ef73d85c21a555c7eed5b79410ec10", size = 148541, upload-time = "2026-04-02T09:27:25.146Z" }, + { url = "https://files.pythonhosted.org/packages/48/77/72dcb0921b2ce86420b2d79d454c7022bf5be40202a2a07906b9f2a35c97/charset_normalizer-3.4.7-cp314-cp314-win_amd64.whl", hash = "sha256:92a0a01ead5e668468e952e4238cccd7c537364eb7d851ab144ab6627dbbe12f", size = 159634, upload-time = "2026-04-02T09:27:26.642Z" }, + { url = "https://files.pythonhosted.org/packages/c6/a3/c2369911cd72f02386e4e340770f6e158c7980267da16af8f668217abaa0/charset_normalizer-3.4.7-cp314-cp314-win_arm64.whl", hash = "sha256:67f6279d125ca0046a7fd386d01b311c6363844deac3e5b069b514ba3e63c246", size = 148384, upload-time = "2026-04-02T09:27:28.271Z" }, + { url = "https://files.pythonhosted.org/packages/94/09/7e8a7f73d24dba1f0035fbbf014d2c36828fc1bf9c88f84093e57d315935/charset_normalizer-3.4.7-cp314-cp314t-macosx_10_15_universal2.whl", hash = "sha256:effc3f449787117233702311a1b7d8f59cba9ced946ba727bdc329ec69028e24", size = 330133, upload-time = "2026-04-02T09:27:29.474Z" }, + { url = "https://files.pythonhosted.org/packages/8d/da/96975ddb11f8e977f706f45cddd8540fd8242f71ecdb5d18a80723dcf62c/charset_normalizer-3.4.7-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:fbccdc05410c9ee21bbf16a35f4c1d16123dcdeb8a1d38f33654fa21d0234f79", size = 216257, upload-time = "2026-04-02T09:27:30.793Z" }, + { url = "https://files.pythonhosted.org/packages/e5/e8/1d63bf8ef2d388e95c64b2098f45f84758f6d102a087552da1485912637b/charset_normalizer-3.4.7-cp314-cp314t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:733784b6d6def852c814bce5f318d25da2ee65dd4839a0718641c696e09a2960", size = 234851, upload-time = "2026-04-02T09:27:32.44Z" }, + { url = "https://files.pythonhosted.org/packages/9b/40/e5ff04233e70da2681fa43969ad6f66ca5611d7e669be0246c4c7aaf6dc8/charset_normalizer-3.4.7-cp314-cp314t-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:a89c23ef8d2c6b27fd200a42aa4ac72786e7c60d40efdc76e6011260b6e949c4", size = 233393, upload-time = "2026-04-02T09:27:34.03Z" }, + { url = "https://files.pythonhosted.org/packages/be/c1/06c6c49d5a5450f76899992f1ee40b41d076aee9279b49cf9974d2f313d5/charset_normalizer-3.4.7-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:6c114670c45346afedc0d947faf3c7f701051d2518b943679c8ff88befe14f8e", size = 223251, upload-time = "2026-04-02T09:27:35.369Z" }, + { url = "https://files.pythonhosted.org/packages/2b/9f/f2ff16fb050946169e3e1f82134d107e5d4ae72647ec8a1b1446c148480f/charset_normalizer-3.4.7-cp314-cp314t-manylinux_2_31_armv7l.whl", hash = "sha256:a180c5e59792af262bf263b21a3c49353f25945d8d9f70628e73de370d55e1e1", size = 206609, upload-time = "2026-04-02T09:27:36.661Z" }, + { url = "https://files.pythonhosted.org/packages/69/d5/a527c0cd8d64d2eab7459784fb4169a0ac76e5a6fc5237337982fd61347e/charset_normalizer-3.4.7-cp314-cp314t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:3c9a494bc5ec77d43cea229c4f6db1e4d8fe7e1bbffa8b6f0f0032430ff8ab44", size = 220014, upload-time = "2026-04-02T09:27:38.019Z" }, + { url = "https://files.pythonhosted.org/packages/7e/80/8a7b8104a3e203074dc9aa2c613d4b726c0e136bad1cc734594b02867972/charset_normalizer-3.4.7-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:8d828b6667a32a728a1ad1d93957cdf37489c57b97ae6c4de2860fa749b8fc1e", size = 218979, upload-time = "2026-04-02T09:27:39.37Z" }, + { url = "https://files.pythonhosted.org/packages/02/9a/b759b503d507f375b2b5c153e4d2ee0a75aa215b7f2489cf314f4541f2c0/charset_normalizer-3.4.7-cp314-cp314t-musllinux_1_2_armv7l.whl", hash = "sha256:cf1493cd8607bec4d8a7b9b004e699fcf8f9103a9284cc94962cb73d20f9d4a3", size = 209238, upload-time = "2026-04-02T09:27:40.722Z" }, + { url = "https://files.pythonhosted.org/packages/c2/4e/0f3f5d47b86bdb79256e7290b26ac847a2832d9a4033f7eb2cd4bcf4bb5b/charset_normalizer-3.4.7-cp314-cp314t-musllinux_1_2_ppc64le.whl", hash = "sha256:0c96c3b819b5c3e9e165495db84d41914d6894d55181d2d108cc1a69bfc9cce0", size = 236110, upload-time = "2026-04-02T09:27:42.33Z" }, + { url = "https://files.pythonhosted.org/packages/96/23/bce28734eb3ed2c91dcf93abeb8a5cf393a7b2749725030bb630e554fdd8/charset_normalizer-3.4.7-cp314-cp314t-musllinux_1_2_riscv64.whl", hash = "sha256:752a45dc4a6934060b3b0dab47e04edc3326575f82be64bc4fc293914566503e", size = 219824, upload-time = "2026-04-02T09:27:43.924Z" }, + { url = "https://files.pythonhosted.org/packages/2c/6f/6e897c6984cc4d41af319b077f2f600fc8214eb2fe2d6bcb79141b882400/charset_normalizer-3.4.7-cp314-cp314t-musllinux_1_2_s390x.whl", hash = "sha256:8778f0c7a52e56f75d12dae53ae320fae900a8b9b4164b981b9c5ce059cd1fcb", size = 233103, upload-time = "2026-04-02T09:27:45.348Z" }, + { url = "https://files.pythonhosted.org/packages/76/22/ef7bd0fe480a0ae9b656189ec00744b60933f68b4f42a7bb06589f6f576a/charset_normalizer-3.4.7-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:ce3412fbe1e31eb81ea42f4169ed94861c56e643189e1e75f0041f3fe7020abe", size = 225194, upload-time = "2026-04-02T09:27:46.706Z" }, + { url = "https://files.pythonhosted.org/packages/c5/a7/0e0ab3e0b5bc1219bd80a6a0d4d72ca74d9250cb2382b7c699c147e06017/charset_normalizer-3.4.7-cp314-cp314t-win32.whl", hash = "sha256:c03a41a8784091e67a39648f70c5f97b5b6a37f216896d44d2cdcb82615339a0", size = 159827, upload-time = "2026-04-02T09:27:48.053Z" }, + { url = "https://files.pythonhosted.org/packages/7a/1d/29d32e0fb40864b1f878c7f5a0b343ae676c6e2b271a2d55cc3a152391da/charset_normalizer-3.4.7-cp314-cp314t-win_amd64.whl", hash = "sha256:03853ed82eeebbce3c2abfdbc98c96dc205f32a79627688ac9a27370ea61a49c", size = 174168, upload-time = "2026-04-02T09:27:49.795Z" }, + { url = "https://files.pythonhosted.org/packages/de/32/d92444ad05c7a6e41fb2036749777c163baf7a0301a040cb672d6b2b1ae9/charset_normalizer-3.4.7-cp314-cp314t-win_arm64.whl", hash = "sha256:c35abb8bfff0185efac5878da64c45dafd2b37fb0383add1be155a763c1f083d", size = 153018, upload-time = "2026-04-02T09:27:51.116Z" }, + { url = "https://files.pythonhosted.org/packages/db/8f/61959034484a4a7c527811f4721e75d02d653a35afb0b6054474d8185d4c/charset_normalizer-3.4.7-py3-none-any.whl", hash = "sha256:3dce51d0f5e7951f8bb4900c257dad282f49190fdbebecd4ba99bcc41fef404d", size = 61958, upload-time = "2026-04-02T09:28:37.794Z" }, +] + +[[package]] +name = "click" +version = "8.4.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "colorama", marker = "sys_platform == 'win32'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/23/e4/796662cd90cf80e3a363c99db2b88e0e394b988a575f60a17e16440cd011/click-8.4.0.tar.gz", hash = "sha256:638f1338fe1235c8f4e008e4a8a254fb5c5fbdcbb40ece3c9142ebb78e792973", size = 350843, upload-time = "2026-05-17T00:47:58.425Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/ee/ae/8e92f8058baf87f6c7d86ee7e457668690195cc77efedb8d3797a06e3940/click-8.4.0-py3-none-any.whl", hash = "sha256:40c50b7c6c6adac2823d411041ec84f3f103f1b280d5e9ce0d7f998995832f81", size = 116147, upload-time = "2026-05-17T00:47:56.842Z" }, +] + +[[package]] +name = "colorama" +version = "0.4.6" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/d8/53/6f443c9a4a8358a93a6792e2acffb9d9d5cb0a5cfd8802644b7b1c9a02e4/colorama-0.4.6.tar.gz", hash = "sha256:08695f5cb7ed6e0531a20572697297273c47b8cae5a63ffc6d6ed5c201be6e44", size = 27697, upload-time = "2022-10-25T02:36:22.414Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/d1/d6/3965ed04c63042e047cb6a3e6ed1a63a35087b6a609aa3a15ed8ac56c221/colorama-0.4.6-py2.py3-none-any.whl", hash = "sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6", size = 25335, upload-time = "2022-10-25T02:36:20.889Z" }, +] + +[[package]] +name = "cryptography" +version = "50.0.1" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "cffi", marker = "platform_python_implementation != 'PyPy'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/bb/ad/5d6702db60b1e40b41ef513b6967ff5848f307d50f8449baf1634f5908f1/cryptography-50.0.1.tar.gz", hash = "sha256:5dd9bda1c12b4162f6ff568eeb5e0ff956c28d14406e875cfe8a63a2d414ff20", size = 880381, upload-time = "2026-08-25T19:45:45.499Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/ba/19/797e2aaac9df6a66f1550f49979dc1b1e39ecd2077501c30efa81e8d5d67/cryptography-50.0.1-cp311-abi3-macosx_11_0_arm64.whl", hash = "sha256:b8f852c65863251b9e3a1b8c150ce21e59b522dbb6a7d4bc80e680d38388e986", size = 4010153, upload-time = "2026-08-25T19:44:03.155Z" }, + { url = "https://files.pythonhosted.org/packages/90/34/9ce9a62ed9dc82ca9fd6a34445b6904af56e5f38b3eae2ed32e49c36053d/cryptography-50.0.1-cp311-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:53e279950892dc102c6b4e52af03ae5ea92fac572a1ddab78ca73a997f62b69f", size = 4723133, upload-time = "2026-08-25T19:44:05.461Z" }, + { url = "https://files.pythonhosted.org/packages/57/26/e6d4fc8512a51a5f9ee7bfdbfb853bce1197087df40c9ad993ad370b846f/cryptography-50.0.1-cp311-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:ff838d62ec1bfce4f9ba7fa16f4a7b554cd8d0c299e6be37502161a660c84eef", size = 4712478, upload-time = "2026-08-25T19:44:07.375Z" }, + { url = "https://files.pythonhosted.org/packages/e6/de/d3cdc2815697aae84126cbd6a030ca7b6b452e28a88b501b836bd3aa7a86/cryptography-50.0.1-cp311-abi3-manylinux_2_28_aarch64.whl", hash = "sha256:e74591e283fe6eb956416c929eb58262a719fe0311fd9054c62c3350ed8760d8", size = 4730726, upload-time = "2026-08-25T19:44:09.294Z" }, + { url = "https://files.pythonhosted.org/packages/55/32/38c0d344b98c06d34b5df8946565a9c0d6dbf32c8e0730a7f05f0a3c6cab/cryptography-50.0.1-cp311-abi3-manylinux_2_28_ppc64le.whl", hash = "sha256:5fe002589592ed749ce77fe0695fcbd3500dd61d7d6db5858a7544c612fa8e45", size = 5353524, upload-time = "2026-08-25T19:44:11.96Z" }, + { url = "https://files.pythonhosted.org/packages/e1/1b/82f0f0d8858d4432be1af790477edf62aef90324041aa07c57e57bef1af7/cryptography-50.0.1-cp311-abi3-manylinux_2_28_x86_64.whl", hash = "sha256:51593d180cf6d179bde5c5d065bed81386b1f381656ae7d042b7ffc87a9895ad", size = 4746720, upload-time = "2026-08-25T19:44:14.051Z" }, + { url = "https://files.pythonhosted.org/packages/29/ba/042ca458b8c64348c768284b5d23e69b92ed53d057ab779fee628564676d/cryptography-50.0.1-cp311-abi3-manylinux_2_31_armv7l.whl", hash = "sha256:359e62deae718bce96170e223fdcb6357e4fbd3bb7a3a75f4430763532560e49", size = 4361866, upload-time = "2026-08-25T19:44:16.167Z" }, + { url = "https://files.pythonhosted.org/packages/39/3b/e96c1ef71edef71057c7e3c3d982ce8fda554e0c52d0cc19c18845cde3eb/cryptography-50.0.1-cp311-abi3-manylinux_2_34_aarch64.whl", hash = "sha256:e2ca8fd1b6b4b82a1c4cb02841d0837e3c12336c2e24b520ab8ab3b969733d8f", size = 4730028, upload-time = "2026-08-25T19:44:18.085Z" }, + { url = "https://files.pythonhosted.org/packages/e3/38/45abd72ef63f2e7d0754a6cacf97bd8b69512ace7f6130d24c39ece65da2/cryptography-50.0.1-cp311-abi3-manylinux_2_34_ppc64le.whl", hash = "sha256:76de83fbd91ac49c0feaaa983d0748fd7a53176afac5fb3bf7478d244f0eb527", size = 5308405, upload-time = "2026-08-25T19:44:20.197Z" }, + { url = "https://files.pythonhosted.org/packages/85/66/6ccca4722987ddedaa7fc9c3f4708af7431f5535666c174350830888c6b7/cryptography-50.0.1-cp311-abi3-manylinux_2_34_x86_64.whl", hash = "sha256:51afcfceb15597cf2635068e4ac9a56b2abde622edde17f37d85fd7b5306497a", size = 4746230, upload-time = "2026-08-25T19:44:22.376Z" }, + { url = "https://files.pythonhosted.org/packages/13/0e/b1f92e013228111413f2e6743948b80bc24dfd3c1b87ba98ceea16f5df89/cryptography-50.0.1-cp311-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:be224a65493ec5b74a158ff22a5522ce4a5ca1e543c647a3a4730d4a09e5f959", size = 4862596, upload-time = "2026-08-25T19:44:24.472Z" }, + { url = "https://files.pythonhosted.org/packages/7e/22/c3654cccc856e9d682817b04ac3ee79731cb09ca6f95996a95c904de2883/cryptography-50.0.1-cp311-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:9ebcdd5519be9b652a46f507817a74591774fc3d6923ac364e4dfa64e36b291b", size = 5014082, upload-time = "2026-08-25T19:44:26.709Z" }, + { url = "https://files.pythonhosted.org/packages/42/8b/cb12b1b60c91b074ca6bf0fdd59aa8f10d8bc5f73af8faece86ef0421b37/cryptography-50.0.1-cp311-abi3-win_amd64.whl", hash = "sha256:aed8db4f6d71c51efb89530e12d9464e7bf2923d46c3205dc794a2a93f8c0648", size = 3842826, upload-time = "2026-08-25T19:44:28.784Z" }, + { url = "https://files.pythonhosted.org/packages/5b/f0/424cb557d99aa86ac55da5e2add02e2882e44047b6264f93ade1b975a993/cryptography-50.0.1-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:30a125032e5642a21ff816e021152bd4e7e94f03eff3f4b7fca41cd22bc3110f", size = 3973525, upload-time = "2026-08-25T19:44:30.7Z" }, + { url = "https://files.pythonhosted.org/packages/4d/72/3a2711d967977ab5fc80b782837c7e8d1ac7445e764c20c381a265c57ef3/cryptography-50.0.1-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:a0b1a59e3a089064a0ec309e9428c8e3ae4e161419d20ac33600767e83fc658a", size = 4708817, upload-time = "2026-08-25T19:44:32.773Z" }, + { url = "https://files.pythonhosted.org/packages/b4/f2/bb1f56e10815b789df0b409a69fa4992ff3d3fef9c72747f4a6b26fed38e/cryptography-50.0.1-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:8921d58f426793c5f1b47f0b59575780de9a095214958d0eb37d909593db8367", size = 4697300, upload-time = "2026-08-25T19:44:35.144Z" }, + { url = "https://files.pythonhosted.org/packages/08/bd/ed5396be499ffcf8807a585bfe38b71a1fbdd1c342b4f9b6d0ef5162a946/cryptography-50.0.1-cp314-cp314t-manylinux_2_28_aarch64.whl", hash = "sha256:a8f40ea47330e71b594a7e246898f93177c259490c63183dbaf9e571d71ed9a5", size = 4716039, upload-time = "2026-08-25T19:44:37.192Z" }, + { url = "https://files.pythonhosted.org/packages/f6/6e/1cf405c5c8e8df7545378048e954792f00b7f2367af8863ce8b8f3e10607/cryptography-50.0.1-cp314-cp314t-manylinux_2_28_ppc64le.whl", hash = "sha256:a255449073358275b64b67d3f595f268bbef70e72b6edb65e0c70c735bf739c9", size = 5332388, upload-time = "2026-08-25T19:44:39.16Z" }, + { url = "https://files.pythonhosted.org/packages/47/92/b4317e8c32c4f47b062f5398bd79106b220a124546f42be83bf32b761e2a/cryptography-50.0.1-cp314-cp314t-manylinux_2_28_x86_64.whl", hash = "sha256:8df2de9102026855887e4587084f6eabd80ed0f345b8ad8a7ac27ab9bf4723e0", size = 4730293, upload-time = "2026-08-25T19:44:41.298Z" }, + { url = "https://files.pythonhosted.org/packages/39/0d/a1e7633e2c744d0f2983320a27e924ef2264c79c56e1a58d5fb0a1cfd413/cryptography-50.0.1-cp314-cp314t-manylinux_2_31_armv7l.whl", hash = "sha256:ac02b07824d4d1001bd4367599f839c19cb171924c796e52c23508ac14c2c0cc", size = 4346031, upload-time = "2026-08-25T19:44:43.245Z" }, + { url = "https://files.pythonhosted.org/packages/88/dd/b215616f9bab3fc18510c78a4e5c9f362d77838503c363dc747c7d4f5c6f/cryptography-50.0.1-cp314-cp314t-manylinux_2_34_aarch64.whl", hash = "sha256:cbf74a81765ee67413503ca6e26dcc4f6f5a519822436cc0a1b97aab6c1b8a17", size = 4715344, upload-time = "2026-08-25T19:44:45.291Z" }, + { url = "https://files.pythonhosted.org/packages/b1/1b/ec3ebd31741d0e963612c4fe43caa39341b9b1e031e469820e42e4c83918/cryptography-50.0.1-cp314-cp314t-manylinux_2_34_ppc64le.whl", hash = "sha256:16c5ecd954b3330ebfb6605eca4fd952da8bef376551d5cc264534e3770a9ee6", size = 5287201, upload-time = "2026-08-25T19:44:47.297Z" }, + { url = "https://files.pythonhosted.org/packages/1a/01/0127d11a762b31a9ee0221894f540318761783f3fdc4bc5d057698caebd5/cryptography-50.0.1-cp314-cp314t-manylinux_2_34_x86_64.whl", hash = "sha256:79bf008d1f9af6071c797ad133e39915dfee7614f18f18f4db9072eb715064a3", size = 4730023, upload-time = "2026-08-25T19:44:49.435Z" }, + { url = "https://files.pythonhosted.org/packages/9e/b9/e7425ebfb599241a0c1d7000f1b466c3062da66c19d9525031315dff7213/cryptography-50.0.1-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:330fbb252391c596f1ae42c5754449dc924e6ad012dca8efe0d703f9f2d12ec6", size = 4847362, upload-time = "2026-08-25T19:44:51.94Z" }, + { url = "https://files.pythonhosted.org/packages/2d/fd/60d0ddf4defa12e482c9d5e0f554384d6e8ab25341fd15f060028fd92e6a/cryptography-50.0.1-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:42be3bb70596b3abe4ac097b75be223e8b3ab614a0e5de068e3dcc54d71d6149", size = 4999247, upload-time = "2026-08-25T19:44:53.876Z" }, + { url = "https://files.pythonhosted.org/packages/4d/56/bc4f2b209e766c93372cfcd59b781a0b2b59700f62a969580415b699c2b2/cryptography-50.0.1-cp314-cp314t-win_amd64.whl", hash = "sha256:f74455bb086a85d5e81246412602aaa97ed095e504cd40dd261ef50be42205bf", size = 3825806, upload-time = "2026-08-25T19:44:56.209Z" }, + { url = "https://files.pythonhosted.org/packages/84/a9/ee16a903f13755e914d1eecc482fe64d1f10761c3960e5d8fa6837377aff/cryptography-50.0.1-cp39-abi3-macosx_11_0_arm64.whl", hash = "sha256:ca83d00d9e69cd5eb63f2e69c3a5a59e0cecae5ae14c6ae0b35830fe3b37bad0", size = 4035307, upload-time = "2026-08-25T19:44:58.305Z" }, + { url = "https://files.pythonhosted.org/packages/5e/a5/9ec7e81e8526c0d7a387d73386b2daed3f39e10d81a85930bd1b6bfba65c/cryptography-50.0.1-cp39-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:05ba322c4da95b262a212c345af888ef2c37c88c0509756ea00a0e6d68850f23", size = 4751900, upload-time = "2026-08-25T19:45:00.401Z" }, + { url = "https://files.pythonhosted.org/packages/7e/3c/0e77bd5ffcf078e9dd27d3074aad6c030d9b10d0bf69329d573c927a188c/cryptography-50.0.1-cp39-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:e22dfed744bd4002e909464cb23d2f0b05c6f3113a79ef2e9864a53db737c733", size = 4738357, upload-time = "2026-08-25T19:45:02.786Z" }, + { url = "https://files.pythonhosted.org/packages/27/3a/3c5f80daa4dcd47323c7af8a2fcb90de27a33564d4fcac69846c0972691a/cryptography-50.0.1-cp39-abi3-manylinux_2_28_aarch64.whl", hash = "sha256:4c4188f7c0cf655be5c06342b817ed0f9595b69ffa2b12026e5353eed29dea88", size = 4758474, upload-time = "2026-08-25T19:45:04.889Z" }, + { url = "https://files.pythonhosted.org/packages/6e/2b/214cf0cf93db9628c3c20c896b229f327f6fb1b20e4b3743d8ad3f00af8b/cryptography-50.0.1-cp39-abi3-manylinux_2_28_ppc64le.whl", hash = "sha256:2ebbfb0f1fed745e91796e3e1080a1440423fdae8ece1b995a1d80883a409054", size = 5375862, upload-time = "2026-08-25T19:45:07.163Z" }, + { url = "https://files.pythonhosted.org/packages/d6/51/3f9701867a46b6c1740c9b52fc4d3bed6cbdcfedcc9b6e64305c07f39cff/cryptography-50.0.1-cp39-abi3-manylinux_2_28_x86_64.whl", hash = "sha256:407fe2b6db00939c05c0e945e9914238f2f0a430974839429dafc82b1ee6bee5", size = 4772942, upload-time = "2026-08-25T19:45:09.396Z" }, + { url = "https://files.pythonhosted.org/packages/0d/5c/13ea642e08e2544d0f5396122055f4820cfacb3203562197b5967125ea97/cryptography-50.0.1-cp39-abi3-manylinux_2_31_armv7l.whl", hash = "sha256:2b34d76a652ea2b6faf777c35df230c5637842cd904e04f16230c3f9f03e4361", size = 4383347, upload-time = "2026-08-25T19:45:11.659Z" }, + { url = "https://files.pythonhosted.org/packages/84/d5/7d1fe1cb93f91c428093ff234e128c89ba8ea61a6f26aab406081f9b996e/cryptography-50.0.1-cp39-abi3-manylinux_2_34_aarch64.whl", hash = "sha256:01f41478cf33fc605a6a089cd56d28b45c6c0b45a1928b61797f2621a04bac71", size = 4758050, upload-time = "2026-08-25T19:45:13.745Z" }, + { url = "https://files.pythonhosted.org/packages/dd/04/557fc5ead96a829e0bc812a3b9dc4a52a2f27e4f7f5950da7ff27653a805/cryptography-50.0.1-cp39-abi3-manylinux_2_34_ppc64le.whl", hash = "sha256:fc3ed7ebd2a8c96f5b166de0ab9b624996bef3b07bbeb19364dfb78222c22c80", size = 5332955, upload-time = "2026-08-25T19:45:16.193Z" }, + { url = "https://files.pythonhosted.org/packages/8c/eb/5d7124083e8d8cda8f5b348f544b71ad6f707ad63193758ef4d8e569da02/cryptography-50.0.1-cp39-abi3-manylinux_2_34_x86_64.whl", hash = "sha256:9dde0a357190eb3b1da1bb9ab750e9c85cba82ca5977aa0836cbb94e92611239", size = 4772694, upload-time = "2026-08-25T19:45:18.315Z" }, + { url = "https://files.pythonhosted.org/packages/63/8e/f1f955e0921dd2b6d22eae7e8d24a4c4b638d10735ffbf6a71f99eb0fcb8/cryptography-50.0.1-cp39-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:fd3718b960d0b5dd213cdf03f3bcb7000e69dda0de8b956061947ff6bcff5558", size = 4888413, upload-time = "2026-08-25T19:45:20.4Z" }, + { url = "https://files.pythonhosted.org/packages/1f/ab/89e2b798d2c3925f82e2bb72d5979f3d2f6da2dd22ef4a8cd8b70d920039/cryptography-50.0.1-cp39-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:2a93d05e34d5f67fba6f891fe85d929999baa7195e853923ea6d7576c9e68c5e", size = 5044355, upload-time = "2026-08-25T19:45:22.353Z" }, + { url = "https://files.pythonhosted.org/packages/99/89/87ef49ffe383ef4e147d27b7bf2088fb0b54ea409dd87b5a89442e5828a5/cryptography-50.0.1-cp39-abi3-win_amd64.whl", hash = "sha256:55d16b1ef3ee0958d893a977b19777887e546c9954ea81b200c3301a864013f2", size = 3875429, upload-time = "2026-08-25T19:45:24.418Z" }, +] + +[[package]] +name = "idna" +version = "3.15" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/82/77/7b3966d0b9d1d31a36ddf1746926a11dface89a83409bf1483f0237aa758/idna-3.15.tar.gz", hash = "sha256:ca962446ea538f7092a95e057da437618e886f4d349216d2b1e294abfdb65fdc", size = 199245, upload-time = "2026-05-12T22:45:57.011Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/d2/23/408243171aa9aaba178d3e2559159c24c1171a641aa83b67bdd3394ead8e/idna-3.15-py3-none-any.whl", hash = "sha256:048adeaf8c2d788c40fee287673ccaa74c24ffd8dcf09ffa555a2fbb59f10ac8", size = 72340, upload-time = "2026-05-12T22:45:55.733Z" }, +] + +[[package]] +name = "iniconfig" +version = "2.3.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/72/34/14ca021ce8e5dfedc35312d08ba8bf51fdd999c576889fc2c24cb97f4f10/iniconfig-2.3.0.tar.gz", hash = "sha256:c76315c77db068650d49c5b56314774a7804df16fee4402c1f19d6d15d8c4730", size = 20503, upload-time = "2025-10-18T21:55:43.219Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/cb/b1/3846dd7f199d53cb17f49cba7e651e9ce294d8497c8c150530ed11865bb8/iniconfig-2.3.0-py3-none-any.whl", hash = "sha256:f631c04d2c48c52b84d0d0549c99ff3859c98df65b3101406327ecc7d53fbf12", size = 7484, upload-time = "2025-10-18T21:55:41.639Z" }, +] + +[[package]] +name = "jinja2" +version = "3.1.6" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "markupsafe" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/df/bf/f7da0350254c0ed7c72f3e33cef02e048281fec7ecec5f032d4aac52226b/jinja2-3.1.6.tar.gz", hash = "sha256:0137fb05990d35f1275a587e9aee6d56da821fc83491a0fb838183be43f66d6d", size = 245115, upload-time = "2025-03-05T20:05:02.478Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/62/a1/3d680cbfd5f4b8f15abc1d571870c5fc3e594bb582bc3b64ea099db13e56/jinja2-3.1.6-py3-none-any.whl", hash = "sha256:85ece4451f492d0c13c5dd7c13a64681a86afae63a5f347908daf103ce6d2f67", size = 134899, upload-time = "2025-03-05T20:05:00.369Z" }, +] + +[[package]] +name = "librt" +version = "0.15.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/36/9b/356320fbae2ac8467e21c5e73e1389c80468e4998c62cc7d3536cc51b614/librt-0.15.0.tar.gz", hash = "sha256:4e66cbe84437497d951b799d3e1551291b6fb3d643820a7014b3655d57a59162", size = 214338, upload-time = "2026-08-07T10:49:42.663Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/ba/39/99c25030e782bdfb7a21be8c05254806a2e4bbb05c8d50c2a2130acbfa05/librt-0.15.0-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:e87bc679f86a99aa3b26e3c78eeb821a247c9a28eae48eaafcc32c3bf4c3bb9e", size = 151021, upload-time = "2026-08-07T10:47:00.057Z" }, + { url = "https://files.pythonhosted.org/packages/14/43/f4b1bd1b2888798a1409808889a25ea1ba49eaabce7d681ed27734c2df9d/librt-0.15.0-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:71599e011ac880e8e45d46047d714871894c7d4ab6f25626f8d4f89da21f368d", size = 155267, upload-time = "2026-08-07T10:47:01.311Z" }, + { url = "https://files.pythonhosted.org/packages/0c/db/3ad9c965c72f1e1d6beeec44ec10a54e17be8ae042fbb4baade16cbadced/librt-0.15.0-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:c802434092b769b1d613ed2e13fac15fbfce1934a74bd10283b03c0fae231cd1", size = 503136, upload-time = "2026-08-07T10:47:02.45Z" }, + { url = "https://files.pythonhosted.org/packages/4b/07/5888a6d76acd62ebce66c61b74d94e9370b9c32929f111e487bb6546f8ed/librt-0.15.0-cp312-cp312-manylinux2014_i686.manylinux_2_17_i686.manylinux_2_28_i686.whl", hash = "sha256:5500eeae393a184d14e1f35645962c27129d20c81afa4069e6ef826ebc2b3aaa", size = 496670, upload-time = "2026-08-07T10:47:03.675Z" }, + { url = "https://files.pythonhosted.org/packages/29/39/ab57cc2f5b276156da02bb7f5a8921bada1cb1993ffec99acf811c602c23/librt-0.15.0-cp312-cp312-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:6ecfc32dfb46fb7b565bcd6abf9412acf978775a998273d22888a6d7953730dd", size = 513688, upload-time = "2026-08-07T10:47:04.981Z" }, + { url = "https://files.pythonhosted.org/packages/a7/b9/bdbb0b648b5c2befb031f4c6f3b1dd857415e8fb492a25a3c764a6681e6c/librt-0.15.0-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:89cc46cfd15022e35084355478c9ac809d90b1152222706ac9a7655ec21df6fa", size = 531904, upload-time = "2026-08-07T10:47:06.211Z" }, + { url = "https://files.pythonhosted.org/packages/93/26/473c2e4b6c104e9e58e27ce95fc8005c8bd4fc36cae4f254371125a92db8/librt-0.15.0-cp312-cp312-manylinux_2_34_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:d5f51401d102c885b9ca509e62c79b1dbff286e1b9b047fde6f763780789356d", size = 524427, upload-time = "2026-08-07T10:47:07.592Z" }, + { url = "https://files.pythonhosted.org/packages/26/60/03b3abb82b41714671b907bf6989b228e31e6a8af52dec82b5b0728dc250/librt-0.15.0-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:cc30523e3f1a23fb7511cc659834a0d01a1042bb9de359bc1c131cc4ec6c9656", size = 543155, upload-time = "2026-08-07T10:47:08.866Z" }, + { url = "https://files.pythonhosted.org/packages/f2/0e/9bb1f0a4affbd0a1888f4f79dc03ed2a299d9a2c26c59ab2a97dcbf11903/librt-0.15.0-cp312-cp312-musllinux_1_2_i686.whl", hash = "sha256:59fe030d8ae4a57e3fb7756bf35a858de74e04066fc8555c53d0af979132af81", size = 546890, upload-time = "2026-08-07T10:47:10.327Z" }, + { url = "https://files.pythonhosted.org/packages/dc/84/6937a280d461f7de6e031ffb02edc2b7c3c90d49d630565ce8ff27cbc5f2/librt-0.15.0-cp312-cp312-musllinux_1_2_ppc64le.whl", hash = "sha256:5a6526a2a956bbb1e4ae3568c82e650fc99119c66bb011ea60715744955a2b4d", size = 555163, upload-time = "2026-08-07T10:47:11.798Z" }, + { url = "https://files.pythonhosted.org/packages/bc/95/2a2853c1ee014bf102116e7f897a04beeaeb2461b45b79af98bdfb95f1ef/librt-0.15.0-cp312-cp312-musllinux_1_2_riscv64.whl", hash = "sha256:85ea21ec6730194d67156b0e0b5430ccb1d61f8b8b907e39b37f9812b74a13f0", size = 535812, upload-time = "2026-08-07T10:47:13.279Z" }, + { url = "https://files.pythonhosted.org/packages/c9/4c/cf9601c1b4c5f09280acd5d83abdb2e68527a2be8257136eb42304218622/librt-0.15.0-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:1e47b8ba865d7ede071a91a7163073bbaeb72541f1ef8a07d512c45c7b5007f2", size = 573688, upload-time = "2026-08-07T10:47:14.727Z" }, + { url = "https://files.pythonhosted.org/packages/47/6d/9ac7cbec46189a7625af4b5acbd25f10d827f4141b2002181848c8418923/librt-0.15.0-cp312-cp312-win32.whl", hash = "sha256:a5207ec414d1c4a2a7231b2086970dc036f94293cdf338190984958a013a42f1", size = 106138, upload-time = "2026-08-07T10:47:15.973Z" }, + { url = "https://files.pythonhosted.org/packages/38/d0/2ae99c83be86ce23f925ac1aeeedc777e97f427c4a8d190c70d0a16e9a87/librt-0.15.0-cp312-cp312-win_amd64.whl", hash = "sha256:73b30cfa976659b3917c8f6153bdb0591c6a9ec6583599fd24a689b690622022", size = 126974, upload-time = "2026-08-07T10:47:17.049Z" }, + { url = "https://files.pythonhosted.org/packages/5d/ef/dd24f9635c730b86b87587967dda7516b1845e8b17684603d31607fed598/librt-0.15.0-cp312-cp312-win_arm64.whl", hash = "sha256:a54cf9e0ef47b96af580849db5471142200568ce1e02cbf416addab551369570", size = 112292, upload-time = "2026-08-07T10:47:18.222Z" }, + { url = "https://files.pythonhosted.org/packages/e7/42/467b53a601b406ccd7b97c1fd54b59cb34f9185ad5ce7e9d5c3c4e8961c8/librt-0.15.0-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:db13ca398005abcbe538deda87b686d9bd08b7001cf40c4c06b444960ae10a26", size = 151029, upload-time = "2026-08-07T10:47:19.312Z" }, + { url = "https://files.pythonhosted.org/packages/3e/e6/36c2299b7a94b84fdd01220d8a777a71be5be0925bb0dbdf71c0a06a34d9/librt-0.15.0-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:aa1f1995789dca3698bc550aaceb09a51bd5df0a057ff84ff15296cd1975b801", size = 155194, upload-time = "2026-08-07T10:47:20.398Z" }, + { url = "https://files.pythonhosted.org/packages/c9/b6/ed5071f9325845e670bd36012757419767fbf56af77ed483077b9e4db541/librt-0.15.0-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:55456ea87d8df21808446d03817be2f65e20391c1c615d9187440dff28cd08dc", size = 502568, upload-time = "2026-08-07T10:47:21.652Z" }, + { url = "https://files.pythonhosted.org/packages/7f/81/6450c67c3615d87704bcbc21323fafc69c799b06a044c447529f725d4b01/librt-0.15.0-cp313-cp313-manylinux2014_i686.manylinux_2_17_i686.manylinux_2_28_i686.whl", hash = "sha256:5a86a5a08c2235316bdb359d5dbb6ce0abfca7fac06363103e2c5af571d92f95", size = 496153, upload-time = "2026-08-07T10:47:22.925Z" }, + { url = "https://files.pythonhosted.org/packages/e1/d6/5f52b722bc75076954b3bfd49be15ea362df4d580c6fb315d0f617100d30/librt-0.15.0-cp313-cp313-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:e56b6a368529bed262da40ce13f8fef590db0479819cca84f16a1f01ac356d0b", size = 513336, upload-time = "2026-08-07T10:47:24.213Z" }, + { url = "https://files.pythonhosted.org/packages/8d/e2/c08fd1d36ce63ea5a12b85c5d37f4550b5f86a692167e41e5a74222607ae/librt-0.15.0-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:234d8d394721fa0d786af15ebf1f3fb7f3ed82fd1cd0cde45c2f247b5d4281d2", size = 531661, upload-time = "2026-08-07T10:47:25.507Z" }, + { url = "https://files.pythonhosted.org/packages/3f/d8/d9482fcbeb177b9eb87bb3899eeb3b42be690313c652f9e146b1d0681fb2/librt-0.15.0-cp313-cp313-manylinux_2_34_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:d8363d7accb0286ac3a0e633f396e93800dafb8150494505daf9515bbda591f3", size = 524487, upload-time = "2026-08-07T10:47:26.79Z" }, + { url = "https://files.pythonhosted.org/packages/10/cc/075171517b41f861753034fbb151b42cfc83bcc853849f24f5e66fd60ccf/librt-0.15.0-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:0f0ee3644d951f31055ad07d77d92520e84505dd7a432cc4cd501dd70ee06785", size = 543201, upload-time = "2026-08-07T10:47:27.999Z" }, + { url = "https://files.pythonhosted.org/packages/b0/03/42c2330f37eeb475b6affeedd06518f60035f323af3a839335e3fc9fef2d/librt-0.15.0-cp313-cp313-musllinux_1_2_i686.whl", hash = "sha256:2cfd1a81a648806e6a7717be4cc4d1bb392fa229752bf8444ba365e381e984d6", size = 546467, upload-time = "2026-08-07T10:47:29.396Z" }, + { url = "https://files.pythonhosted.org/packages/57/1e/1ad4c5638f7e64d8560328bd25c54b409a661bdb6ff254b38ff90744288d/librt-0.15.0-cp313-cp313-musllinux_1_2_ppc64le.whl", hash = "sha256:a6cd22c9da0d866558e46a041f1cc0c2bbb26b61b137b2347fa834c332e1d101", size = 555139, upload-time = "2026-08-07T10:47:30.815Z" }, + { url = "https://files.pythonhosted.org/packages/49/41/39fa7d15db1204cd1cbe6514680fbdc243adf754a0885061308f43afc013/librt-0.15.0-cp313-cp313-musllinux_1_2_riscv64.whl", hash = "sha256:6d5225ef8801e4ea5e482fa9b5dfb891dd9ef6f6d870f1f25d449ca2c70ac218", size = 536050, upload-time = "2026-08-07T10:47:32.222Z" }, + { url = "https://files.pythonhosted.org/packages/1e/88/c6dcf0dd8e26dc0c9a499a2abab8646c86dcaf9ecea9524cb46d3686331a/librt-0.15.0-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:6d28a05796b99f749bf8794f17ba9ba1612d0076b802e9cfc62c554634e9ce3b", size = 573700, upload-time = "2026-08-07T10:47:33.527Z" }, + { url = "https://files.pythonhosted.org/packages/1b/9b/ab54c71a7918a7c34fa5327fb61390a77446a07a146fbfb1165250a61035/librt-0.15.0-cp313-cp313-pyemscripten_2025_0_wasm32.whl", hash = "sha256:2067ff438048cead9d223ca5675bae2a25e520a7c3e6c1498bf9c6892d22caab", size = 82194, upload-time = "2026-08-07T10:47:34.835Z" }, + { url = "https://files.pythonhosted.org/packages/8d/b2/4f9a243bb892395f3becb80789ade13771701091f9f07ab8230247953ba8/librt-0.15.0-cp313-cp313-win32.whl", hash = "sha256:1cd3b721f24c206398b9e26da3c3a9c011e6e89d06f318ba8ebefc30f1003890", size = 106231, upload-time = "2026-08-07T10:47:36.251Z" }, + { url = "https://files.pythonhosted.org/packages/bf/af/64aff4885a40b93132382f2c314647d722574605416504379184ef3045ea/librt-0.15.0-cp313-cp313-win_amd64.whl", hash = "sha256:f395a4a9a03ac062dbe9a9f82e0c720502e590a38feee6a757bc82e9c63afbd8", size = 126996, upload-time = "2026-08-07T10:47:37.453Z" }, + { url = "https://files.pythonhosted.org/packages/27/83/335bccf6c7cb9028cb0b54aead27d9ece3f01f83bc6baa2abace5da655c1/librt-0.15.0-cp313-cp313-win_arm64.whl", hash = "sha256:0a15cb554761247d84a3ec0cbdf4078d70725384f0e4662c0fa3b26266eb60ad", size = 112188, upload-time = "2026-08-07T10:47:38.729Z" }, + { url = "https://files.pythonhosted.org/packages/a8/93/949053fb462eecc4a9a5ee770a81f4b40be7b79538b245545d4aebc6b58b/librt-0.15.0-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:f5de7feedc56337a088eb15cd9fafa9938367362221d8cc62c642b7f94821993", size = 149833, upload-time = "2026-08-07T10:47:39.86Z" }, + { url = "https://files.pythonhosted.org/packages/61/ca/8281aa6cd560a3420e4497729f6b704b53be3eeaaef82d5aeadddaf7441f/librt-0.15.0-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:6c0eb900c0e91f4aebe680845242e614f1864edfd44106380d0752ac29522bf8", size = 154088, upload-time = "2026-08-07T10:47:41.065Z" }, + { url = "https://files.pythonhosted.org/packages/dd/02/1a1662dceaba6a086360891448d5ce9a7d3555976cae59a31a39d744b9c7/librt-0.15.0-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:e8c9a650a188e38bac005048cbe6342e81407782944d01934540ab75e417df21", size = 494215, upload-time = "2026-08-07T10:47:42.388Z" }, + { url = "https://files.pythonhosted.org/packages/69/84/99211619dc656370a3740c33d2b0b6d5a3fb1e73689314f6ed477a397dc4/librt-0.15.0-cp314-cp314-manylinux2014_i686.manylinux_2_17_i686.manylinux_2_28_i686.whl", hash = "sha256:92bfed8deec93df30286b9fe9e3b1dd17329cc076a192b4ee5ec223841d54953", size = 491173, upload-time = "2026-08-07T10:47:43.683Z" }, + { url = "https://files.pythonhosted.org/packages/d4/aa/5448d0b05f4579b635d3899176817ebf561af0e57bacd425b5b1887264c1/librt-0.15.0-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:ec4b19788f835711a2072f9dbe6b03b3bf32ed1f0fb30cf399bdd59d9f0c33fa", size = 505512, upload-time = "2026-08-07T10:47:45.314Z" }, + { url = "https://files.pythonhosted.org/packages/95/82/01940e40b83c43a546c4a3c896cf34ca272a9690899d55914e4827b3dcce/librt-0.15.0-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:d4c7bacb70930f3d0a56f4ecf1be474a1f0d941b01dd73b756f3c256d42cb879", size = 523073, upload-time = "2026-08-07T10:47:46.66Z" }, + { url = "https://files.pythonhosted.org/packages/88/fa/759c0030f3ee371439eb26de34fc745807caf0abb878af7af4b8b7c3dd3d/librt-0.15.0-cp314-cp314-manylinux_2_34_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:3e79f05e4a08b4d880342673312bbc895b56df7765605796f15902eb5367d3ae", size = 515080, upload-time = "2026-08-07T10:47:48.319Z" }, + { url = "https://files.pythonhosted.org/packages/0b/27/894e072228fcb159703c655da69f8cd10dbed489c36e3df7dd032a2483be/librt-0.15.0-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:a417149c0cba4d50b61e992e5a15e69eaf96746609b461cc4ed168aeef6b79dd", size = 534164, upload-time = "2026-08-07T10:47:49.875Z" }, + { url = "https://files.pythonhosted.org/packages/98/a3/0078e91c1f36f8815db17827de15650b9a3fe56c55fbf998c854b34e40d3/librt-0.15.0-cp314-cp314-musllinux_1_2_i686.whl", hash = "sha256:da7a94d6a3411f579d72aa3e3bc5fbca7ed4549f3dbd7e5de3aa567333374285", size = 540616, upload-time = "2026-08-07T10:47:51.408Z" }, + { url = "https://files.pythonhosted.org/packages/86/33/81a29b796dd52a45e9ef7974c7732926e8f10f15b8d2be505665979f896d/librt-0.15.0-cp314-cp314-musllinux_1_2_ppc64le.whl", hash = "sha256:856f743ae607f2c1380eccb566c0038a9fb3eabf0fc2be2704d76d9f73557239", size = 545890, upload-time = "2026-08-07T10:47:52.818Z" }, + { url = "https://files.pythonhosted.org/packages/05/82/8be1baa1350e5d30cfd70ae79d0a6f4dc5862ef47f7bb2808aabc9bb86e5/librt-0.15.0-cp314-cp314-musllinux_1_2_riscv64.whl", hash = "sha256:779a6e7c894737e5983e7790a9c78c4000c30e23c9aada08081bdbea53b0fa60", size = 523287, upload-time = "2026-08-07T10:47:54.165Z" }, + { url = "https://files.pythonhosted.org/packages/c6/4f/d1be6a01a35c20ef734e0e44113f87d4af756a9354a89dcfbe3b4f8af5e1/librt-0.15.0-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:96bb17dbe8bab3c0954fbebfc69ed395599de75b6bbc35e3270a878e15d4dd65", size = 565868, upload-time = "2026-08-07T10:47:55.566Z" }, + { url = "https://files.pythonhosted.org/packages/67/88/649cfa33f5825927b160610f670bdab012a64d627eddb94fa795ea4292fd/librt-0.15.0-cp314-cp314-pyemscripten_2026_0_wasm32.whl", hash = "sha256:7220697efaa6e5348fc3d18ee7f8563d4bfecd9872b37ffb915bfc1d08840622", size = 81619, upload-time = "2026-08-07T10:47:56.886Z" }, + { url = "https://files.pythonhosted.org/packages/22/31/8e88a8d5e48fc8d1a817787fb6811dfff6499acd6c8683dd83934aa6ede0/librt-0.15.0-cp314-cp314-win32.whl", hash = "sha256:f54598964d357b1c5ab77cf5d92f21e598fe0e23cdbe9618480807f81b4eba15", size = 100138, upload-time = "2026-08-07T10:47:58.093Z" }, + { url = "https://files.pythonhosted.org/packages/80/92/20fd6c4b6a1b1a564b076d55cd3d427d8428217d7638dc25a654cc4791d4/librt-0.15.0-cp314-cp314-win_amd64.whl", hash = "sha256:3ff5893a2c23d886aa9ce786de5ac6ddc74aeeaf90743682b74d920e117d2e28", size = 121258, upload-time = "2026-08-07T10:47:59.564Z" }, + { url = "https://files.pythonhosted.org/packages/fc/28/6af430b44d9ebb897b865a3c363b6dcace51357be2347cc0f8f869656a86/librt-0.15.0-cp314-cp314-win_arm64.whl", hash = "sha256:3722a099730704c9a3d70c879fc0f51daec25fe5f1555672d97bc595abeafb95", size = 106467, upload-time = "2026-08-07T10:48:01.097Z" }, + { url = "https://files.pythonhosted.org/packages/7e/aa/b42bb798942ced219f6d63b27e07f91237887a8d0bd0921666db79a13790/librt-0.15.0-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:38c0c7d4b6fc06c3324b3f9162c8391bfc4fd9dde53afe1033ce7edb48d5a714", size = 159523, upload-time = "2026-08-07T10:48:02.442Z" }, + { url = "https://files.pythonhosted.org/packages/75/03/1b53cd4ef904e73b1d828a5f90143bf94a2967d7cfff0b9ccf93e12aa9b4/librt-0.15.0-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:8b2fdd7ead3c995c37940a790690660d0ca006c302db26cc51933f6766866fc3", size = 161638, upload-time = "2026-08-07T10:48:03.725Z" }, + { url = "https://files.pythonhosted.org/packages/ac/c4/9f9c9fba097d49e9e694c2b4dc331df31884645ecbc58a93b4b5fc69d2c5/librt-0.15.0-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:2fde98cf1fc4bac144ce23c2c4c017b924ba714509ea9334977b0b27050c837d", size = 701795, upload-time = "2026-08-07T10:48:05.135Z" }, + { url = "https://files.pythonhosted.org/packages/4c/05/0966840bda0380c8ae167b9043c6230202941cc90ea29c48e096964c765e/librt-0.15.0-cp314-cp314t-manylinux2014_i686.manylinux_2_17_i686.manylinux_2_28_i686.whl", hash = "sha256:e3b461183c5fa7681b48560f91515f53a953122fb30c71e07abc67d7ddf58c38", size = 682147, upload-time = "2026-08-07T10:48:06.555Z" }, + { url = "https://files.pythonhosted.org/packages/18/af/1c47ca573c30ea47d195aec26133af522fea1104afaace028d7b32247ea8/librt-0.15.0-cp314-cp314t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:4bbcc257e3babea20a91715c361b24554ec4e8f51aa578568afc230799fe1a19", size = 696397, upload-time = "2026-08-07T10:48:08.03Z" }, + { url = "https://files.pythonhosted.org/packages/2e/0f/1aed6223d4f9f9d1171a8596ff100ea4c3f7699fea7a4ba657c3e60daa6c/librt-0.15.0-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:b845b8d48088fad0cadc84be4b8fda63203be7e9237b71015b3925443c1f35ab", size = 722542, upload-time = "2026-08-07T10:48:09.569Z" }, + { url = "https://files.pythonhosted.org/packages/c6/22/9e3a929aea456c97d69e6ef3884efea56d4807f97399471cc946baebd8af/librt-0.15.0-cp314-cp314t-manylinux_2_34_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:b30e600e8f337b9bd7f39b86d9fdfedc73cc46e3d0f745931a23a234220bb7e2", size = 729709, upload-time = "2026-08-07T10:48:11.129Z" }, + { url = "https://files.pythonhosted.org/packages/e9/1b/c327ef6018e3a9ca0b8e7c5eddeeb331ba8f9b76c24e126d37d0f6d62faf/librt-0.15.0-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:64b0c8c35aa4c4ed79896359f3e0b285cbe4e610042106500da4811c322cc108", size = 752891, upload-time = "2026-08-07T10:48:12.558Z" }, + { url = "https://files.pythonhosted.org/packages/d7/d1/d5f1ea02c56930087009e39db9b70660a663e76c730b27b925d786718457/librt-0.15.0-cp314-cp314t-musllinux_1_2_i686.whl", hash = "sha256:0da0d94cb802f32a0524653e7201f2cef72d5f700a5407678f5290483d4fcd08", size = 745301, upload-time = "2026-08-07T10:48:14.55Z" }, + { url = "https://files.pythonhosted.org/packages/d9/3c/5f7c585d15ebb2250c73e7c0ee4e9e47be72c65d520c07ddbcdc62037674/librt-0.15.0-cp314-cp314t-musllinux_1_2_ppc64le.whl", hash = "sha256:4a6369168d371207339b1e50d4532b06a7121586141f82599505a3f315751d47", size = 747921, upload-time = "2026-08-07T10:48:16.453Z" }, + { url = "https://files.pythonhosted.org/packages/7f/52/1443a446486eba966bcbca1696b472e4f210320ec42f490a47f48fbf0fdc/librt-0.15.0-cp314-cp314t-musllinux_1_2_riscv64.whl", hash = "sha256:c434e072557ade9cbc642d052c89d031efe47d5c9614523619d0d74a02378e81", size = 727561, upload-time = "2026-08-07T10:48:18.089Z" }, + { url = "https://files.pythonhosted.org/packages/79/91/2270a9380f11725cf83ce1925a5e32dd1dde2be9bba597f25c10a38644e7/librt-0.15.0-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:c7eec6a42018bc1d45763b1c162d3d2bf7c3b9a1b0ed30d3e91dcba390efefcc", size = 774417, upload-time = "2026-08-07T10:48:19.611Z" }, + { url = "https://files.pythonhosted.org/packages/9e/3b/f4b1548d4f5b99186737fe27aec238e9823e8d5d23bf4df007c030689dc5/librt-0.15.0-cp314-cp314t-win32.whl", hash = "sha256:6912fa5e635d74529ac7cdb1bdf6ca3af4453da8d1edbe0110ee1cb4ad407ebf", size = 104381, upload-time = "2026-08-07T10:48:21.048Z" }, + { url = "https://files.pythonhosted.org/packages/80/b6/134afad262def1de04c0843c376d02135f1168af43f22e09a52bd8394727/librt-0.15.0-cp314-cp314t-win_amd64.whl", hash = "sha256:8e11699ed745931c395acd3621b07062e0f840efa6935aad87a64ed0995f0915", size = 127034, upload-time = "2026-08-07T10:48:22.561Z" }, + { url = "https://files.pythonhosted.org/packages/99/5f/1b6846b20572bd699c9e9ec321a5f781845bee477df2aa2a43b28bc40119/librt-0.15.0-cp314-cp314t-win_arm64.whl", hash = "sha256:5d2a91724463bfed4f573cd7a9fdc856d2e230d0c0e5a61416a93481dccd8605", size = 110827, upload-time = "2026-08-07T10:48:23.804Z" }, + { url = "https://files.pythonhosted.org/packages/c6/44/4de9f4ddadb009a55c7758eb5736d62534a7daaf27bd71bc50e64b606b06/librt-0.15.0-cp315-cp315-macosx_10_15_x86_64.whl", hash = "sha256:8443e38dcfcfdbcf5add5118c623efd788d65ac2e25756d6251a54a06a4d0aca", size = 149843, upload-time = "2026-08-07T10:48:25.148Z" }, + { url = "https://files.pythonhosted.org/packages/1f/eb/5d9ab71e30119c44094e0275f38b47dd327aea0f843a080396677029d508/librt-0.15.0-cp315-cp315-macosx_11_0_arm64.whl", hash = "sha256:6d15a29033c57490cfe2069097c6fc4049e4e65ffbb749be7dc453b7c4c68965", size = 154510, upload-time = "2026-08-07T10:48:26.485Z" }, + { url = "https://files.pythonhosted.org/packages/d0/9c/8505d1b8f5e8c19587bd03f7429993b3e9ce5c06819d856bfb11d919374c/librt-0.15.0-cp315-cp315-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:d2c05c729b589e734c09578bf5964be48a911765484840d017bbc84f49d4c4ad", size = 497543, upload-time = "2026-08-07T10:48:28.045Z" }, + { url = "https://files.pythonhosted.org/packages/1d/9a/3a8390775cb095765aded027ac9c63e7c8ea74e731498607544c6505de0e/librt-0.15.0-cp315-cp315-manylinux2014_i686.manylinux_2_17_i686.manylinux_2_28_i686.whl", hash = "sha256:fa60887537e1d0cd2d9982269d33a709bf54b195cd2b9364fc0a758022af5bd9", size = 480452, upload-time = "2026-08-07T10:48:29.531Z" }, + { url = "https://files.pythonhosted.org/packages/e7/40/258a4a7117ee915d66de5cd9b8ade65a440993161107ce3a686f1859955c/librt-0.15.0-cp315-cp315-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:d8bc24219b24c0af375718942ab75e3544b2763085f40f965be4326734ae8328", size = 507768, upload-time = "2026-08-07T10:48:31.007Z" }, + { url = "https://files.pythonhosted.org/packages/6b/c6/2f4dd296c97a0b85b98894519b279408ec9dd602d4f692b1ea0e25dee670/librt-0.15.0-cp315-cp315-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:86a21a7bd3fe3a419512ef424cc1c020f6771d0b29cfddff36d1635a855e63f0", size = 525122, upload-time = "2026-08-07T10:48:32.7Z" }, + { url = "https://files.pythonhosted.org/packages/49/dd/29eab42be13b2bf0ea8cb227135a45d44693e30a7e8b92871981ff56b82b/librt-0.15.0-cp315-cp315-manylinux_2_34_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:dbab647e88d90b3167b91efe7091e248653688ed4337e4f90907a722c7361bb9", size = 520371, upload-time = "2026-08-07T10:48:34.294Z" }, + { url = "https://files.pythonhosted.org/packages/91/ed/4bad71adeca8fe208b775c2a35417fa5a2584c8f4791daaf89a89450fea1/librt-0.15.0-cp315-cp315-musllinux_1_2_aarch64.whl", hash = "sha256:d8edcf6f550e918dca779c069b9e156385c60b406f99fc7641f32c52f7193659", size = 537258, upload-time = "2026-08-07T10:48:35.88Z" }, + { url = "https://files.pythonhosted.org/packages/4c/63/59dba6143fdcc7240c54458b629f3250000a61b8945890fc9efd451b19c5/librt-0.15.0-cp315-cp315-musllinux_1_2_i686.whl", hash = "sha256:8b62076030baa2d8b1501a46bf0e19c27a489aa90671c55665bff7887f7660b0", size = 527432, upload-time = "2026-08-07T10:48:37.466Z" }, + { url = "https://files.pythonhosted.org/packages/ec/21/21a24c6a2327d8362580efebe77286bf47b0f4062ec5ea41766e609d3c7d/librt-0.15.0-cp315-cp315-musllinux_1_2_ppc64le.whl", hash = "sha256:d00d20d1818e82a07a0ee0aa89a98b17ed7916b92441090b683719cb20a59b6d", size = 548108, upload-time = "2026-08-07T10:48:39.384Z" }, + { url = "https://files.pythonhosted.org/packages/5a/6d/fc68c89a7971418b41f9a873623ff935cb864097544c6a2f8ce491c8ef5d/librt-0.15.0-cp315-cp315-musllinux_1_2_riscv64.whl", hash = "sha256:4e6ee93fc3cf848dcbf0cce2eca73d8e7dcd0cc2b6df3a529d57750b30a4c55c", size = 529681, upload-time = "2026-08-07T10:48:41.392Z" }, + { url = "https://files.pythonhosted.org/packages/65/7e/c2d98766124400d722063a630b0fde38a9fc768705d37eecca15c47dc192/librt-0.15.0-cp315-cp315-musllinux_1_2_x86_64.whl", hash = "sha256:32896a0af72508ea979e0acb4e4c04cbeeae04938167950d535c83c45597167d", size = 567736, upload-time = "2026-08-07T10:48:43.124Z" }, + { url = "https://files.pythonhosted.org/packages/55/6c/f8c34a95e3a515c6e1c192b89511e7253c89a7760c6b500d57ffdb8d2dc8/librt-0.15.0-cp315-cp315-pyemscripten_2026_5_wasm32.whl", hash = "sha256:ec3ba415afaf951f6951b1dd16d3c8e4f540065fc382d7e70b823a79567ca374", size = 81673, upload-time = "2026-08-07T10:48:44.645Z" }, + { url = "https://files.pythonhosted.org/packages/c9/9e/e23fa8e78679ec45728188650b39e8ff476c83b691c96f749217df3b1b7c/librt-0.15.0-cp315-cp315-win32.whl", hash = "sha256:d2813ba2503764f0450680c533d13df7cff9b49df1411062eded5f67db4195b9", size = 100081, upload-time = "2026-08-07T10:48:46.171Z" }, + { url = "https://files.pythonhosted.org/packages/e1/dc/3eb4c5e297343f0620a55532cd7c8d764d3001fa2159212dadf480464827/librt-0.15.0-cp315-cp315-win_amd64.whl", hash = "sha256:b87d67e33afaf265262f2a66db578284b88ee2e6fcd224579cb5c15518677ad8", size = 121228, upload-time = "2026-08-07T10:48:47.631Z" }, + { url = "https://files.pythonhosted.org/packages/97/70/43abce19f04e49762f8ec834c8fafee13cc40fd6b94a72a24e534febfcd0/librt-0.15.0-cp315-cp315-win_arm64.whl", hash = "sha256:713bd7df21170b982e729e46870f31d6b437bd1a9b4648cffb529bd3c2ec5c4b", size = 106487, upload-time = "2026-08-07T10:48:49.095Z" }, + { url = "https://files.pythonhosted.org/packages/de/15/83f2deddb9368b8951ec8c9477269b5b9b8bd9bbf15e57402d0f38817dca/librt-0.15.0-cp315-cp315t-macosx_10_15_x86_64.whl", hash = "sha256:3de789c82752730f94782a5ee518baf9c05edf85733aeaf73bb6e518755cdf54", size = 159448, upload-time = "2026-08-07T10:48:50.649Z" }, + { url = "https://files.pythonhosted.org/packages/06/bf/043097353f9b3c73b583d07f6b8e552795463f4bfc8caf85e42eee50c26a/librt-0.15.0-cp315-cp315t-macosx_11_0_arm64.whl", hash = "sha256:e0b5deec9a8664eb722c797241970fd4aa1894d25fda36a1ddac0f7407606bd6", size = 161686, upload-time = "2026-08-07T10:48:52.174Z" }, + { url = "https://files.pythonhosted.org/packages/f4/2a/8ae77f9719d42ce71cd708560a3557b38ac3c17a0383e57f87084de45bbe/librt-0.15.0-cp315-cp315t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:5563302a8359bc2295bb7084d1a8ed1519df96afb30eb2aa4e0bff7b54228988", size = 710668, upload-time = "2026-08-07T10:48:53.782Z" }, + { url = "https://files.pythonhosted.org/packages/61/34/c0436ea134deb9a0d6da80a396a2739a81cb31e0418f7227239e23140898/librt-0.15.0-cp315-cp315t-manylinux2014_i686.manylinux_2_17_i686.manylinux_2_28_i686.whl", hash = "sha256:22d6263b9d39d7bbb286fa791945646e3218f1be2d693e36fb630f1d0e59cd13", size = 679396, upload-time = "2026-08-07T10:48:55.645Z" }, + { url = "https://files.pythonhosted.org/packages/4a/9f/001e0d99aa9250d5cd5715a9081291a20656083459f9019cda15255329e1/librt-0.15.0-cp315-cp315t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:39ffd14646190c454f0d86e0d256b33f00a87a26ab410e619773b841d0e41416", size = 704313, upload-time = "2026-08-07T10:48:57.46Z" }, + { url = "https://files.pythonhosted.org/packages/2d/53/b34fa9d0ff00f136f4d58ebb4c411ff634baed1eb412bb602a2bc8dcafcb/librt-0.15.0-cp315-cp315t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:c47318cd3a61401452de11282242937e3e057c4fd3dbaf601e269d0928a06c0a", size = 729847, upload-time = "2026-08-07T10:48:59.231Z" }, + { url = "https://files.pythonhosted.org/packages/86/ac/fa4d7a424665040e95baf480a6d523446057684b6758624c85338e8a23b2/librt-0.15.0-cp315-cp315t-manylinux_2_34_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:a56a1d4f859a82ca5b99fc4b82c9b027b15e3c455c5cd99e7d0719f27bb20b6c", size = 742736, upload-time = "2026-08-07T10:49:01.151Z" }, + { url = "https://files.pythonhosted.org/packages/8a/f1/e17a9bb5de6fb8c3186ed1a7d68d21618b027ac2d3633e03d3b6109c67ae/librt-0.15.0-cp315-cp315t-musllinux_1_2_aarch64.whl", hash = "sha256:077471b3182db4e17c36ae91555f36a4d2c00080b267f749bcad34a478a9a302", size = 763454, upload-time = "2026-08-07T10:49:03.039Z" }, + { url = "https://files.pythonhosted.org/packages/1d/ec/ecd02cd30935b931b9cdbfed6ab5a099c51b280b4e7baa274da80978ed27/librt-0.15.0-cp315-cp315t-musllinux_1_2_i686.whl", hash = "sha256:411ca4d1b905b860ceba7570dd6717a71dedaddcc4b0f77ece710aa41ee11f8d", size = 743296, upload-time = "2026-08-07T10:49:04.941Z" }, + { url = "https://files.pythonhosted.org/packages/e6/b5/b3c2b8353ce820a4854f78d19321344242f89fa71c975b71132ba9bf242a/librt-0.15.0-cp315-cp315t-musllinux_1_2_ppc64le.whl", hash = "sha256:1256589e0b0adb31751d685a68bce29d73407ddf4ef05d4188f49d5dcf9566d9", size = 756217, upload-time = "2026-08-07T10:49:06.825Z" }, + { url = "https://files.pythonhosted.org/packages/3c/52/6cc22542ba59146b05cca2a656f9ff8bb67e38e63d12c3b0cc183d837bf1/librt-0.15.0-cp315-cp315t-musllinux_1_2_riscv64.whl", hash = "sha256:f42b74a53e5f26a0ba0007411a7455b66c67ce4022a39cc1f56fc4efd65bcbab", size = 741934, upload-time = "2026-08-07T10:49:08.839Z" }, + { url = "https://files.pythonhosted.org/packages/40/32/a04b72b1aa86e3be23b2ecff8c1aad2dcc955bd3956d6d26e7e34267e57a/librt-0.15.0-cp315-cp315t-musllinux_1_2_x86_64.whl", hash = "sha256:291bf73caf78b9e88d6fae9bfd693207ff7d832e2fdbe2cf8e746bc13f5f892b", size = 783763, upload-time = "2026-08-07T10:49:10.661Z" }, + { url = "https://files.pythonhosted.org/packages/6c/f0/89eb11dffbe9279ff37144dec786927314502ae0b114f1449dc78c458aab/librt-0.15.0-cp315-cp315t-win32.whl", hash = "sha256:c16d15ee371643ab48dc8248a3e680ebbeca573a13af2c3dd0c985b142d77162", size = 104313, upload-time = "2026-08-07T10:49:12.305Z" }, + { url = "https://files.pythonhosted.org/packages/6d/4a/1f1978c200f563beda63c36adff2d65bbecb81e365e8e69e572f5f70fbc6/librt-0.15.0-cp315-cp315t-win_amd64.whl", hash = "sha256:dbd605739f228912dc49027cb764456b9757750bdc2b6b7773164db7096c6fd1", size = 126889, upload-time = "2026-08-07T10:49:13.881Z" }, + { url = "https://files.pythonhosted.org/packages/38/a6/800800bfed7b1fb10fc3f3d557785c3854e80d3f7a9800d784b176a1fc2d/librt-0.15.0-cp315-cp315t-win_arm64.whl", hash = "sha256:84d244b00604d17df3fc7736c327892d6bba66181254aa4087be807b6c342bdc", size = 110700, upload-time = "2026-08-07T10:49:15.499Z" }, +] + +[[package]] +name = "macholib" +version = "1.16.4" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "altgraph" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/10/2f/97589876ea967487978071c9042518d28b958d87b17dceb7cdc1d881f963/macholib-1.16.4.tar.gz", hash = "sha256:f408c93ab2e995cd2c46e34fe328b130404be143469e41bc366c807448979362", size = 59427, upload-time = "2025-11-22T08:28:38.373Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/c7/d1/a9f36f8ecdf0fb7c9b1e78c8d7af12b8c8754e74851ac7b94a8305540fc7/macholib-1.16.4-py2.py3-none-any.whl", hash = "sha256:da1a3fa8266e30f0ce7e97c6a54eefaae8edd1e5f86f3eb8b95457cae90265ea", size = 38117, upload-time = "2025-11-22T08:28:36.939Z" }, +] + +[[package]] +name = "markdown-it-py" +version = "4.2.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "mdurl" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/06/ff/7841249c247aa650a76b9ee4bbaeae59370dc8bfd2f6c01f3630c35eb134/markdown_it_py-4.2.0.tar.gz", hash = "sha256:04a21681d6fbb623de53f6f364d352309d4094dd4194040a10fd51833e418d49", size = 82454, upload-time = "2026-05-07T12:08:28.36Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/b3/81/4da04ced5a082363ecfa159c010d200ecbd959ae410c10c0264a38cac0f5/markdown_it_py-4.2.0-py3-none-any.whl", hash = "sha256:9f7ebbcd14fe59494226453aed97c1070d83f8d24b6fc3a3bcf9a38092641c4a", size = 91687, upload-time = "2026-05-07T12:08:27.182Z" }, +] + +[[package]] +name = "markupsafe" +version = "3.0.3" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/7e/99/7690b6d4034fffd95959cbe0c02de8deb3098cc577c67bb6a24fe5d7caa7/markupsafe-3.0.3.tar.gz", hash = "sha256:722695808f4b6457b320fdc131280796bdceb04ab50fe1795cd540799ebe1698", size = 80313, upload-time = "2025-09-27T18:37:40.426Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/5a/72/147da192e38635ada20e0a2e1a51cf8823d2119ce8883f7053879c2199b5/markupsafe-3.0.3-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:d53197da72cc091b024dd97249dfc7794d6a56530370992a5e1a08983ad9230e", size = 11615, upload-time = "2025-09-27T18:36:30.854Z" }, + { url = "https://files.pythonhosted.org/packages/9a/81/7e4e08678a1f98521201c3079f77db69fb552acd56067661f8c2f534a718/markupsafe-3.0.3-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:1872df69a4de6aead3491198eaf13810b565bdbeec3ae2dc8780f14458ec73ce", size = 12020, upload-time = "2025-09-27T18:36:31.971Z" }, + { url = "https://files.pythonhosted.org/packages/1e/2c/799f4742efc39633a1b54a92eec4082e4f815314869865d876824c257c1e/markupsafe-3.0.3-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:3a7e8ae81ae39e62a41ec302f972ba6ae23a5c5396c8e60113e9066ef893da0d", size = 24332, upload-time = "2025-09-27T18:36:32.813Z" }, + { url = "https://files.pythonhosted.org/packages/3c/2e/8d0c2ab90a8c1d9a24f0399058ab8519a3279d1bd4289511d74e909f060e/markupsafe-3.0.3-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:d6dd0be5b5b189d31db7cda48b91d7e0a9795f31430b7f271219ab30f1d3ac9d", size = 22947, upload-time = "2025-09-27T18:36:33.86Z" }, + { url = "https://files.pythonhosted.org/packages/2c/54/887f3092a85238093a0b2154bd629c89444f395618842e8b0c41783898ea/markupsafe-3.0.3-cp312-cp312-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:94c6f0bb423f739146aec64595853541634bde58b2135f27f61c1ffd1cd4d16a", size = 21962, upload-time = "2025-09-27T18:36:35.099Z" }, + { url = "https://files.pythonhosted.org/packages/c9/2f/336b8c7b6f4a4d95e91119dc8521402461b74a485558d8f238a68312f11c/markupsafe-3.0.3-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:be8813b57049a7dc738189df53d69395eba14fb99345e0a5994914a3864c8a4b", size = 23760, upload-time = "2025-09-27T18:36:36.001Z" }, + { url = "https://files.pythonhosted.org/packages/32/43/67935f2b7e4982ffb50a4d169b724d74b62a3964bc1a9a527f5ac4f1ee2b/markupsafe-3.0.3-cp312-cp312-musllinux_1_2_riscv64.whl", hash = "sha256:83891d0e9fb81a825d9a6d61e3f07550ca70a076484292a70fde82c4b807286f", size = 21529, upload-time = "2025-09-27T18:36:36.906Z" }, + { url = "https://files.pythonhosted.org/packages/89/e0/4486f11e51bbba8b0c041098859e869e304d1c261e59244baa3d295d47b7/markupsafe-3.0.3-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:77f0643abe7495da77fb436f50f8dab76dbc6e5fd25d39589a0f1fe6548bfa2b", size = 23015, upload-time = "2025-09-27T18:36:37.868Z" }, + { url = "https://files.pythonhosted.org/packages/2f/e1/78ee7a023dac597a5825441ebd17170785a9dab23de95d2c7508ade94e0e/markupsafe-3.0.3-cp312-cp312-win32.whl", hash = "sha256:d88b440e37a16e651bda4c7c2b930eb586fd15ca7406cb39e211fcff3bf3017d", size = 14540, upload-time = "2025-09-27T18:36:38.761Z" }, + { url = "https://files.pythonhosted.org/packages/aa/5b/bec5aa9bbbb2c946ca2733ef9c4ca91c91b6a24580193e891b5f7dbe8e1e/markupsafe-3.0.3-cp312-cp312-win_amd64.whl", hash = "sha256:26a5784ded40c9e318cfc2bdb30fe164bdb8665ded9cd64d500a34fb42067b1c", size = 15105, upload-time = "2025-09-27T18:36:39.701Z" }, + { url = "https://files.pythonhosted.org/packages/e5/f1/216fc1bbfd74011693a4fd837e7026152e89c4bcf3e77b6692fba9923123/markupsafe-3.0.3-cp312-cp312-win_arm64.whl", hash = "sha256:35add3b638a5d900e807944a078b51922212fb3dedb01633a8defc4b01a3c85f", size = 13906, upload-time = "2025-09-27T18:36:40.689Z" }, + { url = "https://files.pythonhosted.org/packages/38/2f/907b9c7bbba283e68f20259574b13d005c121a0fa4c175f9bed27c4597ff/markupsafe-3.0.3-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:e1cf1972137e83c5d4c136c43ced9ac51d0e124706ee1c8aa8532c1287fa8795", size = 11622, upload-time = "2025-09-27T18:36:41.777Z" }, + { url = "https://files.pythonhosted.org/packages/9c/d9/5f7756922cdd676869eca1c4e3c0cd0df60ed30199ffd775e319089cb3ed/markupsafe-3.0.3-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:116bb52f642a37c115f517494ea5feb03889e04df47eeff5b130b1808ce7c219", size = 12029, upload-time = "2025-09-27T18:36:43.257Z" }, + { url = "https://files.pythonhosted.org/packages/00/07/575a68c754943058c78f30db02ee03a64b3c638586fba6a6dd56830b30a3/markupsafe-3.0.3-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:133a43e73a802c5562be9bbcd03d090aa5a1fe899db609c29e8c8d815c5f6de6", size = 24374, upload-time = "2025-09-27T18:36:44.508Z" }, + { url = "https://files.pythonhosted.org/packages/a9/21/9b05698b46f218fc0e118e1f8168395c65c8a2c750ae2bab54fc4bd4e0e8/markupsafe-3.0.3-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:ccfcd093f13f0f0b7fdd0f198b90053bf7b2f02a3927a30e63f3ccc9df56b676", size = 22980, upload-time = "2025-09-27T18:36:45.385Z" }, + { url = "https://files.pythonhosted.org/packages/7f/71/544260864f893f18b6827315b988c146b559391e6e7e8f7252839b1b846a/markupsafe-3.0.3-cp313-cp313-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:509fa21c6deb7a7a273d629cf5ec029bc209d1a51178615ddf718f5918992ab9", size = 21990, upload-time = "2025-09-27T18:36:46.916Z" }, + { url = "https://files.pythonhosted.org/packages/c2/28/b50fc2f74d1ad761af2f5dcce7492648b983d00a65b8c0e0cb457c82ebbe/markupsafe-3.0.3-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:a4afe79fb3de0b7097d81da19090f4df4f8d3a2b3adaa8764138aac2e44f3af1", size = 23784, upload-time = "2025-09-27T18:36:47.884Z" }, + { url = "https://files.pythonhosted.org/packages/ed/76/104b2aa106a208da8b17a2fb72e033a5a9d7073c68f7e508b94916ed47a9/markupsafe-3.0.3-cp313-cp313-musllinux_1_2_riscv64.whl", hash = "sha256:795e7751525cae078558e679d646ae45574b47ed6e7771863fcc079a6171a0fc", size = 21588, upload-time = "2025-09-27T18:36:48.82Z" }, + { url = "https://files.pythonhosted.org/packages/b5/99/16a5eb2d140087ebd97180d95249b00a03aa87e29cc224056274f2e45fd6/markupsafe-3.0.3-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:8485f406a96febb5140bfeca44a73e3ce5116b2501ac54fe953e488fb1d03b12", size = 23041, upload-time = "2025-09-27T18:36:49.797Z" }, + { url = "https://files.pythonhosted.org/packages/19/bc/e7140ed90c5d61d77cea142eed9f9c303f4c4806f60a1044c13e3f1471d0/markupsafe-3.0.3-cp313-cp313-win32.whl", hash = "sha256:bdd37121970bfd8be76c5fb069c7751683bdf373db1ed6c010162b2a130248ed", size = 14543, upload-time = "2025-09-27T18:36:51.584Z" }, + { url = "https://files.pythonhosted.org/packages/05/73/c4abe620b841b6b791f2edc248f556900667a5a1cf023a6646967ae98335/markupsafe-3.0.3-cp313-cp313-win_amd64.whl", hash = "sha256:9a1abfdc021a164803f4d485104931fb8f8c1efd55bc6b748d2f5774e78b62c5", size = 15113, upload-time = "2025-09-27T18:36:52.537Z" }, + { url = "https://files.pythonhosted.org/packages/f0/3a/fa34a0f7cfef23cf9500d68cb7c32dd64ffd58a12b09225fb03dd37d5b80/markupsafe-3.0.3-cp313-cp313-win_arm64.whl", hash = "sha256:7e68f88e5b8799aa49c85cd116c932a1ac15caaa3f5db09087854d218359e485", size = 13911, upload-time = "2025-09-27T18:36:53.513Z" }, + { url = "https://files.pythonhosted.org/packages/e4/d7/e05cd7efe43a88a17a37b3ae96e79a19e846f3f456fe79c57ca61356ef01/markupsafe-3.0.3-cp313-cp313t-macosx_10_13_x86_64.whl", hash = "sha256:218551f6df4868a8d527e3062d0fb968682fe92054e89978594c28e642c43a73", size = 11658, upload-time = "2025-09-27T18:36:54.819Z" }, + { url = "https://files.pythonhosted.org/packages/99/9e/e412117548182ce2148bdeacdda3bb494260c0b0184360fe0d56389b523b/markupsafe-3.0.3-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:3524b778fe5cfb3452a09d31e7b5adefeea8c5be1d43c4f810ba09f2ceb29d37", size = 12066, upload-time = "2025-09-27T18:36:55.714Z" }, + { url = "https://files.pythonhosted.org/packages/bc/e6/fa0ffcda717ef64a5108eaa7b4f5ed28d56122c9a6d70ab8b72f9f715c80/markupsafe-3.0.3-cp313-cp313t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:4e885a3d1efa2eadc93c894a21770e4bc67899e3543680313b09f139e149ab19", size = 25639, upload-time = "2025-09-27T18:36:56.908Z" }, + { url = "https://files.pythonhosted.org/packages/96/ec/2102e881fe9d25fc16cb4b25d5f5cde50970967ffa5dddafdb771237062d/markupsafe-3.0.3-cp313-cp313t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:8709b08f4a89aa7586de0aadc8da56180242ee0ada3999749b183aa23df95025", size = 23569, upload-time = "2025-09-27T18:36:57.913Z" }, + { url = "https://files.pythonhosted.org/packages/4b/30/6f2fce1f1f205fc9323255b216ca8a235b15860c34b6798f810f05828e32/markupsafe-3.0.3-cp313-cp313t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:b8512a91625c9b3da6f127803b166b629725e68af71f8184ae7e7d54686a56d6", size = 23284, upload-time = "2025-09-27T18:36:58.833Z" }, + { url = "https://files.pythonhosted.org/packages/58/47/4a0ccea4ab9f5dcb6f79c0236d954acb382202721e704223a8aafa38b5c8/markupsafe-3.0.3-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:9b79b7a16f7fedff2495d684f2b59b0457c3b493778c9eed31111be64d58279f", size = 24801, upload-time = "2025-09-27T18:36:59.739Z" }, + { url = "https://files.pythonhosted.org/packages/6a/70/3780e9b72180b6fecb83a4814d84c3bf4b4ae4bf0b19c27196104149734c/markupsafe-3.0.3-cp313-cp313t-musllinux_1_2_riscv64.whl", hash = "sha256:12c63dfb4a98206f045aa9563db46507995f7ef6d83b2f68eda65c307c6829eb", size = 22769, upload-time = "2025-09-27T18:37:00.719Z" }, + { url = "https://files.pythonhosted.org/packages/98/c5/c03c7f4125180fc215220c035beac6b9cb684bc7a067c84fc69414d315f5/markupsafe-3.0.3-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:8f71bc33915be5186016f675cd83a1e08523649b0e33efdb898db577ef5bb009", size = 23642, upload-time = "2025-09-27T18:37:01.673Z" }, + { url = "https://files.pythonhosted.org/packages/80/d6/2d1b89f6ca4bff1036499b1e29a1d02d282259f3681540e16563f27ebc23/markupsafe-3.0.3-cp313-cp313t-win32.whl", hash = "sha256:69c0b73548bc525c8cb9a251cddf1931d1db4d2258e9599c28c07ef3580ef354", size = 14612, upload-time = "2025-09-27T18:37:02.639Z" }, + { url = "https://files.pythonhosted.org/packages/2b/98/e48a4bfba0a0ffcf9925fe2d69240bfaa19c6f7507b8cd09c70684a53c1e/markupsafe-3.0.3-cp313-cp313t-win_amd64.whl", hash = "sha256:1b4b79e8ebf6b55351f0d91fe80f893b4743f104bff22e90697db1590e47a218", size = 15200, upload-time = "2025-09-27T18:37:03.582Z" }, + { url = "https://files.pythonhosted.org/packages/0e/72/e3cc540f351f316e9ed0f092757459afbc595824ca724cbc5a5d4263713f/markupsafe-3.0.3-cp313-cp313t-win_arm64.whl", hash = "sha256:ad2cf8aa28b8c020ab2fc8287b0f823d0a7d8630784c31e9ee5edea20f406287", size = 13973, upload-time = "2025-09-27T18:37:04.929Z" }, + { url = "https://files.pythonhosted.org/packages/33/8a/8e42d4838cd89b7dde187011e97fe6c3af66d8c044997d2183fbd6d31352/markupsafe-3.0.3-cp314-cp314-macosx_10_13_x86_64.whl", hash = "sha256:eaa9599de571d72e2daf60164784109f19978b327a3910d3e9de8c97b5b70cfe", size = 11619, upload-time = "2025-09-27T18:37:06.342Z" }, + { url = "https://files.pythonhosted.org/packages/b5/64/7660f8a4a8e53c924d0fa05dc3a55c9cee10bbd82b11c5afb27d44b096ce/markupsafe-3.0.3-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:c47a551199eb8eb2121d4f0f15ae0f923d31350ab9280078d1e5f12b249e0026", size = 12029, upload-time = "2025-09-27T18:37:07.213Z" }, + { url = "https://files.pythonhosted.org/packages/da/ef/e648bfd021127bef5fa12e1720ffed0c6cbb8310c8d9bea7266337ff06de/markupsafe-3.0.3-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:f34c41761022dd093b4b6896d4810782ffbabe30f2d443ff5f083e0cbbb8c737", size = 24408, upload-time = "2025-09-27T18:37:09.572Z" }, + { url = "https://files.pythonhosted.org/packages/41/3c/a36c2450754618e62008bf7435ccb0f88053e07592e6028a34776213d877/markupsafe-3.0.3-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:457a69a9577064c05a97c41f4e65148652db078a3a509039e64d3467b9e7ef97", size = 23005, upload-time = "2025-09-27T18:37:10.58Z" }, + { url = "https://files.pythonhosted.org/packages/bc/20/b7fdf89a8456b099837cd1dc21974632a02a999ec9bf7ca3e490aacd98e7/markupsafe-3.0.3-cp314-cp314-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:e8afc3f2ccfa24215f8cb28dcf43f0113ac3c37c2f0f0806d8c70e4228c5cf4d", size = 22048, upload-time = "2025-09-27T18:37:11.547Z" }, + { url = "https://files.pythonhosted.org/packages/9a/a7/591f592afdc734f47db08a75793a55d7fbcc6902a723ae4cfbab61010cc5/markupsafe-3.0.3-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:ec15a59cf5af7be74194f7ab02d0f59a62bdcf1a537677ce67a2537c9b87fcda", size = 23821, upload-time = "2025-09-27T18:37:12.48Z" }, + { url = "https://files.pythonhosted.org/packages/7d/33/45b24e4f44195b26521bc6f1a82197118f74df348556594bd2262bda1038/markupsafe-3.0.3-cp314-cp314-musllinux_1_2_riscv64.whl", hash = "sha256:0eb9ff8191e8498cca014656ae6b8d61f39da5f95b488805da4bb029cccbfbaf", size = 21606, upload-time = "2025-09-27T18:37:13.485Z" }, + { url = "https://files.pythonhosted.org/packages/ff/0e/53dfaca23a69fbfbbf17a4b64072090e70717344c52eaaaa9c5ddff1e5f0/markupsafe-3.0.3-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:2713baf880df847f2bece4230d4d094280f4e67b1e813eec43b4c0e144a34ffe", size = 23043, upload-time = "2025-09-27T18:37:14.408Z" }, + { url = "https://files.pythonhosted.org/packages/46/11/f333a06fc16236d5238bfe74daccbca41459dcd8d1fa952e8fbd5dccfb70/markupsafe-3.0.3-cp314-cp314-win32.whl", hash = "sha256:729586769a26dbceff69f7a7dbbf59ab6572b99d94576a5592625d5b411576b9", size = 14747, upload-time = "2025-09-27T18:37:15.36Z" }, + { url = "https://files.pythonhosted.org/packages/28/52/182836104b33b444e400b14f797212f720cbc9ed6ba34c800639d154e821/markupsafe-3.0.3-cp314-cp314-win_amd64.whl", hash = "sha256:bdc919ead48f234740ad807933cdf545180bfbe9342c2bb451556db2ed958581", size = 15341, upload-time = "2025-09-27T18:37:16.496Z" }, + { url = "https://files.pythonhosted.org/packages/6f/18/acf23e91bd94fd7b3031558b1f013adfa21a8e407a3fdb32745538730382/markupsafe-3.0.3-cp314-cp314-win_arm64.whl", hash = "sha256:5a7d5dc5140555cf21a6fefbdbf8723f06fcd2f63ef108f2854de715e4422cb4", size = 14073, upload-time = "2025-09-27T18:37:17.476Z" }, + { url = "https://files.pythonhosted.org/packages/3c/f0/57689aa4076e1b43b15fdfa646b04653969d50cf30c32a102762be2485da/markupsafe-3.0.3-cp314-cp314t-macosx_10_13_x86_64.whl", hash = "sha256:1353ef0c1b138e1907ae78e2f6c63ff67501122006b0f9abad68fda5f4ffc6ab", size = 11661, upload-time = "2025-09-27T18:37:18.453Z" }, + { url = "https://files.pythonhosted.org/packages/89/c3/2e67a7ca217c6912985ec766c6393b636fb0c2344443ff9d91404dc4c79f/markupsafe-3.0.3-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:1085e7fbddd3be5f89cc898938f42c0b3c711fdcb37d75221de2666af647c175", size = 12069, upload-time = "2025-09-27T18:37:19.332Z" }, + { url = "https://files.pythonhosted.org/packages/f0/00/be561dce4e6ca66b15276e184ce4b8aec61fe83662cce2f7d72bd3249d28/markupsafe-3.0.3-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:1b52b4fb9df4eb9ae465f8d0c228a00624de2334f216f178a995ccdcf82c4634", size = 25670, upload-time = "2025-09-27T18:37:20.245Z" }, + { url = "https://files.pythonhosted.org/packages/50/09/c419f6f5a92e5fadde27efd190eca90f05e1261b10dbd8cbcb39cd8ea1dc/markupsafe-3.0.3-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:fed51ac40f757d41b7c48425901843666a6677e3e8eb0abcff09e4ba6e664f50", size = 23598, upload-time = "2025-09-27T18:37:21.177Z" }, + { url = "https://files.pythonhosted.org/packages/22/44/a0681611106e0b2921b3033fc19bc53323e0b50bc70cffdd19f7d679bb66/markupsafe-3.0.3-cp314-cp314t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:f190daf01f13c72eac4efd5c430a8de82489d9cff23c364c3ea822545032993e", size = 23261, upload-time = "2025-09-27T18:37:22.167Z" }, + { url = "https://files.pythonhosted.org/packages/5f/57/1b0b3f100259dc9fffe780cfb60d4be71375510e435efec3d116b6436d43/markupsafe-3.0.3-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:e56b7d45a839a697b5eb268c82a71bd8c7f6c94d6fd50c3d577fa39a9f1409f5", size = 24835, upload-time = "2025-09-27T18:37:23.296Z" }, + { url = "https://files.pythonhosted.org/packages/26/6a/4bf6d0c97c4920f1597cc14dd720705eca0bf7c787aebc6bb4d1bead5388/markupsafe-3.0.3-cp314-cp314t-musllinux_1_2_riscv64.whl", hash = "sha256:f3e98bb3798ead92273dc0e5fd0f31ade220f59a266ffd8a4f6065e0a3ce0523", size = 22733, upload-time = "2025-09-27T18:37:24.237Z" }, + { url = "https://files.pythonhosted.org/packages/14/c7/ca723101509b518797fedc2fdf79ba57f886b4aca8a7d31857ba3ee8281f/markupsafe-3.0.3-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:5678211cb9333a6468fb8d8be0305520aa073f50d17f089b5b4b477ea6e67fdc", size = 23672, upload-time = "2025-09-27T18:37:25.271Z" }, + { url = "https://files.pythonhosted.org/packages/fb/df/5bd7a48c256faecd1d36edc13133e51397e41b73bb77e1a69deab746ebac/markupsafe-3.0.3-cp314-cp314t-win32.whl", hash = "sha256:915c04ba3851909ce68ccc2b8e2cd691618c4dc4c4232fb7982bca3f41fd8c3d", size = 14819, upload-time = "2025-09-27T18:37:26.285Z" }, + { url = "https://files.pythonhosted.org/packages/1a/8a/0402ba61a2f16038b48b39bccca271134be00c5c9f0f623208399333c448/markupsafe-3.0.3-cp314-cp314t-win_amd64.whl", hash = "sha256:4faffd047e07c38848ce017e8725090413cd80cbc23d86e55c587bf979e579c9", size = 15426, upload-time = "2025-09-27T18:37:27.316Z" }, + { url = "https://files.pythonhosted.org/packages/70/bc/6f1c2f612465f5fa89b95bead1f44dcb607670fd42891d8fdcd5d039f4f4/markupsafe-3.0.3-cp314-cp314t-win_arm64.whl", hash = "sha256:32001d6a8fc98c8cb5c947787c5d08b0a50663d139f1305bac5885d98d9b40fa", size = 14146, upload-time = "2025-09-27T18:37:28.327Z" }, +] + +[[package]] +name = "mdurl" +version = "0.1.2" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/d6/54/cfe61301667036ec958cb99bd3efefba235e65cdeb9c84d24a8293ba1d90/mdurl-0.1.2.tar.gz", hash = "sha256:bb413d29f5eea38f31dd4754dd7377d4465116fb207585f97bf925588687c1ba", size = 8729, upload-time = "2022-08-14T12:40:10.846Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/b3/38/89ba8ad64ae25be8de66a6d463314cf1eb366222074cfda9ee839c56a4b4/mdurl-0.1.2-py3-none-any.whl", hash = "sha256:84008a41e51615a49fc9966191ff91509e3c40b939176e643fd50a5c2196b8f8", size = 9979, upload-time = "2022-08-14T12:40:09.779Z" }, +] + +[[package]] +name = "mypy" +version = "2.3.1" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "ast-serialize" }, + { name = "librt", marker = "platform_python_implementation != 'PyPy'" }, + { name = "mypy-extensions" }, + { name = "pathspec" }, + { name = "typing-extensions" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/82/6a/878cc1097d4035f82bd516658d0c528d2a9955bc7b363afcbd0b07fea11b/mypy-2.3.1.tar.gz", hash = "sha256:47c1b1207258513a9d93495f69c8be9de73916186f0e52703e8c461b7a623419", size = 3992554, upload-time = "2026-08-15T03:03:38.549Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/85/da/d6effc4f808a842d91edc22535dc9e799d2ff6e91449168b7f47a0771f54/mypy-2.3.1-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:a32bbbb940af990d3be0b8af321c7b6815bb1b3b48142fe7459b9cc5f58959ff", size = 14047547, upload-time = "2026-08-15T03:02:57.707Z" }, + { url = "https://files.pythonhosted.org/packages/e4/e6/478229701dab76f26485fc8ff5d6f241f393da22447400bbc56f6946aebe/mypy-2.3.1-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:ff715e45b2231a8e85de1d163d1b42791e4d7aab8f5145f85fee1b710b735aff", size = 14216515, upload-time = "2026-08-15T03:01:26.496Z" }, + { url = "https://files.pythonhosted.org/packages/8d/fe/7c42327a3b21e84681f691982cbfe43f334a3685f3b683b72c376476c4fa/mypy-2.3.1-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:858fc57d3d91fa728e33e7ad71def60fc6272694607b306cd3292db53ae39080", size = 15307789, upload-time = "2026-08-15T03:03:31.62Z" }, + { url = "https://files.pythonhosted.org/packages/59/f4/7e597edbe01b5a56fa958ce541302dcaabfed979966f1dffedbea0ea0fc2/mypy-2.3.1-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:851833db876e7b650f93719c74b7879a08e338979c96054fdfc3bfd90a486355", size = 15548831, upload-time = "2026-08-15T03:03:15.55Z" }, + { url = "https://files.pythonhosted.org/packages/a3/52/cb31e084bc0314a1e384bdd677a4b80e55af04ccac077545e2238b9d320a/mypy-2.3.1-cp312-cp312-win_amd64.whl", hash = "sha256:4c5095a327483591c94e0c8d3ef9e50d4ab1369b541eae007c1f23bc2a41f6bb", size = 11226359, upload-time = "2026-08-15T03:03:29.002Z" }, + { url = "https://files.pythonhosted.org/packages/7a/47/88fcf6217b43fa2da81a8c2611370af18141536a4f0294bbf98b457d456d/mypy-2.3.1-cp312-cp312-win_arm64.whl", hash = "sha256:bbfe022634a2a195406bd469e888d2eaf193b02ba7e607391cd7640374aaae3b", size = 10214707, upload-time = "2026-08-15T03:02:48.807Z" }, + { url = "https://files.pythonhosted.org/packages/de/cf/862010ee800ca9c2bd0c4c0dacf0f092e5411824a09b8f97ad4be8fe250e/mypy-2.3.1-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:114dff494000f18bd10d5d95d84b8567b26da60279ecbe838131841df20e635d", size = 13964542, upload-time = "2026-08-15T03:02:21.43Z" }, + { url = "https://files.pythonhosted.org/packages/75/5a/3f3a2107b41e3e92e617e25daaee121413b91e9784bea733131ed4fecc5d/mypy-2.3.1-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:c8637731bb5eee3671eb2c3200827aa3564ed8a9309ecee4d1afe77e6d031bdb", size = 14168922, upload-time = "2026-08-15T03:03:00.351Z" }, + { url = "https://files.pythonhosted.org/packages/8b/41/04dc4fe7e63d7820fa4eff272e95157d30cbea921388f3ab3fe77794cd0b/mypy-2.3.1-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:1c80fbc405ed8020f5ff3802dc18cf060197bcdd3fbdd6a26ef2fd34dfdd5226", size = 15244791, upload-time = "2026-08-15T03:02:31.089Z" }, + { url = "https://files.pythonhosted.org/packages/96/fc/c3053b26b9054949285aa868cb6af8c10e7591541cacd79c5dcc06a1fcf9/mypy-2.3.1-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:84081f538ce27375045c02e3d7f81bd11d853400621ae245d87ce7b6c420ec74", size = 15501627, upload-time = "2026-08-15T03:03:34.128Z" }, + { url = "https://files.pythonhosted.org/packages/70/4e/d77daab008bbc4e5001374d7928f4a260d28f0e6747af444fc4763f7a310/mypy-2.3.1-cp313-cp313-win_amd64.whl", hash = "sha256:e9144ac16fde007096f9563eb2041b4433c2d705c4218edeb79e7e9d01035ee6", size = 11243961, upload-time = "2026-08-15T03:02:11.952Z" }, + { url = "https://files.pythonhosted.org/packages/f0/f8/7eb68c136e4abd30569fe31ef2bfcb7eceae9952cab80017c04cd09f5d0c/mypy-2.3.1-cp313-cp313-win_arm64.whl", hash = "sha256:77ad9529e67dca28e511f5cd5671436584ce91f6d3bac159a353158187b986ac", size = 10213219, upload-time = "2026-08-15T03:02:26.361Z" }, + { url = "https://files.pythonhosted.org/packages/be/c4/42a49d44aeff804edf1b19acce0b49e8bd1a9c57dee9605dd8d980aa43d7/mypy-2.3.1-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:192abaedf75da1bc0b1cef104927e70ec49c1ef0031cc4825c7ee10a438ed24d", size = 13986778, upload-time = "2026-08-15T03:01:33.69Z" }, + { url = "https://files.pythonhosted.org/packages/45/13/9331fd2dfed7194d66c5304072894a8be3e51e9deda6863c1eceaa35a43d/mypy-2.3.1-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:bf678dffd16efcda2c15cbd30e9ecc0081388e29ea23687a88e686ed92638dc3", size = 14188467, upload-time = "2026-08-15T03:02:40.554Z" }, + { url = "https://files.pythonhosted.org/packages/78/f7/f4a34edab45667c5465855dc585a20e87978ffa8aee711445b7239d120c6/mypy-2.3.1-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:8e036f06b41630f4c8a1d48f9ac6aa26acc65f8be089973f5519da643318f03f", size = 15225538, upload-time = "2026-08-15T03:03:09.761Z" }, + { url = "https://files.pythonhosted.org/packages/40/05/534b3590757bd05794f73e07f6666c2a77b8597ffed795c94ce570096aa0/mypy-2.3.1-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:71af9c8a894e862b58e92abb08e53b05a384a1e5e5d6dc7cda59126211a53d82", size = 15480805, upload-time = "2026-08-15T03:01:41.134Z" }, + { url = "https://files.pythonhosted.org/packages/55/da/bdfba852e2562f599624af5bb7d29e36b0b4f526f2b8bac85efe0dd1803d/mypy-2.3.1-cp314-cp314-pyemscripten_2026_0_wasm32.whl", hash = "sha256:3c80cd23d85368bdd9f37d5231dfd97d35bcbf5bf41af96ef3a9b078ad1957f9", size = 7761712, upload-time = "2026-08-15T03:02:36.008Z" }, + { url = "https://files.pythonhosted.org/packages/98/31/60fc64a74cdba4f2a5d642d32317993e479163e1ac7d91b695e5d15e2264/mypy-2.3.1-cp314-cp314-win_amd64.whl", hash = "sha256:4956f34d145e145562a0a0bf367f642bbc85c04ec2baf47ae015947c3169a85d", size = 11423968, upload-time = "2026-08-15T03:02:06.931Z" }, + { url = "https://files.pythonhosted.org/packages/a9/23/eb5950b24cd26ba3b78f87707a275568d633c77dae8e61c9661be6055ca6/mypy-2.3.1-cp314-cp314-win_arm64.whl", hash = "sha256:cfb12e360242d23d91f5e978d94f58ea66acf5804c4fb6f2f794a20d4cb1b595", size = 10399323, upload-time = "2026-08-15T03:02:33.671Z" }, + { url = "https://files.pythonhosted.org/packages/82/c7/f80f4e46c0b9a00eb5f78a79d49dda8bdf56a5230f7257fb33e76be04da7/mypy-2.3.1-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:e5f1c50bb05b64e2026b52867e8d21106f01313c744a2c4ecc34c90d12e8d6e2", size = 15121308, upload-time = "2026-08-15T03:01:46.053Z" }, + { url = "https://files.pythonhosted.org/packages/5d/74/9b04f17c7074cc5188f02fb63a2ca1d43fedf479e84fe3091c39061a1d7f/mypy-2.3.1-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:667196b352f4cf304ded4c10f90cfc179263a1acfb3cdcfa984bdfd340d498bc", size = 15536590, upload-time = "2026-08-15T03:01:35.941Z" }, + { url = "https://files.pythonhosted.org/packages/26/04/c837ef6208e567774e2ed1f863f8ba6ec4817b1b6dd426315e5d559b6ec9/mypy-2.3.1-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:b9c53e395c12cad2c6d4b67d5da7c6057638a132d85c08b73646b18f802a0045", size = 16791074, upload-time = "2026-08-15T03:01:31.073Z" }, + { url = "https://files.pythonhosted.org/packages/37/68/48730230afa45192d5bd429a6a2ff24a6f8dedda90fdf2b221792b54518f/mypy-2.3.1-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:18162b128c3f9c703cd35f5537446900b0d21a2549aa7a95d21380d2ef643fb0", size = 17069183, upload-time = "2026-08-15T03:02:28.566Z" }, + { url = "https://files.pythonhosted.org/packages/1c/ea/ca23fc9c20eeda09a15c9cbcf50015d0e73f409f6ead059e42aa69a608ff/mypy-2.3.1-cp314-cp314t-win_amd64.whl", hash = "sha256:30c0477d4aab7b7f39c8397dc877f2c96b9fe5588ec379f372c56eb63d599f63", size = 12154679, upload-time = "2026-08-15T03:02:04.809Z" }, + { url = "https://files.pythonhosted.org/packages/3b/67/8d982126034990869466f73b8db80dcb2234a7ac39b4dad093e047a79835/mypy-2.3.1-cp314-cp314t-win_arm64.whl", hash = "sha256:6941ab3619377bc3f32ca02876b07d27f216f5201604b664d3937ea0fdd23bb4", size = 10969159, upload-time = "2026-08-15T03:02:38.152Z" }, + { url = "https://files.pythonhosted.org/packages/ee/f7/41e7f2d8117fbc7a7587286162ffe2f688984b69c46ed63cf5f2e4fc3bae/mypy-2.3.1-cp315-cp315-macosx_11_0_arm64.whl", hash = "sha256:6f041a6de52c9217ca125e78ba0a335cb7fd98a1c0580978e49ab2b126f70b57", size = 13990694, upload-time = "2026-08-15T03:03:21.919Z" }, + { url = "https://files.pythonhosted.org/packages/06/85/8f665811a0c8f3bf6fa1d9acd665ec2d97a2bcc453ae68dcd92340941cd6/mypy-2.3.1-cp315-cp315-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:5159ae60f5dbc3a498af5ba8365505808ac8031bc63f9e00304ad545d40bdd9b", size = 14203518, upload-time = "2026-08-15T03:01:48.455Z" }, + { url = "https://files.pythonhosted.org/packages/2d/82/91b866c8546b120bff83b73a439d90d2d63ef3aff113599e6b8e4d566848/mypy-2.3.1-cp315-cp315-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:47a8a7a0a7f6f6e63995c0ac36fa0c07b127413fdc81f0439b7f3dccafd33561", size = 15220224, upload-time = "2026-08-15T03:01:23.577Z" }, + { url = "https://files.pythonhosted.org/packages/c8/78/c226c99208ee40de7c768369fa533f933afa003dfdc606ff021450724e91/mypy-2.3.1-cp315-cp315-musllinux_1_2_x86_64.whl", hash = "sha256:2329c0501293d4e1f33bc15d04d6304d65a1cdda967ee93a05c1e681a3923133", size = 15501512, upload-time = "2026-08-15T03:02:09.453Z" }, + { url = "https://files.pythonhosted.org/packages/a9/e7/7cfb3f106c393979f4cc37ad6c0586044d50401e3c35b0c003e4f3ba6bc9/mypy-2.3.1-cp315-cp315-pyemscripten_2026_5_wasm32.whl", hash = "sha256:bb26deed807bdb0457cf3e3f1cd7c4a1cf9d66864eaf1b4a61e06805d4c6b1f9", size = 7761913, upload-time = "2026-08-15T03:01:55.65Z" }, + { url = "https://files.pythonhosted.org/packages/99/3c/52affefa273b97939a1f474ae4a349c8718635c15b941112dfab4291b0c1/mypy-2.3.1-cp315-cp315-win_amd64.whl", hash = "sha256:375d7013876a8233b2d05be185bfa09f689696cd999ce8b1cfe6acac5c80e8a3", size = 11422533, upload-time = "2026-08-15T03:03:24.101Z" }, + { url = "https://files.pythonhosted.org/packages/2a/b7/75643e70c72a5b346d8a9b1543c967ea8824df2ee3fb7ccba652c272b7bb/mypy-2.3.1-cp315-cp315-win_arm64.whl", hash = "sha256:586b3612214cceabb3c0f588c97e7d1e535393f06a60e912e994f6b3ace97523", size = 10397931, upload-time = "2026-08-15T03:02:55.265Z" }, + { url = "https://files.pythonhosted.org/packages/10/ce/53be21f2d4adfcd26f63f1184a13ed797015ab463853f117e2e11e4d726f/mypy-2.3.1-cp315-cp315t-macosx_11_0_arm64.whl", hash = "sha256:ef0c6335cda9d807f8193d8ff6204a72bc909fa9882aacbca14f43cdb7188306", size = 15118669, upload-time = "2026-08-15T03:02:51.479Z" }, + { url = "https://files.pythonhosted.org/packages/62/43/20de757cd42989d291a17fad607742c4c74e875ce5cea00e5a5225020ac1/mypy-2.3.1-cp315-cp315t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:e598c8c66401d26b150872154a286e6d484cf2789c3bb28a7556806298423021", size = 15545627, upload-time = "2026-08-15T03:03:05.132Z" }, + { url = "https://files.pythonhosted.org/packages/7e/fc/092bdf77ad280eaf501422f0f3b966012b528076cc13e41a774861c907d1/mypy-2.3.1-cp315-cp315t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:eda22fd4efa9dcd39331d1dede9b5b8b8a7fd69af07592e778433da98610d29e", size = 16764157, upload-time = "2026-08-15T03:02:23.958Z" }, + { url = "https://files.pythonhosted.org/packages/94/5c/c94c4d62d909b07f552d0d9356d7acc943825558e602a64822ffa2231536/mypy-2.3.1-cp315-cp315t-musllinux_1_2_x86_64.whl", hash = "sha256:2a0ba2e57847849fb0d1fcdabb32786d223095ed8bc121dfe322bcdb3d9c46bc", size = 17073258, upload-time = "2026-08-15T03:02:14.573Z" }, + { url = "https://files.pythonhosted.org/packages/c0/f7/511a88b89e478053c02d22039bb8f3ce4183efe8fd7a4f0a5910a8bb0a32/mypy-2.3.1-cp315-cp315t-win_amd64.whl", hash = "sha256:3f7e865dd51f235f60a2dbcd8728a1c095f5ca28f095d48a725b84cd935735c4", size = 12135505, upload-time = "2026-08-15T03:02:16.714Z" }, + { url = "https://files.pythonhosted.org/packages/71/bf/02573b56964ecb0f7c644f915f53c325ae15c3faec521c5adf11599a32df/mypy-2.3.1-cp315-cp315t-win_arm64.whl", hash = "sha256:8ad80807dc3ab8ea978b1b2b6e4a657194ace1d4ef03e0e731aff1abd517da29", size = 10962647, upload-time = "2026-08-15T03:01:43.712Z" }, + { url = "https://files.pythonhosted.org/packages/8e/41/9675c7a1e78edecfba0b79e587a52594c56e189368261dc7b3a7fffb9527/mypy-2.3.1-py3-none-any.whl", hash = "sha256:6ed5c7e3419083268e5c9258bd1c1ef91af44a9e89374dbcaf37b775716e72eb", size = 2754338, upload-time = "2026-08-15T03:02:53.4Z" }, +] + +[[package]] +name = "mypy-extensions" +version = "1.1.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/a2/6e/371856a3fb9d31ca8dac321cda606860fa4548858c0cc45d9d1d4ca2628b/mypy_extensions-1.1.0.tar.gz", hash = "sha256:52e68efc3284861e772bbcd66823fde5ae21fd2fdb51c62a211403730b916558", size = 6343, upload-time = "2025-04-22T14:54:24.164Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/79/7b/2c79738432f5c924bef5071f933bcc9efd0473bac3b4aa584a6f7c1c8df8/mypy_extensions-1.1.0-py3-none-any.whl", hash = "sha256:1be4cccdb0f2482337c4743e60421de3a356cd97508abadd57d47403e94f5505", size = 4963, upload-time = "2025-04-22T14:54:22.983Z" }, +] + +[[package]] +name = "packaging" +version = "26.2" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/d7/f1/e7a6dd94a8d4a5626c03e4e99c87f241ba9e350cd9e6d75123f992427270/packaging-26.2.tar.gz", hash = "sha256:ff452ff5a3e828ce110190feff1178bb1f2ea2281fa2075aadb987c2fb221661", size = 228134, upload-time = "2026-04-24T20:15:23.917Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/df/b2/87e62e8c3e2f4b32e5fe99e0b86d576da1312593b39f47d8ceef365e95ed/packaging-26.2-py3-none-any.whl", hash = "sha256:5fc45236b9446107ff2415ce77c807cee2862cb6fac22b8a73826d0693b0980e", size = 100195, upload-time = "2026-04-24T20:15:22.081Z" }, +] + +[[package]] +name = "pathspec" +version = "1.1.1" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/5a/82/42f767fc1c1143d6fd36efb827202a2d997a375e160a71eb2888a925aac1/pathspec-1.1.1.tar.gz", hash = "sha256:17db5ecd524104a120e173814c90367a96a98d07c45b2e10c2f3919fff91bf5a", size = 135180, upload-time = "2026-04-27T01:46:08.907Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/f1/d9/7fb5aa316bc299258e68c73ba3bddbc499654a07f151cba08f6153988714/pathspec-1.1.1-py3-none-any.whl", hash = "sha256:a00ce642f577bf7f473932318056212bc4f8bfdf53128c78bbd5af0b9b20b189", size = 57328, upload-time = "2026-04-27T01:46:07.06Z" }, +] + +[[package]] +name = "pefile" +version = "2024.8.26" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/03/4f/2750f7f6f025a1507cd3b7218691671eecfd0bbebebe8b39aa0fe1d360b8/pefile-2024.8.26.tar.gz", hash = "sha256:3ff6c5d8b43e8c37bb6e6dd5085658d658a7a0bdcd20b6a07b1fcfc1c4e9d632", size = 76008, upload-time = "2024-08-26T20:58:38.155Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/54/16/12b82f791c7f50ddec566873d5bdd245baa1491bac11d15ffb98aecc8f8b/pefile-2024.8.26-py3-none-any.whl", hash = "sha256:76f8b485dcd3b1bb8166f1128d395fa3d87af26360c2358fb75b80019b957c6f", size = 74766, upload-time = "2024-08-26T21:01:02.632Z" }, +] + +[[package]] +name = "pluggy" +version = "1.6.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/f9/e2/3e91f31a7d2b083fe6ef3fa267035b518369d9511ffab804f839851d2779/pluggy-1.6.0.tar.gz", hash = "sha256:7dcc130b76258d33b90f61b658791dede3486c3e6bfb003ee5c9bfb396dd22f3", size = 69412, upload-time = "2025-05-15T12:30:07.975Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/54/20/4d324d65cc6d9205fabedc306948156824eb9f0ee1633355a8f7ec5c66bf/pluggy-1.6.0-py3-none-any.whl", hash = "sha256:e920276dd6813095e9377c0bc5566d94c932c33b27a3e3945d8389c374dd4746", size = 20538, upload-time = "2025-05-15T12:30:06.134Z" }, +] + +[[package]] +name = "portabase-cli" +version = "26.8.12" +source = { virtual = "." } +dependencies = [ + { name = "cryptography" }, + { name = "jinja2" }, + { name = "pyyaml" }, + { name = "questionary" }, + { name = "requests" }, + { name = "rich" }, + { name = "typer" }, +] + +[package.dev-dependencies] +dev = [ + { name = "mypy" }, + { name = "pyinstaller" }, + { name = "pytest" }, + { name = "ruff" }, +] + +[package.metadata] +requires-dist = [ + { name = "cryptography", specifier = ">=44.0.0" }, + { name = "jinja2", specifier = ">=3.1" }, + { name = "pyyaml", specifier = ">=6.0.3" }, + { name = "questionary", specifier = ">=2.1.0" }, + { name = "requests", specifier = ">=2.32.5" }, + { name = "rich", specifier = ">=14.2.0" }, + { name = "typer", specifier = ">=0.20.0" }, +] + +[package.metadata.requires-dev] +dev = [ + { name = "mypy", specifier = ">=1.15" }, + { name = "pyinstaller", specifier = ">=6.17.0" }, + { name = "pytest", specifier = ">=8.3" }, + { name = "ruff", specifier = ">=0.16.0" }, +] + +[[package]] +name = "prompt-toolkit" +version = "3.0.52" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "wcwidth" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/a1/96/06e01a7b38dce6fe1db213e061a4602dd6032a8a97ef6c1a862537732421/prompt_toolkit-3.0.52.tar.gz", hash = "sha256:28cde192929c8e7321de85de1ddbe736f1375148b02f2e17edd840042b1be855", size = 434198, upload-time = "2025-08-27T15:24:02.057Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/84/03/0d3ce49e2505ae70cf43bc5bb3033955d2fc9f932163e84dc0779cc47f48/prompt_toolkit-3.0.52-py3-none-any.whl", hash = "sha256:9aac639a3bbd33284347de5ad8d68ecc044b91a762dc39b7c21095fcd6a19955", size = 391431, upload-time = "2025-08-27T15:23:59.498Z" }, +] + +[[package]] +name = "pycparser" +version = "3.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/1b/7d/92392ff7815c21062bea51aa7b87d45576f649f16458d78b7cf94b9ab2e6/pycparser-3.0.tar.gz", hash = "sha256:600f49d217304a5902ac3c37e1281c9fe94e4d0489de643a9504c5cdfdfc6b29", size = 103492, upload-time = "2026-01-21T14:26:51.89Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/0c/c3/44f3fbbfa403ea2a7c779186dc20772604442dde72947e7d01069cbe98e3/pycparser-3.0-py3-none-any.whl", hash = "sha256:b727414169a36b7d524c1c3e31839a521725078d7b2ff038656844266160a992", size = 48172, upload-time = "2026-01-21T14:26:50.693Z" }, +] + +[[package]] +name = "pygments" +version = "2.20.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/c3/b2/bc9c9196916376152d655522fdcebac55e66de6603a76a02bca1b6414f6c/pygments-2.20.0.tar.gz", hash = "sha256:6757cd03768053ff99f3039c1a36d6c0aa0b263438fcab17520b30a303a82b5f", size = 4955991, upload-time = "2026-03-29T13:29:33.898Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl", hash = "sha256:81a9e26dd42fd28a23a2d169d86d7ac03b46e2f8b59ed4698fb4785f946d0176", size = 1231151, upload-time = "2026-03-29T13:29:30.038Z" }, +] + +[[package]] +name = "pyinstaller" +version = "6.20.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "altgraph" }, + { name = "macholib", marker = "sys_platform == 'darwin'" }, + { name = "packaging" }, + { name = "pefile", marker = "sys_platform == 'win32'" }, + { name = "pyinstaller-hooks-contrib" }, + { name = "pywin32-ctypes", marker = "sys_platform == 'win32'" }, + { name = "setuptools" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/46/60/d03d52e6690d4e9caf333dcd14550cde634ce6c118b3bc8fa3112c3186fd/pyinstaller-6.20.0.tar.gz", hash = "sha256:95c5c7e03d5d61e9dfb8ef259c699cf492bb1041beb6dbe83696608cec07347a", size = 4048728, upload-time = "2026-04-22T20:59:36.96Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/d0/e4/e228d6d1bbb7fd62dc660a8fb202a583b023d3a3624ca95d1a9290ee4d6a/pyinstaller-6.20.0-py3-none-macosx_10_13_universal2.whl", hash = "sha256:bf3be4e1284ee78ddccba5e29f99443a12a7b4673168288ffc4c9d38c6f7b90e", size = 1047642, upload-time = "2026-04-22T20:58:32.006Z" }, + { url = "https://files.pythonhosted.org/packages/ce/bd/afb631bcb3f9040efebd4f6d067f0828b51710818f69fb41a2d4b7787f52/pyinstaller-6.20.0-py3-none-manylinux2014_aarch64.whl", hash = "sha256:72ae9c1fdea134afa791f58bdc9a1934d5c7609753c111e0026bfc272b32b712", size = 742494, upload-time = "2026-04-22T20:58:36.285Z" }, + { url = "https://files.pythonhosted.org/packages/76/08/0729a5bac14754150e5d83b39d87d842eb42b0bffcaa03dbad6252e23a39/pyinstaller-6.20.0-py3-none-manylinux2014_i686.whl", hash = "sha256:1031bcc307f3fbeffd4e162723e64d46dbf591c82dd0997413afb2a07328b941", size = 754191, upload-time = "2026-04-22T20:58:40.603Z" }, + { url = "https://files.pythonhosted.org/packages/e6/82/bc0ee4c7b97db1958eb651e0da9fb1e672e5ae53ca8867fd97701de52906/pyinstaller-6.20.0-py3-none-manylinux2014_ppc64le.whl", hash = "sha256:8df3b3f347659fa2562d8d193a98ad4600133b8b8d07c268df89e4154376750e", size = 751902, upload-time = "2026-04-22T20:58:44.7Z" }, + { url = "https://files.pythonhosted.org/packages/3d/e7/770002d6aaa54173881cb2c49bb195ba67b97bf39bac1cdf320f28401629/pyinstaller-6.20.0-py3-none-manylinux2014_s390x.whl", hash = "sha256:b0d3cc9dd8120d448459bd3880a12e2f9774c51443af49047801446377999a59", size = 748634, upload-time = "2026-04-22T20:58:48.579Z" }, + { url = "https://files.pythonhosted.org/packages/fe/db/68ba1fccb71278b2124fb90b37b7c8c0bc4c1173fba45b94466df3d9cb7f/pyinstaller-6.20.0-py3-none-manylinux2014_x86_64.whl", hash = "sha256:03696bb6350177c6bc23bcaf78e71a33c4a89b6754dd90d1be2f318e978c918b", size = 748490, upload-time = "2026-04-22T20:58:52.749Z" }, + { url = "https://files.pythonhosted.org/packages/03/0f/ac77ffa996a56be3d5c8f85734a007f8347240691657f9704e7de2527fa3/pyinstaller-6.20.0-py3-none-musllinux_1_1_aarch64.whl", hash = "sha256:6357f1699f6af84f37e7367f031d4f68abdba65543b83990c9e8f5a4cebed0b7", size = 747650, upload-time = "2026-04-22T20:58:57.093Z" }, + { url = "https://files.pythonhosted.org/packages/e0/56/1ee91c3a2bc10ca1f36da10a6fd55ff7efc4dec367171eb25992a827874f/pyinstaller-6.20.0-py3-none-musllinux_1_1_x86_64.whl", hash = "sha256:0ab39c690abad26ba148e8f664f0478acc82a733997f4f22e757774832802da9", size = 747413, upload-time = "2026-04-22T20:59:01.174Z" }, + { url = "https://files.pythonhosted.org/packages/d7/55/ae264339996953c4cdf9d89d916a0a8fa26a83cf917a742fff8b9d5f3fe8/pyinstaller-6.20.0-py3-none-win32.whl", hash = "sha256:9a7637e8e44b4387b13667fdcaac86ab6b29c446c16d34d8401539b81838759c", size = 1331584, upload-time = "2026-04-22T20:59:07.201Z" }, + { url = "https://files.pythonhosted.org/packages/76/8c/300f57578882cce259bfb5ae56fda3b69caa3fe9df40a176c719920ea6e2/pyinstaller-6.20.0-py3-none-win_amd64.whl", hash = "sha256:d588844e890ee80c4365867f98146636e1849bbca8e4284bbf0c809aff0f161a", size = 1391851, upload-time = "2026-04-22T20:59:14.024Z" }, + { url = "https://files.pythonhosted.org/packages/8a/ea/b2f8e1642aecda78c0b75c7321f708e49e10bb3c00dd4f148c40761a1527/pyinstaller-6.20.0-py3-none-win_arm64.whl", hash = "sha256:bd53282c0a73e5c95573e1ddc8e5d564d4932bec91efbaed4dc5fdff9c2ae7f2", size = 1332259, upload-time = "2026-04-22T20:59:20.509Z" }, +] + +[[package]] +name = "pyinstaller-hooks-contrib" +version = "2026.5" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "packaging" }, + { name = "setuptools" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/1a/67/f4452d68793fb15beba4f19ef39a38a8822f0da7452b503c400d5a21f5c1/pyinstaller_hooks_contrib-2026.5.tar.gz", hash = "sha256:f066dfca8f7c45ff6336c9cf9fe25b4e48bfeb322a1aa24faaedfb8a8d1b0b08", size = 173689, upload-time = "2026-05-04T22:36:55.124Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/f6/5c/fd465d11da4d12b50d7eb5d2ee2ceb780d8d049dbb489f3828d131e387af/pyinstaller_hooks_contrib-2026.5-py3-none-any.whl", hash = "sha256:ea1535783fbdac4626351709e83f3ea80b681d3a4745763ebb407b5e27342eb9", size = 457314, upload-time = "2026-05-04T22:36:53.598Z" }, +] + +[[package]] +name = "pytest" +version = "9.1.1" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "colorama", marker = "sys_platform == 'win32'" }, + { name = "iniconfig" }, + { name = "packaging" }, + { name = "pluggy" }, + { name = "pygments" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/e4/47/b9efed96c114afcfa3c9d3fe98a76a1d14c74a9e266d397cf6eb64be5e01/pytest-9.1.1.tar.gz", hash = "sha256:1088fbde8f2b49d95a549a195707afa7a76a3ce9bcadc26b6d71f0ffda5fe313", size = 1636369, upload-time = "2026-06-19T10:58:32.857Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/24/25/1de2678b631f5a49215c6c96fff41ba892b0a34df68d6d80292b1b48aa7f/pytest-9.1.1-py3-none-any.whl", hash = "sha256:37a86b45efb9a47a61a36449063e8e18d0cab3161329fc099eb21783169c4f0c", size = 386536, upload-time = "2026-06-19T10:58:31.347Z" }, +] + +[[package]] +name = "pywin32-ctypes" +version = "0.2.3" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/85/9f/01a1a99704853cb63f253eea009390c88e7131c67e66a0a02099a8c917cb/pywin32-ctypes-0.2.3.tar.gz", hash = "sha256:d162dc04946d704503b2edc4d55f3dba5c1d539ead017afa00142c38b9885755", size = 29471, upload-time = "2024-08-14T10:15:34.626Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/de/3d/8161f7711c017e01ac9f008dfddd9410dff3674334c233bde66e7ba65bbf/pywin32_ctypes-0.2.3-py3-none-any.whl", hash = "sha256:8a1513379d709975552d202d942d9837758905c8d01eb82b8bcc30918929e7b8", size = 30756, upload-time = "2024-08-14T10:15:33.187Z" }, +] + +[[package]] +name = "pyyaml" +version = "6.0.3" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/05/8e/961c0007c59b8dd7729d542c61a4d537767a59645b82a0b521206e1e25c2/pyyaml-6.0.3.tar.gz", hash = "sha256:d76623373421df22fb4cf8817020cbb7ef15c725b9d5e45f17e189bfc384190f", size = 130960, upload-time = "2025-09-25T21:33:16.546Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/d1/33/422b98d2195232ca1826284a76852ad5a86fe23e31b009c9886b2d0fb8b2/pyyaml-6.0.3-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:7f047e29dcae44602496db43be01ad42fc6f1cc0d8cd6c83d342306c32270196", size = 182063, upload-time = "2025-09-25T21:32:11.445Z" }, + { url = "https://files.pythonhosted.org/packages/89/a0/6cf41a19a1f2f3feab0e9c0b74134aa2ce6849093d5517a0c550fe37a648/pyyaml-6.0.3-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:fc09d0aa354569bc501d4e787133afc08552722d3ab34836a80547331bb5d4a0", size = 173973, upload-time = "2025-09-25T21:32:12.492Z" }, + { url = "https://files.pythonhosted.org/packages/ed/23/7a778b6bd0b9a8039df8b1b1d80e2e2ad78aa04171592c8a5c43a56a6af4/pyyaml-6.0.3-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:9149cad251584d5fb4981be1ecde53a1ca46c891a79788c0df828d2f166bda28", size = 775116, upload-time = "2025-09-25T21:32:13.652Z" }, + { url = "https://files.pythonhosted.org/packages/65/30/d7353c338e12baef4ecc1b09e877c1970bd3382789c159b4f89d6a70dc09/pyyaml-6.0.3-cp312-cp312-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:5fdec68f91a0c6739b380c83b951e2c72ac0197ace422360e6d5a959d8d97b2c", size = 844011, upload-time = "2025-09-25T21:32:15.21Z" }, + { url = "https://files.pythonhosted.org/packages/8b/9d/b3589d3877982d4f2329302ef98a8026e7f4443c765c46cfecc8858c6b4b/pyyaml-6.0.3-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:ba1cc08a7ccde2d2ec775841541641e4548226580ab850948cbfda66a1befcdc", size = 807870, upload-time = "2025-09-25T21:32:16.431Z" }, + { url = "https://files.pythonhosted.org/packages/05/c0/b3be26a015601b822b97d9149ff8cb5ead58c66f981e04fedf4e762f4bd4/pyyaml-6.0.3-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:8dc52c23056b9ddd46818a57b78404882310fb473d63f17b07d5c40421e47f8e", size = 761089, upload-time = "2025-09-25T21:32:17.56Z" }, + { url = "https://files.pythonhosted.org/packages/be/8e/98435a21d1d4b46590d5459a22d88128103f8da4c2d4cb8f14f2a96504e1/pyyaml-6.0.3-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:41715c910c881bc081f1e8872880d3c650acf13dfa8214bad49ed4cede7c34ea", size = 790181, upload-time = "2025-09-25T21:32:18.834Z" }, + { url = "https://files.pythonhosted.org/packages/74/93/7baea19427dcfbe1e5a372d81473250b379f04b1bd3c4c5ff825e2327202/pyyaml-6.0.3-cp312-cp312-win32.whl", hash = "sha256:96b533f0e99f6579b3d4d4995707cf36df9100d67e0c8303a0c55b27b5f99bc5", size = 137658, upload-time = "2025-09-25T21:32:20.209Z" }, + { url = "https://files.pythonhosted.org/packages/86/bf/899e81e4cce32febab4fb42bb97dcdf66bc135272882d1987881a4b519e9/pyyaml-6.0.3-cp312-cp312-win_amd64.whl", hash = "sha256:5fcd34e47f6e0b794d17de1b4ff496c00986e1c83f7ab2fb8fcfe9616ff7477b", size = 154003, upload-time = "2025-09-25T21:32:21.167Z" }, + { url = "https://files.pythonhosted.org/packages/1a/08/67bd04656199bbb51dbed1439b7f27601dfb576fb864099c7ef0c3e55531/pyyaml-6.0.3-cp312-cp312-win_arm64.whl", hash = "sha256:64386e5e707d03a7e172c0701abfb7e10f0fb753ee1d773128192742712a98fd", size = 140344, upload-time = "2025-09-25T21:32:22.617Z" }, + { url = "https://files.pythonhosted.org/packages/d1/11/0fd08f8192109f7169db964b5707a2f1e8b745d4e239b784a5a1dd80d1db/pyyaml-6.0.3-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:8da9669d359f02c0b91ccc01cac4a67f16afec0dac22c2ad09f46bee0697eba8", size = 181669, upload-time = "2025-09-25T21:32:23.673Z" }, + { url = "https://files.pythonhosted.org/packages/b1/16/95309993f1d3748cd644e02e38b75d50cbc0d9561d21f390a76242ce073f/pyyaml-6.0.3-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:2283a07e2c21a2aa78d9c4442724ec1eb15f5e42a723b99cb3d822d48f5f7ad1", size = 173252, upload-time = "2025-09-25T21:32:25.149Z" }, + { url = "https://files.pythonhosted.org/packages/50/31/b20f376d3f810b9b2371e72ef5adb33879b25edb7a6d072cb7ca0c486398/pyyaml-6.0.3-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:ee2922902c45ae8ccada2c5b501ab86c36525b883eff4255313a253a3160861c", size = 767081, upload-time = "2025-09-25T21:32:26.575Z" }, + { url = "https://files.pythonhosted.org/packages/49/1e/a55ca81e949270d5d4432fbbd19dfea5321eda7c41a849d443dc92fd1ff7/pyyaml-6.0.3-cp313-cp313-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:a33284e20b78bd4a18c8c2282d549d10bc8408a2a7ff57653c0cf0b9be0afce5", size = 841159, upload-time = "2025-09-25T21:32:27.727Z" }, + { url = "https://files.pythonhosted.org/packages/74/27/e5b8f34d02d9995b80abcef563ea1f8b56d20134d8f4e5e81733b1feceb2/pyyaml-6.0.3-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:0f29edc409a6392443abf94b9cf89ce99889a1dd5376d94316ae5145dfedd5d6", size = 801626, upload-time = "2025-09-25T21:32:28.878Z" }, + { url = "https://files.pythonhosted.org/packages/f9/11/ba845c23988798f40e52ba45f34849aa8a1f2d4af4b798588010792ebad6/pyyaml-6.0.3-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:f7057c9a337546edc7973c0d3ba84ddcdf0daa14533c2065749c9075001090e6", size = 753613, upload-time = "2025-09-25T21:32:30.178Z" }, + { url = "https://files.pythonhosted.org/packages/3d/e0/7966e1a7bfc0a45bf0a7fb6b98ea03fc9b8d84fa7f2229e9659680b69ee3/pyyaml-6.0.3-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:eda16858a3cab07b80edaf74336ece1f986ba330fdb8ee0d6c0d68fe82bc96be", size = 794115, upload-time = "2025-09-25T21:32:31.353Z" }, + { url = "https://files.pythonhosted.org/packages/de/94/980b50a6531b3019e45ddeada0626d45fa85cbe22300844a7983285bed3b/pyyaml-6.0.3-cp313-cp313-win32.whl", hash = "sha256:d0eae10f8159e8fdad514efdc92d74fd8d682c933a6dd088030f3834bc8e6b26", size = 137427, upload-time = "2025-09-25T21:32:32.58Z" }, + { url = "https://files.pythonhosted.org/packages/97/c9/39d5b874e8b28845e4ec2202b5da735d0199dbe5b8fb85f91398814a9a46/pyyaml-6.0.3-cp313-cp313-win_amd64.whl", hash = "sha256:79005a0d97d5ddabfeeea4cf676af11e647e41d81c9a7722a193022accdb6b7c", size = 154090, upload-time = "2025-09-25T21:32:33.659Z" }, + { url = "https://files.pythonhosted.org/packages/73/e8/2bdf3ca2090f68bb3d75b44da7bbc71843b19c9f2b9cb9b0f4ab7a5a4329/pyyaml-6.0.3-cp313-cp313-win_arm64.whl", hash = "sha256:5498cd1645aa724a7c71c8f378eb29ebe23da2fc0d7a08071d89469bf1d2defb", size = 140246, upload-time = "2025-09-25T21:32:34.663Z" }, + { url = "https://files.pythonhosted.org/packages/9d/8c/f4bd7f6465179953d3ac9bc44ac1a8a3e6122cf8ada906b4f96c60172d43/pyyaml-6.0.3-cp314-cp314-macosx_10_13_x86_64.whl", hash = "sha256:8d1fab6bb153a416f9aeb4b8763bc0f22a5586065f86f7664fc23339fc1c1fac", size = 181814, upload-time = "2025-09-25T21:32:35.712Z" }, + { url = "https://files.pythonhosted.org/packages/bd/9c/4d95bb87eb2063d20db7b60faa3840c1b18025517ae857371c4dd55a6b3a/pyyaml-6.0.3-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:34d5fcd24b8445fadc33f9cf348c1047101756fd760b4dacb5c3e99755703310", size = 173809, upload-time = "2025-09-25T21:32:36.789Z" }, + { url = "https://files.pythonhosted.org/packages/92/b5/47e807c2623074914e29dabd16cbbdd4bf5e9b2db9f8090fa64411fc5382/pyyaml-6.0.3-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:501a031947e3a9025ed4405a168e6ef5ae3126c59f90ce0cd6f2bfc477be31b7", size = 766454, upload-time = "2025-09-25T21:32:37.966Z" }, + { url = "https://files.pythonhosted.org/packages/02/9e/e5e9b168be58564121efb3de6859c452fccde0ab093d8438905899a3a483/pyyaml-6.0.3-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:b3bc83488de33889877a0f2543ade9f70c67d66d9ebb4ac959502e12de895788", size = 836355, upload-time = "2025-09-25T21:32:39.178Z" }, + { url = "https://files.pythonhosted.org/packages/88/f9/16491d7ed2a919954993e48aa941b200f38040928474c9e85ea9e64222c3/pyyaml-6.0.3-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:c458b6d084f9b935061bc36216e8a69a7e293a2f1e68bf956dcd9e6cbcd143f5", size = 794175, upload-time = "2025-09-25T21:32:40.865Z" }, + { url = "https://files.pythonhosted.org/packages/dd/3f/5989debef34dc6397317802b527dbbafb2b4760878a53d4166579111411e/pyyaml-6.0.3-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:7c6610def4f163542a622a73fb39f534f8c101d690126992300bf3207eab9764", size = 755228, upload-time = "2025-09-25T21:32:42.084Z" }, + { url = "https://files.pythonhosted.org/packages/d7/ce/af88a49043cd2e265be63d083fc75b27b6ed062f5f9fd6cdc223ad62f03e/pyyaml-6.0.3-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:5190d403f121660ce8d1d2c1bb2ef1bd05b5f68533fc5c2ea899bd15f4399b35", size = 789194, upload-time = "2025-09-25T21:32:43.362Z" }, + { url = "https://files.pythonhosted.org/packages/23/20/bb6982b26a40bb43951265ba29d4c246ef0ff59c9fdcdf0ed04e0687de4d/pyyaml-6.0.3-cp314-cp314-win_amd64.whl", hash = "sha256:4a2e8cebe2ff6ab7d1050ecd59c25d4c8bd7e6f400f5f82b96557ac0abafd0ac", size = 156429, upload-time = "2025-09-25T21:32:57.844Z" }, + { url = "https://files.pythonhosted.org/packages/f4/f4/a4541072bb9422c8a883ab55255f918fa378ecf083f5b85e87fc2b4eda1b/pyyaml-6.0.3-cp314-cp314-win_arm64.whl", hash = "sha256:93dda82c9c22deb0a405ea4dc5f2d0cda384168e466364dec6255b293923b2f3", size = 143912, upload-time = "2025-09-25T21:32:59.247Z" }, + { url = "https://files.pythonhosted.org/packages/7c/f9/07dd09ae774e4616edf6cda684ee78f97777bdd15847253637a6f052a62f/pyyaml-6.0.3-cp314-cp314t-macosx_10_13_x86_64.whl", hash = "sha256:02893d100e99e03eda1c8fd5c441d8c60103fd175728e23e431db1b589cf5ab3", size = 189108, upload-time = "2025-09-25T21:32:44.377Z" }, + { url = "https://files.pythonhosted.org/packages/4e/78/8d08c9fb7ce09ad8c38ad533c1191cf27f7ae1effe5bb9400a46d9437fcf/pyyaml-6.0.3-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:c1ff362665ae507275af2853520967820d9124984e0f7466736aea23d8611fba", size = 183641, upload-time = "2025-09-25T21:32:45.407Z" }, + { url = "https://files.pythonhosted.org/packages/7b/5b/3babb19104a46945cf816d047db2788bcaf8c94527a805610b0289a01c6b/pyyaml-6.0.3-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:6adc77889b628398debc7b65c073bcb99c4a0237b248cacaf3fe8a557563ef6c", size = 831901, upload-time = "2025-09-25T21:32:48.83Z" }, + { url = "https://files.pythonhosted.org/packages/8b/cc/dff0684d8dc44da4d22a13f35f073d558c268780ce3c6ba1b87055bb0b87/pyyaml-6.0.3-cp314-cp314t-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:a80cb027f6b349846a3bf6d73b5e95e782175e52f22108cfa17876aaeff93702", size = 861132, upload-time = "2025-09-25T21:32:50.149Z" }, + { url = "https://files.pythonhosted.org/packages/b1/5e/f77dc6b9036943e285ba76b49e118d9ea929885becb0a29ba8a7c75e29fe/pyyaml-6.0.3-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:00c4bdeba853cc34e7dd471f16b4114f4162dc03e6b7afcc2128711f0eca823c", size = 839261, upload-time = "2025-09-25T21:32:51.808Z" }, + { url = "https://files.pythonhosted.org/packages/ce/88/a9db1376aa2a228197c58b37302f284b5617f56a5d959fd1763fb1675ce6/pyyaml-6.0.3-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:66e1674c3ef6f541c35191caae2d429b967b99e02040f5ba928632d9a7f0f065", size = 805272, upload-time = "2025-09-25T21:32:52.941Z" }, + { url = "https://files.pythonhosted.org/packages/da/92/1446574745d74df0c92e6aa4a7b0b3130706a4142b2d1a5869f2eaa423c6/pyyaml-6.0.3-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:16249ee61e95f858e83976573de0f5b2893b3677ba71c9dd36b9cf8be9ac6d65", size = 829923, upload-time = "2025-09-25T21:32:54.537Z" }, + { url = "https://files.pythonhosted.org/packages/f0/7a/1c7270340330e575b92f397352af856a8c06f230aa3e76f86b39d01b416a/pyyaml-6.0.3-cp314-cp314t-win_amd64.whl", hash = "sha256:4ad1906908f2f5ae4e5a8ddfce73c320c2a1429ec52eafd27138b7f1cbe341c9", size = 174062, upload-time = "2025-09-25T21:32:55.767Z" }, + { url = "https://files.pythonhosted.org/packages/f1/12/de94a39c2ef588c7e6455cfbe7343d3b2dc9d6b6b2f40c4c6565744c873d/pyyaml-6.0.3-cp314-cp314t-win_arm64.whl", hash = "sha256:ebc55a14a21cb14062aa4162f906cd962b28e2e9ea38f9b4391244cd8de4ae0b", size = 149341, upload-time = "2025-09-25T21:32:56.828Z" }, +] + +[[package]] +name = "questionary" +version = "2.1.1" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "prompt-toolkit" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/f6/45/eafb0bba0f9988f6a2520f9ca2df2c82ddfa8d67c95d6625452e97b204a5/questionary-2.1.1.tar.gz", hash = "sha256:3d7e980292bb0107abaa79c68dd3eee3c561b83a0f89ae482860b181c8bd412d", size = 25845, upload-time = "2025-08-28T19:00:20.851Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/3c/26/1062c7ec1b053db9e499b4d2d5bc231743201b74051c973dadeac80a8f43/questionary-2.1.1-py3-none-any.whl", hash = "sha256:a51af13f345f1cdea62347589fbb6df3b290306ab8930713bfae4d475a7d4a59", size = 36753, upload-time = "2025-08-28T19:00:19.56Z" }, +] + +[[package]] +name = "requests" +version = "2.34.2" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "certifi" }, + { name = "charset-normalizer" }, + { name = "idna" }, + { name = "urllib3" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/ac/c3/e2a2b89f2d3e2179abd6d00ebd70bff6273f37fb3e0cc209f48b39d00cbf/requests-2.34.2.tar.gz", hash = "sha256:f288924cae4e29463698d6d60bc6a4da69c89185ad1e0bcc4104f584e960b9ed", size = 142856, upload-time = "2026-05-14T19:25:27.735Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/a0/f4/c67b0b3f1b9245e8d266f0f112c500d50e5b4e83cb6f3b71b6528104182a/requests-2.34.2-py3-none-any.whl", hash = "sha256:2a0d60c172f83ac6ab31e4554906c0f3b3588d37b5cb939b1c061f4907e278e0", size = 73075, upload-time = "2026-05-14T19:25:26.443Z" }, +] + +[[package]] +name = "rich" +version = "15.0.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "markdown-it-py" }, + { name = "pygments" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/c0/8f/0722ca900cc807c13a6a0c696dacf35430f72e0ec571c4275d2371fca3e9/rich-15.0.0.tar.gz", hash = "sha256:edd07a4824c6b40189fb7ac9bc4c52536e9780fbbfbddf6f1e2502c31b068c36", size = 230680, upload-time = "2026-04-12T08:24:00.75Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/82/3b/64d4899d73f91ba49a8c18a8ff3f0ea8f1c1d75481760df8c68ef5235bf5/rich-15.0.0-py3-none-any.whl", hash = "sha256:33bd4ef74232fb73fe9279a257718407f169c09b78a87ad3d296f548e27de0bb", size = 310654, upload-time = "2026-04-12T08:24:02.83Z" }, +] + +[[package]] +name = "ruff" +version = "0.16.7" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/82/bb/5a449b9162e49b139d72f61672bd3ac1d790221f796d3304e2241fff4c58/ruff-0.16.7.tar.gz", hash = "sha256:5f71d004ac1263b22fa39462ac5ae618a4b77d58981af2cc79bf79a29c12b1a6", size = 4924184, upload-time = "2026-09-10T18:04:06.336Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/e3/b2/c80aeeb7f9e469c0d63a85d2f1ab6e1ebfbe10ea7a8d2438b7e09e3ff09e/ruff-0.16.7-py3-none-linux_armv6l.whl", hash = "sha256:727307773e7c7f9181d3ed3a2484186e56c1fa1874255911c74585eb2c7c19f9", size = 10048917, upload-time = "2026-09-10T18:03:30.28Z" }, + { url = "https://files.pythonhosted.org/packages/7b/96/20bb7bcae008004df52afcb7ac83432d4a467f2c17b672fe46d26be231c5/ruff-0.16.7-py3-none-macosx_10_12_x86_64.whl", hash = "sha256:9d61c258deabf58f34c67bd4bb4d939c7f2e6b5f0e59c1cdd1cf771b11cde929", size = 10242929, upload-time = "2026-09-10T18:03:32.706Z" }, + { url = "https://files.pythonhosted.org/packages/90/b2/f184b0d5abec02db69cfd7e49b688ae0237554528ca777136c613bf36bee/ruff-0.16.7-py3-none-macosx_11_0_arm64.whl", hash = "sha256:7ab81118df8945e0193d0240712aa4496573595b75185c3636ed825592a0f728", size = 9847245, upload-time = "2026-09-10T18:03:34.509Z" }, + { url = "https://files.pythonhosted.org/packages/eb/2d/db1633a641866ed801e34cc6b60ef236c5e16f9b2124ab1d49cc24a5fe4f/ruff-0.16.7-py3-none-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:4c196c968874fc8019da8e7163de7a1a370f111e2309b4b7dfea0fce950198d0", size = 9961780, upload-time = "2026-09-10T18:03:36.618Z" }, + { url = "https://files.pythonhosted.org/packages/4d/98/edea21e1a3e38dbbc3bf6bb068b863b3b06184cf8533a4c7dbbe208a89d5/ruff-0.16.7-py3-none-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:ac8c3bd0a7e10ad31e6ce51e7a99f3cb772e69aecdd6b9ea7e99b362f62a62c0", size = 9866337, upload-time = "2026-09-10T18:03:38.805Z" }, + { url = "https://files.pythonhosted.org/packages/0b/11/a15e60d4c87b214646f116ca9d204475bf993ee1047459bc9a360fd4d6d1/ruff-0.16.7-py3-none-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:398d3988edde000b5c75dc1b3f584708da9bc990de069c18909142580fec1af9", size = 10562512, upload-time = "2026-09-10T18:03:40.71Z" }, + { url = "https://files.pythonhosted.org/packages/29/42/eaff4c9b6d0c7cdf56df313a17e89ae854f5bbc0b0c8f9cce19be0ab7a8f/ruff-0.16.7-py3-none-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:ce05b62b770a8217c4646a9c4139fca00efe8fe5d71f87df2b243ff20d4584d1", size = 11302938, upload-time = "2026-09-10T18:03:42.607Z" }, + { url = "https://files.pythonhosted.org/packages/5d/43/c75aa59a4ec181fe2ec06cab30e198c1c6d107229a9f008ae3a7c16cabd8/ruff-0.16.7-py3-none-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:af1b576fddb9d9ef2ececfb5fadcd6a624b25070ed85e3cfcfe449fc3ff6a7b9", size = 10840857, upload-time = "2026-09-10T18:03:44.604Z" }, + { url = "https://files.pythonhosted.org/packages/21/33/81f3da371942ea031105ba679d8d6e28ec1660ccd690a45f42d381161356/ruff-0.16.7-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:9ce7f8f22df67c93ed96c717f9128eadb797144ac2bad475cf536f31d6100c55", size = 10370001, upload-time = "2026-09-10T18:03:46.706Z" }, + { url = "https://files.pythonhosted.org/packages/fa/0b/6345fb4dbf6dd0ed1cfe5d18391dc9c3f59cc81622a7b0a65b84b3e730ba/ruff-0.16.7-py3-none-manylinux_2_31_riscv64.whl", hash = "sha256:06d0e93d04f392996435ebd600c153f65b47d73fbec2415aa99c5ee5756b3a5f", size = 10548735, upload-time = "2026-09-10T18:03:48.658Z" }, + { url = "https://files.pythonhosted.org/packages/3f/4d/c5576adf511f92a328e5569dda190ecdd430da51f1a649f3a4a2fd73e21e/ruff-0.16.7-py3-none-musllinux_1_2_aarch64.whl", hash = "sha256:142151a5e7b93c1b11111337142f89dd2fbfee92161225c99a97222f22e32656", size = 10108496, upload-time = "2026-09-10T18:03:50.563Z" }, + { url = "https://files.pythonhosted.org/packages/ff/8c/667d83c16199a17a56adc6b0bd4c3beb5b767a2babcd16a56f76f9be7fd6/ruff-0.16.7-py3-none-musllinux_1_2_armv7l.whl", hash = "sha256:e6651f97a342d8b35d54d8991544ca22169b86dc54111cb604666940c431b750", size = 9860136, upload-time = "2026-09-10T18:03:52.621Z" }, + { url = "https://files.pythonhosted.org/packages/99/75/78d401106731999a1dd20cc5a6961e37e1eb9397a3b589f73f3a5ce146a3/ruff-0.16.7-py3-none-musllinux_1_2_i686.whl", hash = "sha256:ef140c6eb935fa9a84c9c607dfb2cb1b85843c192e79265b0c54f35f557ea8e5", size = 10286290, upload-time = "2026-09-10T18:03:55.207Z" }, + { url = "https://files.pythonhosted.org/packages/68/49/56f9c3a8b755df93a0ad318b2147bf4ef5dae9a7e5ec61c460109c67957f/ruff-0.16.7-py3-none-musllinux_1_2_x86_64.whl", hash = "sha256:53e39506a730fadeee0d998ed5946f30671f0db240c6c7c73bdabbe33604bb6f", size = 10745048, upload-time = "2026-09-10T18:03:57.299Z" }, + { url = "https://files.pythonhosted.org/packages/5f/ea/7f9b938a63ece4bec677ad7f9f7fa02df3383db1949ed93a382441c09a87/ruff-0.16.7-py3-none-win32.whl", hash = "sha256:2ea3470fcebcbc5df2fb0c6f3b90333fa9084c534e0111c038fa4a6ab9f1c4b7", size = 10059082, upload-time = "2026-09-10T18:03:59.632Z" }, + { url = "https://files.pythonhosted.org/packages/39/11/480a6973a927aa653e1cead6a6416008640e03a99d05b34c0434b8c6c366/ruff-0.16.7-py3-none-win_amd64.whl", hash = "sha256:7ac26aca826e9e21d0f1cb25b54ac660760a9fdd094d3e4df9848232be98cfc6", size = 10593368, upload-time = "2026-09-10T18:04:01.999Z" }, + { url = "https://files.pythonhosted.org/packages/8b/4b/51327018d056f0dad2c2238f26d1fb0f53707a9d91b75dea6d1b3039f136/ruff-0.16.7-py3-none-win_arm64.whl", hash = "sha256:aab7f39e2c9df6c596216070f98eef1207b94f8516cca20c808826974971855b", size = 10412401, upload-time = "2026-09-10T18:04:04.098Z" }, +] + +[[package]] +name = "setuptools" +version = "82.0.1" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/4f/db/cfac1baf10650ab4d1c111714410d2fbb77ac5a616db26775db562c8fab2/setuptools-82.0.1.tar.gz", hash = "sha256:7d872682c5d01cfde07da7bccc7b65469d3dca203318515ada1de5eda35efbf9", size = 1152316, upload-time = "2026-03-09T12:47:17.221Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/9d/76/f789f7a86709c6b087c5a2f52f911838cad707cc613162401badc665acfe/setuptools-82.0.1-py3-none-any.whl", hash = "sha256:a59e362652f08dcd477c78bb6e7bd9d80a7995bc73ce773050228a348ce2e5bb", size = 1006223, upload-time = "2026-03-09T12:47:15.026Z" }, +] + +[[package]] +name = "shellingham" +version = "1.5.4" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/58/15/8b3609fd3830ef7b27b655beb4b4e9c62313a4e8da8c676e142cc210d58e/shellingham-1.5.4.tar.gz", hash = "sha256:8dbca0739d487e5bd35ab3ca4b36e11c4078f3a234bfce294b0a0291363404de", size = 10310, upload-time = "2023-10-24T04:13:40.426Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/e0/f9/0595336914c5619e5f28a1fb793285925a8cd4b432c9da0a987836c7f822/shellingham-1.5.4-py2.py3-none-any.whl", hash = "sha256:7ecfff8f2fd72616f7481040475a65b2bf8af90a56c89140852d1120324e8686", size = 9755, upload-time = "2023-10-24T04:13:38.866Z" }, +] + +[[package]] +name = "typer" +version = "0.25.1" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "annotated-doc" }, + { name = "click" }, + { name = "rich" }, + { name = "shellingham" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/e4/51/9aed62104cea109b820bbd6c14245af756112017d309da813ef107d42e7e/typer-0.25.1.tar.gz", hash = "sha256:9616eb8853a09ffeabab1698952f33c6f29ffdbceb4eaeecf571880e8d7664cc", size = 122276, upload-time = "2026-04-30T19:32:16.964Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/3f/f9/2b3ff4e56e5fa7debfaf9eb135d0da96f3e9a1d5b27222223c7296336e5f/typer-0.25.1-py3-none-any.whl", hash = "sha256:75caa44ed46a03fb2dab8808753ffacdbfea88495e74c85a28c5eefcf5f39c89", size = 58409, upload-time = "2026-04-30T19:32:18.271Z" }, +] + +[[package]] +name = "typing-extensions" +version = "4.16.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/f6/cc/6253133b5bb138fc3306cebfbda2c520f545d36b5be2c7255cc528bb45d6/typing_extensions-4.16.0.tar.gz", hash = "sha256:dc983d19a509c94dba722ee6abd33940f7c05a89e243c47e907eb4db6f1a43e5", size = 113555, upload-time = "2026-07-02T08:40:05.92Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/49/d3/b8441a820a491ddfc024b0b0cf0393375b75ea13866d9c66727e54c2fc80/typing_extensions-4.16.0-py3-none-any.whl", hash = "sha256:481caa481374e813c1b176ada14e97f1f67a4539ce9cfeb3f350d78d6370c2e8", size = 45571, upload-time = "2026-07-02T08:40:04.659Z" }, +] + +[[package]] +name = "urllib3" +version = "2.7.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/53/0c/06f8b233b8fd13b9e5ee11424ef85419ba0d8ba0b3138bf360be2ff56953/urllib3-2.7.0.tar.gz", hash = "sha256:231e0ec3b63ceb14667c67be60f2f2c40a518cb38b03af60abc813da26505f4c", size = 433602, upload-time = "2026-05-07T16:13:18.596Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/7f/3e/5db95bcf282c52709639744ca2a8b149baccf648e39c8cc87553df9eae0c/urllib3-2.7.0-py3-none-any.whl", hash = "sha256:9fb4c81ebbb1ce9531cce37674bbc6f1360472bc18ca9a553ede278ef7276897", size = 131087, upload-time = "2026-05-07T16:13:17.151Z" }, +] + +[[package]] +name = "wcwidth" +version = "0.7.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/2c/ee/afaf0f85a9a18fe47a67f1e4422ed6cf1fe642f0ae0a2f81166231303c52/wcwidth-0.7.0.tar.gz", hash = "sha256:90e3a7ea092341c44b99562e75d09e4d5160fe7a3974c6fb842a101a95e7eed0", size = 182132, upload-time = "2026-05-02T16:04:12.653Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/41/52/e465037f5375f43533d1a80b6923955201596a99142ed524d77b571a1418/wcwidth-0.7.0-py3-none-any.whl", hash = "sha256:5d69154c429a82910e241c738cd0e2976fac8a2dd47a1a805f4afed1c0f136f2", size = 110825, upload-time = "2026-05-02T16:04:11.033Z" }, +] From 17b56fec7c7b95414bc654866c605413e0772e45 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Th=C3=A9o=20LAGACHE?= Date: Fri, 11 Sep 2026 14:26:36 +0200 Subject: [PATCH 08/24] feat(agent): show the proposed configuration and confirm before writing The dashboard kept its summary and confirmation through the refactor; the agent lost them when database setup moved to a loop that writes as it goes. Restored for the agent itself, before its files are written, with --yes to skip the prompt like the dashboard. The Edge Key is masked in the summary. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01CbemoDqsmMdRq6PS5Vaup8 --- commands/agent.py | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/commands/agent.py b/commands/agent.py index d06162e..845a333 100644 --- a/commands/agent.py +++ b/commands/agent.py @@ -73,6 +73,10 @@ def run( force: Annotated[ bool, typer.Option("--force", "-f", help="Overwrite an existing folder") ] = False, + yes: Annotated[ + bool, + typer.Option("--yes", "-y", help="Skip the configuration confirmation"), + ] = False, ) -> None: self.ui.banner() self.require_docker(self.docker) @@ -107,6 +111,23 @@ def run( name="host_gateway", ) + self.ui.summary( + [ + ("Agent Name", name), + ("Path", str(path)), + ("Edge Key", env_vars["EDGE_KEY"]), + ("Timezone", env_vars["TZ"]), + ("Polling", f"{env_vars['POLLING']}s"), + ("Host Gateway", "Yes" if gateway else "No"), + ("Files to Create", "docker-compose.yml, .env, databases.json"), + ], + title="PROPOSED CONFIGURATION", + ) + if not yes: + self.confirm_or_abort( + "Apply this configuration and generate files?", default=True + ) + project = AgentProject.create(path, env_vars, host_gateway=gateway) self._write(project) self.ui.success(f"Agent '{name}' created in {path}") From d823cde7f24967228db779e2e9b44e2022626919 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Th=C3=A9o=20LAGACHE?= Date: Fri, 11 Sep 2026 14:44:05 +0200 Subject: [PATCH 09/24] docs: dashboard settings and auth providers design Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01CbemoDqsmMdRq6PS5Vaup8 --- .../2026-09-11-dashboard-settings-design.md | 198 ++++++++++++++++++ 1 file changed, 198 insertions(+) create mode 100644 docs/superpowers/specs/2026-09-11-dashboard-settings-design.md diff --git a/docs/superpowers/specs/2026-09-11-dashboard-settings-design.md b/docs/superpowers/specs/2026-09-11-dashboard-settings-design.md new file mode 100644 index 0000000..559f793 --- /dev/null +++ b/docs/superpowers/specs/2026-09-11-dashboard-settings-design.md @@ -0,0 +1,198 @@ +# Dashboard settings and authentication — Design + +Date : 2026-09-11 +Dépend de : `2026-09-11-cli-refactor-design.md` (état déclaratif, `EnvFile`, `Field`, `Form`, `CommandGroup`). + +## 1. Objectif + +Configurer depuis le CLI ce que le dashboard lit dans son environnement : API et MCP, onboarding, authentification par mot de passe, providers OIDC et OAuth2. À la création et après coup, en interactif et par flags. + +Au passage, une rupture de surface décidée pour être cohérente : un namespace par composant. + +## 2. Surface CLI + +### 2.1 Avant / après + +| Avant | Après | +|---|---| +| `agent NAME` | `agent create NAME` | +| `db add\|remove\|list NAME` | `agent db add\|remove\|list NAME` | +| `dashboard NAME` | `dashboard create NAME` | +| — | `dashboard show NAME` | +| — | `dashboard set NAME KEY VALUE [KEY VALUE…]` | +| — | `dashboard auth add\|remove\|list NAME` | +| `start\|stop\|restart\|logs\|uninstall\|build PATH` | inchangés | +| `config`, `update`, `decrypt` | inchangés | + +Un groupe Typer ne peut pas porter à la fois un argument positionnel et des sous-commandes (vérifié : `dashboard auth` créerait un dashboard nommé `auth`). D'où `create`. + +### 2.2 Compatibilité + +- `db` reste à la racine **une version**, alias de `agent db`, avec un avertissement à chaque appel : `'portabase db' is deprecated, use 'portabase agent db'`. Retiré à la version suivante. +- `portabase agent my-agent` et `portabase dashboard my-dash` produisent une `UsageError` « No such command ». Le catcher de `main.py` la reconnaît (token précédent = `agent` ou `dashboard`) et ajoute le hint `Did you mean: portabase agent create my-agent?`. +- README, `CONTRIBUTING.md`, doc portabase.io et script d'installation à mettre à jour dans la même release. + +### 2.3 Implémentation + +`CommandGroup` gagne `groups: list[CommandGroup]` pour imbriquer (`agent` contient `db`). `AgentCommand` devient `AgentCreateCommand` dans un `AgentCommands(CommandGroup)` ; idem `DashboardCreateCommand` dans `DashboardCommands`. + +## 3. Registre de settings + +Le cœur : une liste déclarative, un endroit à toucher pour ajouter un réglage. + +```python +@dataclass(frozen=True) +class Setting: + field: Field # name, prompt, kind, default, choices, help, validator + env: str # variable écrite dans .env + section: str # "network" | "api" | "onboarding" | "auth" + secret: bool = False # jamais en flag visible sans avertissement, masqué à l'affichage +``` + +`services/dashboard_settings.py` : + +| Section | `field.name` | `env` | kind | défaut dashboard | +|---|---|---|---|---| +| network | `url` | `PROJECT_URL` | text | `http://localhost:` | +| network | `behind_proxy` | `TUSD_BEHIND_PROXY` | bool | false | +| network | `trusted_domains` | `TRUSTED_DOMAINS` | text | — | +| api | `api` | `API_ENABLED` | bool | false | +| api | `openapi` | `OPENAPI_ENABLED` | bool | false | +| api | `mcp` | `MCP_ENABLED` | bool | false | +| onboarding | `skip_onboarding` | `SKIP_ONBOARDING` | bool | false | +| onboarding | `admin_name` | `AUTH_DEFAULT_USER_NAME` | text | — | +| onboarding | `admin_email` | `AUTH_DEFAULT_USER` | text | — | +| onboarding | `admin_password` | `AUTH_DEFAULT_PASSWORD` | secret | — | +| auth | `password_auth` | `AUTH_EMAIL_PASSWORD_ENABLED` | bool | true | +| auth | `signup` | `AUTH_SIGNUP_ENABLED` | bool | — | +| auth | `passkey` | `AUTH_PASSKEY_ENABLED` | bool | — | +| auth | `account_linking` | `AUTH_ALLOW_LINKING` | bool | — | +| auth | `account_unlinking` | `AUTH_ALLOW_UNLINKING` | bool | — | +| auth | `sync_oidc_roles` | `AUTH_SYNC_OIDC_ROLES_ON_LOGIN` | bool | — | +| auth | `role_map` | `AUTH_ROLE_MAP` | text | — | +| auth | `allowed_group` | `ALLOWED_GROUP` | text | — | + +Ce que le registre produit, sans code par réglage : + +- les flags de `dashboard create` : `--api/--no-api` pour un bool, `--url` pour un texte, `--admin-password-stdin` pour un secret ; +- les prompts interactifs, groupés par section ; +- la validation de `dashboard set` : `KEY` doit être un `field.name` du registre, `VALUE` est coercé par `Form` (bool `true/false/yes/no/1/0`, choix, validateur) ; +- l'affichage de `dashboard show`, section par section, secrets masqués. + +Écriture dans `.env` : booléens en `true`/`false`. À la création, seuls les réglages fournis ou différents du défaut sont écrits (le `.env` reste lisible). `set` écrit toujours la valeur demandée. + +`admin_password` porte le validateur documenté : 8 caractères, majuscule, minuscule, chiffre, spécial. + +## 4. Providers d'authentification + +Répétables, donc en sous-commande, comme `agent db`. + +### 4.1 Modèle + +```python +@dataclass(frozen=True) +class AuthProvider: + kind: Literal["oidc", "oauth"] + id: str # providerId : "keycloak", "github" + values: dict[str, str] # champs → valeurs, sans le préfixe +``` + +Stockage dans `.env` par préfixe, ce qui est exactement le mécanisme des bases managées : + +| kind | préfixe | champs | +|---|---|---| +| oidc | `AUTH_OIDC__` | `ID`, `TITLE`, `DESC`, `ICON`, `ISSUER_URL`, `CLIENT`, `SECRET`, `SCOPES`, `PKCE`, `HOST` | +| oauth | `AUTH_SOCIAL__` | `CLIENT`, `SECRET`, `TITLE` | + +`` est l'identifiant en majuscules avec `-` → `_` ; `AUTH_OIDC__ID` reçoit l'identifiant tel que saisi (c'est le `providerId` du callback). Les providers OAuth sont limités aux noms connus du dashboard : `google`, `github`, `discord`, `apple`, `linkedin`, `x`, `reddit`. + +Lecture : `DashboardProject.providers` scanne les clés du `.env` par préfixe et reconstruit la liste. Aucun autre état. + +### 4.2 Commandes + +``` +dashboard auth add NAME oidc ID --issuer URL --client CLIENT (--secret S | --secret-stdin) + [--title T] [--scopes "openid profile email"] [--pkce] [--host H] +dashboard auth add NAME oauth PROVIDER --client CLIENT (--secret S | --secret-stdin) [--title T] +dashboard auth list NAME +dashboard auth remove NAME ID [--yes] +``` + +- `add` sur un `ID` existant → `ValidationError`, hint « remove it first ». +- `remove` fait `env.remove_prefix(...)` puis re-rend. +- `list` affiche kind, id, titre, issuer/provider, et le callback à déclarer chez le fournisseur : `/api/auth/sso/callback/`. +- `--secret` visible accepté avec avertissement, `--secret-stdin` recommandé — même règle que `agent db add`. + +En interactif, `add` sans flags pose les champs du kind via `Form`. + +## 5. Validations croisées + +Dans `DashboardProject.validate()`, appelé avant toute écriture. Ce sont des refus, pas des avertissements : chacune laisse une instance inaccessible. + +| Condition | Erreur | +|---|---| +| `skip_onboarding` sans `admin_email` **et** `admin_password` | « Skipping onboarding needs an initial account: set admin_email and admin_password. » | +| `password_auth = false` et aucun provider | « Disabling password login with no OIDC or OAuth provider would lock everyone out. » | +| au moins un provider et `url` sur `localhost` | « Providers need a public URL for their callback; set url (currently http://localhost:8887). » | +| `auth remove` du dernier provider alors que `password_auth = false` | même refus que la ligne 2 | + +`admin_password` faible → refus par le validateur du champ. + +## 6. Interactif — `dashboard create` + +Le flux actuel reste : port, mode base, timezone, résumé, confirmation. Entre le résumé et la confirmation, une question : + +``` +Configure API, MCP and authentication now? [y/N] +``` + +Non par défaut. Si oui, trois sections courtes, chacune précédée de `ui.section(...)` : + +1. **API** — `api`, `openapi`, `mcp` +2. **Onboarding** — `skip_onboarding` ; si oui, `admin_name`, `admin_email`, `admin_password` (masqué) +3. **Authentication** — `password_auth`, `signup`, `passkey` + +Les providers ne sont pas dans le wizard : hint `Add a login provider with: portabase dashboard auth add NAME oidc …`, comme `agent create` renvoie vers `db add`. + +Le résumé inclut les réglages non-défaut. `--yes` saute la confirmation, `--non-interactive` prend les défauts et ne pose pas la question des sections. + +## 7. Application des changements + +Toute commande mutante (`create`, `set`, `auth add`, `auth remove`) termine par `project.save_state()` puis `renderer.render_dashboard(project).write(path)`. Le template a `env_file: .env`, donc le compose ne change pas — mais `write` est appelé quand même pour garder un seul chemin. + +Puis le message : `Apply with: portabase start NAME`. + +**`restart` est corrigé dans cette spec** : `docker compose restart` ne relit pas `env_file` ni ne crée un service ajouté (bug déjà constaté sur `agent db add`). `RestartCommand` fait `up -d` puis `restart`, pour converger vers l'état déclaré. Le message des commandes mutantes peut alors dire `portabase restart NAME` sans mentir. + +## 8. Ce qui ne change pas + +- `dashboard.yml.j2` : aucune modification, `env_file` suffit. +- `EnvFile`, `Form`, `Field`, `Summary`, `render_dashboard` : réutilisés tels quels. +- Le mode `custom`/`external`/`internal` de la base : inchangé. + +## 9. Hors périmètre + +- SMTP (`SMTP_*`), `RETENTION_CRON`, `STALE_BACKUP_THRESHOLD_HOURS`, `BACKUP_FOLDER_NAME`, `TELEMETRY` du dashboard. Le registre les accepte en une ligne chacun le jour venu. +- Provider OAuth2 générique (endpoints libres) : le dashboard ne le documente pas. +- Un `agent set` : l'agent n'a que `TZ`, `POLLING`, `LOG_LEVEL` ; à ajouter si le besoin apparaît, avec le même registre. +- Tests automatisés : spec séparée. Vérification ici par `render_check` et les parcours non-interactifs. + +## 10. Fichiers + +| Fichier | Action | +|---|---| +| `commands/base.py` | `CommandGroup.groups` pour l'imbrication | +| `commands/agent.py` | `AgentCommands` (groupe) + `AgentCreateCommand` ; `db` devient `agent db` | +| `commands/db.py` | inchangé, enregistré sous `agent` ; alias racine déprécié | +| `commands/dashboard.py` | `DashboardCommands` : `create`, `show`, `set` | +| `commands/dashboard_auth.py` | `add`, `list`, `remove` | +| `commands/lifecycle.py` | `RestartCommand` → `up -d` puis `restart` | +| `services/dashboard_settings.py` | `Setting`, `SETTINGS`, `OAUTH_PROVIDERS`, `OIDC_FIELDS` | +| `services/project.py` | `DashboardProject` : `settings`, `providers`, `validate()`, `set()`, `add_provider()`, `remove_provider()` | +| `main.py` | enregistrement des groupes, hint « did you mean » | +| `README.md`, `.github/CONTRIBUTING.md` | nouvelle surface | + +## 11. Questions ouvertes + +- Le `providerId` OIDC : imposer le même que `` en minuscules, ou le laisser libre via `--id` ? Défaut retenu : identique, pas de flag. +- `dashboard set` accepte plusieurs paires en une commande ; faut-il aussi `dashboard unset KEY` pour revenir au défaut du dashboard (retirer la variable) ? Défaut retenu : oui, trivial avec `env.remove`. From 1b31db7cf3925d656a91d6be7bf13e532191084e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Th=C3=A9o=20LAGACHE?= Date: Fri, 11 Sep 2026 14:45:28 +0200 Subject: [PATCH 10/24] style: name the pre-write panel SUMMARY Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01CbemoDqsmMdRq6PS5Vaup8 --- commands/agent.py | 2 +- commands/dashboard.py | 2 +- docs/superpowers/plans/2026-09-11-plan-4-render-commands.md | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/commands/agent.py b/commands/agent.py index 845a333..456558f 100644 --- a/commands/agent.py +++ b/commands/agent.py @@ -121,7 +121,7 @@ def run( ("Host Gateway", "Yes" if gateway else "No"), ("Files to Create", "docker-compose.yml, .env, databases.json"), ], - title="PROPOSED CONFIGURATION", + title="SUMMARY", ) if not yes: self.confirm_or_abort( diff --git a/commands/dashboard.py b/commands/dashboard.py index 0a58dcc..3d048b3 100644 --- a/commands/dashboard.py +++ b/commands/dashboard.py @@ -146,7 +146,7 @@ def run( ] rows.append(("Files to Create", "docker-compose.yml, .env")) - self.ui.summary(rows, title="PROPOSED CONFIGURATION") + self.ui.summary(rows, title="SUMMARY") if not yes: self.confirm_or_abort( "Apply this configuration and generate files?", default=True diff --git a/docs/superpowers/plans/2026-09-11-plan-4-render-commands.md b/docs/superpowers/plans/2026-09-11-plan-4-render-commands.md index e93ca1c..1856538 100644 --- a/docs/superpowers/plans/2026-09-11-plan-4-render-commands.md +++ b/docs/superpowers/plans/2026-09-11-plan-4-render-commands.md @@ -1349,7 +1349,7 @@ class DashboardCommand(Command): rows += [("DB Host", host), ("DB Name", dbname), ("Connection URL", env_vars["DATABASE_URL"])] rows.append(("Files to Create", "docker-compose.yml, .env")) - self.ui.summary(rows, title="PROPOSED CONFIGURATION") + self.ui.summary(rows, title="SUMMARY") if not yes: self.confirm_or_abort("Apply this configuration and generate files?", default=True) From c59cb08fac3ada61438c17f8aafd0f19b95b63fe Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Th=C3=A9o=20LAGACHE?= Date: Fri, 11 Sep 2026 14:47:25 +0200 Subject: [PATCH 11/24] =?UTF-8?q?feat!:=20one=20namespace=20per=20componen?= =?UTF-8?q?t=20=E2=80=94=20agent=20create,=20agent=20db?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 'portabase agent NAME' becomes 'portabase agent create NAME' and 'db' moves under 'agent'. A Typer group cannot carry both a positional argument and subcommands, and the dashboard is about to grow 'auth' and 'set', so both components get the same shape now, before the refactor ships. 'portabase db' survives one release as a deprecated alias that warns, and the old 'agent NAME' form answers with the new command instead of a bare 'No such command'. restart now runs 'up -d' before 'restart': compose restart neither creates a service added since the last start nor rereads env_file, so the advice printed after 'db add' was wrong. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01CbemoDqsmMdRq6PS5Vaup8 --- commands/agent.py | 38 +++++++++++++++++++++++++++++++++----- commands/base.py | 32 ++++++++++++++++++++++++++++++++ commands/db.py | 2 +- commands/lifecycle.py | 14 ++++++++++++-- main.py | 31 ++++++++++++++++++------------- services/project.py | 2 +- ui/components/hints.py | 2 +- 7 files changed, 98 insertions(+), 23 deletions(-) diff --git a/commands/agent.py b/commands/agent.py index 456558f..377b4bd 100644 --- a/commands/agent.py +++ b/commands/agent.py @@ -5,8 +5,8 @@ import typer -from commands.base import Command -from commands.db import report_write +from commands.base import Command, CommandGroup +from commands.db import DbCommands, report_write from commands.flows.add_database import AddDatabaseFlow from core.errors import ValidationError from core.utils import validate_edge_key @@ -31,8 +31,8 @@ def _edge_key(value: str) -> str: return value -class AgentCommand(Command): - name, help, panel = "agent", "Create a new Portabase Agent instance.", "Creation" +class AgentCreateCommand(Command): + name, help, panel = "create", "Create a new Portabase Agent instance.", "Creation" no_args_is_help = True def __init__( @@ -134,7 +134,7 @@ def run( if self.ui.non_interactive: self.ui.hint( - f"Add databases with: portabase db add {name} " + f"Add databases with: portabase agent db add {name} " "--engine postgresql --mode new" ) else: @@ -164,3 +164,31 @@ def _write(self, project: AgentProject) -> None: project.save_state() report = result.write(project.path) report_write(self.ui, report) + + +class AgentCommands(CommandGroup): + name, help, panel = "agent", "Create and manage Portabase agents.", "Components" + + def __init__( + self, + ui: UI, + telemetry: Telemetry, + docker: DockerRunner, + templates: TemplateRepository, + renderer: ComposeRenderer, + engines: EngineRegistry, + ports: PortAllocator, + ) -> None: + super().__init__(ui, telemetry) + self._create = AgentCreateCommand( + ui, telemetry, docker, templates, renderer, engines, ports + ) + self.db = DbCommands(ui, telemetry, engines, ports, templates, renderer, docker) + + @property + def commands(self) -> list[Command]: + return [self._create] + + @property + def groups(self) -> list[CommandGroup]: + return [self.db] diff --git a/commands/base.py b/commands/base.py index 8aabba8..c849de3 100644 --- a/commands/base.py +++ b/commands/base.py @@ -95,11 +95,43 @@ def __init__(self, ui: UI, telemetry: Telemetry) -> None: @abstractmethod def commands(self) -> list[Command]: ... + @property + def groups(self) -> list[CommandGroup]: + return [] + def build_typer(self) -> typer.Typer: sub = typer.Typer(help=self.help, no_args_is_help=True) for cmd in self.commands: cmd.register(sub) + for group in self.groups: + group.register(sub) return sub def register(self, app: typer.Typer) -> None: app.add_typer(self.build_typer(), name=self.name, rich_help_panel=self.panel) + + +class DeprecatedAlias(CommandGroup): + def __init__( + self, ui: UI, telemetry: Telemetry, target: CommandGroup, *, name: str, use: str + ) -> None: + super().__init__(ui, telemetry) + self.name = name + self.help = f"Deprecated alias of '{use}'." + self.panel = target.panel + self._target = target + self._use = use + + @property + def commands(self) -> list[Command]: + return self._target.commands + + def build_typer(self) -> typer.Typer: + sub = super().build_typer() + ui, use, name = self.ui, self._use, self.name + + @sub.callback() + def _warn() -> None: + ui.warning(f"'portabase {name}' is deprecated, use 'portabase {use}'.") + + return sub diff --git a/commands/db.py b/commands/db.py index 4c5118b..867aaf1 100644 --- a/commands/db.py +++ b/commands/db.py @@ -258,7 +258,7 @@ def run(self, name: NameArg) -> None: class DbCommands(CommandGroup): - name, help, panel = "db", "Manage an agent's databases.", "Configuration" + name, help, panel = "db", "Manage an agent's databases.", "Components" def __init__( self, diff --git a/commands/lifecycle.py b/commands/lifecycle.py index dc2adab..d0583ff 100644 --- a/commands/lifecycle.py +++ b/commands/lifecycle.py @@ -45,8 +45,18 @@ class StopCommand(_ComposeCommand): class RestartCommand(_ComposeCommand): - name, help = "restart", "Restart a Portabase component." - verb, compose_args, done = "Restarting", ["restart"], "Restarted" + name, help = "restart", "Restart a Portabase component, applying config changes." + verb, compose_args, done = "Restarting", ["up", "-d"], "Restarted" + + def run(self, path: PathArg) -> None: + path = self.require_project_dir(path) + self.require_docker(self.docker) + with self.ui.status(f"{self.verb} {path.name}..."): + # `compose restart` neither creates services added since the last + # start nor rereads env_file; `up -d` converges first. + self.docker.compose(path, ["up", "-d"]) + self.docker.compose(path, ["restart"]) + self.ui.success(self.done) class LogsCommand(Command): diff --git a/main.py b/main.py index fe5da03..22ecffd 100644 --- a/main.py +++ b/main.py @@ -1,5 +1,6 @@ import os import platform +import re import sys from dataclasses import dataclass from typing import Annotated @@ -7,11 +8,11 @@ import click import typer -from commands.agent import AgentCommand +from commands.agent import AgentCommands +from commands.base import DeprecatedAlias from commands.build import BuildCommand from commands.config import ConfigCommands from commands.dashboard import DashboardCommand -from commands.db import DbCommands from commands.decrypt import DecryptCommand from commands.lifecycle import ( LogsCommand, @@ -120,11 +121,10 @@ def root( ui.out(ctx.get_help() + "\n") raise typer.Exit() + agent = AgentCommands( + ui, telemetry, docker, templates, renderer, engine_registry, ports + ) commands = [ - AgentCommand( - ui, telemetry, docker, templates, renderer, engine_registry, ports - ), - DashboardCommand(ui, telemetry, docker, templates, renderer, ports), StartCommand(ui, telemetry, docker), StopCommand(ui, telemetry, docker), RestartCommand(ui, telemetry, docker), @@ -134,16 +134,23 @@ def root( DecryptCommand(ui, telemetry), UpdateCommand(ui, telemetry, checker, updater), ] + agent.register(app) + DashboardCommand(ui, telemetry, docker, templates, renderer, ports).register(app) for cmd in commands: cmd.register(app) - - DbCommands( - ui, telemetry, engine_registry, ports, templates, renderer, docker - ).register(app) + DeprecatedAlias(ui, telemetry, agent.db, name="db", use="agent db").register(app) ConfigCommands(ui, telemetry, config).register(app) return app, checker +def _usage_hint(error: click.UsageError) -> str: + group = error.ctx.command.name if error.ctx and error.ctx.command else None + match = re.match(r"No such command '(.+)'", error.format_message()) + if group in ("agent", "dashboard") and match: + return f"Did you mean: portabase {group} create {match.group(1)}?" + return "Run 'portabase --help' for usage." + + def _notify_update( ui: UI, checker: UpdateChecker, settings: Settings, invoked: str | None ) -> None: @@ -188,9 +195,7 @@ def main() -> None: except click.exceptions.Exit as e: exit_code = e.exit_code except click.UsageError as e: - err = ValidationError( - e.format_message(), hint="Run 'portabase --help' for usage." - ) + err = ValidationError(e.format_message(), hint=_usage_hint(e)) ui.error(err) telemetry.error(err) exit_code = err.exit_code diff --git a/services/project.py b/services/project.py index 9436beb..0dd63dd 100644 --- a/services/project.py +++ b/services/project.py @@ -148,7 +148,7 @@ def find(self, id_or_name: str) -> DatabaseSpec: ] if not matches: raise ValidationError( - f"No database matching '{id_or_name}'.", hint="See: portabase db list" + f"No database matching '{id_or_name}'.", hint="See: portabase agent db list" ) if len(matches) > 1: raise ValidationError( diff --git a/ui/components/hints.py b/ui/components/hints.py index 2b6e834..0bb059b 100644 --- a/ui/components/hints.py +++ b/ui/components/hints.py @@ -7,7 +7,7 @@ HINTS = [ "The Edge Key contains the connection details for dashboard and agent communication.", "Portabase uses Docker Compose to isolate your databases.", - "List every configured database with 'portabase db list '.", + "List every configured database with 'portabase agent db list '.", "Running 'portabase stop' will gracefully shut down your containers.", "The agent polls GitHub for configuration updates.", "Logs can be viewed in real time with 'portabase logs '.", From 6627fc8051014a485b044b20acdaa0da8c7671fa Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Th=C3=A9o=20LAGACHE?= Date: Fri, 11 Sep 2026 14:49:02 +0200 Subject: [PATCH 12/24] feat(dashboard): settings registry, auth providers and lock-out checks A declarative registry maps 18 dashboard settings to their .env variable, type, default and prompt; flags, prompts, validation and display derive from it. OIDC and OAuth providers are stored by prefix in .env and read back by scanning it, the same way managed databases are. DashboardProject refuses three states that would leave an instance unreachable: skipping onboarding without an initial account, disabling password login with no provider, and a provider whose callback would point at localhost. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01CbemoDqsmMdRq6PS5Vaup8 --- services/dashboard_settings.py | 260 +++++++++++++++++++++++++++++++++ services/project.py | 107 +++++++++++++- 2 files changed, 366 insertions(+), 1 deletion(-) create mode 100644 services/dashboard_settings.py diff --git a/services/dashboard_settings.py b/services/dashboard_settings.py new file mode 100644 index 0000000..4065c81 --- /dev/null +++ b/services/dashboard_settings.py @@ -0,0 +1,260 @@ +from __future__ import annotations + +import re +from dataclasses import dataclass +from typing import Any, Literal + +from core.errors import ValidationError +from core.fields import Field + +Section = Literal["network", "api", "onboarding", "auth"] +SECTION_TITLES: dict[Section, str] = { + "network": "Network", + "api": "API & MCP", + "onboarding": "Onboarding", + "auth": "Authentication", +} + + +def strong_password(value: str) -> str: + checks = ( + (len(value) >= 8, "at least 8 characters"), + (re.search(r"[a-z]", value), "a lowercase letter"), + (re.search(r"[A-Z]", value), "an uppercase letter"), + (re.search(r"\d", value), "a digit"), + (re.search(r"[^A-Za-z0-9]", value), "a special character"), + ) + missing = [label for ok, label in checks if not ok] + if missing: + raise ValidationError( + "Password too weak.", hint="It needs " + ", ".join(missing) + "." + ) + return value + + +def public_url(value: str) -> str: + if not re.match(r"^https?://[^/\s]+", value): + raise ValidationError( + f"Invalid URL: {value!r}", hint="Expected http(s)://host[:port]" + ) + return value.rstrip("/") + + +@dataclass(frozen=True) +class Setting: + field: Field + env: str + section: Section + secret: bool = False + + @property + def name(self) -> str: + return self.field.name + + +SETTINGS: tuple[Setting, ...] = ( + Setting( + Field( + "url", + "Public URL", + "text", + validator=public_url, + help="Used for links and OAuth/OIDC callbacks.", + ), + "PROJECT_URL", + "network", + ), + Setting( + Field("behind_proxy", "Behind a reverse proxy?", "bool", default=False), + "TUSD_BEHIND_PROXY", + "network", + ), + Setting( + Field("trusted_domains", "Trusted domains (comma-separated)", "text"), + "TRUSTED_DOMAINS", + "network", + ), + Setting( + Field("api", "Enable the REST API (/api/v1)?", "bool", default=False), + "API_ENABLED", + "api", + ), + Setting( + Field("openapi", "Enable OpenAPI spec and Swagger UI?", "bool", default=False), + "OPENAPI_ENABLED", + "api", + ), + Setting( + Field("mcp", "Enable the MCP server (/api/v1/mcp)?", "bool", default=False), + "MCP_ENABLED", + "api", + ), + Setting( + Field( + "skip_onboarding", + "Skip the onboarding wizard?", + "bool", + default=False, + help="Requires an initial account: admin_email and admin_password.", + ), + "SKIP_ONBOARDING", + "onboarding", + ), + Setting( + Field("admin_name", "Initial user name", "text"), + "AUTH_DEFAULT_USER_NAME", + "onboarding", + ), + Setting( + Field("admin_email", "Initial user email", "text"), + "AUTH_DEFAULT_USER", + "onboarding", + ), + Setting( + Field( + "admin_password", + "Initial user password", + "secret", + validator=strong_password, + ), + "AUTH_DEFAULT_PASSWORD", + "onboarding", + secret=True, + ), + Setting( + Field( + "password_auth", + "Allow email/password login?", + "bool", + default=True, + help="Disable only with at least one OIDC or OAuth provider configured.", + ), + "AUTH_EMAIL_PASSWORD_ENABLED", + "auth", + ), + Setting( + Field("signup", "Allow self sign-up?", "bool"), + "AUTH_SIGNUP_ENABLED", + "auth", + ), + Setting( + Field("passkey", "Allow passkey login?", "bool"), + "AUTH_PASSKEY_ENABLED", + "auth", + ), + Setting( + Field("account_linking", "Allow linking provider accounts?", "bool"), + "AUTH_ALLOW_LINKING", + "auth", + ), + Setting( + Field("account_unlinking", "Allow unlinking provider accounts?", "bool"), + "AUTH_ALLOW_UNLINKING", + "auth", + ), + Setting( + Field("sync_oidc_roles", "Sync roles from OIDC on login?", "bool"), + "AUTH_SYNC_OIDC_ROLES_ON_LOGIN", + "auth", + ), + Setting( + Field("role_map", "Role map (remote:portabase,...)", "text"), + "AUTH_ROLE_MAP", + "auth", + ), + Setting( + Field("allowed_group", "Restrict access to this group", "text"), + "ALLOWED_GROUP", + "auth", + ), +) + +BY_NAME: dict[str, Setting] = {s.name: s for s in SETTINGS} + +WIZARD_SECTIONS: tuple[tuple[Section, tuple[str, ...]], ...] = ( + ("api", ("api", "openapi", "mcp")), + ("onboarding", ("skip_onboarding", "admin_name", "admin_email", "admin_password")), + ("auth", ("password_auth", "signup", "passkey")), +) + + +def get(name: str) -> Setting: + try: + return BY_NAME[name] + except KeyError: + raise ValidationError( + f"Unknown setting '{name}'.", + hint="Known: " + ", ".join(BY_NAME), + ) from None + + +def to_env(setting: Setting, value: Any) -> str: + if setting.field.kind == "bool": + return "true" if value else "false" + return str(value) + + +def from_env(setting: Setting, raw: str | None) -> Any: + if raw is None: + return setting.field.default + if setting.field.kind == "bool": + return raw.strip().lower() in ("1", "true", "yes", "on") + return raw + + +OAUTH_PROVIDERS: tuple[str, ...] = ( + "google", + "github", + "discord", + "apple", + "linkedin", + "x", + "reddit", +) + +OIDC_FIELDS: tuple[Field, ...] = ( + Field("issuer", "Issuer / discovery URL", "text", validator=public_url), + Field("client", "Client ID", "text"), + Field("secret", "Client secret", "secret"), + Field("title", "Display name", "text", default=""), + Field("scopes", "Scopes", "text", default=""), + Field("pkce", "Use PKCE?", "bool", default=False), + Field("host", "Host override", "text", default=""), +) + +OAUTH_FIELDS: tuple[Field, ...] = ( + Field("client", "Client ID", "text"), + Field("secret", "Client secret", "secret"), + Field("title", "Display name", "text", default=""), +) + +OIDC_ENV: dict[str, str] = { + "issuer": "ISSUER_URL", + "client": "CLIENT", + "secret": "SECRET", + "title": "TITLE", + "scopes": "SCOPES", + "pkce": "PKCE", + "host": "HOST", +} +OAUTH_ENV: dict[str, str] = {"client": "CLIENT", "secret": "SECRET", "title": "TITLE"} + + +def provider_prefix(kind: str, provider_id: str) -> str: + slug = re.sub(r"[^A-Z0-9]", "_", provider_id.upper()) + return f"AUTH_OIDC_{slug}" if kind == "oidc" else f"AUTH_SOCIAL_{slug}" + + +def validate_provider_id(kind: str, provider_id: str) -> str: + pid = provider_id.strip().lower() + if not re.match(r"^[a-z0-9][a-z0-9-]*$", pid): + raise ValidationError( + f"Invalid provider id {provider_id!r}.", + hint="Use lowercase letters, digits and dashes.", + ) + if kind == "oauth" and pid not in OAUTH_PROVIDERS: + raise ValidationError( + f"Unknown OAuth provider '{pid}'.", + hint="Supported: " + ", ".join(OAUTH_PROVIDERS), + ) + return pid diff --git a/services/project.py b/services/project.py index 0dd63dd..171f010 100644 --- a/services/project.py +++ b/services/project.py @@ -9,10 +9,12 @@ from core.specs import DatabaseSpec from engines.base import DbEngine from engines.sqlite import SqliteEngine +from services import dashboard_settings as ds from services.compose_facts import ComposeFacts from services.envfile import EnvFile ProjectKind = Literal["agent", "dashboard"] +ProviderKind = Literal["oidc", "oauth"] DATABASES_FILE = "databases.json" COMPOSE_FILE = "docker-compose.yml" ENV_FILE = ".env" @@ -148,7 +150,8 @@ def find(self, id_or_name: str) -> DatabaseSpec: ] if not matches: raise ValidationError( - f"No database matching '{id_or_name}'.", hint="See: portabase agent db list" + f"No database matching '{id_or_name}'.", + hint="See: portabase agent db list", ) if len(matches) > 1: raise ValidationError( @@ -194,5 +197,107 @@ def db_mode(self) -> Literal["external", "internal", "custom"]: def project_name(self) -> str: return self.env.get("PROJECT_NAME") or self.path.name + def setting(self, name: str) -> Any: + setting = ds.get(name) + return ds.from_env(setting, self.env.get(setting.env)) + + def settings(self) -> dict[str, Any]: + return {s.name: ds.from_env(s, self.env.get(s.env)) for s in ds.SETTINGS} + + def set(self, name: str, value: Any) -> None: + setting = ds.get(name) + self.env.set(setting.env, ds.to_env(setting, value)) + + def unset(self, name: str) -> None: + self.env.remove(ds.get(name).env) + + @property + def providers(self) -> list[AuthProvider]: + found: dict[tuple[ProviderKind, str], dict[str, str]] = {} + kinds: tuple[tuple[ProviderKind, str], ...] = ( + ("oidc", "AUTH_OIDC_"), + ("oauth", "AUTH_SOCIAL_"), + ) + for key, value in self.env.as_dict().items(): + for kind, prefix in kinds: + if not key.startswith(prefix): + continue + rest = key[len(prefix) :] + env_map = ds.OIDC_ENV if kind == "oidc" else ds.OAUTH_ENV + for field_name, suffix in env_map.items(): + if rest.endswith("_" + suffix): + slug = rest[: -len(suffix) - 1] + found.setdefault((kind, slug), {})[field_name] = value + break + else: + if kind == "oidc" and rest.endswith("_ID"): + found.setdefault((kind, rest[:-3]), {})["id"] = value + providers = [] + for (kind, slug), values in sorted(found.items()): + pid = values.pop("id", slug.lower().replace("_", "-")) + providers.append(AuthProvider(kind=kind, id=pid, values=values)) + return providers + + def add_provider(self, provider: AuthProvider) -> None: + if any(p.id == provider.id for p in self.providers): + raise ValidationError( + f"A provider named '{provider.id}' already exists.", + hint="Remove it first: portabase dashboard auth remove", + ) + prefix = ds.provider_prefix(provider.kind, provider.id) + env_map = ds.OIDC_ENV if provider.kind == "oidc" else ds.OAUTH_ENV + if provider.kind == "oidc": + self.env.set(f"{prefix}_ID", provider.id) + for field_name, value in provider.values.items(): + if value in ("", None, False): + continue + raw = "true" if value is True else str(value) + self.env.set(f"{prefix}_{env_map[field_name]}", raw) + + def remove_provider(self, provider_id: str) -> AuthProvider: + match = next((p for p in self.providers if p.id == provider_id), None) + if match is None: + raise ValidationError( + f"No provider named '{provider_id}'.", + hint="See: portabase dashboard auth list", + ) + self.env.remove_prefix(ds.provider_prefix(match.kind, match.id)) + return match + + def callback_url(self, provider_id: str) -> str: + return f"{self.setting('url')}/api/auth/sso/callback/{provider_id}" + + def validate(self) -> None: + values = self.settings() + providers = self.providers + if values["skip_onboarding"] and not ( + values["admin_email"] and values["admin_password"] + ): + raise ValidationError( + "Skipping onboarding needs an initial account.", + hint="Set admin_email and admin_password.", + ) + if not values["password_auth"] and not providers: + raise ValidationError( + "Disabling password login with no OIDC or OAuth provider " + "would lock everyone out.", + hint="Add a provider first: portabase dashboard auth add", + ) + url = values["url"] or "" + if providers and ("localhost" in url or "127.0.0.1" in url): + raise ValidationError( + f"Login providers need a public URL for their callback " + f"(currently {url}).", + hint="portabase dashboard set NAME url https://your.domain", + ) + def save_state(self) -> None: + self.validate() self.env.save() + + +@dataclass(frozen=True) +class AuthProvider: + kind: ProviderKind + id: str + values: dict[str, Any] From 00d0e794225b3cc3b977e8a1737fe47b6ea25ac9 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Th=C3=A9o=20LAGACHE?= Date: Fri, 11 Sep 2026 14:53:37 +0200 Subject: [PATCH 13/24] feat(dashboard): create/show/set/unset and auth add/list/remove MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 'dashboard' is now a group. create derives its settings flags from the registry — --api, --mcp, --skip-onboarding, --admin-password-stdin, --password-auth/--no-password-auth and the rest — so a new setting is one registry entry. In interactive mode the wizard offers the API, onboarding and authentication sections behind a single question, off by default. show prints settings by section with secrets masked, plus the providers and the callback URL each one needs registered. set validates keys and values through the same registry; unset returns to the dashboard default. auth add|list|remove manage OIDC and OAuth providers stored by prefix in .env. OAuth ids are limited to the names the dashboard recognizes. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01CbemoDqsmMdRq6PS5Vaup8 --- README.md | 15 +- commands/dashboard.py | 273 +++++++++++++++++++++++++++++++++++-- commands/dashboard_auth.py | 203 +++++++++++++++++++++++++++ main.py | 4 +- 4 files changed, 478 insertions(+), 17 deletions(-) create mode 100644 commands/dashboard_auth.py diff --git a/README.md b/README.md index d59ef84..94d171d 100644 --- a/README.md +++ b/README.md @@ -64,10 +64,23 @@ Distributed under the Apache License. See `LICENSE.txt` for more details. +## Commands + +``` +portabase agent create NAME create an agent folder +portabase agent db add|remove|list NAME +portabase dashboard create NAME create a dashboard folder +portabase dashboard show|set|unset NAME +portabase dashboard auth add|list|remove NAME +portabase start|stop|restart|logs|uninstall|build PATH +``` + +`portabase db` still works for one release as an alias of `portabase agent db`. + ## Upgrading from 26.08 or earlier From this release the CLI owns `docker-compose.yml`: it is re-rendered from your -`.env` and `databases.json` whenever you run `portabase db add`, `db remove` or +`.env` and `databases.json` whenever you run `portabase agent db add`, `agent db remove` or `build`. The first time that happens on an older install, the existing file is copied to `docker-compose.legacy.yml` first. diff --git a/commands/dashboard.py b/commands/dashboard.py index 3d048b3..acd214b 100644 --- a/commands/dashboard.py +++ b/commands/dashboard.py @@ -1,23 +1,29 @@ from __future__ import annotations +import inspect import secrets import sys from pathlib import Path -from typing import Annotated +from typing import Annotated, Any from urllib.parse import quote import typer -from commands.base import Command +from commands.base import Command, CommandGroup +from commands.dashboard_auth import DashboardAuthCommands from commands.db import report_write +from core.errors import ValidationError from core.utils import generate_password, slugify_project_name +from services import dashboard_settings as ds from services.docker import DockerRunner +from services.envfile import EnvFile from services.ports import PortAllocator from services.project import DashboardProject from services.renderer import ComposeRenderer from services.telemetry import Telemetry from services.templates import TemplateRepository from ui import UI +from ui.form import Form DB_MODES = ("external", "internal", "custom") MODE_LABELS = { @@ -25,12 +31,69 @@ "internal": "Embedded Database (In-container)", "custom": "Custom/Existing Database", } +PathArg = Annotated[Path, typer.Argument(help="Dashboard folder")] -class DashboardCommand(Command): - name = "dashboard" - help = "Create a new Portabase Dashboard instance." - panel = "Creation" +def _flag(name: str) -> str: + return "--" + name.replace("_", "-") + + +def settings_parameters() -> list[inspect.Parameter]: + params: list[inspect.Parameter] = [] + for setting in ds.SETTINGS: + field = setting.field + flag = _flag(setting.name) + if field.kind == "bool": + ann: Any = Annotated[ + bool | None, typer.Option(f"{flag}/--no-{flag[2:]}", help=field.prompt) + ] + else: + note = " (prefer the -stdin variant)" if setting.secret else "" + ann = Annotated[str | None, typer.Option(flag, help=field.prompt + note)] + params.append( + inspect.Parameter( + setting.name, + inspect.Parameter.KEYWORD_ONLY, + default=None, + annotation=ann, + ) + ) + if setting.secret: + params.append( + inspect.Parameter( + f"{setting.name}_stdin", + inspect.Parameter.KEYWORD_ONLY, + default=False, + annotation=Annotated[ + bool, + typer.Option( + f"{flag}-stdin", + help=f"Read {field.prompt.lower()} from stdin", + ), + ], + ) + ) + return params + + +def read_secret_flags(values: dict[str, Any]) -> dict[str, Any]: + out = dict(values) + for setting in ds.SETTINGS: + if setting.secret and out.pop(f"{setting.name}_stdin", False): + out[setting.name] = sys.stdin.readline().rstrip("\n") + return out + + +def display(name: str, value: Any) -> str: + if ds.get(name).secret: + return "••••••••" + if isinstance(value, bool): + return "Yes" if value else "No" + return str(value) + + +class _DashboardCommand(Command): + panel = "Components" no_args_is_help = True def __init__( @@ -48,6 +111,41 @@ def __init__( self.renderer = renderer self.ports = ports + def write(self, project: DashboardProject) -> None: + with self.ui.status("Rendering configuration..."): + result = self.renderer.render_dashboard(project) + project.save_state() + report = result.write(project.path) + report_write(self.ui, report) + + def apply_settings(self, project: DashboardProject, values: dict[str, Any]) -> None: + form = self.ui.form() + for name, raw in values.items(): + if raw is not None: + project.set(name, form.ask(ds.get(name).field, raw)) + + +class DashboardCreateCommand(_DashboardCommand): + name, help = "create", "Create a new Portabase Dashboard instance." + + def register(self, app: typer.Typer) -> None: + def entry(*args: Any, **kwargs: Any) -> None: + self.run(*args, **kwargs) + + static = [ + p + for p in inspect.signature(self.run, eval_str=True).parameters.values() + if p.kind is not inspect.Parameter.VAR_KEYWORD + ] + signature = inspect.Signature(static + settings_parameters()) + entry.__signature__ = signature # type: ignore[attr-defined] + entry.__annotations__ = { + name: param.annotation for name, param in signature.parameters.items() + } + app.command( + self.name, help=self.help, rich_help_panel=self.panel, no_args_is_help=True + )(self._traced(entry)) + def run( self, name: Annotated[str, typer.Argument(help="Dashboard name (creates a folder)")], @@ -84,6 +182,7 @@ def run( bool, typer.Option("--yes", "-y", help="Skip the configuration confirmation"), ] = False, + **settings: Any, ) -> None: self.ui.banner() self.require_docker(self.docker) @@ -116,7 +215,6 @@ def run( rows = [ ("Dashboard Name", name), ("Path", str(path)), - ("Access URL", env_vars["PROJECT_URL"]), ("Database Setup", MODE_LABELS[mode]), ] @@ -145,6 +243,28 @@ def run( ("Connection URL", env_vars["DATABASE_URL"]), ] + env = EnvFile(path / ".env") + env.merge(env_vars) + project = DashboardProject(path, env) + + provided = read_secret_flags(settings) + self.apply_settings(project, provided) + explicit = any(v is not None for v in provided.values()) + if ( + not self.ui.non_interactive + and not explicit + and self.ui.confirm( + "Configure API, MCP and authentication now?", default=False + ) + ): + self._wizard(form, project) + + rows.append(("Access URL", project.setting("url"))) + rows += [ + (ds.get(k).field.prompt, display(k, v)) + for k, v in project.settings().items() + if k != "url" and project.env.get(ds.get(k).env) is not None + ] rows.append(("Files to Create", "docker-compose.yml, .env")) self.ui.summary(rows, title="SUMMARY") if not yes: @@ -152,13 +272,12 @@ def run( "Apply this configuration and generate files?", default=True ) - project = DashboardProject.create(path, env_vars) - with self.ui.status("Rendering configuration..."): - result = self.renderer.render_dashboard(project) - project.save_state() - report = result.write(path) - report_write(self.ui, report) + path.mkdir(parents=True, exist_ok=True) + self.write(project) self.ui.success(f"Dashboard '{name}' created in {path}") + self.ui.hint( + f"Add a login provider with: portabase dashboard auth add {name} oidc ..." + ) if start or ( not self.ui.non_interactive @@ -166,10 +285,24 @@ def run( ): with self.ui.status("Starting..."): self.docker.compose(path, ["up", "-d"]) - self.ui.success(f"Live at: {env_vars['PROJECT_URL']}") + self.ui.success(f"Live at: {project.setting('url')}") else: self.ui.info(f"Run: portabase start {name}") + def _wizard(self, form: Form, project: DashboardProject) -> None: + for section, names in ds.WIZARD_SECTIONS: + self.ui.section(ds.SECTION_TITLES[section]) + for setting_name in names: + needs_account = ( + section == "onboarding" and setting_name != "skip_onboarding" + ) + if needs_account and not project.setting("skip_onboarding"): + continue + setting = ds.get(setting_name) + value = form.ask(setting.field) + if value != setting.field.default or needs_account: + project.set(setting_name, value) + @staticmethod def _pg_env( db: str, user: str, password: str, host: str, port: int, host_port: int @@ -186,3 +319,115 @@ def _pg_env( "DATABASE_URL": url, "PG_PORT": str(host_port), } + + +class DashboardShowCommand(_DashboardCommand): + name, help = "show", "Show a dashboard's settings and login providers." + + def run(self, path: PathArg) -> None: + project = DashboardProject.load(self.require_project_dir(path)) + values = project.settings() + for section, title in ds.SECTION_TITLES.items(): + rows = [ + (s.field.prompt, display(s.name, values[s.name])) + for s in ds.SETTINGS + if s.section == section and values[s.name] not in (None, "") + ] + if rows: + self.ui.summary(rows, title=title.upper()) + providers = project.providers + if providers: + self.ui.table( + ["Kind", "Id", "Title", "Issuer / provider", "Callback"], + [ + [ + p.kind, + p.id, + p.values.get("title", ""), + p.values.get("issuer", p.id), + project.callback_url(p.id), + ] + for p in providers + ], + title="LOGIN PROVIDERS", + ) + else: + state = "enabled." if values["password_auth"] else "disabled!" + self.ui.hint(f"No login provider. Password login is {state}") + + +class DashboardSetCommand(_DashboardCommand): + name, help = "set", "Change dashboard settings: KEY VALUE [KEY VALUE ...]." + + def run( + self, + path: PathArg, + pairs: Annotated[ + list[str], + typer.Argument(help="KEY VALUE pairs; keys as in 'dashboard show'"), + ], + ) -> None: + if len(pairs) % 2: + raise ValidationError( + "Expected KEY VALUE pairs.", hint="Known keys: " + ", ".join(ds.BY_NAME) + ) + project_path = self.require_project_dir(path) + self.templates.resolve() + project = DashboardProject.load(project_path) + self.apply_settings(project, dict(zip(pairs[::2], pairs[1::2], strict=True))) + self.write(project) + for key in pairs[::2]: + self.ui.success(f"{key} = {display(key, project.setting(key))}") + self.ui.info(f"Apply with: portabase restart {project_path.name}") + + +class DashboardUnsetCommand(_DashboardCommand): + name, help = "unset", "Reset dashboard settings to their default: KEY [KEY ...]." + + def run( + self, + path: PathArg, + keys: Annotated[list[str], typer.Argument(help="Setting keys")], + ) -> None: + project_path = self.require_project_dir(path) + self.templates.resolve() + project = DashboardProject.load(project_path) + for key in keys: + project.unset(key) + self.write(project) + self.ui.success("Reset: " + ", ".join(keys)) + self.ui.info(f"Apply with: portabase restart {project_path.name}") + + +class DashboardCommands(CommandGroup): + name, help, panel = ( + "dashboard", + "Create and manage Portabase dashboards.", + "Components", + ) + + def __init__( + self, + ui: UI, + telemetry: Telemetry, + docker: DockerRunner, + templates: TemplateRepository, + renderer: ComposeRenderer, + ports: PortAllocator, + ) -> None: + super().__init__(ui, telemetry) + self._deps = (ui, telemetry, docker, templates, renderer, ports) + self.auth = DashboardAuthCommands(*self._deps) + + @property + def commands(self) -> list[Command]: + return [ + DashboardCreateCommand(*self._deps), + DashboardShowCommand(*self._deps), + DashboardSetCommand(*self._deps), + DashboardUnsetCommand(*self._deps), + ] + + @property + def groups(self) -> list[CommandGroup]: + return [self.auth] diff --git a/commands/dashboard_auth.py b/commands/dashboard_auth.py new file mode 100644 index 0000000..0c919d0 --- /dev/null +++ b/commands/dashboard_auth.py @@ -0,0 +1,203 @@ +from __future__ import annotations + +import sys +from pathlib import Path +from typing import Annotated, Any + +import typer + +from commands.base import Command, CommandGroup +from commands.db import report_write +from core.errors import ValidationError +from services import dashboard_settings as ds +from services.docker import DockerRunner +from services.ports import PortAllocator +from services.project import AuthProvider, DashboardProject, ProviderKind +from services.renderer import ComposeRenderer +from services.telemetry import Telemetry +from services.templates import TemplateRepository +from ui import UI + +PathArg = Annotated[Path, typer.Argument(help="Dashboard folder")] +KINDS: tuple[ProviderKind, ...] = ("oidc", "oauth") + + +class _AuthCommand(Command): + panel = "Components" + no_args_is_help = True + + def __init__( + self, + ui: UI, + telemetry: Telemetry, + docker: DockerRunner, + templates: TemplateRepository, + renderer: ComposeRenderer, + ports: PortAllocator, + ) -> None: + super().__init__(ui, telemetry) + self.templates = templates + self.renderer = renderer + + def load(self, path: Path) -> DashboardProject: + project_path = self.require_project_dir(path) + self.templates.resolve() + return DashboardProject.load(project_path) + + def write(self, project: DashboardProject) -> None: + with self.ui.status("Rendering configuration..."): + result = self.renderer.render_dashboard(project) + project.save_state() + report = result.write(project.path) + report_write(self.ui, report) + self.ui.info(f"Apply with: portabase restart {project.path.name}") + + +class AuthAddCommand(_AuthCommand): + name, help = "add", "Add an OIDC or OAuth login provider." + + def run( + self, + path: PathArg, + kind: Annotated[str, typer.Argument(help="oidc | oauth")], + provider_id: Annotated[ + str, + typer.Argument( + help="Provider id: any slug for oidc, a known name for oauth " + "(google, github, discord, apple, linkedin, x, reddit)" + ), + ], + client: Annotated[ + str | None, typer.Option("--client", help="Client ID") + ] = None, + secret: Annotated[ + str | None, + typer.Option("--secret", help="Client secret (prefer --secret-stdin)"), + ] = None, + secret_stdin: Annotated[ + bool, + typer.Option("--secret-stdin", help="Read the client secret from stdin"), + ] = False, + issuer: Annotated[ + str | None, typer.Option("--issuer", help="OIDC issuer / discovery URL") + ] = None, + title: Annotated[ + str | None, typer.Option("--title", help="Display name") + ] = None, + scopes: Annotated[ + str | None, typer.Option("--scopes", help="OIDC scopes") + ] = None, + pkce: Annotated[ + bool | None, typer.Option("--pkce/--no-pkce", help="OIDC: use PKCE") + ] = None, + host: Annotated[ + str | None, typer.Option("--host", help="OIDC host override") + ] = None, + ) -> None: + if kind not in KINDS: + raise ValidationError(f"Unknown kind '{kind}'.", hint="Use oidc or oauth.") + pid = ds.validate_provider_id(kind, provider_id) + if secret_stdin: + secret = sys.stdin.readline().rstrip("\n") + elif secret is not None: + self.ui.warning( + "--secret is visible in shell history; prefer --secret-stdin." + ) + + values: dict[str, Any] = { + "client": client, + "secret": secret, + "issuer": issuer, + "title": title, + "scopes": scopes, + "pkce": pkce, + "host": host, + } + fields = ds.OIDC_FIELDS if kind == "oidc" else ds.OAUTH_FIELDS + allowed = {f.name for f in fields} + stray = sorted( + k for k, v in values.items() if v is not None and k not in allowed + ) + if stray: + flags = ", ".join("--" + k for k in stray) + raise ValidationError(f"Not applicable to {kind}: {flags}.") + + project = self.load(path) + answers = self.ui.form().collect(list(fields), values) + provider = AuthProvider(kind=kind, id=pid, values=answers) + project.add_provider(provider) + self.write(project) + self.ui.success(f"Added {kind} provider '{pid}'.") + self.ui.info( + f"Callback URL to register at the provider: {project.callback_url(pid)}" + ) + + +class AuthListCommand(_AuthCommand): + name, help = "list", "List login providers." + + def run(self, path: PathArg) -> None: + project = self.load(path) + providers = project.providers + if not providers: + self.ui.warning("No login provider configured.") + return + self.ui.table( + ["Kind", "Id", "Title", "Issuer / provider", "Callback"], + [ + [ + p.kind, + p.id, + p.values.get("title", ""), + p.values.get("issuer", p.id), + project.callback_url(p.id), + ] + for p in providers + ], + title=f"Login providers for {project.path.name}", + ) + + +class AuthRemoveCommand(_AuthCommand): + name, help = "remove", "Remove a login provider." + + def run( + self, + path: PathArg, + provider_id: Annotated[str, typer.Argument(help="Provider id")], + yes: Annotated[ + bool, typer.Option("--yes", "-y", help="Skip confirmation") + ] = False, + ) -> None: + project = self.load(path) + if not yes: + self.confirm_or_abort( + f"Remove login provider '{provider_id}'?", default=False + ) + removed = project.remove_provider(provider_id) + self.write(project) + self.ui.success(f"Removed {removed.kind} provider '{removed.id}'.") + + +class DashboardAuthCommands(CommandGroup): + name, help, panel = "auth", "Manage a dashboard's login providers.", "Components" + + def __init__( + self, + ui: UI, + telemetry: Telemetry, + docker: DockerRunner, + templates: TemplateRepository, + renderer: ComposeRenderer, + ports: PortAllocator, + ) -> None: + super().__init__(ui, telemetry) + self._deps = (ui, telemetry, docker, templates, renderer, ports) + + @property + def commands(self) -> list[Command]: + return [ + AuthAddCommand(*self._deps), + AuthListCommand(*self._deps), + AuthRemoveCommand(*self._deps), + ] diff --git a/main.py b/main.py index 22ecffd..67ba840 100644 --- a/main.py +++ b/main.py @@ -12,7 +12,7 @@ from commands.base import DeprecatedAlias from commands.build import BuildCommand from commands.config import ConfigCommands -from commands.dashboard import DashboardCommand +from commands.dashboard import DashboardCommands from commands.decrypt import DecryptCommand from commands.lifecycle import ( LogsCommand, @@ -135,7 +135,7 @@ def root( UpdateCommand(ui, telemetry, checker, updater), ] agent.register(app) - DashboardCommand(ui, telemetry, docker, templates, renderer, ports).register(app) + DashboardCommands(ui, telemetry, docker, templates, renderer, ports).register(app) for cmd in commands: cmd.register(app) DeprecatedAlias(ui, telemetry, agent.db, name="db", use="agent db").register(app) From 5e0188c16aa2447063c8eb494852483f6e9714a5 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Th=C3=A9o=20LAGACHE?= Date: Fri, 11 Sep 2026 15:02:20 +0200 Subject: [PATCH 14/24] feat(agent): show/set/unset, and every documented agent variable MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The settings registry now serves both components. The agent gains the five variables its documentation lists but the CLI never exposed — DATA_PATH, TMPDIR, RETRY_ATTEMPTS, RETRY_BACKOFF_MS, SSL_CERT_FILE — plus LOG_LEVEL as a choice; agent create derives its flags from the registry like dashboard create does. host_gateway is a compose fact rather than a variable, so the registry allows env-less settings that map to a project attribute. show, set and unset are one implementation shared by both components, parameterised by the project loader and renderer. The agent template renders whichever optional variables are set, so they reach the container without an env_file (which would also expose the managed database credentials). Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01CbemoDqsmMdRq6PS5Vaup8 --- README.md | 1 + commands/agent.py | 129 +++++++------ commands/dashboard.py | 175 ++++-------------- commands/dashboard_auth.py | 6 +- commands/settings.py | 199 ++++++++++++++++++++ services/auth_providers.py | 64 +++++++ services/dashboard_settings.py | 260 -------------------------- services/project.py | 57 ++++-- services/renderer.py | 3 + services/settings.py | 329 +++++++++++++++++++++++++++++++++ templates/agent.yml.j2 | 3 + 11 files changed, 750 insertions(+), 476 deletions(-) create mode 100644 commands/settings.py create mode 100644 services/auth_providers.py delete mode 100644 services/dashboard_settings.py create mode 100644 services/settings.py diff --git a/README.md b/README.md index 94d171d..7350344 100644 --- a/README.md +++ b/README.md @@ -68,6 +68,7 @@ Distributed under the Apache License. See `LICENSE.txt` for more details. ``` portabase agent create NAME create an agent folder +portabase agent show|set|unset NAME portabase agent db add|remove|list NAME portabase dashboard create NAME create a dashboard folder portabase dashboard show|set|unset NAME diff --git a/commands/agent.py b/commands/agent.py index 377b4bd..397f65c 100644 --- a/commands/agent.py +++ b/commands/agent.py @@ -1,16 +1,24 @@ from __future__ import annotations from pathlib import Path -from typing import Annotated +from typing import Annotated, Any import typer from commands.base import Command, CommandGroup from commands.db import DbCommands, report_write from commands.flows.add_database import AddDatabaseFlow -from core.errors import ValidationError -from core.utils import validate_edge_key +from commands.settings import ( + SetCommand, + UnsetCommand, + apply_settings, + display, + read_secret_flags, + show_settings, + with_settings_flags, +) from engines import EngineRegistry +from services import settings as cfg from services.docker import DockerRunner from services.ports import PortAllocator from services.project import AgentProject @@ -22,17 +30,8 @@ NETWORK = "portabase_network" -def _edge_key(value: str) -> str: - if not validate_edge_key(value): - raise ValidationError( - "Invalid Edge Key.", - hint="Expected Base64 or JSON with serverUrl, agentId, masterKeyB64.", - ) - return value - - class AgentCreateCommand(Command): - name, help, panel = "create", "Create a new Portabase Agent instance.", "Creation" + name, help, panel = "create", "Create a new Portabase Agent instance.", "Components" no_args_is_help = True def __init__( @@ -52,21 +51,14 @@ def __init__( self.engines = engines self.ports = ports + def register(self, app: typer.Typer) -> None: + app.command( + self.name, help=self.help, rich_help_panel=self.panel, no_args_is_help=True + )(self._traced(with_settings_flags(self.run, cfg.AGENT))) + def run( self, name: Annotated[str, typer.Argument(help="Agent name (creates a folder)")], - key: Annotated[str | None, typer.Option("--key", "-k", help="Edge Key")] = None, - tz: Annotated[str | None, typer.Option("--tz", help="Timezone")] = None, - polling: Annotated[ - int | None, typer.Option("--polling", help="Polling frequency in seconds") - ] = None, - host_gateway: Annotated[ - bool | None, - typer.Option( - "--host-gateway/--no-host-gateway", - help="Map localhost to host-gateway", - ), - ] = None, start: Annotated[ bool, typer.Option("--start", "-s", help="Start immediately") ] = False, @@ -77,6 +69,7 @@ def run( bool, typer.Option("--yes", "-y", help="Skip the configuration confirmation"), ] = False, + **settings: Any, ) -> None: self.ui.banner() self.require_docker(self.docker) @@ -88,47 +81,33 @@ def run( self.ui.warning(f"Directory '{name}' already exists.") self.confirm_or_abort("Overwrite?", default=False) + provided = read_secret_flags(cfg.AGENT, settings) form = self.ui.form() + answers = { + s.name: form.ask(s.field, provided.get(s.name)) for s in cfg.AGENT if s.core + } env_vars = { - "EDGE_KEY": form.text( - "Edge Key", value=key, validator=_edge_key, name="key" - ), - "TZ": form.text("Timezone", value=tz, default="UTC", name="tz"), - "POLLING": str( - form.integer( - "Polling frequency (seconds)", - value=polling, - default=5, - name="polling", - ) - ), - "LOG_LEVEL": "info", + s.env: s.to_env(answers[s.name]) for s in cfg.AGENT if s.core and s.env } - gateway = form.confirm( - "Add extra_hosts mapping (localhost -> host-gateway)?", - value=host_gateway, - default=False, - name="host_gateway", - ) - - self.ui.summary( - [ - ("Agent Name", name), - ("Path", str(path)), - ("Edge Key", env_vars["EDGE_KEY"]), - ("Timezone", env_vars["TZ"]), - ("Polling", f"{env_vars['POLLING']}s"), - ("Host Gateway", "Yes" if gateway else "No"), - ("Files to Create", "docker-compose.yml, .env, databases.json"), - ], - title="SUMMARY", - ) + gateway = bool(answers["host_gateway"]) + + rows = [("Agent Name", name), ("Path", str(path))] + rows += [ + (s.field.prompt, display(s, answers[s.name])) for s in cfg.AGENT if s.core + ] + rows.append(("Files to Create", "docker-compose.yml, .env, databases.json")) + self.ui.summary(rows, title="SUMMARY") if not yes: self.confirm_or_abort( "Apply this configuration and generate files?", default=True ) project = AgentProject.create(path, env_vars, host_gateway=gateway) + apply_settings( + self.ui, + project, + {k: v for k, v in provided.items() if not cfg.AGENT.get(k).core}, + ) self._write(project) self.ui.success(f"Agent '{name}' created in {path}") @@ -166,6 +145,27 @@ def _write(self, project: AgentProject) -> None: report_write(self.ui, report) +class AgentShowCommand(Command): + name, help, panel = "show", "Show an agent's settings and databases.", "Components" + no_args_is_help = True + + def __init__(self, ui: UI, telemetry: Telemetry, engines: EngineRegistry) -> None: + super().__init__(ui, telemetry) + self.engines = engines + + def run(self, path: Annotated[Path, typer.Argument(help="Agent folder")]) -> None: + project = AgentProject.load(self.require_project_dir(path)) + show_settings(self.ui, project) + if project.databases: + rows = [ + [d.name, d.engine, self.engines.get(d.engine).describe(d)] + for d in project.databases + ] + self.ui.table(["Name", "Engine", "Where"], rows, title="DATABASES") + else: + self.ui.hint("No database yet: portabase agent db add") + + class AgentCommands(CommandGroup): name, help, panel = "agent", "Create and manage Portabase agents.", "Components" @@ -183,11 +183,24 @@ def __init__( self._create = AgentCreateCommand( ui, telemetry, docker, templates, renderer, engines, ports ) + self._templates, self._renderer, self._engines = templates, renderer, engines self.db = DbCommands(ui, telemetry, engines, ports, templates, renderer, docker) @property def commands(self) -> list[Command]: - return [self._create] + shared = ( + self.ui, + self.telemetry, + self._templates, + AgentProject.load, + self._renderer.render_agent, + ) + return [ + self._create, + AgentShowCommand(self.ui, self.telemetry, self._engines), + SetCommand(*shared), + UnsetCommand(*shared), + ] @property def groups(self) -> list[CommandGroup]: diff --git a/commands/dashboard.py b/commands/dashboard.py index acd214b..1c198eb 100644 --- a/commands/dashboard.py +++ b/commands/dashboard.py @@ -1,6 +1,5 @@ from __future__ import annotations -import inspect import secrets import sys from pathlib import Path @@ -12,9 +11,17 @@ from commands.base import Command, CommandGroup from commands.dashboard_auth import DashboardAuthCommands from commands.db import report_write -from core.errors import ValidationError +from commands.settings import ( + SetCommand, + UnsetCommand, + apply_settings, + display, + read_secret_flags, + show_settings, + with_settings_flags, +) from core.utils import generate_password, slugify_project_name -from services import dashboard_settings as ds +from services import settings as cfg from services.docker import DockerRunner from services.envfile import EnvFile from services.ports import PortAllocator @@ -34,64 +41,6 @@ PathArg = Annotated[Path, typer.Argument(help="Dashboard folder")] -def _flag(name: str) -> str: - return "--" + name.replace("_", "-") - - -def settings_parameters() -> list[inspect.Parameter]: - params: list[inspect.Parameter] = [] - for setting in ds.SETTINGS: - field = setting.field - flag = _flag(setting.name) - if field.kind == "bool": - ann: Any = Annotated[ - bool | None, typer.Option(f"{flag}/--no-{flag[2:]}", help=field.prompt) - ] - else: - note = " (prefer the -stdin variant)" if setting.secret else "" - ann = Annotated[str | None, typer.Option(flag, help=field.prompt + note)] - params.append( - inspect.Parameter( - setting.name, - inspect.Parameter.KEYWORD_ONLY, - default=None, - annotation=ann, - ) - ) - if setting.secret: - params.append( - inspect.Parameter( - f"{setting.name}_stdin", - inspect.Parameter.KEYWORD_ONLY, - default=False, - annotation=Annotated[ - bool, - typer.Option( - f"{flag}-stdin", - help=f"Read {field.prompt.lower()} from stdin", - ), - ], - ) - ) - return params - - -def read_secret_flags(values: dict[str, Any]) -> dict[str, Any]: - out = dict(values) - for setting in ds.SETTINGS: - if setting.secret and out.pop(f"{setting.name}_stdin", False): - out[setting.name] = sys.stdin.readline().rstrip("\n") - return out - - -def display(name: str, value: Any) -> str: - if ds.get(name).secret: - return "••••••••" - if isinstance(value, bool): - return "Yes" if value else "No" - return str(value) - - class _DashboardCommand(Command): panel = "Components" no_args_is_help = True @@ -118,33 +67,14 @@ def write(self, project: DashboardProject) -> None: report = result.write(project.path) report_write(self.ui, report) - def apply_settings(self, project: DashboardProject, values: dict[str, Any]) -> None: - form = self.ui.form() - for name, raw in values.items(): - if raw is not None: - project.set(name, form.ask(ds.get(name).field, raw)) - class DashboardCreateCommand(_DashboardCommand): name, help = "create", "Create a new Portabase Dashboard instance." def register(self, app: typer.Typer) -> None: - def entry(*args: Any, **kwargs: Any) -> None: - self.run(*args, **kwargs) - - static = [ - p - for p in inspect.signature(self.run, eval_str=True).parameters.values() - if p.kind is not inspect.Parameter.VAR_KEYWORD - ] - signature = inspect.Signature(static + settings_parameters()) - entry.__signature__ = signature # type: ignore[attr-defined] - entry.__annotations__ = { - name: param.annotation for name, param in signature.parameters.items() - } app.command( self.name, help=self.help, rich_help_panel=self.panel, no_args_is_help=True - )(self._traced(entry)) + )(self._traced(with_settings_flags(self.run, cfg.DASHBOARD))) def run( self, @@ -247,8 +177,8 @@ def run( env.merge(env_vars) project = DashboardProject(path, env) - provided = read_secret_flags(settings) - self.apply_settings(project, provided) + provided = read_secret_flags(cfg.DASHBOARD, settings) + apply_settings(self.ui, project, provided) explicit = any(v is not None for v in provided.values()) if ( not self.ui.non_interactive @@ -261,9 +191,9 @@ def run( rows.append(("Access URL", project.setting("url"))) rows += [ - (ds.get(k).field.prompt, display(k, v)) - for k, v in project.settings().items() - if k != "url" and project.env.get(ds.get(k).env) is not None + (s.field.prompt, display(s, project.setting(s.name))) + for s in cfg.DASHBOARD + if s.name != "url" and project.env.get(s.env or "") is not None ] rows.append(("Files to Create", "docker-compose.yml, .env")) self.ui.summary(rows, title="SUMMARY") @@ -290,15 +220,15 @@ def run( self.ui.info(f"Run: portabase start {name}") def _wizard(self, form: Form, project: DashboardProject) -> None: - for section, names in ds.WIZARD_SECTIONS: - self.ui.section(ds.SECTION_TITLES[section]) + for section, names in cfg.DASHBOARD_WIZARD: + self.ui.section(cfg.DASHBOARD.sections[section]) for setting_name in names: needs_account = ( section == "onboarding" and setting_name != "skip_onboarding" ) if needs_account and not project.setting("skip_onboarding"): continue - setting = ds.get(setting_name) + setting = cfg.DASHBOARD.get(setting_name) value = form.ask(setting.field) if value != setting.field.default or needs_account: project.set(setting_name, value) @@ -326,15 +256,7 @@ class DashboardShowCommand(_DashboardCommand): def run(self, path: PathArg) -> None: project = DashboardProject.load(self.require_project_dir(path)) - values = project.settings() - for section, title in ds.SECTION_TITLES.items(): - rows = [ - (s.field.prompt, display(s.name, values[s.name])) - for s in ds.SETTINGS - if s.section == section and values[s.name] not in (None, "") - ] - if rows: - self.ui.summary(rows, title=title.upper()) + show_settings(self.ui, project) providers = project.providers if providers: self.ui.table( @@ -352,53 +274,10 @@ def run(self, path: PathArg) -> None: title="LOGIN PROVIDERS", ) else: - state = "enabled." if values["password_auth"] else "disabled!" + state = "enabled." if project.setting("password_auth") else "disabled!" self.ui.hint(f"No login provider. Password login is {state}") -class DashboardSetCommand(_DashboardCommand): - name, help = "set", "Change dashboard settings: KEY VALUE [KEY VALUE ...]." - - def run( - self, - path: PathArg, - pairs: Annotated[ - list[str], - typer.Argument(help="KEY VALUE pairs; keys as in 'dashboard show'"), - ], - ) -> None: - if len(pairs) % 2: - raise ValidationError( - "Expected KEY VALUE pairs.", hint="Known keys: " + ", ".join(ds.BY_NAME) - ) - project_path = self.require_project_dir(path) - self.templates.resolve() - project = DashboardProject.load(project_path) - self.apply_settings(project, dict(zip(pairs[::2], pairs[1::2], strict=True))) - self.write(project) - for key in pairs[::2]: - self.ui.success(f"{key} = {display(key, project.setting(key))}") - self.ui.info(f"Apply with: portabase restart {project_path.name}") - - -class DashboardUnsetCommand(_DashboardCommand): - name, help = "unset", "Reset dashboard settings to their default: KEY [KEY ...]." - - def run( - self, - path: PathArg, - keys: Annotated[list[str], typer.Argument(help="Setting keys")], - ) -> None: - project_path = self.require_project_dir(path) - self.templates.resolve() - project = DashboardProject.load(project_path) - for key in keys: - project.unset(key) - self.write(project) - self.ui.success("Reset: " + ", ".join(keys)) - self.ui.info(f"Apply with: portabase restart {project_path.name}") - - class DashboardCommands(CommandGroup): name, help, panel = ( "dashboard", @@ -421,11 +300,19 @@ def __init__( @property def commands(self) -> list[Command]: + ui, telemetry, _docker, templates, renderer, _ports = self._deps + shared = ( + ui, + telemetry, + templates, + DashboardProject.load, + renderer.render_dashboard, + ) return [ DashboardCreateCommand(*self._deps), DashboardShowCommand(*self._deps), - DashboardSetCommand(*self._deps), - DashboardUnsetCommand(*self._deps), + SetCommand(*shared), + UnsetCommand(*shared), ] @property diff --git a/commands/dashboard_auth.py b/commands/dashboard_auth.py index 0c919d0..461e296 100644 --- a/commands/dashboard_auth.py +++ b/commands/dashboard_auth.py @@ -9,7 +9,7 @@ from commands.base import Command, CommandGroup from commands.db import report_write from core.errors import ValidationError -from services import dashboard_settings as ds +from services import auth_providers as ap from services.docker import DockerRunner from services.ports import PortAllocator from services.project import AuthProvider, DashboardProject, ProviderKind @@ -96,7 +96,7 @@ def run( ) -> None: if kind not in KINDS: raise ValidationError(f"Unknown kind '{kind}'.", hint="Use oidc or oauth.") - pid = ds.validate_provider_id(kind, provider_id) + pid = ap.validate_provider_id(kind, provider_id) if secret_stdin: secret = sys.stdin.readline().rstrip("\n") elif secret is not None: @@ -113,7 +113,7 @@ def run( "pkce": pkce, "host": host, } - fields = ds.OIDC_FIELDS if kind == "oidc" else ds.OAUTH_FIELDS + fields = ap.OIDC_FIELDS if kind == "oidc" else ap.OAUTH_FIELDS allowed = {f.name for f in fields} stray = sorted( k for k, v in values.items() if v is not None and k not in allowed diff --git a/commands/settings.py b/commands/settings.py new file mode 100644 index 0000000..6638570 --- /dev/null +++ b/commands/settings.py @@ -0,0 +1,199 @@ +from __future__ import annotations + +import inspect +import sys +from collections.abc import Callable +from pathlib import Path +from typing import Annotated, Any, Protocol + +import typer + +from commands.base import Command +from commands.db import report_write +from core.errors import ValidationError +from services import settings as cfg +from services.renderer import RenderResult +from services.telemetry import Telemetry +from services.templates import TemplateRepository +from ui import UI + + +class SettingsProject(Protocol): + path: Path + registry: cfg.Registry + + def setting(self, name: str) -> Any: ... + def settings(self) -> dict[str, Any]: ... + def set(self, name: str, value: Any) -> None: ... + def unset(self, name: str) -> None: ... + def save_state(self) -> None: ... + + +def flag_of(name: str) -> str: + return "--" + name.replace("_", "-") + + +def settings_parameters(registry: cfg.Registry) -> list[inspect.Parameter]: + params: list[inspect.Parameter] = [] + for setting in registry: + field = setting.field + flag = flag_of(setting.name) + if field.kind == "bool": + ann: Any = Annotated[ + bool | None, typer.Option(f"{flag}/--no-{flag[2:]}", help=field.prompt) + ] + elif field.kind == "int": + ann = Annotated[int | None, typer.Option(flag, help=field.prompt)] + else: + note = " (prefer the -stdin variant)" if setting.secret else "" + ann = Annotated[str | None, typer.Option(flag, help=field.prompt + note)] + params.append( + inspect.Parameter( + setting.name, + inspect.Parameter.KEYWORD_ONLY, + default=None, + annotation=ann, + ) + ) + if setting.secret: + params.append( + inspect.Parameter( + f"{setting.name}_stdin", + inspect.Parameter.KEYWORD_ONLY, + default=False, + annotation=Annotated[ + bool, + typer.Option( + f"{flag}-stdin", + help=f"Read {field.prompt.lower()} from stdin", + ), + ], + ) + ) + return params + + +def with_settings_flags( + run: Callable[..., None], registry: cfg.Registry +) -> Callable[..., None]: + def entry(*args: Any, **kwargs: Any) -> None: + run(*args, **kwargs) + + static = [ + p + for p in inspect.signature(run, eval_str=True).parameters.values() + if p.kind is not inspect.Parameter.VAR_KEYWORD + ] + signature = inspect.Signature(static + settings_parameters(registry)) + entry.__signature__ = signature # type: ignore[attr-defined] + entry.__annotations__ = { + name: param.annotation for name, param in signature.parameters.items() + } + return entry + + +def read_secret_flags(registry: cfg.Registry, values: dict[str, Any]) -> dict[str, Any]: + out = dict(values) + for setting in registry: + if setting.secret and out.pop(f"{setting.name}_stdin", False): + out[setting.name] = sys.stdin.readline().rstrip("\n") + return out + + +def display(setting: cfg.Setting, value: Any) -> str: + if setting.secret: + return "••••••••" + if isinstance(value, bool): + return "Yes" if value else "No" + return str(value) + + +def apply_settings(ui: UI, project: SettingsProject, values: dict[str, Any]) -> None: + form = ui.form() + for name, raw in values.items(): + if raw is not None: + project.set(name, form.ask(project.registry.get(name).field, raw)) + + +def show_settings(ui: UI, project: SettingsProject) -> None: + values = project.settings() + for section, title in project.registry.sections.items(): + rows = [ + (s.field.prompt, display(s, values[s.name])) + for s in project.registry.in_section(section) + if values[s.name] not in (None, "") + ] + if rows: + ui.summary(rows, title=title.upper()) + + +class _SettingsCommand(Command): + panel = "Components" + no_args_is_help = True + + def __init__( + self, + ui: UI, + telemetry: Telemetry, + templates: TemplateRepository, + load: Callable[[Path], SettingsProject], + render: Callable[[Any], RenderResult], + ) -> None: + super().__init__(ui, telemetry) + self.templates = templates + self._load = load + self._render = render + + def load(self, path: Path) -> SettingsProject: + project_path = self.require_project_dir(path) + self.templates.resolve() + return self._load(project_path) + + def write(self, project: SettingsProject) -> None: + with self.ui.status("Rendering configuration..."): + result = self._render(project) + project.save_state() + report = result.write(project.path) + report_write(self.ui, report) + self.ui.info(f"Apply with: portabase restart {project.path.name}") + + +class SetCommand(_SettingsCommand): + name, help = "set", "Change settings: KEY VALUE [KEY VALUE ...]." + + def run( + self, + path: Annotated[Path, typer.Argument(help="Component folder")], + pairs: Annotated[ + list[str], typer.Argument(help="KEY VALUE pairs; keys as in 'show'") + ], + ) -> None: + project = self.load(path) + if len(pairs) % 2: + raise ValidationError( + "Expected KEY VALUE pairs.", + hint="Known keys: " + ", ".join(project.registry.names()), + ) + apply_settings( + self.ui, project, dict(zip(pairs[::2], pairs[1::2], strict=True)) + ) + self.write(project) + for key in pairs[::2]: + self.ui.success( + f"{key} = {display(project.registry.get(key), project.setting(key))}" + ) + + +class UnsetCommand(_SettingsCommand): + name, help = "unset", "Reset settings to their default: KEY [KEY ...]." + + def run( + self, + path: Annotated[Path, typer.Argument(help="Component folder")], + keys: Annotated[list[str], typer.Argument(help="Setting keys")], + ) -> None: + project = self.load(path) + for key in keys: + project.unset(key) + self.write(project) + self.ui.success("Reset: " + ", ".join(keys)) diff --git a/services/auth_providers.py b/services/auth_providers.py new file mode 100644 index 0000000..2d9fc17 --- /dev/null +++ b/services/auth_providers.py @@ -0,0 +1,64 @@ +from __future__ import annotations + +import re + +from core.errors import ValidationError +from core.fields import Field +from services.settings import public_url + +OAUTH_PROVIDERS: tuple[str, ...] = ( + "google", + "github", + "discord", + "apple", + "linkedin", + "x", + "reddit", +) + +OIDC_FIELDS: tuple[Field, ...] = ( + Field("issuer", "Issuer / discovery URL", "text", validator=public_url), + Field("client", "Client ID", "text"), + Field("secret", "Client secret", "secret"), + Field("title", "Display name", "text", default=""), + Field("scopes", "Scopes", "text", default=""), + Field("pkce", "Use PKCE?", "bool", default=False), + Field("host", "Host override", "text", default=""), +) + +OAUTH_FIELDS: tuple[Field, ...] = ( + Field("client", "Client ID", "text"), + Field("secret", "Client secret", "secret"), + Field("title", "Display name", "text", default=""), +) + +OIDC_ENV: dict[str, str] = { + "issuer": "ISSUER_URL", + "client": "CLIENT", + "secret": "SECRET", + "title": "TITLE", + "scopes": "SCOPES", + "pkce": "PKCE", + "host": "HOST", +} +OAUTH_ENV: dict[str, str] = {"client": "CLIENT", "secret": "SECRET", "title": "TITLE"} + + +def provider_prefix(kind: str, provider_id: str) -> str: + slug = re.sub(r"[^A-Z0-9]", "_", provider_id.upper()) + return f"AUTH_OIDC_{slug}" if kind == "oidc" else f"AUTH_SOCIAL_{slug}" + + +def validate_provider_id(kind: str, provider_id: str) -> str: + pid = provider_id.strip().lower() + if not re.match(r"^[a-z0-9][a-z0-9-]*$", pid): + raise ValidationError( + f"Invalid provider id {provider_id!r}.", + hint="Use lowercase letters, digits and dashes.", + ) + if kind == "oauth" and pid not in OAUTH_PROVIDERS: + raise ValidationError( + f"Unknown OAuth provider '{pid}'.", + hint="Supported: " + ", ".join(OAUTH_PROVIDERS), + ) + return pid diff --git a/services/dashboard_settings.py b/services/dashboard_settings.py deleted file mode 100644 index 4065c81..0000000 --- a/services/dashboard_settings.py +++ /dev/null @@ -1,260 +0,0 @@ -from __future__ import annotations - -import re -from dataclasses import dataclass -from typing import Any, Literal - -from core.errors import ValidationError -from core.fields import Field - -Section = Literal["network", "api", "onboarding", "auth"] -SECTION_TITLES: dict[Section, str] = { - "network": "Network", - "api": "API & MCP", - "onboarding": "Onboarding", - "auth": "Authentication", -} - - -def strong_password(value: str) -> str: - checks = ( - (len(value) >= 8, "at least 8 characters"), - (re.search(r"[a-z]", value), "a lowercase letter"), - (re.search(r"[A-Z]", value), "an uppercase letter"), - (re.search(r"\d", value), "a digit"), - (re.search(r"[^A-Za-z0-9]", value), "a special character"), - ) - missing = [label for ok, label in checks if not ok] - if missing: - raise ValidationError( - "Password too weak.", hint="It needs " + ", ".join(missing) + "." - ) - return value - - -def public_url(value: str) -> str: - if not re.match(r"^https?://[^/\s]+", value): - raise ValidationError( - f"Invalid URL: {value!r}", hint="Expected http(s)://host[:port]" - ) - return value.rstrip("/") - - -@dataclass(frozen=True) -class Setting: - field: Field - env: str - section: Section - secret: bool = False - - @property - def name(self) -> str: - return self.field.name - - -SETTINGS: tuple[Setting, ...] = ( - Setting( - Field( - "url", - "Public URL", - "text", - validator=public_url, - help="Used for links and OAuth/OIDC callbacks.", - ), - "PROJECT_URL", - "network", - ), - Setting( - Field("behind_proxy", "Behind a reverse proxy?", "bool", default=False), - "TUSD_BEHIND_PROXY", - "network", - ), - Setting( - Field("trusted_domains", "Trusted domains (comma-separated)", "text"), - "TRUSTED_DOMAINS", - "network", - ), - Setting( - Field("api", "Enable the REST API (/api/v1)?", "bool", default=False), - "API_ENABLED", - "api", - ), - Setting( - Field("openapi", "Enable OpenAPI spec and Swagger UI?", "bool", default=False), - "OPENAPI_ENABLED", - "api", - ), - Setting( - Field("mcp", "Enable the MCP server (/api/v1/mcp)?", "bool", default=False), - "MCP_ENABLED", - "api", - ), - Setting( - Field( - "skip_onboarding", - "Skip the onboarding wizard?", - "bool", - default=False, - help="Requires an initial account: admin_email and admin_password.", - ), - "SKIP_ONBOARDING", - "onboarding", - ), - Setting( - Field("admin_name", "Initial user name", "text"), - "AUTH_DEFAULT_USER_NAME", - "onboarding", - ), - Setting( - Field("admin_email", "Initial user email", "text"), - "AUTH_DEFAULT_USER", - "onboarding", - ), - Setting( - Field( - "admin_password", - "Initial user password", - "secret", - validator=strong_password, - ), - "AUTH_DEFAULT_PASSWORD", - "onboarding", - secret=True, - ), - Setting( - Field( - "password_auth", - "Allow email/password login?", - "bool", - default=True, - help="Disable only with at least one OIDC or OAuth provider configured.", - ), - "AUTH_EMAIL_PASSWORD_ENABLED", - "auth", - ), - Setting( - Field("signup", "Allow self sign-up?", "bool"), - "AUTH_SIGNUP_ENABLED", - "auth", - ), - Setting( - Field("passkey", "Allow passkey login?", "bool"), - "AUTH_PASSKEY_ENABLED", - "auth", - ), - Setting( - Field("account_linking", "Allow linking provider accounts?", "bool"), - "AUTH_ALLOW_LINKING", - "auth", - ), - Setting( - Field("account_unlinking", "Allow unlinking provider accounts?", "bool"), - "AUTH_ALLOW_UNLINKING", - "auth", - ), - Setting( - Field("sync_oidc_roles", "Sync roles from OIDC on login?", "bool"), - "AUTH_SYNC_OIDC_ROLES_ON_LOGIN", - "auth", - ), - Setting( - Field("role_map", "Role map (remote:portabase,...)", "text"), - "AUTH_ROLE_MAP", - "auth", - ), - Setting( - Field("allowed_group", "Restrict access to this group", "text"), - "ALLOWED_GROUP", - "auth", - ), -) - -BY_NAME: dict[str, Setting] = {s.name: s for s in SETTINGS} - -WIZARD_SECTIONS: tuple[tuple[Section, tuple[str, ...]], ...] = ( - ("api", ("api", "openapi", "mcp")), - ("onboarding", ("skip_onboarding", "admin_name", "admin_email", "admin_password")), - ("auth", ("password_auth", "signup", "passkey")), -) - - -def get(name: str) -> Setting: - try: - return BY_NAME[name] - except KeyError: - raise ValidationError( - f"Unknown setting '{name}'.", - hint="Known: " + ", ".join(BY_NAME), - ) from None - - -def to_env(setting: Setting, value: Any) -> str: - if setting.field.kind == "bool": - return "true" if value else "false" - return str(value) - - -def from_env(setting: Setting, raw: str | None) -> Any: - if raw is None: - return setting.field.default - if setting.field.kind == "bool": - return raw.strip().lower() in ("1", "true", "yes", "on") - return raw - - -OAUTH_PROVIDERS: tuple[str, ...] = ( - "google", - "github", - "discord", - "apple", - "linkedin", - "x", - "reddit", -) - -OIDC_FIELDS: tuple[Field, ...] = ( - Field("issuer", "Issuer / discovery URL", "text", validator=public_url), - Field("client", "Client ID", "text"), - Field("secret", "Client secret", "secret"), - Field("title", "Display name", "text", default=""), - Field("scopes", "Scopes", "text", default=""), - Field("pkce", "Use PKCE?", "bool", default=False), - Field("host", "Host override", "text", default=""), -) - -OAUTH_FIELDS: tuple[Field, ...] = ( - Field("client", "Client ID", "text"), - Field("secret", "Client secret", "secret"), - Field("title", "Display name", "text", default=""), -) - -OIDC_ENV: dict[str, str] = { - "issuer": "ISSUER_URL", - "client": "CLIENT", - "secret": "SECRET", - "title": "TITLE", - "scopes": "SCOPES", - "pkce": "PKCE", - "host": "HOST", -} -OAUTH_ENV: dict[str, str] = {"client": "CLIENT", "secret": "SECRET", "title": "TITLE"} - - -def provider_prefix(kind: str, provider_id: str) -> str: - slug = re.sub(r"[^A-Z0-9]", "_", provider_id.upper()) - return f"AUTH_OIDC_{slug}" if kind == "oidc" else f"AUTH_SOCIAL_{slug}" - - -def validate_provider_id(kind: str, provider_id: str) -> str: - pid = provider_id.strip().lower() - if not re.match(r"^[a-z0-9][a-z0-9-]*$", pid): - raise ValidationError( - f"Invalid provider id {provider_id!r}.", - hint="Use lowercase letters, digits and dashes.", - ) - if kind == "oauth" and pid not in OAUTH_PROVIDERS: - raise ValidationError( - f"Unknown OAuth provider '{pid}'.", - hint="Supported: " + ", ".join(OAUTH_PROVIDERS), - ) - return pid diff --git a/services/project.py b/services/project.py index 171f010..e01fc3b 100644 --- a/services/project.py +++ b/services/project.py @@ -9,7 +9,8 @@ from core.specs import DatabaseSpec from engines.base import DbEngine from engines.sqlite import SqliteEngine -from services import dashboard_settings as ds +from services import auth_providers as ap +from services import settings as cfg from services.compose_facts import ComposeFacts from services.envfile import EnvFile @@ -142,6 +143,38 @@ def remove(self, spec: DatabaseSpec, engine: DbEngine) -> None: if spec.managed and spec.host: self.env.remove_prefix(spec.env_prefix) + registry = cfg.AGENT + + def setting(self, name: str) -> Any: + setting = self.registry.get(name) + if setting.env is None: + return getattr(self, name) + return setting.from_env(self.env.get(setting.env)) + + def settings(self) -> dict[str, Any]: + return {s.name: self.setting(s.name) for s in self.registry} + + def set(self, name: str, value: Any) -> None: + setting = self.registry.get(name) + if setting.env is None: + setattr(self, name, value) + else: + self.env.set(setting.env, setting.to_env(value)) + + def unset(self, name: str) -> None: + setting = self.registry.get(name) + if setting.core: + raise ValidationError(f"'{name}' is required and cannot be unset.") + self.env.remove(setting.env or "") + + @property + def extra_env(self) -> list[str]: + return [ + s.env + for s in self.registry + if s.env and not s.core and self.env.get(s.env) is not None + ] + def find(self, id_or_name: str) -> DatabaseSpec: matches = [ d @@ -197,19 +230,21 @@ def db_mode(self) -> Literal["external", "internal", "custom"]: def project_name(self) -> str: return self.env.get("PROJECT_NAME") or self.path.name + registry = cfg.DASHBOARD + def setting(self, name: str) -> Any: - setting = ds.get(name) - return ds.from_env(setting, self.env.get(setting.env)) + setting = self.registry.get(name) + return setting.from_env(self.env.get(setting.env or "")) def settings(self) -> dict[str, Any]: - return {s.name: ds.from_env(s, self.env.get(s.env)) for s in ds.SETTINGS} + return {s.name: self.setting(s.name) for s in self.registry} def set(self, name: str, value: Any) -> None: - setting = ds.get(name) - self.env.set(setting.env, ds.to_env(setting, value)) + setting = self.registry.get(name) + self.env.set(setting.env or "", setting.to_env(value)) def unset(self, name: str) -> None: - self.env.remove(ds.get(name).env) + self.env.remove(self.registry.get(name).env or "") @property def providers(self) -> list[AuthProvider]: @@ -223,7 +258,7 @@ def providers(self) -> list[AuthProvider]: if not key.startswith(prefix): continue rest = key[len(prefix) :] - env_map = ds.OIDC_ENV if kind == "oidc" else ds.OAUTH_ENV + env_map = ap.OIDC_ENV if kind == "oidc" else ap.OAUTH_ENV for field_name, suffix in env_map.items(): if rest.endswith("_" + suffix): slug = rest[: -len(suffix) - 1] @@ -244,8 +279,8 @@ def add_provider(self, provider: AuthProvider) -> None: f"A provider named '{provider.id}' already exists.", hint="Remove it first: portabase dashboard auth remove", ) - prefix = ds.provider_prefix(provider.kind, provider.id) - env_map = ds.OIDC_ENV if provider.kind == "oidc" else ds.OAUTH_ENV + prefix = ap.provider_prefix(provider.kind, provider.id) + env_map = ap.OIDC_ENV if provider.kind == "oidc" else ap.OAUTH_ENV if provider.kind == "oidc": self.env.set(f"{prefix}_ID", provider.id) for field_name, value in provider.values.items(): @@ -261,7 +296,7 @@ def remove_provider(self, provider_id: str) -> AuthProvider: f"No provider named '{provider_id}'.", hint="See: portabase dashboard auth list", ) - self.env.remove_prefix(ds.provider_prefix(match.kind, match.id)) + self.env.remove_prefix(ap.provider_prefix(match.kind, match.id)) return match def callback_url(self, provider_id: str) -> str: diff --git a/services/renderer.py b/services/renderer.py index 57d748b..969128b 100644 --- a/services/renderer.py +++ b/services/renderer.py @@ -123,6 +123,9 @@ def render_agent( "edge_key_var": _var(env, "EDGE_KEY", inline), "log_level_var": _var(env, "LOG_LEVEL", inline), "polling_var": _var(env, "POLLING", inline), + "extra_env": [ + (name, _var(env, name, inline)) for name in project.extra_env + ], } compose = self.header() + self._render("agent.yml.j2", ctx) databases = [ diff --git a/services/settings.py b/services/settings.py new file mode 100644 index 0000000..375af03 --- /dev/null +++ b/services/settings.py @@ -0,0 +1,329 @@ +from __future__ import annotations + +import re +from collections.abc import Iterator +from dataclasses import dataclass +from typing import Any + +from core.errors import ValidationError +from core.fields import Field +from core.utils import validate_edge_key + + +def strong_password(value: str) -> str: + checks = ( + (len(value) >= 8, "at least 8 characters"), + (re.search(r"[a-z]", value), "a lowercase letter"), + (re.search(r"[A-Z]", value), "an uppercase letter"), + (re.search(r"\d", value), "a digit"), + (re.search(r"[^A-Za-z0-9]", value), "a special character"), + ) + missing = [label for ok, label in checks if not ok] + if missing: + raise ValidationError( + "Password too weak.", hint="It needs " + ", ".join(missing) + "." + ) + return value + + +def public_url(value: str) -> str: + if not re.match(r"^https?://[^/\s]+", value): + raise ValidationError( + f"Invalid URL: {value!r}", hint="Expected http(s)://host[:port]" + ) + return value.rstrip("/") + + +def edge_key(value: str) -> str: + if not validate_edge_key(value): + raise ValidationError( + "Invalid Edge Key.", + hint="Expected Base64 or JSON with serverUrl, agentId, masterKeyB64.", + ) + return value + + +def positive(value: int) -> int: + if value < 1: + raise ValidationError("Expected a positive number.") + return value + + +@dataclass(frozen=True) +class Setting: + field: Field + env: str | None + section: str + secret: bool = False + core: bool = False + + @property + def name(self) -> str: + return self.field.name + + def to_env(self, value: Any) -> str: + if self.field.kind == "bool": + return "true" if value else "false" + return str(value) + + def from_env(self, raw: str | None) -> Any: + if raw is None: + return self.field.default + if self.field.kind == "bool": + return raw.strip().lower() in ("1", "true", "yes", "on") + if self.field.kind == "int": + return int(raw) if raw.strip().lstrip("-").isdigit() else raw + return raw + + +class Registry: + def __init__(self, settings: tuple[Setting, ...], sections: dict[str, str]) -> None: + self._settings = settings + self._by_name = {s.name: s for s in settings} + self.sections = sections + + def __iter__(self) -> Iterator[Setting]: + return iter(self._settings) + + def names(self) -> list[str]: + return list(self._by_name) + + def get(self, name: str) -> Setting: + try: + return self._by_name[name] + except KeyError: + raise ValidationError( + f"Unknown setting '{name}'.", hint="Known: " + ", ".join(self._by_name) + ) from None + + def in_section(self, section: str) -> list[Setting]: + return [s for s in self._settings if s.section == section] + + +AGENT = Registry( + ( + Setting( + Field("key", "Edge Key", "text", validator=edge_key), + "EDGE_KEY", + "agent", + secret=True, + core=True, + ), + Setting( + Field("tz", "Timezone", "text", default="UTC"), "TZ", "agent", core=True + ), + Setting( + Field( + "polling", + "Polling frequency (seconds)", + "int", + default=5, + validator=positive, + ), + "POLLING", + "agent", + core=True, + ), + Setting( + Field( + "log_level", + "Log level", + "choice", + default="info", + choices=("debug", "info", "warn", "error"), + ), + "LOG_LEVEL", + "agent", + core=True, + ), + Setting( + Field( + "host_gateway", + "Map localhost to the Docker host?", + "bool", + default=False, + help="Adds extra_hosts localhost:host-gateway so the agent reaches services on the host.", + ), + None, + "network", + core=True, + ), + Setting( + Field("data_path", "Data path inside the container", "text"), + "DATA_PATH", + "storage", + ), + Setting( + Field("tmpdir", "Temporary archives path", "text"), "TMPDIR", "storage" + ), + Setting( + Field( + "retry_attempts", + "Retry attempts for database operations", + "int", + validator=positive, + ), + "RETRY_ATTEMPTS", + "resilience", + ), + Setting( + Field( + "retry_backoff_ms", + "Base delay between retries (ms)", + "int", + validator=positive, + ), + "RETRY_BACKOFF_MS", + "resilience", + ), + Setting( + Field( + "ssl_cert_file", + "CA bundle path for outgoing TLS", + "text", + help="Replaces the default root store; include the Mozilla roots too.", + ), + "SSL_CERT_FILE", + "network", + ), + ), + { + "agent": "Agent", + "network": "Network", + "storage": "Storage", + "resilience": "Resilience", + }, +) + +DASHBOARD = Registry( + ( + Setting( + Field( + "url", + "Public URL", + "text", + validator=public_url, + help="Used for links and OAuth/OIDC callbacks.", + ), + "PROJECT_URL", + "network", + ), + Setting( + Field("behind_proxy", "Behind a reverse proxy?", "bool", default=False), + "TUSD_BEHIND_PROXY", + "network", + ), + Setting( + Field("trusted_domains", "Trusted domains (comma-separated)", "text"), + "TRUSTED_DOMAINS", + "network", + ), + Setting( + Field("api", "Enable the REST API (/api/v1)?", "bool", default=False), + "API_ENABLED", + "api", + ), + Setting( + Field( + "openapi", "Enable OpenAPI spec and Swagger UI?", "bool", default=False + ), + "OPENAPI_ENABLED", + "api", + ), + Setting( + Field("mcp", "Enable the MCP server (/api/v1/mcp)?", "bool", default=False), + "MCP_ENABLED", + "api", + ), + Setting( + Field( + "skip_onboarding", + "Skip the onboarding wizard?", + "bool", + default=False, + help="Requires an initial account: admin_email and admin_password.", + ), + "SKIP_ONBOARDING", + "onboarding", + ), + Setting( + Field("admin_name", "Initial user name", "text"), + "AUTH_DEFAULT_USER_NAME", + "onboarding", + ), + Setting( + Field("admin_email", "Initial user email", "text"), + "AUTH_DEFAULT_USER", + "onboarding", + ), + Setting( + Field( + "admin_password", + "Initial user password", + "secret", + validator=strong_password, + ), + "AUTH_DEFAULT_PASSWORD", + "onboarding", + secret=True, + ), + Setting( + Field( + "password_auth", + "Allow email/password login?", + "bool", + default=True, + help="Disable only with at least one OIDC or OAuth provider configured.", + ), + "AUTH_EMAIL_PASSWORD_ENABLED", + "auth", + ), + Setting( + Field("signup", "Allow self sign-up?", "bool"), + "AUTH_SIGNUP_ENABLED", + "auth", + ), + Setting( + Field("passkey", "Allow passkey login?", "bool"), + "AUTH_PASSKEY_ENABLED", + "auth", + ), + Setting( + Field("account_linking", "Allow linking provider accounts?", "bool"), + "AUTH_ALLOW_LINKING", + "auth", + ), + Setting( + Field("account_unlinking", "Allow unlinking provider accounts?", "bool"), + "AUTH_ALLOW_UNLINKING", + "auth", + ), + Setting( + Field("sync_oidc_roles", "Sync roles from OIDC on login?", "bool"), + "AUTH_SYNC_OIDC_ROLES_ON_LOGIN", + "auth", + ), + Setting( + Field("role_map", "Role map (remote:portabase,...)", "text"), + "AUTH_ROLE_MAP", + "auth", + ), + Setting( + Field("allowed_group", "Restrict access to this group", "text"), + "ALLOWED_GROUP", + "auth", + ), + ), + { + "network": "Network", + "api": "API & MCP", + "onboarding": "Onboarding", + "auth": "Authentication", + }, +) + +DASHBOARD_WIZARD: tuple[tuple[str, tuple[str, ...]], ...] = ( + ("api", ("api", "openapi", "mcp")), + ("onboarding", ("skip_onboarding", "admin_name", "admin_email", "admin_password")), + ("auth", ("password_auth", "signup", "passkey")), +) diff --git a/templates/agent.yml.j2 b/templates/agent.yml.j2 index d865063..be89b10 100644 --- a/templates/agent.yml.j2 +++ b/templates/agent.yml.j2 @@ -19,6 +19,9 @@ services: EDGE_KEY: "{{ edge_key_var }}" LOG_LEVEL: "{{ log_level_var }}" POLLING: "{{ polling_var }}" +{%- for name, value in extra_env %} + {{ name }}: "{{ value }}" +{%- endfor %} networks: - portabase {% for s in services %} From f35b34bfae8c058d5863a398c854c2e3c30d33e1 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Th=C3=A9o=20LAGACHE?= Date: Fri, 11 Sep 2026 15:04:20 +0200 Subject: [PATCH 15/24] feat(agent): ca_bundle mounts an internal CA and points SSL_CERT_FILE at it MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The agent uses rustls, which ignores the system CA store, so the only way to trust an internal CA is SSL_CERT_FILE — and the file it names must exist inside the container. Setting the variable alone would have produced an agent that fails at startup. 'agent set NAME ca_bundle ./ca-bundle.crt' now bind-mounts the host file read-only at a fixed path and sets SSL_CERT_FILE to that path. The host path lives in .env as CA_BUNDLE for compose interpolation only; it is not passed to the container. set refuses a path that does not exist, and the help says why the bundle must include the Mozilla roots. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01CbemoDqsmMdRq6PS5Vaup8 --- services/project.py | 18 +++++++++++++++++- services/renderer.py | 2 ++ services/settings.py | 17 ++++++++++++----- templates/agent.yml.j2 | 6 ++++++ 4 files changed, 37 insertions(+), 6 deletions(-) diff --git a/services/project.py b/services/project.py index e01fc3b..3ea5559 100644 --- a/services/project.py +++ b/services/project.py @@ -124,6 +124,12 @@ def sqlite_mounts(self) -> list[tuple[str, str]]: return mounts def validate(self) -> None: + bundle = self.ca_bundle + if bundle and not (self.path / bundle).exists() and not Path(bundle).exists(): + raise ValidationError( + f"CA bundle not found: {bundle}", + hint=f"Path is resolved from {self.path}; use an absolute path otherwise.", + ) seen: set[str] = set() for d in self.managed: if d.host in seen: @@ -172,9 +178,18 @@ def extra_env(self) -> list[str]: return [ s.env for s in self.registry - if s.env and not s.core and self.env.get(s.env) is not None + if s.env + and s.container_env + and not s.core + and self.env.get(s.env) is not None ] + CA_BUNDLE_IN_CONTAINER = "/etc/ssl/certs/portabase-ca-bundle.crt" + + @property + def ca_bundle(self) -> str | None: + return self.env.get("CA_BUNDLE") + def find(self, id_or_name: str) -> DatabaseSpec: matches = [ d @@ -193,6 +208,7 @@ def find(self, id_or_name: str) -> DatabaseSpec: return matches[0] def save_state(self) -> None: + self.validate() self.env.save() diff --git a/services/renderer.py b/services/renderer.py index 969128b..0267b02 100644 --- a/services/renderer.py +++ b/services/renderer.py @@ -126,6 +126,8 @@ def render_agent( "extra_env": [ (name, _var(env, name, inline)) for name in project.extra_env ], + "ca_bundle": _var(env, "CA_BUNDLE", inline) if project.ca_bundle else None, + "ca_bundle_in_container": project.CA_BUNDLE_IN_CONTAINER, } compose = self.header() + self._render("agent.yml.j2", ctx) databases = [ diff --git a/services/settings.py b/services/settings.py index 375af03..a26acd8 100644 --- a/services/settings.py +++ b/services/settings.py @@ -56,6 +56,7 @@ class Setting: section: str secret: bool = False core: bool = False + container_env: bool = True @property def name(self) -> str: @@ -178,13 +179,19 @@ def in_section(self, section: str) -> list[Setting]: ), Setting( Field( - "ssl_cert_file", - "CA bundle path for outgoing TLS", - "text", - help="Replaces the default root store; include the Mozilla roots too.", + "ca_bundle", + "CA bundle on this host (for an internal CA)", + "path", + help=( + "The file is mounted read-only and SSL_CERT_FILE points at it. " + "It REPLACES the root store, so concatenate the Mozilla roots " + "with your CA: cat /etc/ssl/certs/ca-certificates.crt my-ca.crt " + "> ca-bundle.crt" + ), ), - "SSL_CERT_FILE", + "CA_BUNDLE", "network", + container_env=False, ), ), { diff --git a/templates/agent.yml.j2 b/templates/agent.yml.j2 index be89b10..5bdce60 100644 --- a/templates/agent.yml.j2 +++ b/templates/agent.yml.j2 @@ -10,6 +10,9 @@ services: {%- if docker_socket %} - /var/run/docker.sock:/var/run/docker.sock {%- endif %} +{%- if ca_bundle %} + - {{ ca_bundle }}:{{ ca_bundle_in_container }}:ro +{%- endif %} {%- if host_gateway %} extra_hosts: - "localhost:host-gateway" @@ -22,6 +25,9 @@ services: {%- for name, value in extra_env %} {{ name }}: "{{ value }}" {%- endfor %} +{%- if ca_bundle %} + SSL_CERT_FILE: "{{ ca_bundle_in_container }}" +{%- endif %} networks: - portabase {% for s in services %} From 86089dc2bf6f90ef2fcb03585a84766451752570 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Th=C3=A9o=20LAGACHE?= Date: Fri, 11 Sep 2026 15:06:13 +0200 Subject: [PATCH 16/24] refactor(agent): keep only SSL_CERT_FILE in .env; the bundle path lives in the mount The host path of the CA bundle is not a container variable, so it no longer goes in .env. It is a compose fact like host_gateway: written as the bind-mount source and read back from the existing compose on load. SSL_CERT_FILE stays in .env with the in-container path, which is what the agent consumes. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01CbemoDqsmMdRq6PS5Vaup8 --- services/compose_facts.py | 16 ++++++++++++++++ services/project.py | 25 +++++++++++++++++-------- services/renderer.py | 11 ++++++++--- services/settings.py | 4 +--- templates/agent.yml.j2 | 2 +- 5 files changed, 43 insertions(+), 15 deletions(-) diff --git a/services/compose_facts.py b/services/compose_facts.py index f54bd7f..6b3568a 100644 --- a/services/compose_facts.py +++ b/services/compose_facts.py @@ -5,6 +5,7 @@ import yaml GENERATED_MARKER = "# Generated by Portabase CLI" +CA_BUNDLE_IN_CONTAINER = "/etc/ssl/certs/portabase-ca-bundle.crt" class ComposeFacts: @@ -41,3 +42,18 @@ def host_gateway(self) -> bool: if isinstance(extra, dict): return any("host-gateway" in str(v) for v in extra.values()) return False + + @property + def ca_bundle(self) -> str | None: + for volume in self._service("agent").get("volumes") or []: + if isinstance(volume, str): + parts = volume.split(":") + if len(parts) >= 2 and parts[1] == CA_BUNDLE_IN_CONTAINER: + return parts[0] + elif ( + isinstance(volume, dict) + and volume.get("target") == CA_BUNDLE_IN_CONTAINER + ): + source = volume.get("source") + return str(source) if source else None + return None diff --git a/services/project.py b/services/project.py index 3ea5559..77b430f 100644 --- a/services/project.py +++ b/services/project.py @@ -11,7 +11,7 @@ from engines.sqlite import SqliteEngine from services import auth_providers as ap from services import settings as cfg -from services.compose_facts import ComposeFacts +from services.compose_facts import CA_BUNDLE_IN_CONTAINER, ComposeFacts from services.envfile import EnvFile ProjectKind = Literal["agent", "dashboard"] @@ -93,6 +93,7 @@ def load(cls, path: Path) -> AgentProject: databases = [spec_from_entry(e, env) for e in entries if isinstance(e, dict)] facts = ComposeFacts(path / COMPOSE_FILE) project = cls(path, env, databases, facts.host_gateway) + project._ca_bundle = facts.ca_bundle project.validate() return project @@ -171,24 +172,32 @@ def unset(self, name: str) -> None: setting = self.registry.get(name) if setting.core: raise ValidationError(f"'{name}' is required and cannot be unset.") - self.env.remove(setting.env or "") + if setting.env is None: + setattr(self, name, None) + else: + self.env.remove(setting.env) @property def extra_env(self) -> list[str]: return [ s.env for s in self.registry - if s.env - and s.container_env - and not s.core - and self.env.get(s.env) is not None + if s.env and not s.core and self.env.get(s.env) is not None ] - CA_BUNDLE_IN_CONTAINER = "/etc/ssl/certs/portabase-ca-bundle.crt" + _ca_bundle: str | None = None @property def ca_bundle(self) -> str | None: - return self.env.get("CA_BUNDLE") + return self._ca_bundle + + @ca_bundle.setter + def ca_bundle(self, host_path: str | None) -> None: + self._ca_bundle = host_path or None + if host_path: + self.env.set("SSL_CERT_FILE", CA_BUNDLE_IN_CONTAINER) + else: + self.env.remove("SSL_CERT_FILE") def find(self, id_or_name: str) -> DatabaseSpec: matches = [ diff --git a/services/renderer.py b/services/renderer.py index 0267b02..ac0043a 100644 --- a/services/renderer.py +++ b/services/renderer.py @@ -15,7 +15,11 @@ from core.errors import TemplateError from core.specs import DatabaseSpec from engines import EngineRegistry -from services.compose_facts import GENERATED_MARKER, ComposeFacts +from services.compose_facts import ( + CA_BUNDLE_IN_CONTAINER, + GENERATED_MARKER, + ComposeFacts, +) from services.envfile import EnvFile from services.project import ( COMPOSE_FILE, @@ -126,8 +130,9 @@ def render_agent( "extra_env": [ (name, _var(env, name, inline)) for name in project.extra_env ], - "ca_bundle": _var(env, "CA_BUNDLE", inline) if project.ca_bundle else None, - "ca_bundle_in_container": project.CA_BUNDLE_IN_CONTAINER, + "ca_bundle": project.ca_bundle, + "ca_bundle_in_container": CA_BUNDLE_IN_CONTAINER, + "ssl_cert_file_var": _var(env, "SSL_CERT_FILE", inline), } compose = self.header() + self._render("agent.yml.j2", ctx) databases = [ diff --git a/services/settings.py b/services/settings.py index a26acd8..d7a3002 100644 --- a/services/settings.py +++ b/services/settings.py @@ -56,7 +56,6 @@ class Setting: section: str secret: bool = False core: bool = False - container_env: bool = True @property def name(self) -> str: @@ -189,9 +188,8 @@ def in_section(self, section: str) -> list[Setting]: "> ca-bundle.crt" ), ), - "CA_BUNDLE", + None, "network", - container_env=False, ), ), { diff --git a/templates/agent.yml.j2 b/templates/agent.yml.j2 index 5bdce60..4732866 100644 --- a/templates/agent.yml.j2 +++ b/templates/agent.yml.j2 @@ -26,7 +26,7 @@ services: {{ name }}: "{{ value }}" {%- endfor %} {%- if ca_bundle %} - SSL_CERT_FILE: "{{ ca_bundle_in_container }}" + SSL_CERT_FILE: "{{ ssl_cert_file_var }}" {%- endif %} networks: - portabase From 4658f7441cd031a9a1a55e35c60fa0d2a1779414 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Th=C3=A9o=20LAGACHE?= Date: Fri, 11 Sep 2026 15:10:43 +0200 Subject: [PATCH 17/24] feat(dashboard): end create with a hint on adding OIDC/OAuth providers Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01CbemoDqsmMdRq6PS5Vaup8 --- commands/dashboard.py | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/commands/dashboard.py b/commands/dashboard.py index 1c198eb..2bcf720 100644 --- a/commands/dashboard.py +++ b/commands/dashboard.py @@ -205,9 +205,6 @@ def run( path.mkdir(parents=True, exist_ok=True) self.write(project) self.ui.success(f"Dashboard '{name}' created in {path}") - self.ui.hint( - f"Add a login provider with: portabase dashboard auth add {name} oidc ..." - ) if start or ( not self.ui.non_interactive @@ -219,6 +216,16 @@ def run( else: self.ui.info(f"Run: portabase start {name}") + self.ui.print("") + self.ui.hint("Single sign-on (OIDC / OAuth) can be added at any time:") + self.ui.hint(f" portabase dashboard set {name} url https://your.domain") + self.ui.hint( + f" portabase dashboard auth add {name} oidc keycloak --issuer URL ..." + ) + self.ui.hint( + f" portabase dashboard auth add {name} oauth github --client ID ..." + ) + def _wizard(self, form: Form, project: DashboardProject) -> None: for section, names in cfg.DASHBOARD_WIZARD: self.ui.section(cfg.DASHBOARD.sections[section]) From ea37e4e45d9548ecb44118924cadf82f3566f55d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Th=C3=A9o=20LAGACHE?= Date: Fri, 11 Sep 2026 15:21:16 +0200 Subject: [PATCH 18/24] feat(dashboard): prompt for provider kind and id in auth add/remove Rename commands/dashboard_auth.py to commands/auth.py. Both arguments of 'dashboard auth add' become optional and are asked interactively when omitted (kind as a choice, OAuth provider from the known list, OIDC id as free text). 'dashboard auth remove' without an id offers the configured providers as a selection list. Non-interactive mode keeps failing with 'Missing --kind' / 'Missing --id'. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01CbemoDqsmMdRq6PS5Vaup8 --- commands/{dashboard_auth.py => auth.py} | 47 ++++++++++++++----- commands/dashboard.py | 2 +- .../2026-09-11-dashboard-settings-design.md | 2 +- 3 files changed, 37 insertions(+), 14 deletions(-) rename commands/{dashboard_auth.py => auth.py} (78%) diff --git a/commands/dashboard_auth.py b/commands/auth.py similarity index 78% rename from commands/dashboard_auth.py rename to commands/auth.py index 461e296..f26e9c4 100644 --- a/commands/dashboard_auth.py +++ b/commands/auth.py @@ -59,14 +59,16 @@ class AuthAddCommand(_AuthCommand): def run( self, path: PathArg, - kind: Annotated[str, typer.Argument(help="oidc | oauth")], + kind: Annotated[ + str | None, typer.Argument(help="oidc | oauth (asked if omitted)") + ] = None, provider_id: Annotated[ - str, + str | None, typer.Argument( help="Provider id: any slug for oidc, a known name for oauth " "(google, github, discord, apple, linkedin, x, reddit)" ), - ], + ] = None, client: Annotated[ str | None, typer.Option("--client", help="Client ID") ] = None, @@ -94,9 +96,18 @@ def run( str | None, typer.Option("--host", help="OIDC host override") ] = None, ) -> None: - if kind not in KINDS: - raise ValidationError(f"Unknown kind '{kind}'.", hint="Use oidc or oauth.") - pid = ap.validate_provider_id(kind, provider_id) + form = self.ui.form() + picked = form.choice("Provider kind", list(KINDS), value=kind, name="kind") + provider_kind: ProviderKind = "oidc" if picked == "oidc" else "oauth" + if provider_kind == "oauth": + provider_id = form.choice( + "OAuth provider", list(ap.OAUTH_PROVIDERS), value=provider_id, name="id" + ) + else: + provider_id = form.text( + "Provider id (slug, e.g. keycloak)", value=provider_id, name="id" + ) + pid = ap.validate_provider_id(provider_kind, provider_id) if secret_stdin: secret = sys.stdin.readline().rstrip("\n") elif secret is not None: @@ -113,21 +124,21 @@ def run( "pkce": pkce, "host": host, } - fields = ap.OIDC_FIELDS if kind == "oidc" else ap.OAUTH_FIELDS + fields = ap.OIDC_FIELDS if provider_kind == "oidc" else ap.OAUTH_FIELDS allowed = {f.name for f in fields} stray = sorted( k for k, v in values.items() if v is not None and k not in allowed ) if stray: flags = ", ".join("--" + k for k in stray) - raise ValidationError(f"Not applicable to {kind}: {flags}.") + raise ValidationError(f"Not applicable to {provider_kind}: {flags}.") project = self.load(path) - answers = self.ui.form().collect(list(fields), values) - provider = AuthProvider(kind=kind, id=pid, values=answers) + answers = form.collect(list(fields), values) + provider = AuthProvider(kind=provider_kind, id=pid, values=answers) project.add_provider(provider) self.write(project) - self.ui.success(f"Added {kind} provider '{pid}'.") + self.ui.success(f"Added {provider_kind} provider '{pid}'.") self.ui.info( f"Callback URL to register at the provider: {project.callback_url(pid)}" ) @@ -164,12 +175,24 @@ class AuthRemoveCommand(_AuthCommand): def run( self, path: PathArg, - provider_id: Annotated[str, typer.Argument(help="Provider id")], + provider_id: Annotated[ + str | None, typer.Argument(help="Provider id (asked if omitted)") + ] = None, yes: Annotated[ bool, typer.Option("--yes", "-y", help="Skip confirmation") ] = False, ) -> None: project = self.load(path) + if provider_id is None: + providers = project.providers + if not providers: + self.ui.warning("No login provider to remove.") + return + choices = [f"{p.id} ({p.kind})" for p in providers] + picked = self.ui.form().choice( + "Which provider to remove?", choices, name="id" + ) + provider_id = providers[choices.index(picked)].id if not yes: self.confirm_or_abort( f"Remove login provider '{provider_id}'?", default=False diff --git a/commands/dashboard.py b/commands/dashboard.py index 2bcf720..a38c5e2 100644 --- a/commands/dashboard.py +++ b/commands/dashboard.py @@ -8,8 +8,8 @@ import typer +from commands.auth import DashboardAuthCommands from commands.base import Command, CommandGroup -from commands.dashboard_auth import DashboardAuthCommands from commands.db import report_write from commands.settings import ( SetCommand, diff --git a/docs/superpowers/specs/2026-09-11-dashboard-settings-design.md b/docs/superpowers/specs/2026-09-11-dashboard-settings-design.md index 559f793..97096eb 100644 --- a/docs/superpowers/specs/2026-09-11-dashboard-settings-design.md +++ b/docs/superpowers/specs/2026-09-11-dashboard-settings-design.md @@ -185,7 +185,7 @@ Puis le message : `Apply with: portabase start NAME`. | `commands/agent.py` | `AgentCommands` (groupe) + `AgentCreateCommand` ; `db` devient `agent db` | | `commands/db.py` | inchangé, enregistré sous `agent` ; alias racine déprécié | | `commands/dashboard.py` | `DashboardCommands` : `create`, `show`, `set` | -| `commands/dashboard_auth.py` | `add`, `list`, `remove` | +| `commands/auth.py` | `add`, `list`, `remove` | | `commands/lifecycle.py` | `RestartCommand` → `up -d` puis `restart` | | `services/dashboard_settings.py` | `Setting`, `SETTINGS`, `OAUTH_PROVIDERS`, `OIDC_FIELDS` | | `services/project.py` | `DashboardProject` : `settings`, `providers`, `validate()`, `set()`, `add_provider()`, `remove_provider()` | From 932b94257ddb4a43b906480ec92a1f64fa493080 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Th=C3=A9o=20LAGACHE?= Date: Fri, 11 Sep 2026 15:21:37 +0200 Subject: [PATCH 19/24] chore: drop a leftover comment in main.py Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01CbemoDqsmMdRq6PS5Vaup8 --- main.py | 2 -- 1 file changed, 2 deletions(-) diff --git a/main.py b/main.py index 67ba840..1ed4388 100644 --- a/main.py +++ b/main.py @@ -50,8 +50,6 @@ def from_env(cls, argv: list[str]) -> "Settings": no_color=bool(os.environ.get("NO_COLOR")) or "--no-color" in argv, ) if settings.no_color: - # Typer renders --help with its own Rich console, which only honors - # the NO_COLOR convention; --help is handled before any callback runs. os.environ["NO_COLOR"] = "1" return settings From 8d1ef4e0b307bcf414ce5f94b1070b91e43214b1 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Th=C3=A9o=20LAGACHE?= Date: Fri, 11 Sep 2026 15:39:47 +0200 Subject: [PATCH 20/24] feat(dashboard): prompt for provider kind and id in auth add/remove Rename commands/dashboard_auth.py to commands/auth.py. Both arguments of 'dashboard auth add' become optional and are asked interactively when omitted (kind as a choice, OAuth provider from the known list, OIDC id as free text). 'dashboard auth remove' without an id offers the configured providers as a selection list. Non-interactive mode keeps failing with 'Missing --kind' / 'Missing --id'. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01CbemoDqsmMdRq6PS5Vaup8 --- commands/{dashboard_auth.py => auth.py} | 47 ++++++++++++++----- commands/dashboard.py | 2 +- .../2026-09-11-dashboard-settings-design.md | 2 +- main.py | 2 - 4 files changed, 37 insertions(+), 16 deletions(-) rename commands/{dashboard_auth.py => auth.py} (78%) diff --git a/commands/dashboard_auth.py b/commands/auth.py similarity index 78% rename from commands/dashboard_auth.py rename to commands/auth.py index 461e296..f26e9c4 100644 --- a/commands/dashboard_auth.py +++ b/commands/auth.py @@ -59,14 +59,16 @@ class AuthAddCommand(_AuthCommand): def run( self, path: PathArg, - kind: Annotated[str, typer.Argument(help="oidc | oauth")], + kind: Annotated[ + str | None, typer.Argument(help="oidc | oauth (asked if omitted)") + ] = None, provider_id: Annotated[ - str, + str | None, typer.Argument( help="Provider id: any slug for oidc, a known name for oauth " "(google, github, discord, apple, linkedin, x, reddit)" ), - ], + ] = None, client: Annotated[ str | None, typer.Option("--client", help="Client ID") ] = None, @@ -94,9 +96,18 @@ def run( str | None, typer.Option("--host", help="OIDC host override") ] = None, ) -> None: - if kind not in KINDS: - raise ValidationError(f"Unknown kind '{kind}'.", hint="Use oidc or oauth.") - pid = ap.validate_provider_id(kind, provider_id) + form = self.ui.form() + picked = form.choice("Provider kind", list(KINDS), value=kind, name="kind") + provider_kind: ProviderKind = "oidc" if picked == "oidc" else "oauth" + if provider_kind == "oauth": + provider_id = form.choice( + "OAuth provider", list(ap.OAUTH_PROVIDERS), value=provider_id, name="id" + ) + else: + provider_id = form.text( + "Provider id (slug, e.g. keycloak)", value=provider_id, name="id" + ) + pid = ap.validate_provider_id(provider_kind, provider_id) if secret_stdin: secret = sys.stdin.readline().rstrip("\n") elif secret is not None: @@ -113,21 +124,21 @@ def run( "pkce": pkce, "host": host, } - fields = ap.OIDC_FIELDS if kind == "oidc" else ap.OAUTH_FIELDS + fields = ap.OIDC_FIELDS if provider_kind == "oidc" else ap.OAUTH_FIELDS allowed = {f.name for f in fields} stray = sorted( k for k, v in values.items() if v is not None and k not in allowed ) if stray: flags = ", ".join("--" + k for k in stray) - raise ValidationError(f"Not applicable to {kind}: {flags}.") + raise ValidationError(f"Not applicable to {provider_kind}: {flags}.") project = self.load(path) - answers = self.ui.form().collect(list(fields), values) - provider = AuthProvider(kind=kind, id=pid, values=answers) + answers = form.collect(list(fields), values) + provider = AuthProvider(kind=provider_kind, id=pid, values=answers) project.add_provider(provider) self.write(project) - self.ui.success(f"Added {kind} provider '{pid}'.") + self.ui.success(f"Added {provider_kind} provider '{pid}'.") self.ui.info( f"Callback URL to register at the provider: {project.callback_url(pid)}" ) @@ -164,12 +175,24 @@ class AuthRemoveCommand(_AuthCommand): def run( self, path: PathArg, - provider_id: Annotated[str, typer.Argument(help="Provider id")], + provider_id: Annotated[ + str | None, typer.Argument(help="Provider id (asked if omitted)") + ] = None, yes: Annotated[ bool, typer.Option("--yes", "-y", help="Skip confirmation") ] = False, ) -> None: project = self.load(path) + if provider_id is None: + providers = project.providers + if not providers: + self.ui.warning("No login provider to remove.") + return + choices = [f"{p.id} ({p.kind})" for p in providers] + picked = self.ui.form().choice( + "Which provider to remove?", choices, name="id" + ) + provider_id = providers[choices.index(picked)].id if not yes: self.confirm_or_abort( f"Remove login provider '{provider_id}'?", default=False diff --git a/commands/dashboard.py b/commands/dashboard.py index 2bcf720..a38c5e2 100644 --- a/commands/dashboard.py +++ b/commands/dashboard.py @@ -8,8 +8,8 @@ import typer +from commands.auth import DashboardAuthCommands from commands.base import Command, CommandGroup -from commands.dashboard_auth import DashboardAuthCommands from commands.db import report_write from commands.settings import ( SetCommand, diff --git a/docs/superpowers/specs/2026-09-11-dashboard-settings-design.md b/docs/superpowers/specs/2026-09-11-dashboard-settings-design.md index 559f793..97096eb 100644 --- a/docs/superpowers/specs/2026-09-11-dashboard-settings-design.md +++ b/docs/superpowers/specs/2026-09-11-dashboard-settings-design.md @@ -185,7 +185,7 @@ Puis le message : `Apply with: portabase start NAME`. | `commands/agent.py` | `AgentCommands` (groupe) + `AgentCreateCommand` ; `db` devient `agent db` | | `commands/db.py` | inchangé, enregistré sous `agent` ; alias racine déprécié | | `commands/dashboard.py` | `DashboardCommands` : `create`, `show`, `set` | -| `commands/dashboard_auth.py` | `add`, `list`, `remove` | +| `commands/auth.py` | `add`, `list`, `remove` | | `commands/lifecycle.py` | `RestartCommand` → `up -d` puis `restart` | | `services/dashboard_settings.py` | `Setting`, `SETTINGS`, `OAUTH_PROVIDERS`, `OIDC_FIELDS` | | `services/project.py` | `DashboardProject` : `settings`, `providers`, `validate()`, `set()`, `add_provider()`, `remove_provider()` | diff --git a/main.py b/main.py index 67ba840..1ed4388 100644 --- a/main.py +++ b/main.py @@ -50,8 +50,6 @@ def from_env(cls, argv: list[str]) -> "Settings": no_color=bool(os.environ.get("NO_COLOR")) or "--no-color" in argv, ) if settings.no_color: - # Typer renders --help with its own Rich console, which only honors - # the NO_COLOR convention; --help is handled before any callback runs. os.environ["NO_COLOR"] = "1" return settings From 77167e592cdf957ed2021e048f08a612061148e6 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Th=C3=A9o=20LAGACHE?= Date: Mon, 14 Sep 2026 14:42:17 +0200 Subject: [PATCH 21/24] add: new cli --- .github/CONTRIBUTING.md | 9 + .github/workflows/ci.yml | 19 +- README.md | 2 +- commands/agent.py | 26 +- commands/auth.py | 22 +- commands/base.py | 26 - commands/build.py | 2 +- commands/dashboard.py | 20 +- commands/db.py | 26 +- commands/decrypt.py | 14 +- commands/flows/add_database.py | 12 +- commands/lifecycle.py | 4 +- commands/settings.py | 12 +- commands/update.py | 6 +- core/config.py | 8 +- core/utils.py | 8 +- core/version.py | 30 +- .../plans/2026-09-11-plan-1-ci-hygiene.md | 954 ------- ...2026-09-11-plan-2-foundations-lifecycle.md | 2385 ----------------- .../2026-09-11-plan-3-templates-engines.md | 1944 -------------- .../2026-09-11-plan-4-render-commands.md | 1935 ------------- .../specs/2026-09-11-cli-refactor-design.md | 592 ---- .../2026-09-11-dashboard-settings-design.md | 198 -- engines/base.py | 26 +- engines/mongodb.py | 11 +- engines/mssql.py | 7 +- engines/redis.py | 6 +- engines/valkey.py | 6 +- main.py | 36 +- pyproject.toml | 7 +- scripts/render_check.py | 215 -- services/compose_facts.py | 4 +- services/docker.py | 24 +- services/envfile.py | 41 +- services/http.py | 58 +- services/ports.py | 6 +- services/project.py | 61 +- services/renderer.py | 25 +- services/settings.py | 4 +- services/templates.py | 14 +- services/updater.py | 21 +- templates/dashboard.yml.j2 | 14 +- templates/engines/firebird.yml.j2 | 10 +- templates/engines/mariadb.yml.j2 | 8 +- templates/engines/mongodb.yml.j2 | 6 +- templates/engines/mssql.yml.j2 | 4 +- templates/engines/mysql.yml.j2 | 8 +- templates/engines/postgresql-cluster.yml.j2 | 6 +- templates/engines/postgresql.yml.j2 | 6 +- templates/engines/redis.yml.j2 | 2 +- templates/engines/valkey.yml.j2 | 4 +- tests/__init__.py | 0 tests/conftest.py | 90 + tests/core/__init__.py | 0 tests/core/config.py | 38 + tests/core/crypto.py | 135 + tests/core/errors.py | 53 + tests/core/fields.py | 29 + tests/core/specs.py | 33 + tests/core/utils.py | 99 + tests/core/version.py | 99 + tests/engines/__init__.py | 0 tests/engines/base.py | 106 + tests/engines/docker_volume.py | 84 + tests/engines/firebird.py | 145 + tests/engines/mariadb.py | 131 + tests/engines/mongodb.py | 132 + tests/engines/mssql.py | 120 + tests/engines/mysql.py | 126 + tests/engines/postgresql.py | 139 + tests/engines/postgresql_cluster.py | 135 + tests/engines/redis.py | 143 + tests/engines/registry.py | 47 + tests/engines/sqlite.py | 108 + tests/engines/valkey.py | 141 + tests/services/__init__.py | 0 tests/services/auth_providers.py | 49 + tests/services/compose_facts.py | 76 + tests/services/docker.py | 23 + tests/services/envfile.py | 125 + tests/services/http.py | 147 + tests/services/ports.py | 41 + tests/services/project.py | 382 +++ tests/services/renderer.py | 220 ++ tests/services/settings.py | 144 + tests/services/telemetry.py | 29 + tests/services/templates.py | 46 + tests/services/updater.py | 271 ++ tests/structure.py | 89 + tests/support.py | 139 + ui/components/progress.py | 2 +- ui/components/prompt.py | 4 +- ui/components/table.py | 6 +- ui/form.py | 34 +- 94 files changed, 4303 insertions(+), 8551 deletions(-) delete mode 100644 docs/superpowers/plans/2026-09-11-plan-1-ci-hygiene.md delete mode 100644 docs/superpowers/plans/2026-09-11-plan-2-foundations-lifecycle.md delete mode 100644 docs/superpowers/plans/2026-09-11-plan-3-templates-engines.md delete mode 100644 docs/superpowers/plans/2026-09-11-plan-4-render-commands.md delete mode 100644 docs/superpowers/specs/2026-09-11-cli-refactor-design.md delete mode 100644 docs/superpowers/specs/2026-09-11-dashboard-settings-design.md delete mode 100644 scripts/render_check.py create mode 100644 tests/__init__.py create mode 100644 tests/conftest.py create mode 100644 tests/core/__init__.py create mode 100644 tests/core/config.py create mode 100644 tests/core/crypto.py create mode 100644 tests/core/errors.py create mode 100644 tests/core/fields.py create mode 100644 tests/core/specs.py create mode 100644 tests/core/utils.py create mode 100644 tests/core/version.py create mode 100644 tests/engines/__init__.py create mode 100644 tests/engines/base.py create mode 100644 tests/engines/docker_volume.py create mode 100644 tests/engines/firebird.py create mode 100644 tests/engines/mariadb.py create mode 100644 tests/engines/mongodb.py create mode 100644 tests/engines/mssql.py create mode 100644 tests/engines/mysql.py create mode 100644 tests/engines/postgresql.py create mode 100644 tests/engines/postgresql_cluster.py create mode 100644 tests/engines/redis.py create mode 100644 tests/engines/registry.py create mode 100644 tests/engines/sqlite.py create mode 100644 tests/engines/valkey.py create mode 100644 tests/services/__init__.py create mode 100644 tests/services/auth_providers.py create mode 100644 tests/services/compose_facts.py create mode 100644 tests/services/docker.py create mode 100644 tests/services/envfile.py create mode 100644 tests/services/http.py create mode 100644 tests/services/ports.py create mode 100644 tests/services/project.py create mode 100644 tests/services/renderer.py create mode 100644 tests/services/settings.py create mode 100644 tests/services/telemetry.py create mode 100644 tests/services/templates.py create mode 100644 tests/services/updater.py create mode 100644 tests/structure.py create mode 100644 tests/support.py diff --git a/.github/CONTRIBUTING.md b/.github/CONTRIBUTING.md index 76e16d0..735b9dc 100644 --- a/.github/CONTRIBUTING.md +++ b/.github/CONTRIBUTING.md @@ -168,6 +168,15 @@ uv run python main.py config channel stable # or: beta uv run python main.py update ``` +### Running the tests + +Unit tests live in `tests/`, mirroring `core/`, `services/` and `engines/`. They call +the functions directly: no Docker, no network, no built binary. + +```bash +uv run pytest +uv run ruff check . && uv run ruff format --check . && uv run mypy +``` --- ## Reporting Issues diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index cae7298..9b0883d 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -28,8 +28,8 @@ jobs: run: uv run ruff format --check . - name: types run: uv run mypy - - name: templates - run: uv run python scripts/render_check.py --templates templates + - name: tests + run: uv run pytest steps: - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4 - uses: ./.github/actions/setup @@ -48,18 +48,3 @@ jobs: - name: Build id: build uses: ./.github/actions/build - - # No PORTABASE_TEMPLATES_DIR: the binary must run on its own bundled copy. - - name: End-to-end agent creation - env: - BINARY: ${{ github.workspace }}/${{ steps.build.outputs.path }} - EDGE_KEY: eyJzZXJ2ZXJVcmwiOiJodHRwOi8veCIsImFnZW50SWQiOiJhIiwibWFzdGVyS2V5QjY0IjoiayJ9 - run: | - set -euo pipefail - cd "$(mktemp -d)" - "$BINARY" --help > /dev/null - "$BINARY" --non-interactive agent smoke --key "$EDGE_KEY" - "$BINARY" --non-interactive db add smoke --engine postgresql --mode new - "$BINARY" --non-interactive db list smoke - "$BINARY" --non-interactive build smoke --diff - cd smoke && docker compose config --quiet diff --git a/README.md b/README.md index 7350344..252a001 100644 --- a/README.md +++ b/README.md @@ -76,7 +76,7 @@ portabase dashboard auth add|list|remove NAME portabase start|stop|restart|logs|uninstall|build PATH ``` -`portabase db` still works for one release as an alias of `portabase agent db`. +`portabase db` was removed: database commands only apply to an agent, use `portabase agent db`. ## Upgrading from 26.08 or earlier diff --git a/commands/agent.py b/commands/agent.py index 397f65c..176bb0d 100644 --- a/commands/agent.py +++ b/commands/agent.py @@ -84,16 +84,22 @@ def run( provided = read_secret_flags(cfg.AGENT, settings) form = self.ui.form() answers = { - s.name: form.ask(s.field, provided.get(s.name)) for s in cfg.AGENT if s.core + setting.name: form.ask(setting.field, provided.get(setting.name)) + for setting in cfg.AGENT + if setting.core } env_vars = { - s.env: s.to_env(answers[s.name]) for s in cfg.AGENT if s.core and s.env + setting.env: setting.to_env(answers[setting.name]) + for setting in cfg.AGENT + if setting.core and setting.env } gateway = bool(answers["host_gateway"]) rows = [("Agent Name", name), ("Path", str(path))] rows += [ - (s.field.prompt, display(s, answers[s.name])) for s in cfg.AGENT if s.core + (setting.field.prompt, display(setting, answers[setting.name])) + for setting in cfg.AGENT + if setting.core ] rows.append(("Files to Create", "docker-compose.yml, .env, databases.json")) self.ui.summary(rows, title="SUMMARY") @@ -106,7 +112,11 @@ def run( apply_settings( self.ui, project, - {k: v for k, v in provided.items() if not cfg.AGENT.get(k).core}, + { + key: value + for key, value in provided.items() + if not cfg.AGENT.get(key).core + }, ) self._write(project) self.ui.success(f"Agent '{name}' created in {path}") @@ -158,8 +168,12 @@ def run(self, path: Annotated[Path, typer.Argument(help="Agent folder")]) -> Non show_settings(self.ui, project) if project.databases: rows = [ - [d.name, d.engine, self.engines.get(d.engine).describe(d)] - for d in project.databases + [ + database.name, + database.engine, + self.engines.get(database.engine).describe(database), + ] + for database in project.databases ] self.ui.table(["Name", "Engine", "Where"], rows, title="DATABASES") else: diff --git a/commands/auth.py b/commands/auth.py index f26e9c4..cdab244 100644 --- a/commands/auth.py +++ b/commands/auth.py @@ -125,12 +125,14 @@ def run( "host": host, } fields = ap.OIDC_FIELDS if provider_kind == "oidc" else ap.OAUTH_FIELDS - allowed = {f.name for f in fields} + allowed = {field.name for field in fields} stray = sorted( - k for k, v in values.items() if v is not None and k not in allowed + key + for key, value in values.items() + if value is not None and key not in allowed ) if stray: - flags = ", ".join("--" + k for k in stray) + flags = ", ".join("--" + name for name in stray) raise ValidationError(f"Not applicable to {provider_kind}: {flags}.") project = self.load(path) @@ -157,13 +159,13 @@ def run(self, path: PathArg) -> None: ["Kind", "Id", "Title", "Issuer / provider", "Callback"], [ [ - p.kind, - p.id, - p.values.get("title", ""), - p.values.get("issuer", p.id), - project.callback_url(p.id), + provider.kind, + provider.id, + provider.values.get("title", ""), + provider.values.get("issuer", provider.id), + project.callback_url(provider.id), ] - for p in providers + for provider in providers ], title=f"Login providers for {project.path.name}", ) @@ -188,7 +190,7 @@ def run( if not providers: self.ui.warning("No login provider to remove.") return - choices = [f"{p.id} ({p.kind})" for p in providers] + choices = [f"{provider.id} ({provider.kind})" for provider in providers] picked = self.ui.form().choice( "Which provider to remove?", choices, name="id" ) diff --git a/commands/base.py b/commands/base.py index c849de3..a36ed75 100644 --- a/commands/base.py +++ b/commands/base.py @@ -109,29 +109,3 @@ def build_typer(self) -> typer.Typer: def register(self, app: typer.Typer) -> None: app.add_typer(self.build_typer(), name=self.name, rich_help_panel=self.panel) - - -class DeprecatedAlias(CommandGroup): - def __init__( - self, ui: UI, telemetry: Telemetry, target: CommandGroup, *, name: str, use: str - ) -> None: - super().__init__(ui, telemetry) - self.name = name - self.help = f"Deprecated alias of '{use}'." - self.panel = target.panel - self._target = target - self._use = use - - @property - def commands(self) -> list[Command]: - return self._target.commands - - def build_typer(self) -> typer.Typer: - sub = super().build_typer() - ui, use, name = self.ui, self._use, self.name - - @sub.callback() - def _warn() -> None: - ui.warning(f"'portabase {name}' is deprecated, use 'portabase {use}'.") - - return sub diff --git a/commands/build.py b/commands/build.py index 4b5465e..85316df 100644 --- a/commands/build.py +++ b/commands/build.py @@ -92,7 +92,7 @@ def run( report = result.write(target) report_write(self.ui, report) self.ui.success( - f"Rendered {', '.join(p.name for p in report.wrote)} in {target}" + f"Rendered {', '.join(path.name for path in report.wrote)} in {target}" ) if kind == "agent" and output is None: self.ui.info(f"Restart to apply: portabase restart {path.name}") diff --git a/commands/dashboard.py b/commands/dashboard.py index a38c5e2..78d16b7 100644 --- a/commands/dashboard.py +++ b/commands/dashboard.py @@ -179,7 +179,7 @@ def run( provided = read_secret_flags(cfg.DASHBOARD, settings) apply_settings(self.ui, project, provided) - explicit = any(v is not None for v in provided.values()) + explicit = any(value is not None for value in provided.values()) if ( not self.ui.non_interactive and not explicit @@ -191,9 +191,9 @@ def run( rows.append(("Access URL", project.setting("url"))) rows += [ - (s.field.prompt, display(s, project.setting(s.name))) - for s in cfg.DASHBOARD - if s.name != "url" and project.env.get(s.env or "") is not None + (setting.field.prompt, display(setting, project.setting(setting.name))) + for setting in cfg.DASHBOARD + if setting.name != "url" and project.env.get(setting.env or "") is not None ] rows.append(("Files to Create", "docker-compose.yml, .env")) self.ui.summary(rows, title="SUMMARY") @@ -270,13 +270,13 @@ def run(self, path: PathArg) -> None: ["Kind", "Id", "Title", "Issuer / provider", "Callback"], [ [ - p.kind, - p.id, - p.values.get("title", ""), - p.values.get("issuer", p.id), - project.callback_url(p.id), + provider.kind, + provider.id, + provider.values.get("title", ""), + provider.values.get("issuer", provider.id), + project.callback_url(provider.id), ] - for p in providers + for provider in providers ], title="LOGIN PROVIDERS", ) diff --git a/commands/db.py b/commands/db.py index 867aaf1..bbee8ab 100644 --- a/commands/db.py +++ b/commands/db.py @@ -183,7 +183,10 @@ def run( return if target is None: - choices = [f"{d.name} ({d.engine}) [{d.id[:8]}]" for d in project.databases] + choices = [ + f"{database.name} ({database.engine}) [{database.id[:8]}]" + for database in project.databases + ] picked = self.ui.form().choice( "Which database to remove?", choices, name="id" ) @@ -231,23 +234,26 @@ def run(self, name: NameArg) -> None: self.ui.warning("No databases configured.") return rows = [] - for d in project.databases: - engine = self.engines.get(d.engine) + for database in project.databases: + engine = self.engines.get(database.engine) opts = ", ".join( - f"{k}={v}" for k, v in engine.non_default_options(d).items() + f"{key}={value}" + for key, value in engine.non_default_options(database).items() ) user = ( - "N/A" if d.engine in ("sqlite", "docker-volume") else (d.username or "") + "N/A" + if database.engine in ("sqlite", "docker-volume") + else (database.username or "") ) rows.append( [ - d.name, - d.database or "", - d.engine, - engine.describe(d), + database.name, + database.database or "", + database.engine, + engine.describe(database), user, opts, - d.id[:8] + "...", + database.id[:8] + "...", ] ) self.ui.table( diff --git a/commands/decrypt.py b/commands/decrypt.py index 0b06a9b..f1c08f5 100644 --- a/commands/decrypt.py +++ b/commands/decrypt.py @@ -72,15 +72,19 @@ def _single( out = output_path.resolve() try: decrypt_enc_file(enc_path, out, master_key) - except OSError as e: - raise DecryptionError(f"I/O error on {enc_path.name}: {e}", cause=e) from e + except OSError as error: + raise DecryptionError( + f"I/O error on {enc_path.name}: {error}", cause=error + ) from error self.ui.success(f"Decrypted {enc_path.name} → {out}") def _folder( self, in_dir: Path, output_path: Path | None, master_key: bytes ) -> None: enc_files = sorted( - p for p in in_dir.iterdir() if p.is_file() and p.suffix == ENC_SUFFIX + path + for path in in_dir.iterdir() + if path.is_file() and path.suffix == ENC_SUFFIX ) if not enc_files: self.ui.warning(f"No {ENC_SUFFIX} files found in {in_dir}.") @@ -101,8 +105,8 @@ def _folder( out = out_dir / default_output_for(enc_path) try: decrypt_enc_file(enc_path, out, master_key) - except (DecryptionError, OSError) as e: - failures.append((enc_path.name, str(e))) + except (DecryptionError, OSError) as error: + failures.append((enc_path.name, str(error))) succeeded = len(enc_files) - len(failures) self.ui.info( f"Done: {succeeded} succeeded, {len(failures)} failed " diff --git a/commands/flows/add_database.py b/commands/flows/add_database.py index 878c4eb..3098795 100644 --- a/commands/flows/add_database.py +++ b/commands/flows/add_database.py @@ -96,7 +96,7 @@ def _collect_options( self, form: Form, engine: DbEngine, provided: dict[str, str] ) -> dict[str, Any]: option_fields = engine.option_fields() - known = {f.name for f in option_fields} + known = {field.name for field in option_fields} unknown = set(provided) - known if unknown: raise ValidationError( @@ -115,14 +115,16 @@ def _collect_options( def _reject_irrelevant( values: dict[str, Any], fields: list[Field], engine: DbEngine, mode: str ) -> None: - relevant = {f.name for f in fields} | FLOW_KEYS + relevant = {field.name for field in fields} | FLOW_KEYS extra = sorted( - k for k, v in values.items() if v is not None and k not in relevant + key + for key, value in values.items() + if value is not None and key not in relevant ) if not extra: return - flags = ", ".join("--" + k.replace("_", "-") for k in extra) - applicable = ", ".join("--" + f.name.replace("_", "-") for f in fields) + flags = ", ".join("--" + key.replace("_", "-") for key in extra) + applicable = ", ".join("--" + field.name.replace("_", "-") for field in fields) raise ValidationError( f"Option(s) not applicable to {engine.key} in '{mode}' mode: {flags}.", hint=f"Applicable: {applicable}" diff --git a/commands/lifecycle.py b/commands/lifecycle.py index d0583ff..b277dd9 100644 --- a/commands/lifecycle.py +++ b/commands/lifecycle.py @@ -109,6 +109,6 @@ def run( self.docker.compose(path, ["down", "-v"]) try: shutil.rmtree(path) - except OSError as e: - self.ui.warning(f"Could not remove directory: {e}") + except OSError as error: + self.ui.warning(f"Could not remove directory: {error}") self.ui.success("Uninstalled") diff --git a/commands/settings.py b/commands/settings.py index 6638570..07ded24 100644 --- a/commands/settings.py +++ b/commands/settings.py @@ -80,9 +80,9 @@ def entry(*args: Any, **kwargs: Any) -> None: run(*args, **kwargs) static = [ - p - for p in inspect.signature(run, eval_str=True).parameters.values() - if p.kind is not inspect.Parameter.VAR_KEYWORD + parameter + for parameter in inspect.signature(run, eval_str=True).parameters.values() + if parameter.kind is not inspect.Parameter.VAR_KEYWORD ] signature = inspect.Signature(static + settings_parameters(registry)) entry.__signature__ = signature # type: ignore[attr-defined] @@ -119,9 +119,9 @@ def show_settings(ui: UI, project: SettingsProject) -> None: values = project.settings() for section, title in project.registry.sections.items(): rows = [ - (s.field.prompt, display(s, values[s.name])) - for s in project.registry.in_section(section) - if values[s.name] not in (None, "") + (setting.field.prompt, display(setting, values[setting.name])) + for setting in project.registry.in_section(section) + if values[setting.name] not in (None, "") ] if rows: ui.summary(rows, title=title.upper()) diff --git a/commands/update.py b/commands/update.py index dd6cafb..04a7eba 100644 --- a/commands/update.py +++ b/commands/update.py @@ -56,10 +56,10 @@ def run(self) -> None: def _latest(self) -> Release: try: release = self.checker.fetch_latest() - except NetworkError as e: + except NetworkError as error: raise UpdateError( - "Could not fetch latest release data from GitHub.", cause=e - ) from e + "Could not fetch latest release data from GitHub.", cause=error + ) from error if release is None: raise UpdateError("No release found for this channel.") return release diff --git a/core/config.py b/core/config.py index 1e4c418..da07a34 100644 --- a/core/config.py +++ b/core/config.py @@ -17,8 +17,8 @@ def all(self) -> dict: if not self.path.exists(): return {} try: - with open(self.path, encoding="utf-8") as f: - data = json.load(f) + with open(self.path, encoding="utf-8") as file: + data = json.load(file) except (OSError, json.JSONDecodeError): return {} return data if isinstance(data, dict) else {} @@ -31,8 +31,8 @@ def set(self, key: str, value) -> None: data[key] = value self.path.parent.mkdir(parents=True, exist_ok=True) tmp = self.path.with_suffix(".json.tmp") - with open(tmp, "w", encoding="utf-8") as f: - json.dump(data, f, indent=2) + with open(tmp, "w", encoding="utf-8") as file: + json.dump(data, file, indent=2) os.replace(tmp, self.path) @property diff --git a/core/utils.py b/core/utils.py index c676f18..c7eaaad 100644 --- a/core/utils.py +++ b/core/utils.py @@ -31,6 +31,10 @@ def generate_password(length: int = 16) -> str: return "".join(password) +def escape_yaml_double_quoted(value: str) -> str: + return value.replace("\\", "\\\\").replace('"', '\\"') + + def slugify_project_name(value: str, fallback: str = "portabase") -> str: slug = re.sub(r"[^a-z0-9_-]+", "-", value.lower()) slug = slug.strip("-_") @@ -52,6 +56,8 @@ def validate_edge_key(key: str) -> bool: return False required_fields = ["serverUrl", "agentId", "masterKeyB64"] - return all(field in data for field in required_fields) + return isinstance(data, dict) and all( + field in data for field in required_fields + ) except TypeError: return False diff --git a/core/version.py b/core/version.py index 0ea8299..eea5cfa 100644 --- a/core/version.py +++ b/core/version.py @@ -7,7 +7,9 @@ from pathlib import Path UNKNOWN = "unknown" -_PRE = re.compile(r"^(\d+)\.(\d+)\.(\d+)(?:[-.]?(rc|alpha|beta|a|b)(\d*))?$", re.I) +_PRE = re.compile( + r"^(\d+)\.(\d+)\.(\d+)(?:[-.]?(rc|alpha|beta|a|b)(\d*)(?:\.(\d+))?)?$", re.I +) @lru_cache(maxsize=1) @@ -15,23 +17,25 @@ def current_version() -> str: try: bundled = getattr(sys, "_MEIPASS", None) base = Path(bundled) if bundled else Path(__file__).parent.parent - with open(base / "pyproject.toml", "rb") as f: - return tomllib.load(f)["project"]["version"] + with open(base / "pyproject.toml", "rb") as file: + return tomllib.load(file)["project"]["version"] except (FileNotFoundError, KeyError, tomllib.TOMLDecodeError, AttributeError): return UNKNOWN def is_prerelease(version: str) -> bool: - m = _PRE.match(version.strip().lstrip("v")) - return bool(m and m.group(4)) + match = _PRE.match(version.strip().lstrip("v")) + return bool(match and match.group(4)) -def parse_version(version: str) -> tuple[int, int, int, int, int]: - m = _PRE.match(version.strip().lstrip("v")) - if not m: - return (0, 0, 0, 0, 0) - major, minor, patch = (int(m.group(i)) for i in (1, 2, 3)) - tag = (m.group(4) or "").lower() +def parse_version(version: str) -> tuple[int, int, int, int, int, int]: + match = _PRE.match(version.strip().lstrip("v")) + if not match: + return (0, 0, 0, 0, 0, 0) + major, minor, patch = (int(match.group(group)) for group in (1, 2, 3)) + tag = (match.group(4) or "").lower() rank = {"alpha": 0, "a": 0, "beta": 1, "b": 1, "rc": 2, "": 3}[tag] - num = int(m.group(5)) if m.group(5) else 0 - return (major, minor, patch, rank, num) + num, sub = match.group(5), match.group(6) + if not num: + num, sub = sub, None # "beta.2" is beta 2, like "beta2" + return (major, minor, patch, rank, int(num or 0), int(sub or 0)) diff --git a/docs/superpowers/plans/2026-09-11-plan-1-ci-hygiene.md b/docs/superpowers/plans/2026-09-11-plan-1-ci-hygiene.md deleted file mode 100644 index 9f0648a..0000000 --- a/docs/superpowers/plans/2026-09-11-plan-1-ci-hygiene.md +++ /dev/null @@ -1,954 +0,0 @@ -# Plan 1 — CI, hygiène et release (chantier A) - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** CI de PR bloquante (lint, secrets, sécurité pipeline, build smoke), workflows durcis et pinnés, `./release` remplacé par `bump.yml` — sans changer une ligne de comportement du CLI. - -**Architecture:** Un workflow `ci.yml` sur PR/push main avec des jobs indépendants. Les workflows de release existants restent structurellement identiques, seulement pinnés par SHA et restreints en permissions. La configuration ruff vit dans `pyproject.toml` avec des exclusions explicites pour le code legacy qui sera supprimé aux plans 2–4. - -**Tech Stack:** GitHub Actions, uv 0.9, ruff 0.16, pytest, PyInstaller 6.17, gitleaks-action v2, getplumber/plumber, Dependabot. - -**Spec:** `docs/superpowers/specs/2026-09-11-cli-refactor-design.md` — sections 9 (CI, sécurité, release) et 10 (étape A). - -## Global Constraints - -- Python `>=3.12` (pyproject actuel). Ne pas changer. -- Aucune modification de comportement du CLI dans ce plan. Seuls `pyproject.toml`, `.gitignore`, `.github/**`, `.gitleaks.toml` et le formatage (`ruff format`) bougent. -- Toutes les `uses:` pinnées par SHA complet + commentaire `# vX.Y.Z`. -- `permissions: {}` au top de chaque workflow ; permissions explicites par job. -- Le job `test` existe mais ne collecte aucun test (réservé à la spec tests). -- Aucun test unitaire dans ce plan (consigne utilisateur). Chaque tâche a des étapes de vérification exécutables. -- Commits en Conventional Commits (`chore`, `ci`, `build`, `style`). -- `gh` n'est pas authentifié sur ce poste : les appels `gh api` sur dépôts publics fonctionnent, `gh` sur `Portabase/cli` (protection de branche, secrets) ne fonctionne pas. Vérifier ces points dans l'interface GitHub. - -SHAs résolus le 2026-09-11 (à réutiliser tels quels) : - -| Action | Tag | SHA | -|---|---|---| -| actions/checkout | v4 | `11d5960a326750d5838078e36cf38b85af677262` | -| actions/upload-artifact | v4 | `ea165f8d65b6e75b540449e92b4886f43607fa02` | -| actions/download-artifact | v4 | `d3f86a106a0bac45b974a628896c90dbdf5c8093` | -| actions/attest-build-provenance | v2 | `e8998f949152b193b063cb0ec769d69d929409be` | -| astral-sh/setup-uv | v3 | `caf0cab7a618c569241d31dcd442f54681755d39` | -| astral-sh/ruff-action | v3 | `4919ec5cf1f49eff0871dbcea0da843445b837e6` | -| softprops/action-gh-release | v2 | `3bb12739c298aeb8a4eeaf626c5b8d85266b0e65` | -| mikepenz/release-changelog-builder-action | v5 | `c9dc8369bccbc41e0ac887f8fd674f5925d315f7` | -| gitleaks/gitleaks-action | v2 | `ff98106e4c7b2bc287b24eaf42907196329070c7` | -| getplumber/plumber | (doc officielle) | `3feac69e925e9771f8a495f4177af754d568c1ad` | - -Pour re-résoudre un SHA : `gh api repos///git/ref/tags/ --jq .object.sha` (si `.object.type == "tag"`, résoudre encore via `repos///git/tags/ --jq .object.sha`). - ---- - -## File Structure - -| Fichier | Action | Responsabilité | -|---|---|---| -| `pyproject.toml` | modifier | deps runtime/dev, config ruff, config pytest | -| `.gitignore` | modifier | retirer `uv.lock` (tracké, requis par `--frozen`) | -| `commands/*.py`, `core/*.py`, `main.py` | reformater seulement | `ruff format` mécanique, aucun changement sémantique | -| `.gitleaks.toml` | créer | allowlist des faux positifs | -| `.github/workflows/ci.yml` | créer | lint, test, gitleaks, plumber, build-smoke | -| `.github/workflows/python.yml` | modifier | pin SHA, permissions, `--frozen`, attestation | -| `.github/workflows/github.yml` | modifier | pin SHA, permissions par job | -| `.github/workflows/templates-upload.yml` | modifier | pin SHA, permissions, s3cmd sans `~/.s3cfg` | -| `.github/workflows/release.yml`, `release-candidate.yml` | modifier | `permissions: {}` top-level, retirer `packages: write` | -| `.github/dependabot.yml` | créer | github-actions + uv hebdo | -| `.github/workflows/bump.yml` | créer | remplace `./release` | -| `release` | supprimer | — | -| `.github/CONTRIBUTING.md` | modifier | procédure de release | - ---- - -### Task 1 : `pyproject.toml` — dépendances, ruff, pytest - -**Files:** -- Modify: `pyproject.toml` -- Modify: `.gitignore` - -**Interfaces:** -- Produces: commandes `uv run ruff check .`, `uv run ruff format --check .`, `uv run pytest` utilisables localement et en CI ; groupe `dev` avec `pyinstaller`, `ruff`, `pytest`. - -- [ ] **Step 1: Réécrire `pyproject.toml`** - -Remplacer le contenu intégral par : - -```toml -[project] -name = "portabase-cli" -version = "26.07.6" -description = "The official command line interface (CLI) for managing and deploying Portabase instances with ease." -readme = "README.md" -requires-python = ">=3.12" -dependencies = [ - "typer>=0.20.0", - "rich>=14.2.0", - "questionary>=2.1.0", - "requests>=2.32.5", - "pyyaml>=6.0.3", -] - -[dependency-groups] -dev = [ - "pyinstaller>=6.17.0", - "ruff>=0.16.0", - "pytest>=8.3", -] - -[tool.ruff] -target-version = "py312" -line-length = 88 -extend-exclude = [".venv", "build", "dist"] - -[tool.ruff.lint] -select = [ - "E", "F", "W", # pycodestyle / pyflakes - "I", # isort - "UP", # pyupgrade - "B", # bugbear - "BLE", # blind except - "S110", # try-except-pass - "E722", # bare except - "TID251", # banned imports (activé au plan 2 : rich.prompt, typer.prompt) - "SIM", - "TRY201", - "PLW1510", # subprocess.run sans check= -] -ignore = [ - "B008", # typer.Argument(...) / typer.Option(...) en défaut : idiome Typer - "E501", # line length géré par ruff format -] - -# Code legacy supprimé aux plans 2-4. Ne pas étendre cette liste : tout nouveau -# fichier doit passer sans exception. -[tool.ruff.lint.per-file-ignores] -"commands/agent.py" = ["BLE001", "E722", "S110", "SIM102"] -"commands/db.py" = ["BLE001", "E722", "S110"] -"commands/dashboard.py" = ["BLE001"] -"commands/common.py" = ["BLE001", "PLW1510"] -"core/config.py" = ["BLE001", "E722", "S110"] -"core/utils.py" = ["BLE001", "E722", "S110", "PLR1730"] -"core/updater.py" = ["BLE001", "TRY201"] -"core/network.py" = ["BLE001"] - -[tool.ruff.lint.isort] -known-first-party = ["commands", "core", "templates"] - -[tool.pytest.ini_options] -testpaths = ["tests"] -addopts = "-q" -``` - -- [ ] **Step 2: Retirer `uv.lock` de `.gitignore`** - -`.gitignore` devient : - -``` -dist/ -build/ -.venv/ -__pycache__/ -*.spec -``` - -- [ ] **Step 3: Régénérer le lock et synchroniser** - -Run: `uv lock && uv sync --all-groups` -Expected: `uv.lock` mis à jour (pyinstaller passe en groupe dev, ruff et pytest ajoutés), `.venv` contient `ruff` et `pytest`. - -- [ ] **Step 4: Vérifier que le CLI démarre toujours** - -Run: `uv run python main.py --version` -Expected: `Portabase CLI version: 26.07.6` (plus éventuel message de mise à jour). - -- [ ] **Step 5: Vérifier le lint** - -Run: `uv run ruff check .` -Expected: `All checks passed!`. Si des erreurs subsistent, ajuster **uniquement** `per-file-ignores` pour les fichiers legacy listés ; ne pas modifier le code Python. - -- [ ] **Step 6: Commit** - -```bash -git add pyproject.toml uv.lock .gitignore -git commit -m "build: move pyinstaller to dev group, add ruff and pytest config - -Legacy files get per-file-ignores for bare/blind excepts; those files are -rewritten in later plans and the ignores are removed with them." -``` - ---- - -### Task 2 : Formatage mécanique - -**Files:** -- Modify: tous les fichiers signalés par `ruff format --check` (5 fichiers au 2026-09-11) - -**Interfaces:** -- Produces: `uv run ruff format --check .` passe. - -- [ ] **Step 1: Lister les fichiers à reformater** - -Run: `uv run ruff format --check .` -Expected: `5 files would be reformatted, 17 files already formatted` (nombres indicatifs). - -- [ ] **Step 2: Appliquer** - -Run: `uv run ruff format .` - -- [ ] **Step 3: Vérifier que rien de sémantique n'a changé** - -Run: `git diff --stat && uv run python main.py --help` -Expected: diff uniquement sur espaces/quotes/retours à la ligne ; `--help` affiche les commandes `agent`, `dashboard`, `start`, `stop`, `restart`, `logs`, `uninstall`, `db`, `config`, `update`. - -- [ ] **Step 4: Vérifier lint + format ensemble** - -Run: `uv run ruff check . && uv run ruff format --check .` -Expected: les deux passent. - -- [ ] **Step 5: Commit** - -```bash -git add -A commands core main.py -git commit -m "style: apply ruff format" -``` - ---- - -### Task 3 : `.gitleaks.toml` - -**Files:** -- Create: `.gitleaks.toml` - -**Interfaces:** -- Produces: config lue par `gitleaks/gitleaks-action` (Task 4) et par `gitleaks detect` en local. - -- [ ] **Step 1: Créer le fichier** - -```toml -# Gitleaks configuration for Portabase CLI. -# Extends the default ruleset; only adds allowlists for known false positives. - -title = "portabase-cli" - -[extend] -useDefault = true - -[allowlist] -description = "Known false positives" -paths = [ - # Compose templates contain PASSWORD=${...} placeholders, never real secrets. - '''templates/.*''', - '''\.github/assets/templates/.*''', - # Lock file: hashes only. - '''uv\.lock''', -] -regexes = [ - # Compose interpolation placeholders. - '''\$\{[A-Z0-9_]+\}''', - # Test/fixture edge keys are base64 JSON with these field names, not credentials. - '''"masterKeyB64"''', -] -``` - -- [ ] **Step 2: Scanner l'historique en local** - -Run: `uvx --from gitleaks gitleaks detect --source . --config .gitleaks.toml --redact --no-banner || docker run --rm -v "$PWD:/repo" -w /repo ghcr.io/gitleaks/gitleaks:v8 detect --source . --config .gitleaks.toml --redact --no-banner` - -(Le binaire gitleaks n'est pas distribué via PyPI ; la première commande échouera, la seconde via Docker fonctionne. Si aucun des deux n'est disponible, passer : la CI fera le scan à la Task 4.) - -Expected: `no leaks found`. Si des fuites réelles sont trouvées dans l'historique : **s'arrêter et le signaler** — ne pas allowlister, ne pas réécrire l'historique sans décision explicite. - -- [ ] **Step 3: Commit** - -```bash -git add .gitleaks.toml -git commit -m "ci: add gitleaks config with template placeholder allowlist" -``` - ---- - -### Task 4 : `ci.yml` — lint, test, gitleaks, plumber, build-smoke - -**Files:** -- Create: `.github/workflows/ci.yml` - -**Interfaces:** -- Consumes: config ruff/pytest de Task 1, `.gitleaks.toml` de Task 3. -- Produces: check requis `CI / lint`, `CI / test`, `CI / gitleaks`, `CI / plumber`, `CI / build-smoke` sur chaque PR. Le job `build-smoke` sera enrichi au Plan 4 (invocation `agent --non-interactive`). - -- [ ] **Step 1: Créer le workflow** - -```yaml -name: CI - -on: - pull_request: - push: - branches: [main] - -permissions: {} - -concurrency: - group: ci-${{ github.ref }} - cancel-in-progress: true - -jobs: - lint: - name: lint - runs-on: ubuntu-latest - permissions: - contents: read - steps: - - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4 - - uses: astral-sh/setup-uv@caf0cab7a618c569241d31dcd442f54681755d39 # v3 - - name: Install - run: uv sync --frozen --all-groups - - name: Ruff check - run: uv run ruff check . --output-format=github - - name: Ruff format - run: uv run ruff format --check . - - test: - name: test - runs-on: ubuntu-latest - permissions: - contents: read - steps: - - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4 - - uses: astral-sh/setup-uv@caf0cab7a618c569241d31dcd442f54681755d39 # v3 - - name: Install - run: uv sync --frozen --all-groups - - name: Pytest - # Exit code 5 = no tests collected. Accepted until the test suite exists. - run: | - set +e - uv run pytest - code=$? - set -e - if [ "$code" -ne 0 ] && [ "$code" -ne 5 ]; then exit "$code"; fi - - gitleaks: - name: gitleaks - runs-on: ubuntu-latest - permissions: - contents: read - steps: - - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4 - with: - fetch-depth: 0 - - uses: gitleaks/gitleaks-action@ff98106e4c7b2bc287b24eaf42907196329070c7 # v2 - env: - GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} - GITLEAKS_CONFIG: .gitleaks.toml - - plumber: - name: plumber - runs-on: ubuntu-24.04 - permissions: - contents: read - security-events: write - steps: - - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4 - - uses: getplumber/plumber@3feac69e925e9771f8a495f4177af754d568c1ad - with: - score-push: false - upload-sarif: true - # First run: observe only. Tighten to min-score once the baseline is known. - soft-fail: true - - build-smoke: - name: build-smoke - runs-on: ubuntu-latest - permissions: - contents: read - steps: - - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4 - - uses: astral-sh/setup-uv@caf0cab7a618c569241d31dcd442f54681755d39 # v3 - - name: Install - run: uv sync --frozen --all-groups - - name: Build binary - run: | - rm -rf build dist *.spec - uv run pyinstaller \ - --onefile \ - --name portabase_smoke \ - --paths=. \ - --collect-all rich \ - --collect-all requests \ - --collect-data certifi \ - --add-data "pyproject.toml:." \ - main.py - - name: Smoke - run: | - ./dist/portabase_smoke --version - ./dist/portabase_smoke --help -``` - -- [ ] **Step 2: Valider la syntaxe YAML localement** - -Run: `uv run python -c "import yaml,sys; yaml.safe_load(open('.github/workflows/ci.yml')); print('ok')"` -Expected: `ok`. - -- [ ] **Step 3: Vérifier localement ce que fera le job lint** - -Run: `uv sync --frozen --all-groups && uv run ruff check . --output-format=github && uv run ruff format --check .` -Expected: aucune sortie d'erreur. - -- [ ] **Step 4: Vérifier localement ce que fera le job test** - -Run: `uv run pytest; echo "exit=$?"` -Expected: `exit=5` (aucun test collecté). - -- [ ] **Step 5: Vérifier localement ce que fera build-smoke** - -Run: `rm -rf build dist *.spec && uv run pyinstaller --onefile --name portabase_smoke --paths=. --collect-all rich --collect-all requests --collect-data certifi --add-data "pyproject.toml:." main.py && ./dist/portabase_smoke --version` -Expected: `Portabase CLI version: 26.07.6`. Puis `rm -rf build dist *.spec`. - -- [ ] **Step 6: Commit** - -```bash -git add .github/workflows/ci.yml -git commit -m "ci: add PR workflow (lint, test, gitleaks, plumber, build smoke)" -``` - ---- - -### Task 5 : Durcir `python.yml` (build binaires) - -**Files:** -- Modify: `.github/workflows/python.yml` - -**Interfaces:** -- Consumes: appelé par `release.yml` / `release-candidate.yml` via `workflow_call`. -- Produces: artefacts `portabase__` inchangés + attestation de provenance. - -- [ ] **Step 1: Réécrire le workflow** - -```yaml -name: Build Python Binaries - -on: - workflow_call: - -permissions: {} - -jobs: - build: - name: Build for ${{ matrix.os }} (${{ matrix.arch }}) - runs-on: ${{ matrix.runner }} - permissions: - contents: read - id-token: write - attestations: write - strategy: - matrix: - include: - - os: linux - arch: amd64 - runner: ubuntu-latest - - os: linux - arch: arm64 - runner: ubuntu-24.04-arm - - os: macos - arch: arm64 - runner: macos-latest - - os: macos - arch: amd64 - runner: macos-15-intel - - steps: - - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4 - - - uses: astral-sh/setup-uv@caf0cab7a618c569241d31dcd442f54681755d39 # v3 - - - name: Install - run: uv sync --frozen --all-groups - - - name: Build binary - run: | - rm -rf build dist *.spec - uv run pyinstaller \ - --onefile \ - --name portabase_${{ matrix.os }}_${{ matrix.arch }} \ - --paths=. \ - --collect-all rich \ - --collect-all requests \ - --collect-data certifi \ - --add-data "pyproject.toml:." \ - main.py - - - name: Smoke - run: ./dist/portabase_${{ matrix.os }}_${{ matrix.arch }} --version - - - name: Attest provenance - uses: actions/attest-build-provenance@e8998f949152b193b063cb0ec769d69d929409be # v2 - with: - subject-path: dist/portabase_${{ matrix.os }}_${{ matrix.arch }} - - - name: Upload artifacts - uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4 - with: - name: portabase_${{ matrix.os }}_${{ matrix.arch }} - path: dist/portabase_${{ matrix.os }}_${{ matrix.arch }} -``` - -Changements par rapport à l'actuel : `uv python install` remplacé par `uv sync --frozen` (respecte `.python-version` et le lock) ; étape `Smoke` ; attestation ; permissions explicites. - -- [ ] **Step 2: Valider YAML** - -Run: `uv run python -c "import yaml; yaml.safe_load(open('.github/workflows/python.yml')); print('ok')"` -Expected: `ok`. - -- [ ] **Step 3: Commit** - -```bash -git add .github/workflows/python.yml -git commit -m "ci: pin actions, scope permissions, attest binaries in build workflow" -``` - ---- - -### Task 6 : Durcir `github.yml` (release GitHub + Discord) - -**Files:** -- Modify: `.github/workflows/github.yml` - -**Interfaces:** -- Consumes: artefacts de Task 5. -- Produces: release GitHub identique à aujourd'hui. - -- [ ] **Step 1: Modifier uniquement l'en-tête et les `uses:`** - -Remplacer le bloc `jobs:` d'en-tête et les trois `uses:` ; le reste (changelog config, script Discord) reste identique. - -En-tête (après le bloc `on:` existant, avant `jobs:`) — ajouter : - -```yaml -permissions: {} -``` - -Job : - -```yaml -jobs: - create-release: - runs-on: ubuntu-latest - permissions: - contents: write - steps: - - name: Check out the repo - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4 - with: - fetch-depth: 0 - - - name: Download artifacts - if: inputs.artifact_name != '' - uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4 - with: - pattern: ${{ inputs.artifact_name }} - path: dist - merge-multiple: true -``` - -Et plus bas : - -```yaml - - name: Build Changelog - id: build_changelog - uses: mikepenz/release-changelog-builder-action@c9dc8369bccbc41e0ac887f8fd674f5925d315f7 # v5 -``` - -```yaml - - name: Create GitHub Release - uses: softprops/action-gh-release@3bb12739c298aeb8a4eeaf626c5b8d85266b0e65 # v2 -``` - -- [ ] **Step 2: Vérifier qu'aucun `@vN` non pinné ne reste** - -Run: `grep -nE 'uses: .*@v[0-9]' .github/workflows/github.yml` -Expected: aucune sortie. - -- [ ] **Step 3: Valider YAML** - -Run: `uv run python -c "import yaml; yaml.safe_load(open('.github/workflows/github.yml')); print('ok')"` -Expected: `ok`. - -- [ ] **Step 4: Commit** - -```bash -git add .github/workflows/github.yml -git commit -m "ci: pin actions and scope permissions in release workflow" -``` - ---- - -### Task 7 : Durcir `templates-upload.yml` (S3 sans fichier de credentials) - -**Files:** -- Modify: `.github/workflows/templates-upload.yml` - -**Interfaces:** -- Produces: même arborescence S3 qu'aujourd'hui (`cli/public/templates//` et `latest/`). La source reste `.github/assets/templates/` jusqu'au Plan 3 qui la déplace vers `templates/` et ajoute le manifest. - -- [ ] **Step 1: Réécrire le workflow** - -```yaml -name: Upload Templates to S3 - -on: - workflow_call: - inputs: - version: - required: true - type: string - is_prerelease: - required: true - type: boolean - secrets: - S3_ENDPOINT: - required: true - S3_ACCESS_KEY: - required: true - S3_SECRET_KEY: - required: true - S3_BUCKET: - required: true - -permissions: {} - -jobs: - upload: - runs-on: ubuntu-latest - permissions: - contents: read - env: - # s3cmd reads these flags; no config file is written to disk. - S3CMD_ARGS: >- - --access_key=${{ secrets.S3_ACCESS_KEY }} - --secret_key=${{ secrets.S3_SECRET_KEY }} - --host=${{ secrets.S3_ENDPOINT }} - --host-bucket=%(bucket)s.${{ secrets.S3_ENDPOINT }} - --ssl - steps: - - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4 - - - name: Install s3cmd - run: sudo apt-get update && sudo apt-get install -y s3cmd - - - name: Upload versioned templates - run: | - CLEAN_VERSION="${{ inputs.version }}" - CLEAN_VERSION="${CLEAN_VERSION#v}" - s3cmd $S3CMD_ARGS sync .github/assets/templates/ \ - "s3://${{ secrets.S3_BUCKET }}/cli/public/templates/${CLEAN_VERSION}/" --acl-public - - - name: Upload latest templates (stable only) - if: ${{ !inputs.is_prerelease }} - run: | - s3cmd $S3CMD_ARGS sync .github/assets/templates/ \ - "s3://${{ secrets.S3_BUCKET }}/cli/public/templates/latest/" --acl-public -``` - -Note : les secrets passés en arguments de ligne de commande sont masqués dans les logs par GitHub (`***`). C'est le compromis retenu ; l'alternative (`~/.s3cfg`) laisse les secrets en clair sur le disque du runner. - -- [ ] **Step 2: Valider YAML** - -Run: `uv run python -c "import yaml; yaml.safe_load(open('.github/workflows/templates-upload.yml')); print('ok')"` -Expected: `ok`. - -- [ ] **Step 3: Commit** - -```bash -git add .github/workflows/templates-upload.yml -git commit -m "ci: pass S3 credentials to s3cmd as flags instead of writing ~/.s3cfg" -``` - ---- - -### Task 8 : Permissions top-level sur `release.yml` et `release-candidate.yml` - -**Files:** -- Modify: `.github/workflows/release.yml:10-13` -- Modify: `.github/workflows/release-candidate.yml:12-15` - -**Interfaces:** -- Produces: workflows appelants avec permissions minimales ; les jobs `uses:` héritent des permissions déclarées dans les workflows appelés (Tasks 5–7). - -- [ ] **Step 1: Dans les deux fichiers, remplacer** - -```yaml -permissions: - contents: write - packages: write -``` - -par - -```yaml -permissions: - contents: write - id-token: write - attestations: write - security-events: write -``` - -Un workflow appelant doit déclarer au moins les permissions que les workflows appelés demandent (`contents: write` pour la release, `id-token`/`attestations` pour l'attestation). `packages: write` n'était utilisé par aucun job. - -- [ ] **Step 2: Vérifier** - -Run: `grep -n "packages" .github/workflows/*.yml` -Expected: aucune sortie. - -- [ ] **Step 3: Commit** - -```bash -git add .github/workflows/release.yml .github/workflows/release-candidate.yml -git commit -m "ci: drop unused packages permission, declare attestation permissions" -``` - ---- - -### Task 9 : Dependabot - -**Files:** -- Create: `.github/dependabot.yml` - -**Interfaces:** -- Produces: PRs hebdomadaires pour les SHAs d'actions et les dépendances uv. - -- [ ] **Step 1: Créer le fichier** - -```yaml -version: 2 -updates: - - package-ecosystem: github-actions - directory: / - schedule: - interval: weekly - groups: - actions: - patterns: ["*"] - commit-message: - prefix: "ci" - - - package-ecosystem: uv - directory: / - schedule: - interval: weekly - groups: - python: - patterns: ["*"] - commit-message: - prefix: "build" -``` - -- [ ] **Step 2: Valider YAML** - -Run: `uv run python -c "import yaml; yaml.safe_load(open('.github/dependabot.yml')); print('ok')"` -Expected: `ok`. - -- [ ] **Step 3: Commit** - -```bash -git add .github/dependabot.yml -git commit -m "ci: enable dependabot for actions and uv" -``` - ---- - -### Task 10 : `bump.yml` remplace `./release` - -**Files:** -- Create: `.github/workflows/bump.yml` -- Delete: `release` -- Modify: `.github/CONTRIBUTING.md` - -**Interfaces:** -- Produces: déclenchement manuel qui commit `chore(release): X`, tague `X` et pousse. Le push du tag déclenche `release.yml` ou `release-candidate.yml` selon le motif, exactement comme le script. - -- [ ] **Step 1: Créer le workflow** - -```yaml -name: Bump version - -on: - workflow_dispatch: - inputs: - version: - description: "Version (e.g. 26.09.0 or 26.09.0rc1). No leading v." - required: true - type: string - channel: - description: "stable: only from main. rc: any branch." - required: true - type: choice - options: [stable, rc] - default: rc - -permissions: {} - -jobs: - bump: - runs-on: ubuntu-latest - permissions: - contents: write - steps: - - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4 - with: - fetch-depth: 0 - # Use a PAT if branch protection blocks GITHUB_TOKEN pushes to main. - token: ${{ secrets.GITHUB_TOKEN }} - - - name: Validate version against channel - env: - VERSION: ${{ inputs.version }} - CHANNEL: ${{ inputs.channel }} - REF: ${{ github.ref_name }} - run: | - set -euo pipefail - if [[ "$VERSION" == v* ]]; then - echo "::error::Version must not start with 'v'"; exit 1 - fi - if [[ "$CHANNEL" == "stable" ]]; then - if [[ "$REF" != "main" ]]; then - echo "::error::stable releases are only allowed from main (got $REF)"; exit 1 - fi - if [[ ! "$VERSION" =~ ^[0-9]+\.[0-9]+\.[0-9]+$ ]]; then - echo "::error::stable version must match X.Y.Z"; exit 1 - fi - else - if [[ ! "$VERSION" =~ ^[0-9]+\.[0-9]+\.[0-9]+([-.]?(rc|alpha|beta|a|b)[0-9]*(\.[0-9]+)?)$ ]]; then - echo "::error::rc version must match X.Y.Z(rc|a|b|alpha|beta)N"; exit 1 - fi - fi - if git rev-parse "$VERSION" >/dev/null 2>&1; then - echo "::error::Tag $VERSION already exists"; exit 1 - fi - - - name: Update version files - env: - VERSION: ${{ inputs.version }} - run: | - set -euo pipefail - DATE=$(date -u +%F) - sed -i "s/^version = \".*\"/version = \"$VERSION\"/" pyproject.toml - if [ -f CITATION.cff ]; then - sed -i "s/^version: .*/version: $VERSION/" CITATION.cff - sed -i "s/^date-released: .*/date-released: \"$DATE\"/" CITATION.cff - fi - git diff --stat - - - name: Commit, tag, push - env: - VERSION: ${{ inputs.version }} - run: | - set -euo pipefail - git config user.name "github-actions[bot]" - git config user.email "41898282+github-actions[bot]@users.noreply.github.com" - git add pyproject.toml CITATION.cff - if git diff --cached --quiet; then - echo "No version change to commit" - else - git commit -m "chore(release): $VERSION" - fi - git tag -a "$VERSION" -m "Release $VERSION" - git push origin HEAD - git push origin "$VERSION" -``` - -Différences avec le script : pas de `package.json` / `Cargo.toml` (absents du dépôt) ; `git add .` remplacé par un `add` ciblé ; identité bot. - -Point à vérifier dans l'interface GitHub (Settings → Branches) : si `main` exige une PR, `git push origin HEAD` sera refusé pour `GITHUB_TOKEN`. Deux solutions : (a) autoriser `github-actions[bot]` à contourner la règle ; (b) créer un PAT fine-grained (Contents: write) stocké en secret `RELEASE_TOKEN` et remplacer `token: ${{ secrets.GITHUB_TOKEN }}` par `token: ${{ secrets.RELEASE_TOKEN }}`. Note : un push effectué avec `GITHUB_TOKEN` ne déclenche **pas** d'autres workflows par design GitHub — **le push du tag ne déclenchera donc pas `release.yml`**. Avec un PAT (`RELEASE_TOKEN`), il le déclenche. → **Utiliser un PAT est obligatoire** pour que le tag lance la release. Créer le secret avant le premier usage. - -- [ ] **Step 2: Remplacer le token par le PAT** - -Dans le workflow ci-dessus, `token: ${{ secrets.GITHUB_TOKEN }}` → `token: ${{ secrets.RELEASE_TOKEN }}` et supprimer le commentaire au-dessus. Le secret `RELEASE_TOKEN` (fine-grained PAT, dépôt `Portabase/cli`, permissions Contents: Read and write, Metadata: Read) doit être créé par un mainteneur dans Settings → Secrets → Actions. - -- [ ] **Step 3: Supprimer le script** - -Run: `git rm release` - -- [ ] **Step 4: Documenter dans CONTRIBUTING.md** - -Ajouter une section à la fin de `.github/CONTRIBUTING.md` : - -```markdown -## Releasing - -Releases are cut from GitHub Actions, never from a local machine. - -1. Open **Actions → Bump version → Run workflow**. -2. Pick the branch (`main` for stable, any branch for a release candidate). -3. Enter the version without a leading `v` (`26.09.0` for stable, `26.09.0rc1` for a candidate) and the matching channel. -4. The workflow commits `chore(release): `, creates the tag and pushes. The tag triggers the build, the GitHub release, the Discord notification and the template upload. - -Stable versions must match `X.Y.Z` and can only be cut from `main`. -``` - -- [ ] **Step 5: Valider YAML** - -Run: `uv run python -c "import yaml; yaml.safe_load(open('.github/workflows/bump.yml')); print('ok')"` -Expected: `ok`. - -- [ ] **Step 6: Commit** - -```bash -git add .github/workflows/bump.yml .github/CONTRIBUTING.md -git commit -m "ci: replace ./release script with bump workflow" -``` - ---- - -### Task 11 : Vérification de bout en bout sur GitHub - -**Files:** aucun. - -**Interfaces:** -- Consumes: tout ce qui précède. - -- [ ] **Step 1: Pousser une branche et ouvrir une PR** - -```bash -git checkout -b ci/hygiene -git push -u origin ci/hygiene -gh pr create --fill --title "ci: PR workflow, pinned actions, bump workflow" --body "Implements plan 1 (chantier A) of docs/superpowers/specs/2026-09-11-cli-refactor-design.md. No CLI behaviour change." -``` - -(`gh` non authentifié ici : créer la PR depuis l'interface si la commande échoue.) - -- [ ] **Step 2: Vérifier les checks** - -Expected dans l'onglet Checks : `lint`, `test`, `gitleaks`, `plumber`, `build-smoke` tous verts. `plumber` publie un rapport SARIF dans Security → Code scanning ; noter le score obtenu. - -- [ ] **Step 3: Si `plumber` remonte des findings sur les workflows** - -Les traiter dans la même PR si triviaux (permission manquante, action non pinnée oubliée). Sinon ouvrir une issue avec la liste et laisser `soft-fail: true`. - -- [ ] **Step 4: Créer le secret `RELEASE_TOKEN`** - -Settings → Secrets and variables → Actions → New repository secret. PAT fine-grained, dépôt `Portabase/cli`, Contents: Read and write, Metadata: Read. - -- [ ] **Step 5: Merger, puis tester `bump.yml` avec un rc jetable** - -Actions → Bump version → branche `main`, version `26.07.7rc1`, channel `rc`. Expected : commit `chore(release): 26.07.7rc1` sur `main`, tag créé, `release-candidate.yml` déclenché, binaires attestés publiés en pre-release, templates uploadés sous `templates/26.07.7rc1/`. - -- [ ] **Step 6: Rendre les checks requis** - -Settings → Branches → `main` → Require status checks : `lint`, `test`, `gitleaks`, `build-smoke`. Laisser `plumber` non requis tant que `soft-fail: true`. - ---- - -## Self-review - -**Spec coverage (§9, §10 A) :** -- 9.1 `ci.yml` : lint ✔ (T4), test vide ✔ (T4), gitleaks ✔ (T3, T4), plumber ✔ (T4), build-smoke `--version` ✔ (T4 ; l'invocation `agent --non-interactive` arrive au Plan 4), `render-check` et `engines-check` → Plan 3 (dépendent des templates `.j2` et du registre). -- 9.2 pin SHA ✔ (T4–T8), Dependabot ✔ (T9), `permissions: {}` ✔, `packages: write` retiré ✔ (T8), `~/.s3cfg` supprimé ✔ (T7), attestation ✔ (T5). -- 9.3 `bump.yml` ✔ (T10), `./release` supprimé ✔, pas de release-please ✔, question branch protection → T10/T11. `templates-hotfix.yml` et manifest → Plan 3. -- 9.4 `pyproject.toml` ✔ (T1) ; `jinja2` ajouté au Plan 3 quand il est utilisé. -- 10 A : shippable stable ✔ (T11 step 5 le prouve avec un rc). - -**Placeholder scan :** aucun TBD/TODO. Toutes les étapes ont leur contenu ou leur commande. - -**Type consistency :** noms de jobs identiques entre T4 et T11 (`lint`, `test`, `gitleaks`, `plumber`, `build-smoke`) ; secret `RELEASE_TOKEN` cohérent T10/T11 ; SHAs identiques entre tâches. - -**Écart connu :** T1 `per-file-ignores` liste des fichiers/règles déduits du run ruff du 2026-09-11 ; si ruff remonte une règle non listée sur un fichier legacy, l'ajouter à la liste de ce fichier (pas de correction de code). diff --git a/docs/superpowers/plans/2026-09-11-plan-2-foundations-lifecycle.md b/docs/superpowers/plans/2026-09-11-plan-2-foundations-lifecycle.md deleted file mode 100644 index 51f8cae..0000000 --- a/docs/superpowers/plans/2026-09-11-plan-2-foundations-lifecycle.md +++ /dev/null @@ -1,2385 +0,0 @@ -# Plan 2 — Fondations et lifecycle (chantiers B + C) - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** Poser les fondations POO (erreurs, `ui/`, services d'infrastructure, `Command`, catcher, télémétrie no-op, updater sans auto-update) et réécrire les commandes de lifecycle/config/update dessus, tout en gardant `agent`, `dashboard` et `db` sur l'ancien code via un adaptateur — le CLI reste shippable en stable à la fin. - -**Architecture:** `main.py` construit les dépendances (`UI`, `Telemetry`, `GlobalConfig`, `HttpClient`, `DockerRunner`) et les injecte dans des classes `Command` enregistrées sur Typer. Un seul `try` dans `main()` traduit `PortabaseError` en message + code de sortie. Les commandes legacy sont enregistrées telles quelles par `LegacyCommand` ; elles continuent d'importer `core.utils.console` jusqu'au Plan 4. - -**Tech Stack:** Python 3.12, Typer 0.25 / Click 8.4, Rich 15, questionary 2.1, requests. - -**Spec:** `docs/superpowers/specs/2026-09-11-cli-refactor-design.md` — sections 3, 4.1, 7, 8, 10 (B, C). - -## Global Constraints - -- Prérequis : Plan 1 exécuté (ruff configuré, CI en place). -- Règle de dépendance descendante : `commands → services, engines, ui, core` ; `services → engines, core` (jamais `ui`) ; `ui → core` ; `core → rien`. Vérifiée par ruff `TID251` (Task 12). -- `rich.prompt`, `typer.prompt`, `typer.confirm`, `print` interdits hors `ui/` (ruff `TID251`, activé Task 12 avec exceptions legacy). -- `typer.Exit` n'est levé nulle part hors des fichiers legacy ; le nouveau code lève `PortabaseError`. -- Pas de tests unitaires (consigne). Chaque tâche a des vérifications exécutables ; les commandes Docker sont vérifiées avec un dossier agent réel si Docker est disponible, sinon sur leurs chemins d'erreur. -- Ne pas toucher `commands/agent.py`, `commands/db.py`, `commands/dashboard.py`, `commands/decrypt.py`, `core/crypto.py`, `core/network.py`, `core/docker.py`, `templates/compose.py` (réécrits ou supprimés au Plan 4). `decrypt` (ajouté en 26.08.12) est enregistré via `LegacyCommand` comme `agent`/`dashboard`. `core/utils.py` : seulement retirer `current_version` (Task 2). -- Déviation spec assumée : `Field` vit dans `core/fields.py` (partagé par `ui.Form` et `engines`), pas dans `engines/base.py`. -- Nom de la clé de config existante conservé : `update_channel` (valeurs `stable` / `beta`). -- Commits Conventional Commits, un par tâche minimum. - ---- - -## File Structure - -| Fichier | Action | Responsabilité | -|---|---|---| -| `core/errors.py` | créer | hiérarchie `PortabaseError` | -| `core/version.py` | créer | `current_version()`, `parse_version()`, `is_prerelease()` | -| `core/utils.py` | modifier | retirer `current_version` (re-export pour legacy) | -| `core/config.py` | modifier | ajouter classe `GlobalConfig` ; fonctions legacy conservées | -| `core/fields.py` | créer | `Field` | -| `ui/theme.py` | créer | `PALETTE`, `RICH_THEME`, `QUESTIONARY_STYLE` | -| `ui/components/base.py` | créer | `Component` | -| `ui/components/hints.py` | créer | `HINTS`, `Hint` | -| `ui/components/message.py` | créer | `Message` | -| `ui/components/banner.py` | créer | `Banner` | -| `ui/components/section.py` | créer | `Section` | -| `ui/components/status.py` | créer | `Status` | -| `ui/components/progress.py` | créer | `Progress` (téléchargement) | -| `ui/components/prompt.py` | créer | `Prompt` (questionary) | -| `ui/form.py` | créer | `Form` | -| `ui/__init__.py` | créer | façade `UI` | -| `services/http.py` | créer | `HttpClient` | -| `services/docker.py` | créer | `DockerRunner` | -| `services/telemetry.py` | créer | `Telemetry`, `NoopTelemetry`, `ConsoleTelemetry`, `TelemetryHub`, `TelemetryFactory` | -| `services/updater.py` | créer | `Release`, `UpdateChecker`, `Updater` | -| `commands/base.py` | créer | `Command`, `CommandGroup`, `LegacyCommand` | -| `commands/lifecycle.py` | créer | `Start/Stop/Restart/Logs/Uninstall` | -| `commands/config.py` | réécrire | `ConfigCommands` | -| `commands/update.py` | créer | `UpdateCommand` | -| `main.py` | réécrire | `Settings`, `build_app`, `main` | -| `commands/common.py`, `core/updater.py` | supprimer | — | -| `pyproject.toml` | modifier | `TID251`, per-file-ignores mis à jour | - ---- - -### Task 1 : `core/errors.py` - -**Files:** -- Create: `core/errors.py` - -**Interfaces:** -- Produces: `PortabaseError(message, *, hint=None, cause=None)` avec attributs `message`, `hint`, `cause`, classe-attributs `code: str`, `exit_code: int` ; sous-classes `UserAbort`, `ValidationError`, `ConfigError`, `DockerError`, `TemplateError`, `NetworkError`, `UpdateError`. - -- [ ] **Step 1: Écrire le module** - -```python -"""Exception hierarchy. Every error the CLI reports to a user is one of these.""" - -from __future__ import annotations - - -class PortabaseError(Exception): - code: str = "E_GENERIC" - exit_code: int = 1 - - def __init__( - self, - message: str, - *, - hint: str | None = None, - cause: BaseException | None = None, - ) -> None: - super().__init__(message) - self.message = message - self.hint = hint - self.cause = cause - if cause is not None: - self.__cause__ = cause - - def __str__(self) -> str: - return self.message - - -class UserAbort(PortabaseError): - code = "E_ABORT" - exit_code = 130 - - def __init__(self, message: str = "Cancelled.", **kwargs) -> None: - super().__init__(message, **kwargs) - - -class ValidationError(PortabaseError): - code = "E_VALIDATION" - exit_code = 2 - - -class ConfigError(PortabaseError): - code = "E_CONFIG" - exit_code = 3 - - -class DockerError(PortabaseError): - code = "E_DOCKER" - exit_code = 4 - - -class TemplateError(PortabaseError): - code = "E_TEMPLATE" - exit_code = 5 - - -class NetworkError(PortabaseError): - code = "E_NETWORK" - exit_code = 6 - - -class UpdateError(PortabaseError): - code = "E_UPDATE" - exit_code = 7 -``` - -- [ ] **Step 2: Vérifier** - -Run: `uv run python -c "from core.errors import *; e = DockerError('daemon down', hint='start it'); print(e.code, e.exit_code, e, e.hint); assert isinstance(e, PortabaseError)"` -Expected: `E_DOCKER 4 daemon down start it`. - -- [ ] **Step 3: Commit** - -```bash -git add core/errors.py -git commit -m "feat(core): add PortabaseError hierarchy with stable codes and exit codes" -``` - ---- - -### Task 2 : `core/version.py` et `core/fields.py` - -**Files:** -- Create: `core/version.py` -- Create: `core/fields.py` -- Modify: `core/utils.py:197-214` (fonction `current_version`) - -**Interfaces:** -- Produces: `current_version() -> str` ; `parse_version(v: str) -> tuple` ; `is_prerelease(v: str) -> bool` ; `Field` dataclass. - -- [ ] **Step 1: Écrire `core/version.py`** - -```python -"""CLI version helpers. Version is read from the bundled pyproject.toml.""" - -from __future__ import annotations - -import re -import sys -import tomllib -from functools import lru_cache -from pathlib import Path - -UNKNOWN = "unknown" -_PRE = re.compile(r"^(\d+)\.(\d+)\.(\d+)(?:[-.]?(rc|alpha|beta|a|b)(\d*))?$", re.I) - - -@lru_cache(maxsize=1) -def current_version() -> str: - try: - base = Path(sys._MEIPASS) if getattr(sys, "frozen", False) else Path(__file__).parent.parent - with open(base / "pyproject.toml", "rb") as f: - return tomllib.load(f)["project"]["version"] - except (FileNotFoundError, KeyError, tomllib.TOMLDecodeError, AttributeError): - return UNKNOWN - - -def is_prerelease(version: str) -> bool: - m = _PRE.match(version.strip().lstrip("v")) - return bool(m and m.group(4)) - - -def parse_version(version: str) -> tuple[int, int, int, int, int]: - """Sortable tuple. Pre-releases sort before the final release of the same number. - - (major, minor, patch, pre_rank, pre_number) — pre_rank: 0 alpha/a, 1 beta/b, 2 rc, 3 final. - """ - m = _PRE.match(version.strip().lstrip("v")) - if not m: - return (0, 0, 0, 0, 0) - major, minor, patch = (int(m.group(i)) for i in (1, 2, 3)) - tag = (m.group(4) or "").lower() - rank = {"alpha": 0, "a": 0, "beta": 1, "b": 1, "rc": 2, "": 3}[tag] - num = int(m.group(5)) if m.group(5) else 0 - return (major, minor, patch, rank, num) -``` - -- [ ] **Step 2: Écrire `core/fields.py`** - -```python -"""Declarative input field. Used by ui.Form to prompt or validate a value.""" - -from __future__ import annotations - -from collections.abc import Callable -from dataclasses import dataclass -from typing import Any, Literal - -FieldKind = Literal["text", "int", "secret", "bool", "choice", "path"] - - -@dataclass(frozen=True) -class Field: - name: str - prompt: str - kind: FieldKind = "text" - default: Any = None - choices: tuple[str, ...] = () - help: str | None = None - validator: Callable[[Any], Any] | None = None - - @property - def flag(self) -> str: - return "--" + self.name.replace("_", "-") -``` - -- [ ] **Step 3: Retirer `current_version` de `core/utils.py`** - -Supprimer la fonction `current_version` (lignes ~197–214) et ajouter en tête des imports : - -```python -from core.version import current_version # noqa: F401 — re-export for legacy modules -``` - -`core/network.py` et `core/updater.py` importent `current_version` depuis `core.utils` ; le re-export les garde fonctionnels jusqu'à leur suppression. - -- [ ] **Step 4: Vérifier** - -Run: `uv run python -c "from core.version import *; print(current_version(), is_prerelease('26.09.0rc1'), parse_version('26.09.0rc1') < parse_version('26.09.0'), parse_version('26.10.0') > parse_version('26.9.9'))" && uv run python main.py --version` -Expected: `26.07.6 True True True` puis `Portabase CLI version: 26.07.6`. - -- [ ] **Step 5: Commit** - -```bash -git add core/version.py core/fields.py core/utils.py -git commit -m "feat(core): add version helpers and Field descriptor" -``` - ---- - -### Task 3 : `GlobalConfig` - -**Files:** -- Modify: `core/config.py` - -**Interfaces:** -- Produces: `GlobalConfig(path: Path = GLOBAL_CONFIG_FILE)` avec `get(key, default=None)`, `set(key, value)`, `all() -> dict`, `cache_dir: Path` (`~/.portabase/cache`) ; propriétés typées `update_channel: str | None`, `telemetry: bool`, `telemetry_endpoint: str | None`. -- Les fonctions module-level existantes restent (legacy). - -- [ ] **Step 1: Ajouter la classe en fin de `core/config.py`** - -```python -class GlobalConfig: - """~/.portabase/config.json. Unknown keys are preserved.""" - - KNOWN_KEYS = ("update_channel", "telemetry", "telemetry_endpoint") - - def __init__(self, path: Path = GLOBAL_CONFIG_FILE) -> None: - self.path = path - self.cache_dir = path.parent / "cache" - - def all(self) -> dict: - if not self.path.exists(): - return {} - try: - with open(self.path, encoding="utf-8") as f: - data = json.load(f) - except (OSError, json.JSONDecodeError): - return {} - return data if isinstance(data, dict) else {} - - def get(self, key: str, default=None): - return self.all().get(key, default) - - def set(self, key: str, value) -> None: - data = self.all() - data[key] = value - self.path.parent.mkdir(parents=True, exist_ok=True) - tmp = self.path.with_suffix(".json.tmp") - with open(tmp, "w", encoding="utf-8") as f: - json.dump(data, f, indent=2) - os.replace(tmp, self.path) - - @property - def update_channel(self) -> str | None: - return self.get("update_channel") - - @property - def telemetry(self) -> bool: - return str(self.get("telemetry", "false")).lower() in ("1", "true", "yes") - - @property - def telemetry_endpoint(self) -> str | None: - return self.get("telemetry_endpoint") -``` - -Ajouter au-dessus des fonctions legacy le commentaire : - -```python -# --- Legacy helpers below: used by commands/agent.py, db.py, dashboard.py, core/updater.py. -# --- Removed in plan 4. New code uses GlobalConfig. -``` - -- [ ] **Step 2: Vérifier** - -Run: `uv run python -c " -from pathlib import Path; import tempfile -from core.config import GlobalConfig -c = GlobalConfig(Path(tempfile.mkdtemp())/'config.json') -print(c.all(), c.telemetry); c.set('update_channel','beta'); c.set('telemetry', True) -print(c.update_channel, c.telemetry, c.all())"` -Expected: `{} False` puis `beta True {'update_channel': 'beta', 'telemetry': True}`. - -- [ ] **Step 3: Commit** - -```bash -git add core/config.py -git commit -m "feat(core): add GlobalConfig class over ~/.portabase/config.json" -``` - ---- - -### Task 4 : `ui/theme.py` et composants d'affichage - -**Files:** -- Create: `ui/__init__.py` (vide pour l'instant, rempli Task 6) -- Create: `ui/theme.py` -- Create: `ui/components/__init__.py` (vide) -- Create: `ui/components/base.py` -- Create: `ui/components/hints.py` -- Create: `ui/components/message.py` -- Create: `ui/components/banner.py` -- Create: `ui/components/section.py` -- Create: `ui/components/status.py` -- Create: `ui/components/progress.py` - -**Interfaces:** -- Produces: `Component(console)` ; `Hint(console).random() -> str` ; `Message(console).success/info/warning(text)`, `.error(exc: PortabaseError, *, verbose: bool, unexpected: bool)` ; `Banner(console)()` ; `Section(console)(title)` ; `Status(console)(text) -> ContextManager` ; `Progress(console).download(description, total) -> ContextManager[Callable[[int], None]]`. - -- [ ] **Step 1: `ui/theme.py`** - -```python -"""Single source of visual tokens. Rich theme and questionary style derive from PALETTE.""" - -from __future__ import annotations - -from questionary import Style -from rich.theme import Theme - -PALETTE = { - "brand": "#ff6600", - "accent": "#5f00d7", - "info": "cyan", - "warning": "magenta", - "danger": "red", - "success": "green", - "muted": "grey50", -} - -RICH_THEME = Theme( - { - "info": f"dim {PALETTE['info']}", - "warning": PALETTE["warning"], - "danger": f"bold {PALETTE['danger']}", - "success": f"bold {PALETTE['success']}", - "title": f"bold white on {PALETTE['accent']}", - "key": f"bold {PALETTE['brand']}", - "value": "white", - "hint": f"italic {PALETTE['muted']}", - "brand": f"bold {PALETTE['brand']}", - } -) - -QUESTIONARY_STYLE = Style( - [ - ("qmark", f"fg:{PALETTE['brand']} bold"), - ("question", "bold"), - ("pointer", f"fg:{PALETTE['brand']} bold"), - ("highlighted", f"fg:black bg:{PALETTE['brand']} bold"), - ("selected", f"fg:{PALETTE['brand']} bold"), - ("answer", f"fg:{PALETTE['brand']}"), - ] -) - -QUESTIONARY_STYLE_PLAIN = Style([]) -``` - -- [ ] **Step 2: `ui/components/base.py`** - -```python -from __future__ import annotations - -from rich.console import Console - - -class Component: - """Stateless renderable bound to a console. Instantiate per call.""" - - def __init__(self, console: Console) -> None: - self.console = console -``` - -- [ ] **Step 3: `ui/components/hints.py`** - -Reprendre la liste `HINTS` de `core/utils.py:42-66` telle quelle. - -```python -from __future__ import annotations - -import random - -from ui.components.base import Component - -HINTS = [ - "The Edge Key contains the connection details for dashboard and agent communication.", - "Portabase uses Docker Compose to isolate your databases.", - "You can list all configured databases using 'portabase db list '.", - "Running 'portabase stop' will gracefully shut down your containers.", - "The agent polls the github for configuration updates.", - "Logs can be viewed in real-time with 'portabase logs '.", - "Custom environment variables can be added to the generated .env file.", - "Need to update? Use 'portabase update' to get the latest version.", - "You can add multiple databases to a single agent during setup.", - "Portabase Dashboard provides a web interface to manage your infrastructure.", - "Is Docker not running? The CLI will offer to start it for you!", - "All configurations are stored locally in the component's folder.", - "The 'portabase restart' command is useful after manual .env modifications.", - "Portabase is open-source! Check our GitHub to contribute.", - "Using the --start flag with 'agent' or 'dashboard' skips the final prompt.", - "Internal databases are automatically backed up when using volumes.", - "The dashboard requires a PostgreSQL database to store its own data.", - "You can change the update channel to 'beta' in the config for early features.", - "Portabase network ensures secure communication between your containers.", - "Lost your Edge Key? You can find it in the dashboard.", - "The 'portabase uninstall' command safely removes containers and their data.", - "Use 'portabase --version' to check your current installation details.", - "The 'databases.json' file keeps track of all managed database instances.", -] - - -class Hint(Component): - def random(self) -> str: - return f"[hint]{random.choice(HINTS)}[/hint]" - - def __call__(self, text: str | None = None) -> None: - self.console.print(f"[hint]{text}[/hint]" if text else self.random()) -``` - -- [ ] **Step 4: `ui/components/message.py`** - -```python -from __future__ import annotations - -import traceback - -from core.errors import PortabaseError -from ui.components.base import Component - - -class Message(Component): - def success(self, text: str) -> None: - self.console.print(f"[success]✔ {text}[/success]") - - def info(self, text: str) -> None: - self.console.print(f"[info]ℹ {text}[/info]") - - def warning(self, text: str) -> None: - self.console.print(f"[warning]⚠ {text}[/warning]") - - def error(self, exc: PortabaseError, *, verbose: bool = False, unexpected: bool = False) -> None: - label = "Unexpected error" if unexpected else "Error" - self.console.print(f"[danger]✖ {label}:[/danger] {exc.message}") - if exc.hint: - self.console.print(f" [hint]↳ {exc.hint}[/hint]") - if verbose or unexpected: - self.console.print(f" [hint]code: {exc.code}[/hint]") - if verbose and exc.cause is not None: - self.console.print(f" [hint]cause: {type(exc.cause).__name__}: {exc.cause}[/hint]") - if verbose: - self.console.print("".join(traceback.format_exception(exc)), highlight=False, markup=False) -``` - -- [ ] **Step 5: `ui/components/banner.py`** - -```python -from __future__ import annotations - -from rich.align import Align - -from ui.components.base import Component -from ui.components.hints import Hint - -BANNER = """ -[brand]█▀█ █▀█ █▀█ ▀█▀ ▄▀█ █▄▄ ▄▀█ █▀ █▀▀[/brand] -[brand]█▀▀ █▄█ █▀▄ █ █▀█ █▄█ █▀█ ▄█ ██▄[/brand] -[hint]Deploy your infrastructure anywhere.[/hint] -""" - - -class Banner(Component): - def __call__(self) -> None: - self.console.print(Align.center(BANNER)) - self.console.print(Align.center(Hint(self.console).random() + "\n")) -``` - -- [ ] **Step 6: `ui/components/section.py`** - -```python -from __future__ import annotations - -from rich.panel import Panel - -from ui.components.base import Component - - -class Section(Component): - def __call__(self, title: str) -> None: - self.console.print("") - self.console.print(Panel(f"[bold]{title}[/bold]", style="cyan", expand=False)) -``` - -- [ ] **Step 7: `ui/components/status.py`** - -```python -from __future__ import annotations - -from contextlib import AbstractContextManager - -from ui.components.base import Component -from ui.components.hints import Hint - - -class Status(Component): - def __call__(self, text: str, *, spinner: str = "dots") -> AbstractContextManager: - message = f"[bold magenta]{text}[/bold magenta]\n{Hint(self.console).random()}" - return self.console.status(message, spinner=spinner) -``` - -- [ ] **Step 8: `ui/components/progress.py`** - -```python -from __future__ import annotations - -from collections.abc import Callable, Iterator -from contextlib import contextmanager - -from rich.progress import ( - BarColumn, - DownloadColumn, - Progress as RichProgress, - SpinnerColumn, - TextColumn, - TransferSpeedColumn, -) - -from ui.components.base import Component -from ui.components.hints import Hint - - -class Progress(Component): - @contextmanager - def download(self, description: str, total: int) -> Iterator[Callable[[int], None]]: - """Yields an advance(n_bytes) callable.""" - with RichProgress( - SpinnerColumn(), - TextColumn("[progress.description]{task.description}\n" + Hint(self.console).random()), - BarColumn(), - DownloadColumn(), - TransferSpeedColumn(), - console=self.console, - ) as progress: - task = progress.add_task(description, total=total or None) - yield lambda n: progress.update(task, advance=n) -``` - -- [ ] **Step 9: Vérifier le rendu** - -Run: `uv run python -c " -from rich.console import Console -from ui.theme import RICH_THEME -from ui.components.message import Message -from ui.components.banner import Banner -from ui.components.section import Section -from core.errors import DockerError -c = Console(theme=RICH_THEME) -Banner(c)(); Section(c)('Database Setup') -m = Message(c); m.success('ok'); m.info('note'); m.warning('careful') -m.error(DockerError('daemon down', hint='run: sudo systemctl start docker')) -m.error(DockerError('daemon down', hint='x', cause=RuntimeError('boom')), verbose=True)"` -Expected: bannière orange, panneau cyan, quatre messages avec icônes ✔ ℹ ⚠ ✖, hint indenté, puis le bloc verbose avec `code: E_DOCKER`, `cause: RuntimeError: boom` et une traceback. - -- [ ] **Step 10: Commit** - -```bash -git add ui/ -git commit -m "feat(ui): add theme tokens and display components" -``` - ---- - -### Task 5 : `ui/components/prompt.py` et `ui/form.py` - -**Files:** -- Create: `ui/components/prompt.py` -- Create: `ui/form.py` - -**Interfaces:** -- Consumes: `Field` (Task 2), `UserAbort`/`ValidationError` (Task 1), `QUESTIONARY_STYLE` (Task 4). -- Produces: `Prompt(console, style)` avec `text/integer/secret/confirm/select/path` renvoyant `None` sur Ctrl-C ; `Form(prompt, non_interactive)` avec `ask(field, value)`, `collect(fields, values) -> dict`, raccourcis `text/integer/secret/confirm/choice`. - -- [ ] **Step 1: `ui/components/prompt.py`** - -```python -from __future__ import annotations - -from collections.abc import Sequence - -import questionary -from questionary import Style -from rich.console import Console - -from ui.components.base import Component - - -class Prompt(Component): - """Thin wrapper over questionary. Every method returns None when the user aborts (Ctrl-C).""" - - def __init__(self, console: Console, style: Style) -> None: - super().__init__(console) - self.style = style - - def text(self, message: str, *, default: str | None = None) -> str | None: - return questionary.text(message, default=default or "", style=self.style).ask() - - def integer(self, message: str, *, default: int | None = None) -> int | None: - answer = questionary.text( - message, - default="" if default is None else str(default), - validate=lambda v: v.strip().lstrip("-").isdigit() or "Enter a whole number", - style=self.style, - ).ask() - return None if answer is None else int(answer) - - def secret(self, message: str) -> str | None: - return questionary.password(message, style=self.style).ask() - - def confirm(self, message: str, *, default: bool = False) -> bool | None: - return questionary.confirm(message, default=default, style=self.style).ask() - - def select(self, message: str, choices: Sequence[str], *, default: str | None = None) -> str | None: - return questionary.select(message, choices=list(choices), default=default, style=self.style).ask() - - def path(self, message: str, *, default: str | None = None) -> str | None: - return questionary.path(message, default=default or "", style=self.style).ask() -``` - -- [ ] **Step 2: `ui/form.py`** - -```python -"""Flag → prompt → default → error. The only place that knows about non-interactive mode.""" - -from __future__ import annotations - -from collections.abc import Callable, Sequence -from typing import Any - -from core.errors import UserAbort, ValidationError -from core.fields import Field -from ui.components.prompt import Prompt - -_TRUE = {"1", "true", "yes", "y", "on"} -_FALSE = {"0", "false", "no", "n", "off"} - - -class Form: - def __init__(self, prompt: Prompt, non_interactive: bool) -> None: - self.prompt = prompt - self.non_interactive = non_interactive - self._askers: dict[str, Callable[[Field], Any]] = { - "text": lambda f: self.prompt.text(f.prompt, default=f.default), - "int": lambda f: self.prompt.integer(f.prompt, default=f.default), - "secret": lambda f: self.prompt.secret(f.prompt), - "bool": lambda f: self.prompt.confirm(f.prompt, default=bool(f.default)), - "choice": lambda f: self.prompt.select(f.prompt, f.choices, default=f.default), - "path": lambda f: self.prompt.path(f.prompt, default=f.default), - } - - # ---- core ------------------------------------------------------------- - - def ask(self, field: Field, value: Any | None = None) -> Any: - if value is not None: - return self._coerce_and_validate(field, value) - if self.non_interactive: - if field.default is not None: - return self._coerce_and_validate(field, field.default) - raise ValidationError( - f"Missing {field.flag}", - hint=f"Required in non-interactive mode: {field.prompt}", - ) - return self._ask_until_valid(field) - - def collect(self, fields: Sequence[Field], values: dict[str, Any]) -> dict[str, Any]: - return {f.name: self.ask(f, values.get(f.name)) for f in fields} - - # ---- shortcuts -------------------------------------------------------- - - def text(self, prompt: str, *, value=None, default=None, validator=None, name="value") -> str: - return self.ask(Field(name, prompt, "text", default=default, validator=validator), value) - - def integer(self, prompt: str, *, value=None, default=None, validator=None, name="value") -> int: - return self.ask(Field(name, prompt, "int", default=default, validator=validator), value) - - def secret(self, prompt: str, *, value=None, validator=None, name="value") -> str: - return self.ask(Field(name, prompt, "secret", validator=validator), value) - - def confirm(self, prompt: str, *, value=None, default: bool = False, name="value") -> bool: - return self.ask(Field(name, prompt, "bool", default=default), value) - - def choice(self, prompt: str, choices: Sequence[str], *, value=None, default=None, name="value") -> str: - return self.ask(Field(name, prompt, "choice", default=default, choices=tuple(choices)), value) - - # ---- internals -------------------------------------------------------- - - def _ask_until_valid(self, field: Field) -> Any: - if field.help: - self.prompt.console.print(f"[info]ℹ {field.help}[/info]") - while True: - answer = self._askers[field.kind](field) - if answer is None: - raise UserAbort() - try: - return self._coerce_and_validate(field, answer) - except ValidationError as e: - self.prompt.console.print(f"[danger]✖ {e.message}[/danger]") - - def _coerce_and_validate(self, field: Field, value: Any) -> Any: - value = self._coerce(field, value) - if field.kind == "choice" and value not in field.choices: - raise ValidationError( - f"Invalid value for {field.flag}: {value!r}", - hint="Choices: " + ", ".join(field.choices), - ) - if field.validator is not None: - value = field.validator(value) # raises ValidationError - return value - - @staticmethod - def _coerce(field: Field, value: Any) -> Any: - if field.kind == "int" and not isinstance(value, int): - try: - return int(str(value).strip()) - except ValueError as e: - raise ValidationError(f"{field.flag} must be a whole number, got {value!r}") from e - if field.kind == "bool" and not isinstance(value, bool): - s = str(value).strip().lower() - if s in _TRUE: - return True - if s in _FALSE: - return False - raise ValidationError(f"{field.flag} must be true or false, got {value!r}") - if field.kind in ("text", "secret", "path", "choice"): - return str(value) - return value -``` - -- [ ] **Step 3: Vérifier le mode non-interactif (sans terminal)** - -Run: `uv run python -c " -from rich.console import Console -from ui.theme import QUESTIONARY_STYLE -from ui.components.prompt import Prompt -from ui.form import Form -from core.fields import Field -from core.errors import ValidationError -f = Form(Prompt(Console(), QUESTIONARY_STYLE), non_interactive=True) -print(f.text('Timezone', value=None, default='UTC'), f.integer('Polling', value='7'), f.confirm('Gateway?', value='yes')) -print(f.collect([Field('engine','Engine','choice',choices=('a','b')), Field('port','Port','int',default=5432)], {'engine':'a'})) -try: f.text('Edge key') -except ValidationError as e: print('OK:', e.message, '|', e.hint) -try: f.choice('Mode', ['new','existing'], value='bogus') -except ValidationError as e: print('OK:', e.message, '|', e.hint)"` -Expected : -``` -UTC 7 True -{'engine': 'a', 'port': 5432} -OK: Missing --value | Required in non-interactive mode: Edge key -OK: Invalid value for --value: 'bogus' | Choices: new, existing -``` - -- [ ] **Step 4: Vérifier le mode interactif (terminal requis)** - -Run: `uv run python -c " -from rich.console import Console -from ui.theme import QUESTIONARY_STYLE -from ui.components.prompt import Prompt -from ui.form import Form -f = Form(Prompt(Console(), QUESTIONARY_STYLE), non_interactive=False) -print(f.choice('Mode', ['new','existing'], default='new')) -print(f.integer('Port', default=5432))"` -Répondre aux deux prompts. Puis relancer et faire Ctrl-C au premier prompt. -Expected: valeurs saisies affichées ; sur Ctrl-C, traceback se terminant par `core.errors.UserAbort: Cancelled.` (le catcher n'est pas encore branché — attendu). - -- [ ] **Step 5: Commit** - -```bash -git add ui/components/prompt.py ui/form.py -git commit -m "feat(ui): add questionary Prompt and Form with non-interactive resolution" -``` - ---- - -### Task 6 : Façade `UI` - -**Files:** -- Modify: `ui/__init__.py` - -**Interfaces:** -- Produces: `UI(console=None, *, non_interactive=False, verbose=False, no_color=False)` ; `configure(**kwargs)` ; `banner()`, `success/info/warning(text)`, `error(exc, unexpected=False)`, `hint(text=None)`, `section(title)`, `status(text)`, `progress()`, `confirm(q, default=False, value=None) -> bool`, `form() -> Form`, `print(renderable)`. Attribut `console`. - -- [ ] **Step 1: Écrire la façade** - -```python -"""Facade: the only thing `commands/` imports from ui. Rich and questionary stay inside ui/.""" - -from __future__ import annotations - -from rich.console import Console - -from core.errors import PortabaseError -from ui.components.banner import Banner -from ui.components.hints import Hint -from ui.components.message import Message -from ui.components.progress import Progress -from ui.components.prompt import Prompt -from ui.components.section import Section -from ui.components.status import Status -from ui.form import Form -from ui.theme import QUESTIONARY_STYLE, QUESTIONARY_STYLE_PLAIN, RICH_THEME - - -class UI: - def __init__( - self, - console: Console | None = None, - *, - non_interactive: bool = False, - verbose: bool = False, - no_color: bool = False, - ) -> None: - self.non_interactive = non_interactive - self.verbose = verbose - self.no_color = no_color - self.console = console or self._make_console() - - def configure(self, *, non_interactive: bool | None = None, verbose: bool | None = None, no_color: bool | None = None) -> None: - if non_interactive is not None: - self.non_interactive = non_interactive - if verbose is not None: - self.verbose = verbose - if no_color is not None and no_color != self.no_color: - self.no_color = no_color - self.console = self._make_console() - - def _make_console(self) -> Console: - return Console(theme=RICH_THEME, no_color=self.no_color) - - # ---- output ----------------------------------------------------------- - - def print(self, renderable) -> None: - self.console.print(renderable) - - def banner(self) -> None: - Banner(self.console)() - - def success(self, text: str) -> None: - Message(self.console).success(text) - - def info(self, text: str) -> None: - Message(self.console).info(text) - - def warning(self, text: str) -> None: - Message(self.console).warning(text) - - def error(self, exc: PortabaseError, *, unexpected: bool = False) -> None: - Message(self.console).error(exc, verbose=self.verbose, unexpected=unexpected) - - def hint(self, text: str | None = None) -> None: - Hint(self.console)(text) - - def section(self, title: str) -> None: - Section(self.console)(title) - - def status(self, text: str): - return Status(self.console)(text) - - def progress(self) -> Progress: - return Progress(self.console) - - # ---- input ------------------------------------------------------------ - - def form(self) -> Form: - style = QUESTIONARY_STYLE_PLAIN if self.no_color else QUESTIONARY_STYLE - return Form(Prompt(self.console, style), self.non_interactive) - - def confirm(self, question: str, *, default: bool = False, value: bool | None = None) -> bool: - return self.form().confirm(question, value=value, default=default) -``` - -- [ ] **Step 2: Vérifier** - -Run: `uv run python -c " -from ui import UI -ui = UI(non_interactive=True) -ui.banner(); ui.section('Test'); ui.success('a'); ui.warning('b'); ui.hint() -print('confirm default:', ui.confirm('Really?', default=False)) -with ui.status('Working...'): import time; time.sleep(0.5) -ui.configure(no_color=True); ui.success('no color')"` -Expected: rendu, `confirm default: False` sans prompt, spinner 0,5 s, dernière ligne sans couleur. - -- [ ] **Step 3: Commit** - -```bash -git add ui/__init__.py -git commit -m "feat(ui): add UI facade" -``` - ---- - -### Task 7 : `services/http.py` et `services/docker.py` - -**Files:** -- Create: `services/__init__.py` (vide) -- Create: `services/http.py` -- Create: `services/docker.py` - -**Interfaces:** -- Produces: - - `HttpClient(timeout=10.0)` : `get_json(url) -> Any`, `get_text(url) -> str`, `download(url, dest: Path, on_progress: Callable[[int], None] | None = None, *, timeout=30.0) -> int` (octets), `head_content_length(url) -> int | None`. Lèvent `NetworkError`. - - `DockerRunner(docker_bin: str | None = None)` : `available() -> bool`, `daemon_running() -> bool`, `start_daemon() -> bool`, `ensure_network(name)`, `compose(cwd, args, *, check=True, capture=False) -> subprocess.CompletedProcess`, `project_name(cwd) -> str`. Lèvent `DockerError`. - -- [ ] **Step 1: `services/http.py`** - -```python -"""requests wrapper. Every failure becomes NetworkError; nothing else leaks out.""" - -from __future__ import annotations - -from collections.abc import Callable -from pathlib import Path -from typing import Any - -import requests - -from core.errors import NetworkError - -_HINT = "Check your internet connection or proxy settings." - - -class HttpClient: - def __init__(self, timeout: float = 10.0, user_agent: str = "portabase-cli") -> None: - self.timeout = timeout - self.session = requests.Session() - self.session.headers["User-Agent"] = user_agent - - def get_json(self, url: str) -> Any: - try: - r = self.session.get(url, timeout=self.timeout) - r.raise_for_status() - return r.json() - except requests.RequestException as e: - raise NetworkError(f"GET {url} failed: {e}", hint=_HINT, cause=e) from e - except ValueError as e: - raise NetworkError(f"GET {url}: response is not JSON", cause=e) from e - - def get_text(self, url: str) -> str: - try: - r = self.session.get(url, timeout=self.timeout) - r.raise_for_status() - return r.text - except requests.RequestException as e: - raise NetworkError(f"GET {url} failed: {e}", hint=_HINT, cause=e) from e - - def status(self, url: str) -> int: - """HTTP status without raising on 4xx/5xx. Network failure still raises.""" - try: - return self.session.get(url, timeout=self.timeout, stream=True).status_code - except requests.RequestException as e: - raise NetworkError(f"GET {url} failed: {e}", hint=_HINT, cause=e) from e - - def download( - self, - url: str, - dest: Path, - on_progress: Callable[[int], None] | None = None, - *, - timeout: float = 30.0, - ) -> int: - written = 0 - try: - with self.session.get(url, stream=True, timeout=timeout) as r: - r.raise_for_status() - with open(dest, "wb") as f: - for chunk in r.iter_content(chunk_size=64 * 1024): - if not chunk: - continue - f.write(chunk) - written += len(chunk) - if on_progress: - on_progress(len(chunk)) - except requests.RequestException as e: - dest.unlink(missing_ok=True) - raise NetworkError(f"Download of {url} failed: {e}", hint=_HINT, cause=e) from e - return written - - def content_length(self, url: str) -> int | None: - try: - r = self.session.head(url, timeout=self.timeout, allow_redirects=True) - value = r.headers.get("content-length") - return int(value) if value else None - except (requests.RequestException, ValueError): - return None -``` - -- [ ] **Step 2: `services/docker.py`** - -Reprend `core/docker.py` + `check_system`/`start_docker` de `core/utils.py`, sans aucune sortie terminal. - -```python -"""Docker CLI runner. No terminal output; callers decide what to show.""" - -from __future__ import annotations - -import platform -import shutil -import subprocess -import time -from pathlib import Path - -from core.errors import DockerError -from core.utils import slugify_project_name - -_START_COMMANDS = { - "Linux": ["sudo", "systemctl", "start", "docker"], - "Darwin": ["open", "--background", "-a", "Docker"], - "Windows": ["cmd", "/c", "start", "docker"], -} - - -class DockerRunner: - def __init__(self, docker_bin: str | None = None) -> None: - self._bin = docker_bin - - @property - def binary(self) -> str: - if self._bin is None: - found = shutil.which("docker") - if found is None: - raise DockerError( - "Docker not found (binary missing).", - hint="Install Docker: https://docs.docker.com/get-docker/", - ) - self._bin = found - return self._bin - - def available(self) -> bool: - return shutil.which("docker") is not None - - def daemon_running(self) -> bool: - try: - subprocess.run( - [self.binary, "info"], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, check=True - ) - return True - except (subprocess.CalledProcessError, OSError): - return False - - def start_daemon(self, *, wait_seconds: int = 20) -> bool: - cmd = _START_COMMANDS.get(platform.system()) - if cmd is None: - return False - try: - subprocess.run(cmd, check=True) - except (subprocess.CalledProcessError, OSError) as e: - raise DockerError(f"Failed to start Docker: {e}", cause=e) from e - deadline = time.monotonic() + wait_seconds - while time.monotonic() < deadline: - if self.daemon_running(): - return True - time.sleep(2) - return False - - def ensure_network(self, name: str) -> None: - inspect = subprocess.run( - [self.binary, "network", "inspect", name], - stdout=subprocess.DEVNULL, - stderr=subprocess.DEVNULL, - check=False, - ) - if inspect.returncode == 0: - return - try: - subprocess.run([self.binary, "network", "create", name], stdout=subprocess.DEVNULL, check=True) - except subprocess.CalledProcessError as e: - raise DockerError(f"Could not create Docker network '{name}'.", cause=e) from e - - @staticmethod - def project_name(cwd: Path) -> str: - return slugify_project_name(cwd.resolve().name) - - def compose( - self, - cwd: Path, - args: list[str], - *, - check: bool = True, - capture: bool = False, - ) -> subprocess.CompletedProcess: - cmd = [self.binary, "compose", "-p", self.project_name(cwd), *args] - try: - return subprocess.run( - cmd, - cwd=cwd, - check=check, - capture_output=capture, - text=capture, - ) - except subprocess.CalledProcessError as e: - raise DockerError( - f"docker compose {' '.join(args)} failed (exit {e.returncode}).", - hint=f"Run it manually in {cwd} to see the full output.", - cause=e, - ) from e -``` - -- [ ] **Step 3: Vérifier** - -Run: `uv run python -c " -from services.http import HttpClient -from services.docker import DockerRunner -from core.errors import NetworkError, DockerError -h = HttpClient(timeout=5) -print(type(h.get_json('https://api.github.com/repos/Portabase/cli')).__name__) -try: h.get_json('https://127.0.0.1:1/nope') -except NetworkError as e: print('NetworkError OK:', e.code) -d = DockerRunner(); print('docker available:', d.available(), '| daemon:', d.available() and d.daemon_running()) -try: DockerRunner(docker_bin='/nonexistent').compose(__import__('pathlib').Path('.'), ['version']) -except (DockerError, OSError) as e: print('error path OK:', type(e).__name__)"` -Expected: `dict`, `NetworkError OK: E_NETWORK`, état Docker local, `error path OK: FileNotFoundError` ou `DockerError` (les deux acceptables ici ; `OSError` est traité au niveau commande, Task 9). - -- [ ] **Step 4: Commit** - -```bash -git add services/ -git commit -m "feat(services): add HttpClient and DockerRunner" -``` - ---- - -### Task 8 : `services/telemetry.py` - -**Files:** -- Create: `services/telemetry.py` - -**Interfaces:** -- Produces: `Telemetry` ABC (`session(**attrs)`, `span(name, **attrs)`, `event(name, **attrs)`, `error(exc, unexpected=False)`, `flush()`) ; `NoopTelemetry` ; `ConsoleTelemetry(stream=sys.stderr)` ; `TelemetryHub(inner)` avec `.set(inner)` ; `TelemetryFactory.build(config: GlobalConfig, *, debug: bool) -> TelemetryHub`. - -- [ ] **Step 1: Écrire le module** - -```python -"""Telemetry contract. Noop by default; OTel exporter can be plugged later without touching callers. - -Never record: agent names, paths, keys, credentials, file contents. -""" - -from __future__ import annotations - -import sys -import time -from abc import ABC, abstractmethod -from collections.abc import Iterator -from contextlib import contextmanager -from typing import Any, TextIO - -from core.config import GlobalConfig - - -class Telemetry(ABC): - @abstractmethod - def session(self, **attrs: Any): - """Context manager: root span for one CLI invocation.""" - - @abstractmethod - def span(self, name: str, **attrs: Any): - """Context manager: child span.""" - - @abstractmethod - def event(self, name: str, **attrs: Any) -> None: ... - - @abstractmethod - def error(self, exc: BaseException, *, unexpected: bool = False) -> None: ... - - def flush(self) -> None: - return None - - -class NoopTelemetry(Telemetry): - @contextmanager - def session(self, **attrs: Any) -> Iterator[None]: - yield - - @contextmanager - def span(self, name: str, **attrs: Any) -> Iterator[None]: - yield - - def event(self, name: str, **attrs: Any) -> None: - return None - - def error(self, exc: BaseException, *, unexpected: bool = False) -> None: - return None - - -class ConsoleTelemetry(Telemetry): - """--debug: prints spans and events to stderr. Development aid, not an exporter.""" - - def __init__(self, stream: TextIO = sys.stderr) -> None: - self.stream = stream - self._depth = 0 - - def _log(self, line: str) -> None: - self.stream.write(" " * self._depth + f"[telemetry] {line}\n") - self.stream.flush() - - @contextmanager - def session(self, **attrs: Any) -> Iterator[None]: - with self.span("session", **attrs): - yield - - @contextmanager - def span(self, name: str, **attrs: Any) -> Iterator[None]: - self._log(f"▶ {name} {attrs}") - self._depth += 1 - start = time.perf_counter() - try: - yield - finally: - self._depth -= 1 - self._log(f"◀ {name} {time.perf_counter() - start:.3f}s") - - def event(self, name: str, **attrs: Any) -> None: - self._log(f"• {name} {attrs}") - - def error(self, exc: BaseException, *, unexpected: bool = False) -> None: - code = getattr(exc, "code", type(exc).__name__) - self._log(f"✖ error code={code} unexpected={unexpected}") - - -class TelemetryHub(Telemetry): - """Delegates to a swappable implementation. Commands hold the hub; main swaps the inner.""" - - def __init__(self, inner: Telemetry | None = None) -> None: - self.inner: Telemetry = inner or NoopTelemetry() - - def set(self, inner: Telemetry) -> None: - self.inner = inner - - def session(self, **attrs: Any): - return self.inner.session(**attrs) - - def span(self, name: str, **attrs: Any): - return self.inner.span(name, **attrs) - - def event(self, name: str, **attrs: Any) -> None: - self.inner.event(name, **attrs) - - def error(self, exc: BaseException, *, unexpected: bool = False) -> None: - self.inner.error(exc, unexpected=unexpected) - - def flush(self) -> None: - self.inner.flush() - - -class TelemetryFactory: - @staticmethod - def build(config: GlobalConfig, *, debug: bool = False) -> TelemetryHub: - if debug: - return TelemetryHub(ConsoleTelemetry()) - # Opt-in and endpoint present → OTel exporter (future plan). Until then: noop. - return TelemetryHub(NoopTelemetry()) -``` - -- [ ] **Step 2: Vérifier** - -Run: `uv run python -c " -from services.telemetry import * -from core.config import GlobalConfig -hub = TelemetryFactory.build(GlobalConfig(), debug=True) -with hub.session(cli_version='x'): - with hub.span('command.start', command='start'): - hub.event('compose', args='up') - hub.error(RuntimeError('boom'), unexpected=True) -hub.set(NoopTelemetry()) -with hub.span('silent'): pass -print('ok')"` -Expected: lignes `[telemetry]` imbriquées sur stderr pour session/command/event/error, rien pour `silent`, puis `ok`. - -- [ ] **Step 3: Commit** - -```bash -git add services/telemetry.py -git commit -m "feat(services): add Telemetry contract with noop, console and hub implementations" -``` - ---- - -### Task 9 : `commands/base.py` — `Command`, `CommandGroup`, `LegacyCommand` - -**Files:** -- Create: `commands/base.py` - -**Interfaces:** -- Consumes: `UI`, `Telemetry`, `DockerRunner`, erreurs. -- Produces: - - `Command(ui, telemetry)` : attributs de classe `name`, `help`, `panel`, `no_args_is_help=False` ; `register(app)` ; `run(...)` abstraite ; helpers `require_docker(docker)`, `require_project_dir(path) -> Path`. - - `CommandGroup(ui, telemetry)` : `name`, `help`, `commands: list[Command]`, `typer() -> typer.Typer`, `register(app)`. - - `LegacyCommand(ui, telemetry, fn, *, name, help, panel, no_args_is_help=True)`. - -- [ ] **Step 1: Écrire le module** - -```python -"""Command base classes. Typer registers bound `run` methods; dependencies come via constructors.""" - -from __future__ import annotations - -import functools -from abc import ABC, abstractmethod -from collections.abc import Callable -from pathlib import Path - -import typer - -from core.errors import ConfigError, DockerError, UserAbort -from services.docker import DockerRunner -from services.telemetry import Telemetry -from ui import UI - - -class Command(ABC): - name: str - help: str - panel: str = "General" - no_args_is_help: bool = False - - def __init__(self, ui: UI, telemetry: Telemetry) -> None: - self.ui = ui - self.telemetry = telemetry - - # ---- registration ----------------------------------------------------- - - def register(self, app: typer.Typer) -> None: - app.command( - self.name, - help=self.help, - rich_help_panel=self.panel, - no_args_is_help=self.no_args_is_help, - )(self._traced(self.run)) - - def _traced(self, fn: Callable) -> Callable: - @functools.wraps(fn) - def wrapper(*args, **kwargs): - with self.telemetry.span(f"command.{self.name}"): - return fn(*args, **kwargs) - - return wrapper - - @abstractmethod - def run(self, *args, **kwargs) -> None: ... - - # ---- shared helpers --------------------------------------------------- - - def require_docker(self, docker: DockerRunner) -> None: - """Binary present and daemon up, offering to start it when interactive.""" - if not docker.available(): - raise DockerError( - "Docker not found (binary missing).", - hint="Install Docker: https://docs.docker.com/get-docker/", - ) - if docker.daemon_running(): - return - self.ui.warning("Docker is installed but the daemon is not running.") - if self.ui.confirm("Do you want to try starting Docker?", default=False): - with self.ui.status("Waiting for Docker to start..."): - started = docker.start_daemon() - if started: - self.ui.success("Docker started successfully.") - return - raise DockerError("Docker is required to continue.", hint="Start the Docker daemon and retry.") - - @staticmethod - def require_project_dir(path: Path) -> Path: - path = path.resolve() - if not (path / "docker-compose.yml").exists(): - raise ConfigError( - f"No Portabase configuration found in: {path}", - hint="Expected a docker-compose.yml created by 'portabase agent' or 'portabase dashboard'.", - ) - return path - - def confirm_or_abort(self, question: str, *, default: bool = False, value: bool | None = None) -> None: - if not self.ui.confirm(question, default=default, value=value): - raise UserAbort() - - -class CommandGroup: - name: str - help: str - panel: str = "General" - - def __init__(self, ui: UI, telemetry: Telemetry) -> None: - self.ui = ui - self.telemetry = telemetry - - @property - @abstractmethod - def commands(self) -> list[Command]: ... - - def typer(self) -> typer.Typer: - sub = typer.Typer(help=self.help, no_args_is_help=True) - for cmd in self.commands: - cmd.register(sub) - return sub - - def register(self, app: typer.Typer) -> None: - app.add_typer(self.typer(), name=self.name, rich_help_panel=self.panel) - - -class LegacyCommand(Command): - """Adapter for the pre-refactor function-style commands. Removed in plan 4.""" - - def __init__( - self, - ui: UI, - telemetry: Telemetry, - fn: Callable, - *, - name: str, - help: str, - panel: str, - no_args_is_help: bool = True, - ) -> None: - super().__init__(ui, telemetry) - self.name, self.help, self.panel, self.no_args_is_help = name, help, panel, no_args_is_help - self._fn = fn - - def register(self, app: typer.Typer) -> None: - app.command( - self.name, - help=self.help, - rich_help_panel=self.panel, - no_args_is_help=self.no_args_is_help, - )(self._traced(self._fn)) - - def run(self, *args, **kwargs) -> None: - return self._fn(*args, **kwargs) -``` - -Note : `_traced` utilise `functools.wraps`, donc Typer voit la signature de la fonction d'origine (`__wrapped__`) — c'est ce qui permet d'envelopper sans casser l'introspection des paramètres. - -- [ ] **Step 2: Vérifier l'introspection Typer à travers `_traced`** - -Run: `uv run python -c " -from typing import Annotated -import typer -from commands.base import Command -from ui import UI -from services.telemetry import NoopTelemetry -class Hello(Command): - name, help, panel = 'hello', 'Say hello', 'Test' - def run(self, name: Annotated[str, typer.Argument()], loud: Annotated[bool, typer.Option('--loud')] = False): - print('hello', name.upper() if loud else name) -app = typer.Typer(add_completion=False) -@app.callback() -def root(): pass -Hello(UI(), NoopTelemetry()).register(app) -app(['hello', 'bob', '--loud'], standalone_mode=False)"` -Expected: `hello BOB`. - -- [ ] **Step 3: Commit** - -```bash -git add commands/base.py -git commit -m "feat(commands): add Command, CommandGroup and LegacyCommand base classes" -``` - ---- - -### Task 10 : `services/updater.py` - -**Files:** -- Create: `services/updater.py` - -**Interfaces:** -- Consumes: `HttpClient`, `GlobalConfig`, `core.version`. -- Produces: `Release(tag, assets: dict[str, str], prerelease: bool)` ; `UpdateChecker(http, config, current: str)` : `include_prerelease -> bool`, `latest(force=False) -> Release | None` (cache 24 h, `None` si réseau KO), `available() -> str | None` (tag plus récent ou `None`) ; `Updater(http, current: str)` : `asset_name() -> str`, `target_path() -> Path`, `download(release, on_progress) -> Path` (vérifie sha256 via `checksums.txt`), `install(tmp: Path, target: Path) -> None`. - -- [ ] **Step 1: Écrire le module** - -```python -"""Update check (notify only) and manual update with checksum verification.""" - -from __future__ import annotations - -import hashlib -import json -import os -import platform -import shutil -import subprocess -import sys -import tempfile -import time -from collections.abc import Callable -from dataclasses import dataclass -from pathlib import Path - -from core.config import GlobalConfig -from core.errors import NetworkError, UpdateError -from core.version import UNKNOWN, is_prerelease, parse_version -from services.http import HttpClient - -GITHUB_REPO = "Portabase/cli" -RELEASES_URL = f"https://api.github.com/repos/{GITHUB_REPO}/releases" -CACHE_TTL = 24 * 3600 - - -@dataclass(frozen=True) -class Release: - tag: str - assets: dict[str, str] # name -> browser_download_url - prerelease: bool - - @classmethod - def from_api(cls, data: dict) -> Release: - return cls( - tag=str(data.get("tag_name", "")).lstrip("v"), - assets={a["name"]: a["browser_download_url"] for a in data.get("assets", [])}, - prerelease=bool(data.get("prerelease", False)), - ) - - -def platform_asset_name() -> str: - system = platform.system().lower() - system = "macos" if system == "darwin" else system - machine = platform.machine().lower() - arch = "arm64" if machine in ("arm64", "aarch64") else "amd64" - name = f"portabase_{system}_{arch}" - return name + ".exe" if system == "windows" else name - - -def is_frozen() -> bool: - return bool(getattr(sys, "frozen", False)) - - -class UpdateChecker: - def __init__(self, http: HttpClient, config: GlobalConfig, current: str) -> None: - self.http = http - self.config = config - self.current = current - self.cache_file = config.cache_dir / "release.json" - - @property - def include_prerelease(self) -> bool: - channel = self.config.update_channel - if channel: - return channel == "beta" - return is_prerelease(self.current) - - def fetch_latest(self) -> Release | None: - """Network call. Returns None when nothing is published.""" - if self.include_prerelease: - releases = self.http.get_json(RELEASES_URL) - return Release.from_api(releases[0]) if releases else None - return Release.from_api(self.http.get_json(f"{RELEASES_URL}/latest")) - - def latest(self, *, force: bool = False) -> Release | None: - """Cached 24h. Returns None on any network failure — never raises.""" - if not force: - cached = self._read_cache() - if cached is not None: - return cached - try: - release = self.fetch_latest() - except NetworkError: - return None - if release is not None: - self._write_cache(release) - return release - - def available(self, *, force: bool = False) -> str | None: - if self.current == UNKNOWN: - return None - release = self.latest(force=force) - if release is None: - return None - if parse_version(release.tag) > parse_version(self.current): - return release.tag - return None - - def _read_cache(self) -> Release | None: - try: - with open(self.cache_file, encoding="utf-8") as f: - data = json.load(f) - if time.time() - float(data.get("checked_at", 0)) > CACHE_TTL: - return None - if data.get("channel_pre") != self.include_prerelease: - return None - return Release(tag=data["tag"], assets=data.get("assets", {}), prerelease=bool(data.get("prerelease"))) - except (OSError, ValueError, KeyError): - return None - - def _write_cache(self, release: Release) -> None: - try: - self.cache_file.parent.mkdir(parents=True, exist_ok=True) - with open(self.cache_file, "w", encoding="utf-8") as f: - json.dump( - { - "checked_at": time.time(), - "channel_pre": self.include_prerelease, - "tag": release.tag, - "assets": release.assets, - "prerelease": release.prerelease, - }, - f, - ) - except OSError: - pass - - -class Updater: - CHECKSUMS_ASSET = "checksums.txt" - - def __init__(self, http: HttpClient, current: str) -> None: - self.http = http - self.current = current - - def target_path(self) -> Path: - if is_frozen(): - return Path(sys.executable).resolve() - if platform.system().lower() == "windows": - return Path(os.environ.get("APPDATA", "")) / "Portabase" / "portabase.exe" - default = Path("/usr/local/bin/portabase") - return default if default.exists() else Path.home() / ".local" / "bin" / "portabase" - - def download(self, release: Release, on_progress: Callable[[int], None] | None = None) -> Path: - name = platform_asset_name() - url = release.assets.get(name) - if url is None: - raise UpdateError( - f"No binary for this platform ({name}) in release {release.tag}.", - hint="Available: " + ", ".join(sorted(release.assets)) if release.assets else None, - ) - fd, tmp = tempfile.mkstemp(prefix="portabase_update_") - os.close(fd) - tmp_path = Path(tmp) - try: - self.http.download(url, tmp_path, on_progress, timeout=60) - self._verify(release, name, tmp_path) - except Exception: - tmp_path.unlink(missing_ok=True) - raise - return tmp_path - - def expected_size(self, release: Release) -> int | None: - url = release.assets.get(platform_asset_name()) - return self.http.content_length(url) if url else None - - def _verify(self, release: Release, name: str, path: Path) -> None: - url = release.assets.get(self.CHECKSUMS_ASSET) - if url is None: - raise UpdateError(f"Release {release.tag} has no {self.CHECKSUMS_ASSET}; refusing to install.") - expected = None - for line in self.http.get_text(url).splitlines(): - parts = line.split() - if len(parts) == 2 and parts[1].lstrip("*") == name: - expected = parts[0].lower() - if expected is None: - raise UpdateError(f"{name} not listed in {self.CHECKSUMS_ASSET}; refusing to install.") - digest = hashlib.sha256() - with open(path, "rb") as f: - for chunk in iter(lambda: f.read(1 << 20), b""): - digest.update(chunk) - if digest.hexdigest() != expected: - raise UpdateError("Checksum mismatch for downloaded binary; refusing to install.") - - def install(self, tmp: Path, target: Path) -> None: - system = platform.system().lower() - if system != "windows": - tmp.chmod(0o755) - target.parent.mkdir(parents=True, exist_ok=True) - backup = target.with_name(target.name + ".old") - writable = os.access(target.parent, os.W_OK) and (not target.exists() or os.access(target, os.W_OK)) - try: - if writable or system == "windows": - if target.exists(): - backup.unlink(missing_ok=True) - target.rename(backup) - shutil.move(str(tmp), str(target)) - else: - if target.exists(): - subprocess.run(["sudo", "mv", str(target), str(backup)], check=True) - subprocess.run(["sudo", "mv", str(tmp), str(target)], check=True) - subprocess.run(["sudo", "chmod", "+x", str(target)], check=True) - except (OSError, subprocess.CalledProcessError) as e: - raise UpdateError(f"Could not install to {target}: {e}", cause=e) from e -``` - -- [ ] **Step 2: Vérifier le checker (réseau requis)** - -Run: `uv run python -c " -from pathlib import Path; import tempfile -from services.http import HttpClient -from services.updater import UpdateChecker, platform_asset_name -from core.config import GlobalConfig -cfg = GlobalConfig(Path(tempfile.mkdtemp())/'config.json') -c = UpdateChecker(HttpClient(), cfg, '0.0.1') -r = c.latest(force=True); print('latest:', r.tag, 'pre:', r.prerelease, 'assets:', len(r.assets)) -print('cached:', c.latest().tag == r.tag, '| available from 0.0.1:', c.available()) -print('asset for this machine:', platform_asset_name(), platform_asset_name() in r.assets)"` -Expected: tag de la dernière release stable (ex. `26.07.6`), `cached: True`, `available from 0.0.1: `, asset présent `True` sur linux/macos. - -- [ ] **Step 3: Commit** - -```bash -git add services/updater.py -git commit -m "feat(services): add UpdateChecker (notify, cached) and Updater with checksum verification" -``` - ---- - -### Task 11 : `commands/lifecycle.py`, `commands/config.py`, `commands/update.py` - -**Files:** -- Create: `commands/lifecycle.py` -- Modify: `commands/config.py` (réécriture complète) -- Create: `commands/update.py` -- Delete: `commands/common.py` (Task 12, après bascule de `main.py`) - -**Interfaces:** -- Consumes: `Command`, `CommandGroup`, `DockerRunner`, `UpdateChecker`, `Updater`, `GlobalConfig`. -- Produces: classes `StartCommand`, `StopCommand`, `RestartCommand`, `LogsCommand`, `UninstallCommand` (constructeur `(ui, telemetry, docker)`) ; `ConfigCommands(ui, telemetry, config)` groupe `config` avec `show`, `get`, `set`, `channel` ; `UpdateCommand(ui, telemetry, checker, updater)`. - -- [ ] **Step 1: `commands/lifecycle.py`** - -```python -"""start / stop / restart / logs / uninstall. No rendering: work on any folder with a compose file.""" - -from __future__ import annotations - -import shutil -from pathlib import Path -from typing import Annotated - -import typer - -from commands.base import Command -from services.docker import DockerRunner -from services.telemetry import Telemetry -from ui import UI - -PathArg = Annotated[Path, typer.Argument(help="Path to the component folder")] - - -class _ComposeCommand(Command): - panel = "Lifecycle" - no_args_is_help = True - verb: str - compose_args: list[str] - done: str - - def __init__(self, ui: UI, telemetry: Telemetry, docker: DockerRunner) -> None: - super().__init__(ui, telemetry) - self.docker = docker - - def run(self, path: PathArg) -> None: - path = self.require_project_dir(path) - self.require_docker(self.docker) - with self.ui.status(f"{self.verb} {path.name}..."): - self.docker.compose(path, self.compose_args) - self.ui.success(self.done) - - -class StartCommand(_ComposeCommand): - name, help = "start", "Start a Portabase component." - verb, compose_args, done = "Starting", ["up", "-d"], "Started" - - -class StopCommand(_ComposeCommand): - name, help = "stop", "Stop a Portabase component." - verb, compose_args, done = "Stopping", ["stop"], "Stopped" - - -class RestartCommand(_ComposeCommand): - name, help = "restart", "Restart a Portabase component." - verb, compose_args, done = "Restarting", ["restart"], "Restarted" - - -class LogsCommand(Command): - name, help, panel = "logs", "View logs of a Portabase component.", "Lifecycle" - no_args_is_help = True - - def __init__(self, ui: UI, telemetry: Telemetry, docker: DockerRunner) -> None: - super().__init__(ui, telemetry) - self.docker = docker - - def run( - self, - path: PathArg, - follow: Annotated[bool, typer.Option("--follow/--no-follow", "-f", help="Follow log output")] = True, - ) -> None: - path = self.require_project_dir(path) - self.require_docker(self.docker) - args = ["logs", "-f"] if follow else ["logs"] - try: - self.docker.compose(path, args, check=False) - except KeyboardInterrupt: - pass - - -class UninstallCommand(Command): - name, help, panel = "uninstall", "Uninstall and delete a Portabase component.", "Lifecycle" - no_args_is_help = True - - def __init__(self, ui: UI, telemetry: Telemetry, docker: DockerRunner) -> None: - super().__init__(ui, telemetry) - self.docker = docker - - def run( - self, - path: PathArg, - force: Annotated[bool, typer.Option("--force", "-f", help="Skip confirmation")] = False, - ) -> None: - path = self.require_project_dir(path) - self.require_docker(self.docker) - if not force: - self.ui.warning(f"This will delete containers, volumes and all data in {path}.") - self.confirm_or_abort("Are you sure?", default=False) - with self.ui.status("Uninstalling..."): - self.docker.compose(path, ["down", "-v"]) - try: - shutil.rmtree(path) - except OSError as e: - self.ui.warning(f"Could not remove directory: {e}") - self.ui.success("Uninstalled") -``` - -- [ ] **Step 2: `commands/config.py` (réécriture)** - -```python -"""Global configuration (~/.portabase/config.json).""" - -from __future__ import annotations - -from typing import Annotated - -import typer - -from commands.base import Command, CommandGroup -from core.config import GlobalConfig -from core.errors import ValidationError -from services.telemetry import Telemetry -from ui import UI - -CHANNELS = ("stable", "beta") -BOOL_KEYS = ("telemetry",) - - -class _ConfigCommand(Command): - panel = "Configuration" - - def __init__(self, ui: UI, telemetry: Telemetry, config: GlobalConfig) -> None: - super().__init__(ui, telemetry) - self.config = config - - -class ConfigShow(_ConfigCommand): - name, help = "show", "Show the current configuration." - - def run(self) -> None: - data = self.config.all() - self.ui.info(f"Configuration file: {self.config.path}") - for key in GlobalConfig.KNOWN_KEYS: - value = data.get(key, "[hint]unset[/hint]") - self.ui.print(f" [key]{key}[/key]: {value}") - for key in sorted(set(data) - set(GlobalConfig.KNOWN_KEYS)): - self.ui.print(f" [key]{key}[/key]: {data[key]} [hint](unknown key)[/hint]") - - -class ConfigGet(_ConfigCommand): - name, help = "get", "Print one configuration value." - no_args_is_help = True - - def run(self, key: Annotated[str, typer.Argument(help="Configuration key")]) -> None: - value = self.config.get(key) - if value is None: - raise ValidationError(f"'{key}' is not set.", hint="Known keys: " + ", ".join(GlobalConfig.KNOWN_KEYS)) - self.ui.print(str(value)) - - -class ConfigSet(_ConfigCommand): - name, help = "set", "Set a configuration value." - no_args_is_help = True - - def run( - self, - key: Annotated[str, typer.Argument(help="Configuration key")], - value: Annotated[str, typer.Argument(help="Value")], - ) -> None: - if key == "update_channel" and value not in CHANNELS: - raise ValidationError(f"Invalid channel '{value}'.", hint="Choose 'stable' or 'beta'.") - stored: object = value - if key in BOOL_KEYS: - lowered = value.lower() - if lowered not in ("true", "false", "1", "0", "yes", "no"): - raise ValidationError(f"'{key}' expects true or false.") - stored = lowered in ("true", "1", "yes") - self.config.set(key, stored) - self.ui.success(f"{key} = {stored}") - - -class ConfigChannel(_ConfigCommand): - """Kept for compatibility with the previous `config channel ` command.""" - - name, help = "channel", "Set the update channel (stable or beta)." - no_args_is_help = True - - def run(self, name: Annotated[str, typer.Argument(help="stable or beta")]) -> None: - ConfigSet(self.ui, self.telemetry, self.config).run("update_channel", name.lower()) - - -class ConfigCommands(CommandGroup): - name, help, panel = "config", "Manage global CLI configuration.", "Configuration" - - def __init__(self, ui: UI, telemetry: Telemetry, config: GlobalConfig) -> None: - super().__init__(ui, telemetry) - self.config = config - - @property - def commands(self) -> list[Command]: - deps = (self.ui, self.telemetry, self.config) - return [ConfigShow(*deps), ConfigGet(*deps), ConfigSet(*deps), ConfigChannel(*deps)] -``` - -- [ ] **Step 3: `commands/update.py`** - -```python -"""Manual update. Auto-update is gone; main.py only prints a notice after commands.""" - -from __future__ import annotations - -from commands.base import Command -from core.errors import UpdateError -from core.version import UNKNOWN, parse_version -from services.telemetry import Telemetry -from services.updater import Release, UpdateChecker, Updater, is_frozen -from ui import UI - - -class UpdateCommand(Command): - name, help, panel = "update", "Update the CLI to the latest version.", "System" - - def __init__(self, ui: UI, telemetry: Telemetry, checker: UpdateChecker, updater: Updater) -> None: - super().__init__(ui, telemetry) - self.checker = checker - self.updater = updater - - def run(self) -> None: - if not is_frozen(): - self.ui.warning("The update command is only available for the binary version of Portabase CLI.") - self.ui.info("If you installed from source, use [bold]git pull[/bold] to update.") - return - - current = self.checker.current - release = self._latest() - if release.tag == current: - self.ui.success(f"Portabase CLI is already up to date ({current}).") - return - if current != UNKNOWN and parse_version(release.tag) < parse_version(current): - self.ui.warning(f"Current version ({current}) is newer than the latest remote version ({release.tag}).") - self.confirm_or_abort("Continue with the downgrade?", default=False) - - target = self.updater.target_path() - self.ui.info(f"Updating Portabase CLI from {current} to {release.tag}") - self.ui.info(f"Target installation path: {target}") - - total = self.updater.expected_size(release) or 0 - with self.ui.progress().download(f"Downloading {release.tag}...", total) as advance: - tmp = self.updater.download(release, advance) - self.updater.install(tmp, target) - self.ui.success(f"Successfully updated to {release.tag}!") - - def _latest(self) -> Release: - try: - release = self.checker.fetch_latest() - except Exception as e: # NetworkError - raise UpdateError("Could not fetch latest release data from GitHub.", cause=e) from e - if release is None: - raise UpdateError("No release found for this channel.") - return release -``` - -- [ ] **Step 4: Vérifier le lint des nouveaux fichiers** - -Run: `uv run ruff check commands/lifecycle.py commands/config.py commands/update.py commands/base.py services ui core` -Expected: `All checks passed!`. (`except Exception` dans `_latest` : remplacer par `except NetworkError` en important `NetworkError` depuis `core.errors` si ruff `BLE001` se plaint — c'est de toute façon plus précis.) - -- [ ] **Step 5: Commit** - -```bash -git add commands/lifecycle.py commands/config.py commands/update.py -git commit -m "feat(commands): rewrite lifecycle, config and update commands as classes" -``` - ---- - -### Task 12 : `main.py` — câblage, catcher, bascule - -**Files:** -- Modify: `main.py` (réécriture complète) -- Delete: `commands/common.py`, `core/updater.py` -- Modify: `pyproject.toml` (`TID251`, per-file-ignores) - -**Interfaces:** -- Consumes: tout ce qui précède + fonctions legacy `commands.agent.agent`, `commands.dashboard.dashboard`, `commands.db.app`. -- Produces: `Settings`, `build_app(ui, telemetry, config, settings) -> tuple[typer.Typer, UpdateChecker]`, `main() -> None`. - -- [ ] **Step 1: Réécrire `main.py`** - -```python -"""Entry point. Builds dependencies, registers commands, owns the single error boundary.""" - -from __future__ import annotations - -import os -import platform -import sys -from dataclasses import dataclass -from typing import Annotated - -import click -import typer - -from commands import agent as legacy_agent -from commands import dashboard as legacy_dashboard -from commands import db as legacy_db -from commands import decrypt as legacy_decrypt -from commands.base import LegacyCommand -from commands.config import ConfigCommands -from commands.lifecycle import LogsCommand, RestartCommand, StartCommand, StopCommand, UninstallCommand -from commands.update import UpdateCommand -from core.config import GlobalConfig -from core.errors import PortabaseError, UserAbort, ValidationError -from core.version import current_version -from services.docker import DockerRunner -from services.http import HttpClient -from services.telemetry import ConsoleTelemetry, TelemetryFactory, TelemetryHub -from services.updater import UpdateChecker, Updater, is_frozen -from ui import UI - - -@dataclass -class Settings: - non_interactive: bool = False - verbose: bool = False - debug: bool = False - no_color: bool = False - - @classmethod - def from_env(cls) -> Settings: - return cls( - non_interactive=os.environ.get("PORTABASE_NON_INTERACTIVE", "").lower() in ("1", "true", "yes") - or not sys.stdin.isatty(), - no_color=bool(os.environ.get("NO_COLOR")), - ) - - -def build_app( - ui: UI, telemetry: TelemetryHub, config: GlobalConfig, settings: Settings -) -> tuple[typer.Typer, UpdateChecker]: - app = typer.Typer(no_args_is_help=True, add_completion=False, rich_markup_mode="rich") - http = HttpClient() - docker = DockerRunner() - version = current_version() - checker = UpdateChecker(http, config, version) - updater = Updater(http, version) - - def version_callback(value: bool) -> None: - if value: - ui.print(f"Portabase CLI version: {version}") - latest = checker.available(force=True) - if latest: - ui.warning(f"A new version is available: [bold]{latest}[/bold]") - raise typer.Exit() - - @app.callback() - def root( - _version: Annotated[ - bool | None, - typer.Option("--version", help="Show the version and exit.", callback=version_callback, is_eager=True), - ] = None, - verbose: Annotated[bool, typer.Option("--verbose", help="Show error causes and tracebacks.")] = False, - debug: Annotated[bool, typer.Option("--debug", help="Verbose plus telemetry trace on stderr.")] = False, - no_color: Annotated[bool, typer.Option("--no-color", help="Disable colours.")] = False, - non_interactive: Annotated[ - bool, - typer.Option("--non-interactive", envvar="PORTABASE_NON_INTERACTIVE", help="Never prompt; fail on missing input."), - ] = False, - ) -> None: - """Portabase CLI to manage agents, dashboards and databases.""" - settings.verbose = verbose or debug - settings.debug = debug - settings.no_color = settings.no_color or no_color - settings.non_interactive = settings.non_interactive or non_interactive - ui.configure(verbose=settings.verbose, no_color=settings.no_color, non_interactive=settings.non_interactive) - if debug: - telemetry.set(ConsoleTelemetry()) - - commands = [ - LegacyCommand(ui, telemetry, legacy_agent.agent, name="agent", help="Create a new Portabase Agent instance.", panel="Creation"), - LegacyCommand(ui, telemetry, legacy_dashboard.dashboard, name="dashboard", help="Create a new Portabase Dashboard instance.", panel="Creation"), - LegacyCommand(ui, telemetry, legacy_decrypt.decrypt, name="decrypt", help="Decrypt Portabase .enc backup files (single file or folder).", panel="Configuration"), - StartCommand(ui, telemetry, docker), - StopCommand(ui, telemetry, docker), - RestartCommand(ui, telemetry, docker), - LogsCommand(ui, telemetry, docker), - UninstallCommand(ui, telemetry, docker), - UpdateCommand(ui, telemetry, checker, updater), - ] - for cmd in commands: - cmd.register(app) - - app.add_typer(legacy_db.app, name="db", rich_help_panel="Configuration") # legacy, replaced in plan 4 - ConfigCommands(ui, telemetry, config).register(app) - - return app, checker - - -def _notify_update(ui: UI, checker: UpdateChecker, settings: Settings, invoked: str | None) -> None: - if not is_frozen() or settings.non_interactive or invoked in ("update", None): - return - latest = checker.available() - if latest: - ui.print("") - ui.warning(f"A new version of Portabase CLI is available: [bold]{latest}[/bold] (current: {checker.current})") - ui.info("Run [bold]portabase update[/bold] to update.") - - -def main() -> None: - settings = Settings.from_env() - config = GlobalConfig() - ui = UI(non_interactive=settings.non_interactive, no_color=settings.no_color) - telemetry = TelemetryFactory.build(config, debug=False) - app, checker = build_app(ui, telemetry, config, settings) - invoked = next((a for a in sys.argv[1:] if not a.startswith("-")), None) - exit_code = 0 - - try: - with telemetry.session(cli_version=current_version(), os=platform.system()): - app(standalone_mode=False) - except UserAbort as e: - ui.warning(e.message) - telemetry.event("abort") - exit_code = e.exit_code - except PortabaseError as e: - ui.error(e) - telemetry.error(e) - exit_code = e.exit_code - except click.exceptions.NoArgsIsHelpError: - exit_code = 0 # help already printed by Typer - except click.exceptions.Exit as e: # typer.Exit from legacy code or --help - exit_code = e.exit_code - except click.exceptions.Abort: # typer.Abort from legacy code - ui.warning("Cancelled.") - exit_code = 130 - except click.UsageError as e: - err = ValidationError(e.format_message(), hint="Run 'portabase --help' for usage.") - ui.error(err) - telemetry.error(err) - exit_code = err.exit_code - except KeyboardInterrupt: - ui.console.print("") - ui.warning("Cancelled.") - exit_code = 130 - except Exception as e: # noqa: BLE001 — last resort: a bug, not an expected error - wrapped = PortabaseError("Unexpected error: " + str(e), cause=e) - ui.error(wrapped, unexpected=True) - telemetry.error(e, unexpected=True) - exit_code = 1 - finally: - telemetry.flush() - - if exit_code == 0: - _notify_update(ui, checker, settings, invoked) - raise SystemExit(exit_code) - - -if __name__ == "__main__": - main() -``` - -- [ ] **Step 2: Supprimer les modules remplacés** - -Run: `git rm commands/common.py core/updater.py` - -Puis vérifier qu'aucun import ne subsiste : -Run: `grep -rn "commands.common\|core.updater\|check_for_updates\|update_cli" --include=*.py . | grep -v ".venv"` -Expected: aucune sortie. - -- [ ] **Step 3: Mettre à jour `pyproject.toml`** - -Remplacer le bloc `[tool.ruff.lint.per-file-ignores]` par : - -```toml -# Code legacy supprimé au plan 4. Ne pas étendre cette liste. -[tool.ruff.lint.per-file-ignores] -"commands/agent.py" = ["BLE001", "E722", "S110", "SIM102", "TID251"] -"commands/db.py" = ["BLE001", "E722", "S110", "TID251"] -"commands/dashboard.py" = ["BLE001", "TID251"] -"commands/decrypt.py" = ["B904", "TID251"] -"core/crypto.py" = ["BLE001", "SIM105"] -"core/config.py" = ["BLE001", "E722", "S110"] -"core/utils.py" = ["BLE001", "E722", "S110", "PLR1730", "TID251"] -"core/network.py" = ["BLE001", "TID251"] -"main.py" = ["TID251"] - -[tool.ruff.lint.flake8-tidy-imports.banned-api] -"rich.prompt".msg = "Use ui.form() / ui.confirm() instead." -"rich.console".msg = "Only ui/ may build a Console. Use the UI facade." -"typer.prompt".msg = "Use ui.form() instead." -"typer.confirm".msg = "Use ui.confirm() instead." -``` - -Et ajouter `"TID251"` dans `select` s'il n'y est pas déjà (il y est depuis Plan 1). `main.py` importe `click` et `typer.Exit`, pas de prompt : `TID251` sur `main.py` est là uniquement pour `ui.console.print` ? Non — `rich.console` n'y est pas importé. Retirer `"main.py" = ["TID251"]` si `ruff check` passe sans. - -Ajouter `known-first-party = ["commands", "core", "services", "ui", "templates"]` dans `[tool.ruff.lint.isort]`. - -- [ ] **Step 4: Lint complet** - -Run: `uv run ruff check . && uv run ruff format --check .` -Expected: passe. Sinon `uv run ruff format .` puis corriger les erreurs signalées **dans les nouveaux fichiers uniquement**. - -- [ ] **Step 5: Vérifier l'aide et les erreurs de saisie** - -Run: `uv run python main.py; echo "exit=$?"` -Expected: aide affichée, `exit=0`. - -Run: `uv run python main.py --help | head -30` -Expected: panneaux `Creation` (agent, dashboard), `Lifecycle` (start, stop, restart, logs, uninstall), `Configuration` (decrypt, db, config), `System` (update) ; options `--version`, `--verbose`, `--debug`, `--no-color`, `--non-interactive`. - -Run: `uv run python main.py start; echo "exit=$?"` -Expected: aide de `start` (no_args_is_help), `exit=0`. - -Run: `uv run python main.py bogus; echo "exit=$?"` -Expected: `✖ Error: No such command 'bogus'.` + hint, `exit=2`. - -Run: `uv run python main.py start /tmp/does-not-exist; echo "exit=$?"` -Expected: `✖ Error: No Portabase configuration found in: /tmp/does-not-exist` + hint, `exit=3`. - -Run: `uv run python main.py --verbose start /tmp/does-not-exist 2>&1 | grep -c "code: E_CONFIG"` -Expected: `1`. - -- [ ] **Step 6: Vérifier config** - -Run: `uv run python main.py config show && uv run python main.py config set update_channel beta && uv run python main.py config get update_channel && uv run python main.py config channel stable && uv run python main.py config set update_channel nope; echo "exit=$?"` -Expected: affichage, `✔ update_channel = beta`, `beta`, `✔ update_channel = stable`, puis `✖ Error: Invalid channel 'nope'.` `exit=2`. - -- [ ] **Step 7: Vérifier update et --version (non-frozen)** - -Run: `uv run python main.py update; echo "exit=$?"; uv run python main.py --version; echo "exit=$?"` -Expected: avertissement "only available for the binary version", `exit=0` ; version puis éventuellement "A new version is available", `exit=0`. - -- [ ] **Step 8: Vérifier le mode non-interactif et Ctrl-C** - -Run: `uv run python main.py --non-interactive uninstall /tmp/does-not-exist; echo "exit=$?"` -Expected: `E_CONFIG`, `exit=3` (l'erreur dossier précède la confirmation). - -Créer un faux projet : `mkdir -p /tmp/pb-fake && touch /tmp/pb-fake/docker-compose.yml`. -Run: `uv run python main.py --non-interactive uninstall /tmp/pb-fake; echo "exit=$?"` -Expected (Docker présent) : confirm par défaut `False` → `⚠ Cancelled.` `exit=130`, dossier intact. (Docker absent : `E_DOCKER`, `exit=4`.) - -Run: `uv run python main.py uninstall /tmp/pb-fake` puis Ctrl-C au prompt. -Expected: `⚠ Cancelled.`, `exit=130`, pas de traceback. - -- [ ] **Step 9: Vérifier les commandes legacy à travers le catcher** - -Run: `uv run python main.py agent; echo "exit=$?"` puis `uv run python main.py db list /tmp/does-not-exist; echo "exit=$?"` -Expected: aide de `agent` `exit=0` ; message legacy `No Portabase configuration found` (ancien style) et `exit=1` (via `typer.Exit(1)` → `click.exceptions.Exit`). - -- [ ] **Step 10: Vérifier le lifecycle réel (si Docker disponible)** - -```bash -cd /tmp && rm -rf pb-smoke && uv --directory /home/soluce/Documents/PROJETS/Portabase/cli run python /home/soluce/Documents/PROJETS/Portabase/cli/main.py dashboard pb-smoke --port 8899 -``` -Répondre `internal` au choix DB, `N` à "Start dashboard now?". Puis : - -```bash -M=/home/soluce/Documents/PROJETS/Portabase/cli/main.py -uv --directory /home/soluce/Documents/PROJETS/Portabase/cli run python $M start /tmp/pb-smoke -uv --directory /home/soluce/Documents/PROJETS/Portabase/cli run python $M logs /tmp/pb-smoke --no-follow | tail -3 -uv --directory /home/soluce/Documents/PROJETS/Portabase/cli run python $M restart /tmp/pb-smoke -uv --directory /home/soluce/Documents/PROJETS/Portabase/cli run python $M stop /tmp/pb-smoke -uv --directory /home/soluce/Documents/PROJETS/Portabase/cli run python $M uninstall /tmp/pb-smoke --force -ls /tmp/pb-smoke 2>&1 -``` -Expected: `✔ Started`, quelques lignes de logs, `✔ Restarted`, `✔ Stopped`, `✔ Uninstalled`, `No such file or directory`. - -- [ ] **Step 11: Commit** - -```bash -git add main.py pyproject.toml -git commit -m "refactor: wire commands through DI container and single error boundary - -Lifecycle, config and update run on the new Command classes; agent, -dashboard and db stay on legacy code behind LegacyCommand until plan 4. -Auto-update is replaced by a post-command notice." -``` - ---- - -### Task 13 : Build smoke, PR - -**Files:** aucun nouveau. - -- [ ] **Step 1: Binaire local** - -Run: `rm -rf build dist *.spec && uv run pyinstaller --onefile --name portabase_smoke --paths=. --collect-all rich --collect-all requests --collect-data certifi --add-data "pyproject.toml:." main.py && ./dist/portabase_smoke --version && ./dist/portabase_smoke config show && ./dist/portabase_smoke start /tmp/nope; echo "exit=$?"; rm -rf build dist *.spec` -Expected: version, config, `E_CONFIG` `exit=3`. Aucun `ModuleNotFoundError` (questionary, services, ui embarqués via `--paths=.`). - -- [ ] **Step 2: PR** - -```bash -git checkout -b refactor/foundations -git push -u origin refactor/foundations -``` -Ouvrir la PR « refactor: foundations (errors, ui, services, Command) + lifecycle rewrite ». Checks Plan 1 verts attendus. - -- [ ] **Step 3: Release candidate (optionnel mais recommandé)** - -Après merge : Actions → Bump version → `26.08.0rc1`, channel `rc`. Installer le binaire rc sur une machine avec une install existante et dérouler `start/stop/logs/restart` + `--version` (la notification de mise à jour après commande s'affiche seulement en binaire). - ---- - -## Self-review - -**Spec coverage :** -- §3 structure : `core/errors`, `core/version`, `core/config` (GlobalConfig), `ui/*`, `services/{http,docker,telemetry,updater}`, `commands/{base,lifecycle,config,update}`, `main.py` ✔. `services/{envfile,ports,templates,renderer,project,compose_facts}`, `engines/`, `commands/{agent,dashboard,build,db,flows}` → Plans 3–4. `core/fields.py` : déviation documentée. -- §4.1 `Command`, `register`, `_traced`, injection ✔ (T9). `Annotated` ✔. -- §7 ui : tokens ✔, composants Banner/Message/Section/Status/Hint/Prompt ✔ + Progress (appelant : update). `Summary`, `DataTable`, `Diff` → Plan 4 (appelants). `Form` ✔ avec flag→prompt→défaut→erreur, `UserAbort` sur `None` ✔. `NO_COLOR` ✔. Pas de prompt sous status : respecté dans lifecycle (confirm avant status). -- §8.1 hiérarchie et codes ✔. §8.2 catcher, `standalone_mode=False`, mapping click ✔ (T12). §8.3 télémétrie contrat + noop + console + hub ✔ ; opt-in config lu par `TelemetryFactory` (endpoint ignoré tant qu'aucun exporter — documenté). §8.4 updater : notif après commande, cache 24 h, silencieux offline, checksum ✔. -- §10 B+C : shippable, legacy via `LegacyCommand` ✔. - -**Placeholders :** aucun. - -**Cohérence des types :** `UI.confirm(question, *, default, value)` utilisé par `Command.confirm_or_abort` et `require_docker` ✔ ; `Telemetry.span` context manager utilisé par `_traced` ✔ ; `UpdateChecker.available(force=)` utilisé par `version_callback` et `_notify_update` ✔ ; `Updater.expected_size/download/install` utilisés par `UpdateCommand` ✔ ; `HttpClient.content_length` utilisé par `Updater.expected_size` ✔ (`head_content_length` cité dans l'interface T7 = `content_length` ; nom retenu : `content_length`). - -**Écarts connus :** -- `UpdateCommand._latest` : utiliser `except NetworkError` (T11 step 4). diff --git a/docs/superpowers/plans/2026-09-11-plan-3-templates-engines.md b/docs/superpowers/plans/2026-09-11-plan-3-templates-engines.md deleted file mode 100644 index 3b5be5d..0000000 --- a/docs/superpowers/plans/2026-09-11-plan-3-templates-engines.md +++ /dev/null @@ -1,1944 +0,0 @@ -# Plan 3 — Templates Jinja2 et moteurs DB (chantier D) - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** Introduire les templates Jinja2 versionnés (source `templates/` à la racine, manifest, cache, `TemplateRepository`), le registre de moteurs DB en classes, et les garde-fous CI (`render-check`, `engines-check`, manifest à l'upload, hotfix) — sans encore brancher le rendu sur les commandes (Plan 4). Le CLI reste fonctionnel : les commandes legacy continuent de lire `agent.yml` / `dashboard.yml` (conservés dans `templates/` jusqu'au Plan 4). - -**Architecture:** `TemplateRepository` résout une version → dossier local (`./templates` en dev, cache `~/.portabase/cache/templates//` en binaire), vérifie un `manifest.json` (sha256) et expose des `jinja2.Template`. Chaque `DbEngine` déclare ses champs, génère un `DatabaseSpec`, produit ses variables `.env`, son contexte de template et sa projection `databases.json`. `render_check.py` rend chaque template avec des fixtures et valide le YAML puis `docker compose config`. - -**Tech Stack:** Jinja2 3.1, PyYAML, Python 3.12, GitHub Actions, s3cmd, jq. - -**Spec:** `docs/superpowers/specs/2026-09-11-cli-refactor-design.md` — sections 5.4, 5.6, 6, 6.1, 9.1 (`render-check`, `engines-check`), 9.3 (hotfix, manifest), 10 (D). - -## Global Constraints - -- Prérequis : Plans 1 et 2 exécutés. -- Règle de dépendance : `engines → core` uniquement. `services → engines, core`. Vérifié par revue ; ruff ne le détecte pas. -- Déviations spec assumées : - - `DatabaseSpec` vit dans `core/specs.py` (produit par `engines`, consommé par `services`), pas dans `services/project.py`. - - Pas de `mysql.yml.j2` : le moteur `mysql` utilise `engines/mariadb.yml.j2`, comme le code legacy (image `mariadb:latest`). Changer d'image casserait les volumes des installs existantes. -- Les fichiers legacy `agent.yml` et `dashboard.yml` sont déplacés tels quels dans `templates/` et restent uploadés (le code legacy les fetch sous `/`). Supprimés au Plan 4. -- Conventions de nommage legacy conservées à l'identique (service `db-pg-`, `db-mongo-auth-`, db `pg_`, user `admin`, firebird user `alice` / `mirror.fdb`, mssql `sa` / `master` / name `MSSQL`, redis/valkey `database: "0"`), pour que les nouvelles installs ressemblent aux anciennes. -- `generate_password` retire `$` et `` ` `` des symboles (Task 1). -- Pas de tests unitaires. `render_check.py` est la vérification exécutable de ce plan et devient un job CI. -- Aucune commande utilisateur ne change dans ce plan. - ---- - -## File Structure - -| Fichier | Action | Responsabilité | -|---|---|---| -| `core/utils.py` | modifier | `generate_password` sans `$`/`` ` `` | -| `core/specs.py` | créer | `DatabaseSpec` | -| `services/ports.py` | créer | `PortAllocator` | -| `engines/__init__.py` | créer | `registry` | -| `engines/base.py` | créer | `DbEngine` | -| `engines/registry.py` | créer | `EngineRegistry` | -| `engines/sql.py` | créer | `StandardSqlEngine`, `PostgresEngine`, `PostgresClusterEngine`, `MySqlEngine`, `MariaDbEngine`, `MssqlEngine`, `FirebirdEngine` | -| `engines/redis.py` | créer | `RedisEngine` | -| `engines/valkey.py` | créer | `ValkeyEngine` | -| `engines/mongo.py` | créer | `MongoEngine` | -| `engines/sqlite.py` | créer | `SqliteEngine` | -| `engines/docker_volume.py` | créer | `DockerVolumeEngine` | -| `templates/agent.yml.j2`, `dashboard.yml.j2`, `engines/*.yml.j2` | créer | templates Jinja2 | -| `templates/engines.map.json` | créer | clé moteur → template | -| `templates/agent.yml`, `dashboard.yml` | déplacer depuis `.github/assets/templates/` | legacy | -| `services/templates.py` | créer | `Manifest`, `TemplateRepository` | -| `scripts/render_check.py` | créer | validation des templates | -| `.github/workflows/ci.yml` | modifier | jobs `render-check`, `engines-check` | -| `.github/workflows/templates-upload.yml` | modifier | source `templates/`, manifest | -| `.github/workflows/templates-hotfix.yml` | créer | re-upload d'une version | -| `pyproject.toml` | modifier | `jinja2` | -| `.gitleaks.toml` | modifier | chemin `templates/` déjà allowlisté ; retirer `.github/assets/templates` | - ---- - -### Task 1 : `core/specs.py`, `services/ports.py`, mot de passe - -**Files:** -- Create: `core/specs.py` -- Create: `services/ports.py` -- Modify: `core/utils.py:70-92` (`generate_password`) - -**Interfaces:** -- Produces: `DatabaseSpec` (frozen dataclass) avec `env_prefix`, `is_service`, `with_options()` ; `PortAllocator().free() -> int` ; `generate_password(length=16)` sans `$` ni `` ` ``. - -- [ ] **Step 1: `core/specs.py`** - -```python -"""Typed view of one databases.json entry plus what the CLI needs to render it.""" - -from __future__ import annotations - -from dataclasses import dataclass, field, replace -from typing import Any - - -@dataclass(frozen=True) -class DatabaseSpec: - id: str - engine: str - name: str - managed: bool = False # True: a Compose service is rendered for it - host: str | None = None # service name when managed, remote host otherwise - port: int | None = None # container/remote port (what the agent connects to) - host_port: int | None = None # published port on the Docker host (managed only) - database: str | None = None - username: str | None = None - password: str | None = None - root_password: str | None = None # firebird - path: str | None = None # sqlite - volume: str | None = None # docker-volume - container: str | None = None # docker-volume - options: dict[str, Any] = field(default_factory=dict) - - @property - def env_prefix(self) -> str: - if not self.host: - raise ValueError("env_prefix requires a host/service name") - return self.host.upper().replace("-", "_") - - @property - def auth(self) -> bool: - return bool(self.password) - - def with_options(self, options: dict[str, Any]) -> DatabaseSpec: - return replace(self, options=dict(options)) -``` - -- [ ] **Step 2: `services/ports.py`** - -```python -"""Free TCP port allocation. Remembers ports handed out during the process to avoid duplicates.""" - -from __future__ import annotations - -import socket - - -class PortAllocator: - def __init__(self) -> None: - self._given: set[int] = set() - - def free(self) -> int: - for _ in range(50): - with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s: - s.bind(("", 0)) - port = s.getsockname()[1] - if port not in self._given: - self._given.add(port) - return port - raise RuntimeError("Could not allocate a free port") - - -class FixedPortAllocator(PortAllocator): - """Deterministic ports for render checks and fixtures.""" - - def __init__(self, start: int = 40000) -> None: - super().__init__() - self._next = start - - def free(self) -> int: - port = self._next - self._next += 1 - return port -``` - -- [ ] **Step 3: Corriger `generate_password` dans `core/utils.py`** - -Remplacer la ligne `symbols = "!@#$%^&*()-_=+[]{}|;:,.<>?"` par : - -```python - # No '$' (Compose interpolation / shell), no '`' or quotes (shell command args in templates). - symbols = "!@#%^&*()-_=+[]{}|;:,.<>?" -``` - -- [ ] **Step 4: Vérifier** - -Run: `uv run python -c " -from core.specs import DatabaseSpec -from services.ports import PortAllocator, FixedPortAllocator -from core.utils import generate_password -s = DatabaseSpec(id='1', engine='postgresql', name='x', managed=True, host='db-pg-a1f2', password='p') -print(s.env_prefix, s.auth, s.with_options({'a':1}).options) -p = PortAllocator(); a, b = p.free(), p.free(); print(a != b, FixedPortAllocator().free()) -pw = generate_password(); print(len(pw), '\$' not in pw and '\`' not in pw)"` -Expected: `DB_PG_A1F2 True {'a': 1}`, `True 40000`, `16 True`. - -- [ ] **Step 5: Commit** - -```bash -git add core/specs.py services/ports.py core/utils.py -git commit -m "feat: add DatabaseSpec, PortAllocator; drop shell-unsafe symbols from generated passwords" -``` - ---- - -### Task 2 : `engines/base.py` et `engines/registry.py` - -**Files:** -- Create: `engines/__init__.py` (rempli Task 4) -- Create: `engines/base.py` -- Create: `engines/registry.py` - -**Interfaces:** -- Produces: `DbEngine` ABC : - - classe-attributs `key`, `display`, `default_port: int | None`, `template: str | None`, `auth_variants=False`, `warning=None`, `has_modes=True` - - `fields_existing() -> list[Field]`, `fields_new() -> list[Field]`, `option_fields() -> list[Field]` - - `generate(*, auth: bool, ports: PortAllocator, answers: dict) -> DatabaseSpec` - - `from_existing(answers: dict) -> DatabaseSpec` - - `env_vars(spec) -> dict[str, str]` - - `template_ctx(spec, *, inline: bool = False) -> dict` - - `agent_entry(spec) -> dict` - - `describe(spec) -> str` (pour `db list` : « host:port », « Local File », « volume: x ») - - helpers `new_id()`, `service_name(slug, auth)`, `var(spec, suffix, value, inline)` -- `EngineRegistry(engines)` : `get(key)`, `keys()`, `choices()`, `__iter__`. - -- [ ] **Step 1: `engines/base.py`** - -```python -"""DbEngine: everything the CLI needs to know about one database engine.""" - -from __future__ import annotations - -import secrets -import uuid -from abc import ABC, abstractmethod -from typing import Any - -from core.fields import Field -from core.specs import DatabaseSpec -from services.ports import PortAllocator - -STANDARD_EXISTING_FIELDS = ( - Field("host", "Host", "text", default="localhost"), - Field("port", "Port", "int"), # default filled per engine - Field("database", "Database Name", "text"), - Field("username", "Username", "text"), - Field("password", "Password", "secret"), -) - - -class DbEngine(ABC): - key: str - display: str - default_port: int | None = None - template: str | None = None # e.g. "engines/postgresql.yml.j2"; None: no Compose service - auth_variants: bool = False # offer with-auth / no-auth when creating a container - warning: str | None = None # shown before collecting answers - has_modes: bool = True # new/existing choice applies - - # ---- declarations ----------------------------------------------------- - - def fields_existing(self) -> list[Field]: - return [ - Field("port", "Port", "int", default=self.default_port) if f.name == "port" else f - for f in STANDARD_EXISTING_FIELDS - ] - - def fields_new(self) -> list[Field]: - return [] - - def option_fields(self) -> list[Field]: - return [] - - # ---- construction ----------------------------------------------------- - - @abstractmethod - def generate(self, *, auth: bool, ports: PortAllocator, answers: dict[str, Any]) -> DatabaseSpec: ... - - def from_existing(self, answers: dict[str, Any]) -> DatabaseSpec: - return DatabaseSpec( - id=self.new_id(), - engine=self.key, - name=answers.get("label") or "External DB", - managed=False, - host=answers["host"], - port=int(answers["port"]), - database=answers["database"], - username=answers["username"], - password=answers["password"], - ) - - # ---- rendering inputs ------------------------------------------------- - - def env_vars(self, spec: DatabaseSpec) -> dict[str, str]: - """Variables written to .env for a managed service. Default: PORT, DB, USER, PASS.""" - p = spec.env_prefix - return { - f"{p}_PORT": str(spec.host_port), - f"{p}_DB": spec.database or "", - f"{p}_USER": spec.username or "", - f"{p}_PASS": spec.password or "", - } - - def template_ctx(self, spec: DatabaseSpec, *, inline: bool = False) -> dict[str, Any]: - return { - "name": spec.host, - "volume": f"{spec.host}-data", - "auth": spec.auth, - "port_var": self.var(spec, "PORT", spec.host_port, inline), - "db_var": self.var(spec, "DB", spec.database, inline), - "user_var": self.var(spec, "USER", spec.username, inline), - "password_var": self.var(spec, "PASS", spec.password, inline), - } - - def agent_entry(self, spec: DatabaseSpec) -> dict[str, Any]: - """Projection to databases.json. Same shape as the legacy CLI.""" - entry: dict[str, Any] = { - "name": spec.name, - "database": self.agent_database(spec), - "type": self.key, - "username": spec.username or "", - "password": spec.password or "", - "port": spec.port, - "host": spec.host, - "generated_id": spec.id, - } - options = self.non_default_options(spec) - if options: - entry["options"] = options - return entry - - def agent_database(self, spec: DatabaseSpec) -> str: - return spec.database or "" - - def describe(self, spec: DatabaseSpec) -> str: - return f"{spec.host}:{spec.port}" - - # ---- helpers ---------------------------------------------------------- - - def non_default_options(self, spec: DatabaseSpec) -> dict[str, Any]: - defaults = {f.name: f.default for f in self.option_fields()} - return {k: v for k, v in spec.options.items() if k in defaults and v != defaults[k]} - - @staticmethod - def new_id() -> str: - return str(uuid.uuid4()) - - @staticmethod - def service_name(slug: str, auth: bool = False) -> str: - suffix = "auth-" if auth else "" - return f"db-{slug}-{suffix}{secrets.token_hex(2)}" - - @staticmethod - def var(spec: DatabaseSpec, suffix: str, value: Any, inline: bool) -> str: - return str(value if value is not None else "") if inline else f"${{{spec.env_prefix}_{suffix}}}" -``` - -- [ ] **Step 2: `engines/registry.py`** - -```python -from __future__ import annotations - -from collections.abc import Iterable, Iterator - -from core.errors import ValidationError -from engines.base import DbEngine - - -class EngineRegistry: - def __init__(self, engines: Iterable[DbEngine]) -> None: - self._by_key: dict[str, DbEngine] = {} - for engine in engines: - if engine.key in self._by_key: - raise ValueError(f"Duplicate engine key: {engine.key}") - self._by_key[engine.key] = engine - - def get(self, key: str) -> DbEngine: - try: - return self._by_key[key] - except KeyError: - raise ValidationError( - f"Unknown engine '{key}'.", - hint="Available: " + ", ".join(self.keys()), - ) from None - - def keys(self) -> list[str]: - return list(self._by_key) - - def choices(self) -> list[str]: - return self.keys() - - def __iter__(self) -> Iterator[DbEngine]: - return iter(self._by_key.values()) - - def __contains__(self, key: str) -> bool: - return key in self._by_key -``` - -- [ ] **Step 3: Vérifier** - -Run: `uv run python -c " -from engines.base import DbEngine -from engines.registry import EngineRegistry -from core.errors import ValidationError -print([f.name for f in DbEngine.fields_existing(type('E',(DbEngine,),{'key':'x','display':'X','default_port':1,'generate':lambda *a,**k: None})())]) -try: EngineRegistry([]).get('nope') -except ValidationError as e: print(e.message, '|', e.hint)"` -Expected: `['host', 'port', 'database', 'username', 'password']` puis `Unknown engine 'nope'. | Available: `. - -- [ ] **Step 4: Commit** - -```bash -git add engines/ -git commit -m "feat(engines): add DbEngine base class and EngineRegistry" -``` - ---- - -### Task 3 : Moteurs SQL (`engines/sql.py`) - -**Files:** -- Create: `engines/sql.py` - -**Interfaces:** -- Produces: `StandardSqlEngine` et sous-classes `PostgresEngine` (`postgresql`), `PostgresClusterEngine` (`postgresql-cluster`), `MySqlEngine` (`mysql`), `MariaDbEngine` (`mariadb`), `MssqlEngine` (`mssql`), `FirebirdEngine` (`firebird`). - -- [ ] **Step 1: Écrire le module** - -```python -"""SQL engines rendered as Compose services. Naming mirrors the legacy CLI.""" - -from __future__ import annotations - -import secrets -from typing import Any - -from core.fields import Field -from core.specs import DatabaseSpec -from core.utils import generate_password -from engines.base import DbEngine -from services.ports import PortAllocator - - -class StandardSqlEngine(DbEngine): - slug: str # service name fragment: db--xxxx - db_prefix: str # generated database name: _xxxxxxxx - default_user = "admin" - - def generate(self, *, auth: bool, ports: PortAllocator, answers: dict[str, Any]) -> DatabaseSpec: - db_name = f"{self.db_prefix}_{secrets.token_hex(4)}" - return DatabaseSpec( - id=self.new_id(), - engine=self.key, - name=db_name, - managed=True, - host=self.service_name(self.slug), - port=self.default_port, - host_port=ports.free(), - database=db_name, - username=self.default_user, - password=generate_password(16), - options=dict(answers.get("options", {})), - ) - - -class PostgresEngine(StandardSqlEngine): - key, display, default_port = "postgresql", "PostgreSQL", 5432 - template, slug, db_prefix = "engines/postgresql.yml.j2", "pg", "pg" - - def option_fields(self) -> list[Field]: - return [ - Field( - "keep_ownership", - "Keep ownership?", - "bool", - default=False, - help=( - "When enabled, omits --no-owner and --no-privileges from the dump. Ownership and role " - "assignments are preserved. By default these flags are applied to keep restores portable " - "across users and environments." - ), - ), - Field( - "clean_mode", - "Clean mode", - "choice", - default="clean", - choices=("clean", "none", "drop_schemas", "drop_database"), - help=( - "How the target database is cleaned before a restore. clean: pg_restore --clean --if-exists. " - "none: no pre-clean. drop_schemas: drop every non-system schema CASCADE (works on managed " - "Postgres). drop_database: DROP DATABASE + CREATE DATABASE — requires CREATEDB or superuser; " - "most managed providers do not allow it." - ), - ), - ] - - -class PostgresClusterEngine(StandardSqlEngine): - key, display, default_port = "postgresql-cluster", "PostgreSQL Cluster", 5432 - template, slug, db_prefix = "engines/postgresql.yml.j2", "pg", "pg" - warning = ( - "Postgres Cluster requires a superuser. Cluster backup/restore uses pg_dumpall, which dumps all " - "databases and global objects (roles, tablespaces). The provided user must be a Postgres superuser." - ) - - -class MariaDbEngine(StandardSqlEngine): - key, display, default_port = "mariadb", "MariaDB", 3306 - template, slug, db_prefix = "engines/mariadb.yml.j2", "mariadb", "mysql" - - -class MySqlEngine(MariaDbEngine): - """Legacy behaviour: a 'mysql' container is a MariaDB image. Kept for volume compatibility.""" - - key, display = "mysql", "MySQL" - - -class MssqlEngine(StandardSqlEngine): - key, display, default_port = "mssql", "Microsoft SQL Server", 1433 - template, slug, db_prefix = "engines/mssql.yml.j2", "mssql", "master" - - def generate(self, *, auth: bool, ports: PortAllocator, answers: dict[str, Any]) -> DatabaseSpec: - return DatabaseSpec( - id=self.new_id(), - engine=self.key, - name="MSSQL", - managed=True, - host=self.service_name(self.slug), - port=self.default_port, - host_port=ports.free(), - database="master", - username="sa", - password=generate_password(16), - ) - - def env_vars(self, spec: DatabaseSpec) -> dict[str, str]: - p = spec.env_prefix - return {f"{p}_PORT": str(spec.host_port), f"{p}_PASS": spec.password or ""} - - -class FirebirdEngine(StandardSqlEngine): - key, display, default_port = "firebird", "Firebird", 3050 - template, slug, db_prefix = "engines/firebird.yml.j2", "firebird", "fb" - DATA_DIR = "/var/lib/firebird/data" - - def generate(self, *, auth: bool, ports: PortAllocator, answers: dict[str, Any]) -> DatabaseSpec: - db_file = "mirror.fdb" - return DatabaseSpec( - id=self.new_id(), - engine=self.key, - name=db_file, - managed=True, - host=self.service_name(self.slug), - port=self.default_port, - host_port=ports.free(), - database=f"{self.DATA_DIR}/{db_file}", - username="alice", - password=generate_password(16), - root_password=generate_password(16), - ) - - def env_vars(self, spec: DatabaseSpec) -> dict[str, str]: - base = super().env_vars(spec) - # Compose template expects the bare file name; databases.json carries the container path. - base[f"{spec.env_prefix}_DB"] = (spec.database or "").rsplit("/", 1)[-1] - base[f"{spec.env_prefix}_ROOT_PASS"] = spec.root_password or "" - return base - - def template_ctx(self, spec: DatabaseSpec, *, inline: bool = False) -> dict[str, Any]: - ctx = super().template_ctx(spec, inline=inline) - ctx["db_var"] = self.var(spec, "DB", (spec.database or "").rsplit("/", 1)[-1], inline) - ctx["root_password_var"] = self.var(spec, "ROOT_PASS", spec.root_password, inline) - return ctx -``` - -- [ ] **Step 2: Vérifier** - -Run: `uv run python -c " -from engines.sql import * -from services.ports import FixedPortAllocator -p = FixedPortAllocator() -for E in (PostgresEngine, MySqlEngine, MssqlEngine, FirebirdEngine): - e = E(); s = e.generate(auth=True, ports=p, answers={'options': {'clean_mode': 'none'}}) - print(E.key, s.host[:9], sorted(e.env_vars(s)), e.agent_entry(s).get('options'), e.template_ctx(s)['port_var']) -print(PostgresEngine().agent_entry(PostgresEngine().generate(auth=True, ports=p, answers={})).get('options'))"` -Expected (hex variable) : -``` -postgresql db-pg-xxx ['DB_PG_XXXX_DB', 'DB_PG_XXXX_PASS', 'DB_PG_XXXX_PORT', 'DB_PG_XXXX_USER'] {'clean_mode': 'none'} ${DB_PG_XXXX_PORT} -mysql db-mariad [... 4 vars] None ... -mssql db-mssql- [..._PASS, ..._PORT] None ... -firebird db-fireb [..._DB, ..._PASS, ..._PORT, ..._ROOT_PASS, ..._USER] None ... -None -``` -La dernière ligne : options par défaut → pas de clé `options`. - -- [ ] **Step 3: Commit** - -```bash -git add engines/sql.py -git commit -m "feat(engines): add SQL engines (postgresql, cluster, mysql, mariadb, mssql, firebird)" -``` - ---- - -### Task 4 : Redis, Valkey, Mongo, SQLite, Docker volume, registre - -**Files:** -- Create: `engines/redis.py`, `engines/valkey.py`, `engines/mongo.py`, `engines/sqlite.py`, `engines/docker_volume.py` -- Modify: `engines/__init__.py` - -**Interfaces:** -- Produces: `RedisEngine`, `ValkeyEngine`, `MongoEngine`, `SqliteEngine`, `DockerVolumeEngine` ; `engines.registry: EngineRegistry` (instance module-level) ; `engines.ALL: tuple[DbEngine, ...]`. - -- [ ] **Step 1: `engines/redis.py`** - -```python -from __future__ import annotations - -import secrets -from typing import Any - -from core.fields import Field -from core.specs import DatabaseSpec -from core.utils import generate_password -from engines.base import DbEngine -from services.ports import PortAllocator - - -class RedisEngine(DbEngine): - key, display, default_port = "redis", "Redis", 6379 - template = "engines/redis.yml.j2" - auth_variants = True - - def fields_existing(self) -> list[Field]: - return [ - Field("host", "Host", "text", default="localhost"), - Field("port", "Port", "int", default=self.default_port), - Field("database", "Database index", "text", default="0"), - Field("username", "Username (empty if none)", "text", default=""), - Field("password", "Password (empty if none)", "text", default=""), - ] - - def generate(self, *, auth: bool, ports: PortAllocator, answers: dict[str, Any]) -> DatabaseSpec: - return DatabaseSpec( - id=self.new_id(), - engine=self.key, - name=f"redis_{secrets.token_hex(4)}", - managed=True, - host=self.service_name("redis", auth), - port=self.default_port, - host_port=ports.free(), - database="0", - username="", - password=generate_password(16) if auth else None, - ) - - def env_vars(self, spec: DatabaseSpec) -> dict[str, str]: - p = spec.env_prefix - out = {f"{p}_PORT": str(spec.host_port)} - if spec.auth: - out[f"{p}_PASS"] = spec.password or "" - return out - - def agent_database(self, spec: DatabaseSpec) -> str: - return spec.database or "0" -``` - -- [ ] **Step 2: `engines/valkey.py`** - -Identique à Redis sauf identité et template : - -```python -from __future__ import annotations - -import secrets -from typing import Any - -from core.fields import Field -from core.specs import DatabaseSpec -from core.utils import generate_password -from engines.base import DbEngine -from services.ports import PortAllocator - - -class ValkeyEngine(DbEngine): - key, display, default_port = "valkey", "Valkey", 6379 - template = "engines/valkey.yml.j2" - auth_variants = True - - def fields_existing(self) -> list[Field]: - return [ - Field("host", "Host", "text", default="localhost"), - Field("port", "Port", "int", default=self.default_port), - Field("database", "Database index", "text", default="0"), - Field("username", "Username (empty if none)", "text", default=""), - Field("password", "Password (empty if none)", "text", default=""), - ] - - def generate(self, *, auth: bool, ports: PortAllocator, answers: dict[str, Any]) -> DatabaseSpec: - return DatabaseSpec( - id=self.new_id(), - engine=self.key, - name=f"valkey_{secrets.token_hex(4)}", - managed=True, - host=self.service_name("valkey", auth), - port=self.default_port, - host_port=ports.free(), - database="0", - username="", - password=generate_password(16) if auth else None, - ) - - def env_vars(self, spec: DatabaseSpec) -> dict[str, str]: - p = spec.env_prefix - out = {f"{p}_PORT": str(spec.host_port)} - if spec.auth: - out[f"{p}_PASS"] = spec.password or "" - return out - - def agent_database(self, spec: DatabaseSpec) -> str: - return spec.database or "0" -``` - -- [ ] **Step 3: `engines/mongo.py`** - -```python -from __future__ import annotations - -import secrets -from typing import Any - -from core.specs import DatabaseSpec -from core.utils import generate_password -from engines.base import DbEngine -from services.ports import PortAllocator - - -class MongoEngine(DbEngine): - key, display, default_port = "mongodb", "MongoDB", 27017 - template = "engines/mongodb.yml.j2" - auth_variants = True - - def generate(self, *, auth: bool, ports: PortAllocator, answers: dict[str, Any]) -> DatabaseSpec: - db_name = f"mongo_{secrets.token_hex(4)}" - return DatabaseSpec( - id=self.new_id(), - engine=self.key, - name=db_name, - managed=True, - host=self.service_name("mongo", auth), - port=self.default_port, - host_port=ports.free(), - database=db_name, - username="admin" if auth else "", - password=generate_password(16) if auth else None, - ) - - def env_vars(self, spec: DatabaseSpec) -> dict[str, str]: - p = spec.env_prefix - out = {f"{p}_PORT": str(spec.host_port), f"{p}_DB": spec.database or ""} - if spec.auth: - out[f"{p}_USER"] = spec.username or "" - out[f"{p}_PASS"] = spec.password or "" - return out -``` - -- [ ] **Step 4: `engines/sqlite.py`** - -```python -"""SQLite: a file mounted into the agent. No Compose service.""" - -from __future__ import annotations - -from typing import Any - -from core.fields import Field -from core.specs import DatabaseSpec -from engines.base import DbEngine -from services.ports import PortAllocator - -CONFIG_DIR = "/config" - - -class SqliteEngine(DbEngine): - key, display = "sqlite", "SQLite" - template = None - auth_variants = False - - def fields_existing(self) -> list[Field]: - return [Field("path", "Database Path (relative or absolute)", "text")] - - def fields_new(self) -> list[Field]: - return [Field("name", "Database Name", "text", default="local")] - - def generate(self, *, auth: bool, ports: PortAllocator, answers: dict[str, Any]) -> DatabaseSpec: - name = str(answers.get("name") or "local") - if not name.endswith(".sqlite"): - name += ".sqlite" - return DatabaseSpec( - id=self.new_id(), - engine=self.key, - name=name, - managed=False, - path=name, # relative: ./name mounted to /config/name - database=f"{CONFIG_DIR}/{name}", - ) - - def from_existing(self, answers: dict[str, Any]) -> DatabaseSpec: - raw = str(answers["path"]) - absolute = raw.startswith("/") - return DatabaseSpec( - id=self.new_id(), - engine=self.key, - name=answers.get("label") or "External DB", - managed=False, - path=raw, - database=raw if absolute else f"{CONFIG_DIR}/{raw}", - ) - - @staticmethod - def mount_for(spec: DatabaseSpec) -> tuple[str, str] | None: - """(host_path, container_path) if the file must be bind-mounted into the agent.""" - if spec.database and spec.database.startswith(f"{CONFIG_DIR}/"): - rel = spec.database[len(CONFIG_DIR) + 1 :] - return (f"./{rel}", spec.database) - return None - - def env_vars(self, spec: DatabaseSpec) -> dict[str, str]: - return {} - - def agent_entry(self, spec: DatabaseSpec) -> dict[str, Any]: - return {"name": spec.name, "database": spec.database, "type": self.key, "generated_id": spec.id} - - def describe(self, spec: DatabaseSpec) -> str: - return "Local File" -``` - -- [ ] **Step 5: `engines/docker_volume.py`** - -```python -"""Docker volume backup target. Requires the Docker socket on the agent. No Compose service.""" - -from __future__ import annotations - -from typing import Any - -from core.fields import Field -from core.specs import DatabaseSpec -from engines.base import DbEngine -from services.ports import PortAllocator - - -class DockerVolumeEngine(DbEngine): - key, display = "docker-volume", "Docker Volume" - template = None - has_modes = False - warning = "Requires the Docker socket. It will be mounted on the agent (/var/run/docker.sock)." - - def fields_existing(self) -> list[Field]: - return [ - Field("volume", "Volume Name (e.g. databases_sqlite-data)", "text"), - Field("container", "Container Name (optional, enables auto-restart after restore)", "text", default=""), - ] - - def fields_new(self) -> list[Field]: - return self.fields_existing() - - def generate(self, *, auth: bool, ports: PortAllocator, answers: dict[str, Any]) -> DatabaseSpec: - return self.from_existing(answers) - - def from_existing(self, answers: dict[str, Any]) -> DatabaseSpec: - return DatabaseSpec( - id=self.new_id(), - engine=self.key, - name=answers.get("label") or "Docker Volume", - managed=False, - volume=str(answers["volume"]).strip(), - container=(str(answers.get("container") or "").strip() or None), - ) - - def env_vars(self, spec: DatabaseSpec) -> dict[str, str]: - return {} - - def agent_entry(self, spec: DatabaseSpec) -> dict[str, Any]: - entry = {"name": spec.name, "type": self.key, "volume_name": spec.volume, "generated_id": spec.id} - if spec.container: - entry["container_name"] = spec.container - return entry - - def describe(self, spec: DatabaseSpec) -> str: - return f"volume: {spec.volume}" -``` - -- [ ] **Step 6: `engines/__init__.py`** - -```python -"""Engine registry. Explicit imports keep PyInstaller happy (no dynamic discovery).""" - -from __future__ import annotations - -from engines.docker_volume import DockerVolumeEngine -from engines.mongo import MongoEngine -from engines.redis import RedisEngine -from engines.registry import EngineRegistry -from engines.sql import ( - FirebirdEngine, - MariaDbEngine, - MssqlEngine, - MySqlEngine, - PostgresClusterEngine, - PostgresEngine, -) -from engines.sqlite import SqliteEngine -from engines.valkey import ValkeyEngine - -ALL = ( - PostgresEngine(), - PostgresClusterEngine(), - MySqlEngine(), - MariaDbEngine(), - SqliteEngine(), - FirebirdEngine(), - MongoEngine(), - RedisEngine(), - ValkeyEngine(), - MssqlEngine(), - DockerVolumeEngine(), -) - -registry = EngineRegistry(ALL) - -__all__ = ["ALL", "EngineRegistry", "registry"] -``` - -L'ordre = ordre d'affichage dans le select (identique au legacy). - -- [ ] **Step 7: Vérifier** - -Run: `uv run python -c " -from engines import registry -from services.ports import FixedPortAllocator -p = FixedPortAllocator() -print(registry.keys()) -for e in registry: - if e.template is None: continue - for auth in ((True, False) if e.auth_variants else (True,)): - s = e.generate(auth=auth, ports=p, answers={}) - ctx = e.template_ctx(s); assert ctx['name'] == s.host and set(e.env_vars(s)) >= {s.env_prefix + '_PORT'} - print(f'{e.key:20} auth={auth!s:5} {s.host:24} env={len(e.env_vars(s))} entry.db={e.agent_entry(s)[\"database\"]!r}') -sq = registry.get('sqlite'); s = sq.generate(auth=False, ports=p, answers={'name':'x'}); print(sq.agent_entry(s), sq.mount_for(s)) -dv = registry.get('docker-volume'); print(dv.agent_entry(dv.from_existing({'volume':'v','container':''})))"` -Expected: 11 clés dans l'ordre legacy ; une ligne par moteur/variante avec `entry.db` = `'0'` pour redis/valkey, `'master'` mssql, `/var/lib/firebird/data/mirror.fdb` firebird ; sqlite `{'name': 'x.sqlite', 'database': '/config/x.sqlite', 'type': 'sqlite', 'generated_id': ...} ('./x.sqlite', '/config/x.sqlite')` ; docker-volume sans `container_name`. - -- [ ] **Step 8: Commit** - -```bash -git add engines/ -git commit -m "feat(engines): add redis, valkey, mongodb, sqlite, docker-volume engines and registry" -``` - ---- - -### Task 5 : Templates Jinja2 - -**Files:** -- Create: `templates/agent.yml.j2`, `templates/dashboard.yml.j2` -- Create: `templates/engines/postgresql.yml.j2`, `mariadb.yml.j2`, `mssql.yml.j2`, `firebird.yml.j2`, `mongodb.yml.j2`, `redis.yml.j2`, `valkey.yml.j2` -- Create: `templates/engines.map.json` -- Move: `.github/assets/templates/agent.yml` → `templates/agent.yml`, `dashboard.yml` → `templates/dashboard.yml` -- Modify: `pyproject.toml` (`jinja2`), `.gitleaks.toml` - -**Interfaces:** -- Produces: contrat de contexte. - - `agent.yml.j2` : `host_gateway: bool`, `docker_socket: bool`, `mounts: list[{host, container}]`, `services: list[{name, volume, body}]`, `tz_var, edge_key_var, log_level_var, polling_var: str`. - - `dashboard.yml.j2` : `db_mode: "external"|"internal"|"custom"`, `project_name_var, host_port_var, tz_var, log_level_var, project_secret_var, project_url_var, pg_port_var, postgres_db_var, postgres_user_var, postgres_password_var: str`. - - `engines/*.yml.j2` : `name, volume, auth, port_var, db_var, user_var, password_var` (+ `root_password_var` firebird). - -- [ ] **Step 1: Ajouter Jinja2** - -Run: `uv add "jinja2>=3.1"` -Expected: `pyproject.toml` et `uv.lock` mis à jour. - -- [ ] **Step 2: Déplacer les templates legacy** - -Run: `mkdir -p templates/engines && git mv .github/assets/templates/agent.yml templates/agent.yml && git mv .github/assets/templates/dashboard.yml templates/dashboard.yml && rmdir .github/assets/templates 2>/dev/null; ls templates` - -- [ ] **Step 3: `templates/agent.yml.j2`** - -```jinja -services: - agent: - restart: unless-stopped - image: portabase/agent:latest - volumes: - - ./databases.json:/config/config.json -{%- for m in mounts %} - - {{ m.host }}:{{ m.container }} -{%- endfor %} -{%- if docker_socket %} - - /var/run/docker.sock:/var/run/docker.sock -{%- endif %} -{%- if host_gateway %} - extra_hosts: - - "localhost:host-gateway" -{%- endif %} - environment: - TZ: "{{ tz_var }}" - EDGE_KEY: "{{ edge_key_var }}" - LOG_LEVEL: "{{ log_level_var }}" - POLLING: "{{ polling_var }}" - networks: - - portabase -{% for s in services %} -{{ s.body }} -{%- endfor %} -{% if services %} -volumes: -{%- for s in services %} - {{ s.volume }}: -{%- endfor %} -{% endif %} -networks: - portabase: - name: portabase_network - external: true -``` - -- [ ] **Step 4: `templates/dashboard.yml.j2`** - -```jinja -name: {{ project_name_var }} -services: - portabase: - container_name: {{ project_name_var }}-app - image: portabase/portabase:latest - restart: unless-stopped - env_file: - - .env - ports: - - "{{ host_port_var }}:80" - environment: - - TZ={{ tz_var }} - - LOG_LEVEL={{ log_level_var }} - - PROJECT_SECRET={{ project_secret_var }} - - PROJECT_URL={{ project_url_var }} - volumes: - - portabase-data:/data -{%- if db_mode == "external" %} - depends_on: - db: - condition: service_healthy -{%- endif %} - healthcheck: - test: ["CMD-SHELL", "curl -f http://localhost/api/health"] - interval: 30s - timeout: 5s - retries: 3 - start_period: 60s -{%- if db_mode == "external" %} - db: - container_name: {{ project_name_var }}-pg - image: postgres:17-alpine - restart: unless-stopped - ports: - - "{{ pg_port_var }}:5432" - volumes: - - postgres-data:/var/lib/postgresql/data - environment: - - POSTGRES_DB={{ postgres_db_var }} - - POSTGRES_USER={{ postgres_user_var }} - - POSTGRES_PASSWORD={{ postgres_password_var }} - healthcheck: - test: ["CMD-SHELL", "pg_isready -U {{ postgres_user_var }} -d {{ postgres_db_var }}"] - interval: 10s - timeout: 5s - retries: 5 -{%- endif %} -volumes: -{%- if db_mode == "external" %} - postgres-data: -{%- endif %} - portabase-data: -``` - -- [ ] **Step 5: Templates moteurs** - -`templates/engines/postgresql.yml.j2` : - -```jinja - {{ name }}: - image: postgres:17-alpine - restart: unless-stopped - networks: - - portabase - ports: - - "{{ port_var }}:5432" - volumes: - - {{ volume }}:/var/lib/postgresql/data - environment: - - POSTGRES_DB={{ db_var }} - - POSTGRES_USER={{ user_var }} - - POSTGRES_PASSWORD={{ password_var }} - healthcheck: - test: ["CMD-SHELL", "pg_isready -U {{ user_var }} -d {{ db_var }}"] - interval: 10s - timeout: 5s - retries: 5 -``` - -`templates/engines/mariadb.yml.j2` : - -```jinja - {{ name }}: - image: mariadb:latest - restart: unless-stopped - networks: - - portabase - ports: - - "{{ port_var }}:3306" - environment: - - MYSQL_DATABASE={{ db_var }} - - MYSQL_USER={{ user_var }} - - MYSQL_PASSWORD={{ password_var }} - - MYSQL_RANDOM_ROOT_PASSWORD=yes - volumes: - - {{ volume }}:/var/lib/mysql - healthcheck: - test: ["CMD-SHELL", "mariadb-admin ping -h localhost -u {{ user_var }} -p{{ password_var }}"] - interval: 10s - timeout: 5s - retries: 5 -``` - -`templates/engines/mssql.yml.j2` : - -```jinja - {{ name }}: - image: mcr.microsoft.com/azure-sql-edge:latest - restart: unless-stopped - networks: - - portabase - ports: - - "{{ port_var }}:1433" - environment: - - ACCEPT_EULA=Y - - MSSQL_SA_PASSWORD={{ password_var }} - volumes: - - {{ volume }}:/var/opt/mssql - healthcheck: - test: ["CMD-SHELL", "cat /proc/net/tcp6 | grep -q '059901' || exit 1"] - interval: 10s - timeout: 5s - retries: 20 -``` - -`templates/engines/firebird.yml.j2` : - -```jinja - {{ name }}: - image: firebirdsql/firebird - restart: unless-stopped - networks: - - portabase - ports: - - "{{ port_var }}:3050" - volumes: - - {{ volume }}:/var/lib/firebird/data - environment: - - FIREBIRD_DATABASE={{ db_var }} - - FIREBIRD_USER={{ user_var }} - - FIREBIRD_PASSWORD={{ password_var }} - - FIREBIRD_ROOT_PASSWORD={{ root_password_var }} - - FIREBIRD_DATABASE_DEFAULT_CHARSET=UTF8 - healthcheck: - test: ["CMD-SHELL", "nc -z localhost 3050"] - interval: 10s - timeout: 5s - retries: 5 -``` - -`templates/engines/mongodb.yml.j2` : - -```jinja - {{ name }}: - image: mongo:latest - restart: unless-stopped - networks: - - portabase - ports: - - "{{ port_var }}:27017" - environment: -{%- if auth %} - - MONGO_INITDB_ROOT_USERNAME={{ user_var }} - - MONGO_INITDB_ROOT_PASSWORD={{ password_var }} -{%- endif %} - - MONGO_INITDB_DATABASE={{ db_var }} -{%- if auth %} - command: mongod --auth -{%- endif %} - volumes: - - {{ volume }}:/data/db - healthcheck: - test: ["CMD-SHELL", "mongosh --eval 'db.runCommand({ping:1})' --quiet"] - interval: 10s - timeout: 5s - retries: 5 -``` - -`templates/engines/redis.yml.j2` : - -```jinja - {{ name }}: - image: redis:latest - restart: unless-stopped - ports: - - "{{ port_var }}:6379" - volumes: - - {{ volume }}:/data -{%- if auth %} - environment: - - REDIS_PASSWORD={{ password_var }} - command: ["redis-server", "--requirepass", "{{ password_var }}", "--appendonly", "yes"] -{%- else %} - command: ["redis-server", "--appendonly", "yes"] -{%- endif %} - networks: - - portabase - - default - healthcheck: - test: ["CMD-SHELL", "redis-cli {% if auth %}-a {{ password_var }} {% endif %}ping | grep PONG"] - interval: 10s - timeout: 5s - retries: 5 -``` - -`templates/engines/valkey.yml.j2` : - -```jinja - {{ name }}: - image: valkey/valkey:latest - restart: unless-stopped -{%- if auth %} - command: --requirepass "{{ password_var }}" -{%- else %} - environment: - - ALLOW_EMPTY_PASSWORD=yes -{%- endif %} - ports: - - "{{ port_var }}:6379" - volumes: - - {{ volume }}:/data - networks: - - portabase - - default - healthcheck: - test: ["CMD-SHELL", "valkey-cli {% if auth %}-a {{ password_var }} {% endif %}ping | grep PONG"] - interval: 10s - timeout: 5s - retries: 5 -``` - -Différence assumée vs snippets legacy : `restart: unless-stopped` ajouté sur redis et valkey (spec §5.7). - -- [ ] **Step 6: `templates/engines.map.json`** - -```json -{ - "postgresql": "engines/postgresql.yml.j2", - "postgresql-cluster": "engines/postgresql.yml.j2", - "mysql": "engines/mariadb.yml.j2", - "mariadb": "engines/mariadb.yml.j2", - "mssql": "engines/mssql.yml.j2", - "firebird": "engines/firebird.yml.j2", - "mongodb": "engines/mongodb.yml.j2", - "redis": "engines/redis.yml.j2", - "valkey": "engines/valkey.yml.j2" -} -``` - -- [ ] **Step 7: `.gitleaks.toml`** - -Retirer la ligne `'''\.github/assets/templates/.*''',` de `paths`. - -- [ ] **Step 8: Rendu manuel de contrôle** - -Run: `uv run python -c " -import jinja2, yaml -env = jinja2.Environment(loader=jinja2.FileSystemLoader('templates'), undefined=jinja2.StrictUndefined, keep_trailing_newline=True, autoescape=False) -body = env.get_template('engines/redis.yml.j2').render(name='db-redis-auth-ab12', volume='db-redis-auth-ab12-data', auth=True, port_var='\${DB_REDIS_AUTH_AB12_PORT}', db_var='', user_var='', password_var='\${DB_REDIS_AUTH_AB12_PASS}') -out = env.get_template('agent.yml.j2').render(host_gateway=True, docker_socket=True, mounts=[{'host':'./x.sqlite','container':'/config/x.sqlite'}], services=[{'name':'db-redis-auth-ab12','volume':'db-redis-auth-ab12-data','body':body}], tz_var='\${TZ}', edge_key_var='\${EDGE_KEY}', log_level_var='\${LOG_LEVEL}', polling_var='\${POLLING}') -print(out); d = yaml.safe_load(out); print(sorted(d['services']), d['volumes'], d['services']['agent']['extra_hosts']) -for mode in ('external','internal','custom'): - o = env.get_template('dashboard.yml.j2').render(db_mode=mode, project_name_var='pb', host_port_var='8887', tz_var='\${TZ}', log_level_var='\${LOG_LEVEL}', project_secret_var='\${PROJECT_SECRET}', project_url_var='\${PROJECT_URL}', pg_port_var='\${PG_PORT}', postgres_db_var='\${POSTGRES_DB}', postgres_user_var='\${POSTGRES_USER}', postgres_password_var='\${POSTGRES_PASSWORD}') - print(mode, sorted(yaml.safe_load(o)['services']), sorted(yaml.safe_load(o)['volumes']))"` -Expected: compose agent imprimé avec socket, extra_hosts, mount sqlite, service redis ; `['agent', 'db-redis-auth-ab12'] {'db-redis-auth-ab12-data': None} ['localhost:host-gateway']` ; dashboard `external ['db', 'portabase'] ['portabase-data', 'postgres-data']`, `internal ['portabase'] ['portabase-data']`, `custom ['portabase'] ['portabase-data']`. - -- [ ] **Step 9: Commit** - -```bash -git add templates/ pyproject.toml uv.lock .gitleaks.toml -git commit -m "feat(templates): add Jinja2 compose templates at repo root, move legacy templates" -``` - ---- - -### Task 6 : `services/templates.py` — `Manifest`, `TemplateRepository` - -**Files:** -- Create: `services/templates.py` - -**Interfaces:** -- Consumes: `HttpClient`, `GlobalConfig.cache_dir`, `core.version`, `TemplateError`. -- Produces: - - `Manifest(schema, version, files: dict[str, FileEntry], engines: dict[str, str], generated_at, commit)` avec `from_json(data)`, `from_directory(dir, version)`. - - `TemplateRepository(http, cache_dir, version, base_url=TEMPLATE_BASE_URL, local_dir: Path | None = None)` : `resolve() -> Path` (dossier prêt, fetch si besoin), `get(name) -> jinja2.Template`, `engine_template(key) -> jinja2.Template`, `manifest -> Manifest`, propriété `source: str` (`local` / `cache` / `remote`). - - `TemplateRepository.from_environment(http, config) -> TemplateRepository` : lit `PORTABASE_TEMPLATES_DIR`, `PORTABASE_TEMPLATES_VERSION`, détection dev (`./templates` à côté de `main.py` si non frozen). - - `TEMPLATE_BASE_URL` importée depuis `core/config.py` (inchangée). - -- [ ] **Step 1: Écrire le module** - -```python -"""Versioned remote templates with manifest verification and a local cache. - -Resolution order: explicit local dir (dev) → cache hit → remote fetch. No 'latest' fallback: -a CLI version only ever renders with the templates published for that exact version. -""" - -from __future__ import annotations - -import hashlib -import json -import os -import sys -from dataclasses import dataclass -from pathlib import Path - -import jinja2 - -from core.config import TEMPLATE_BASE_URL, GlobalConfig -from core.errors import NetworkError, TemplateError -from core.version import UNKNOWN, current_version -from services.http import HttpClient - -MANIFEST_NAME = "manifest.json" -SUPPORTED_SCHEMA = 1 - - -@dataclass(frozen=True) -class FileEntry: - sha256: str - size: int - - -@dataclass(frozen=True) -class Manifest: - schema: int - version: str - files: dict[str, FileEntry] - engines: dict[str, str] - generated_at: str = "" - commit: str = "" - - @classmethod - def from_json(cls, data: dict) -> Manifest: - try: - schema = int(data["schema"]) - if schema != SUPPORTED_SCHEMA: - raise TemplateError( - f"Unsupported template manifest schema {schema} (this CLI supports {SUPPORTED_SCHEMA}).", - hint="Update the CLI: portabase update", - ) - files = { - name: FileEntry(sha256=str(e["sha256"]).lower(), size=int(e["size"])) - for name, e in data["files"].items() - } - return cls( - schema=schema, - version=str(data["version"]), - files=files, - engines=dict(data.get("engines", {})), - generated_at=str(data.get("generated_at", "")), - commit=str(data.get("commit", "")), - ) - except (KeyError, TypeError, ValueError) as e: - raise TemplateError("Template manifest is malformed.", cause=e) from e - - @classmethod - def from_directory(cls, directory: Path, version: str) -> Manifest: - """Manifest computed from a local directory (dev mode / render checks).""" - files = {} - for path in sorted(directory.rglob("*.j2")): - rel = path.relative_to(directory).as_posix() - files[rel] = FileEntry(sha256=_sha256(path), size=path.stat().st_size) - engines_map = directory / "engines.map.json" - engines = json.loads(engines_map.read_text(encoding="utf-8")) if engines_map.exists() else {} - return cls(schema=SUPPORTED_SCHEMA, version=version, files=files, engines=engines) - - -def _sha256(path: Path) -> str: - h = hashlib.sha256() - with open(path, "rb") as f: - for chunk in iter(lambda: f.read(1 << 20), b""): - h.update(chunk) - return h.hexdigest() - - -class TemplateRepository: - def __init__( - self, - http: HttpClient, - cache_dir: Path, - version: str, - *, - base_url: str = TEMPLATE_BASE_URL, - local_dir: Path | None = None, - ) -> None: - self.http = http - self.version = version - self.base_url = base_url.rstrip("/") - self.local_dir = local_dir - self.cache_dir = cache_dir / "templates" / version - self._manifest: Manifest | None = None - self._env: jinja2.Environment | None = None - self.source = "unresolved" - - # ---- construction ----------------------------------------------------- - - @classmethod - def from_environment(cls, http: HttpClient, config: GlobalConfig) -> TemplateRepository: - version = os.environ.get("PORTABASE_TEMPLATES_VERSION") or current_version() - local = os.environ.get("PORTABASE_TEMPLATES_DIR") - local_dir = Path(local) if local else None - if local_dir is None and not getattr(sys, "frozen", False): - candidate = Path(__file__).resolve().parent.parent / "templates" - if (candidate / "agent.yml.j2").exists(): - local_dir = candidate - return cls(http, config.cache_dir, version, local_dir=local_dir) - - # ---- resolution ------------------------------------------------------- - - @property - def manifest(self) -> Manifest: - if self._manifest is None: - self.resolve() - assert self._manifest is not None - return self._manifest - - def resolve(self) -> Path: - """Ensure a verified template directory exists locally and return it.""" - if self.local_dir is not None: - if not (self.local_dir / "agent.yml.j2").exists(): - raise TemplateError(f"Template directory {self.local_dir} has no agent.yml.j2.") - self._manifest = Manifest.from_directory(self.local_dir, self.version) - self.source = "local" - return self.local_dir - - if self.version == UNKNOWN: - raise TemplateError( - "Cannot resolve template version (CLI version unknown).", - hint="Set PORTABASE_TEMPLATES_DIR to a local templates folder or PORTABASE_TEMPLATES_VERSION.", - ) - - remote_manifest = self._fetch_manifest() - if remote_manifest is None: - cached = self._cached_manifest() - if cached is None: - raise TemplateError( - f"Templates for version {self.version} are unavailable and not cached.", - hint="Check your internet connection, or set PORTABASE_TEMPLATES_DIR.", - ) - self._manifest = cached - self.source = "cache" - self._verify_cache_complete(cached) - return self.cache_dir - - if remote_manifest.version != self.version: - raise TemplateError( - f"Template manifest is for version {remote_manifest.version}, expected {self.version}." - ) - self._sync(remote_manifest) - self._manifest = remote_manifest - self.source = "remote" - return self.cache_dir - - # ---- access ----------------------------------------------------------- - - def get(self, name: str) -> jinja2.Template: - directory = self.resolve() - if name not in self.manifest.files: - raise TemplateError(f"Template '{name}' is not part of version {self.version}.") - if self._env is None: - self._env = jinja2.Environment( - loader=jinja2.FileSystemLoader(str(directory)), - undefined=jinja2.StrictUndefined, - keep_trailing_newline=True, - autoescape=False, - ) - try: - return self._env.get_template(name) - except jinja2.TemplateError as e: - raise TemplateError(f"Template '{name}' failed to load: {e}", cause=e) from e - - def engine_template(self, engine_key: str) -> jinja2.Template: - name = self.manifest.engines.get(engine_key) - if name is None: - raise TemplateError(f"No template mapped for engine '{engine_key}' in version {self.version}.") - return self.get(name) - - # ---- internals -------------------------------------------------------- - - def _url(self, name: str) -> str: - return f"{self.base_url}/{self.version}/{name}" - - def _fetch_manifest(self) -> Manifest | None: - try: - return Manifest.from_json(self.http.get_json(self._url(MANIFEST_NAME))) - except NetworkError: - return None - - def _cached_manifest(self) -> Manifest | None: - path = self.cache_dir / MANIFEST_NAME - if not path.exists(): - return None - try: - return Manifest.from_json(json.loads(path.read_text(encoding="utf-8"))) - except (OSError, ValueError, TemplateError): - return None - - def _verify_cache_complete(self, manifest: Manifest) -> None: - for name, entry in manifest.files.items(): - path = self.cache_dir / name - if not path.exists() or _sha256(path) != entry.sha256: - raise TemplateError( - f"Cached template '{name}' is missing or corrupt and the network is unavailable.", - hint="Reconnect and retry; the cache will be refreshed.", - ) - - def _sync(self, manifest: Manifest) -> None: - self.cache_dir.mkdir(parents=True, exist_ok=True) - for name, entry in manifest.files.items(): - path = self.cache_dir / name - if path.exists() and path.stat().st_size == entry.size and _sha256(path) == entry.sha256: - continue - path.parent.mkdir(parents=True, exist_ok=True) - tmp = path.with_suffix(path.suffix + ".tmp") - try: - self.http.download(self._url(name), tmp) - except NetworkError as e: - raise TemplateError(f"Could not download template '{name}'.", cause=e) from e - if tmp.stat().st_size != entry.size or _sha256(tmp) != entry.sha256: - tmp.unlink(missing_ok=True) - raise TemplateError(f"Template '{name}' failed integrity check (sha256 mismatch).") - os.replace(tmp, path) - for stale in self.cache_dir.rglob("*.j2"): - if stale.relative_to(self.cache_dir).as_posix() not in manifest.files: - stale.unlink(missing_ok=True) - (self.cache_dir / MANIFEST_NAME).write_text( - json.dumps( - { - "schema": manifest.schema, - "version": manifest.version, - "generated_at": manifest.generated_at, - "commit": manifest.commit, - "files": {n: {"sha256": e.sha256, "size": e.size} for n, e in manifest.files.items()}, - "engines": manifest.engines, - }, - indent=2, - ), - encoding="utf-8", - ) -``` - -`TEMPLATE_BASE_URL` reste définie dans `core/config.py` (le legacy `core/network.py` l'importe de là) ; `services/templates.py` l'importe depuis `core.config`. Pas de circularité : `core` n'importe jamais `services`. - -- [ ] **Step 2: Vérifier en mode local (dev)** - -Run: `uv run python -c " -from pathlib import Path -from services.http import HttpClient -from services.templates import TemplateRepository -from core.config import GlobalConfig -r = TemplateRepository.from_environment(HttpClient(), GlobalConfig()) -print(r.source, r.resolve(), r.source, len(r.manifest.files), r.manifest.engines['mysql']) -print(r.engine_template('redis').render(name='n', volume='v', auth=False, port_var='1', db_var='', user_var='', password_var='')[:40].strip())"` -Expected: `unresolved /templates local 9 engines/mariadb.yml.j2` puis `n:` (début du service rendu). - -- [ ] **Step 3: Vérifier le mode remote contre un serveur local** - -```bash -# Terminal 1 — publie templates/ comme S3 sous la version 99.0.0 avec un manifest -mkdir -p /tmp/pb-s3/99.0.0 && cp -r templates/. /tmp/pb-s3/99.0.0/ && cd /tmp/pb-s3/99.0.0 && \ -uv --directory /home/soluce/Documents/PROJETS/Portabase/cli run python -c " -import json, hashlib, pathlib -files = {p.as_posix(): {'sha256': hashlib.sha256(p.read_bytes()).hexdigest(), 'size': p.stat().st_size} for p in sorted(pathlib.Path('.').rglob('*.j2'))} -json.dump({'schema':1,'version':'99.0.0','generated_at':'now','commit':'x','files':files,'engines':json.load(open('engines.map.json'))}, open('manifest.json','w'), indent=2)" && \ -cd /tmp/pb-s3 && python3 -m http.server 8765 -``` - -Terminal 2 : -```bash -uv run python -c " -import tempfile; from pathlib import Path -from services.http import HttpClient -from services.templates import TemplateRepository -cache = Path(tempfile.mkdtemp()) -r = TemplateRepository(HttpClient(), cache, '99.0.0', base_url='http://127.0.0.1:8765') -print(r.resolve(), r.source, sorted(p.name for p in (cache/'templates'/'99.0.0').iterdir())) -r2 = TemplateRepository(HttpClient(), cache, '99.0.0', base_url='http://127.0.0.1:8765'); r2.resolve(); print('second:', r2.source) -r3 = TemplateRepository(HttpClient(), cache, '99.0.0', base_url='http://127.0.0.1:1'); r3.resolve(); print('offline:', r3.source) -try: TemplateRepository(HttpClient(), Path(tempfile.mkdtemp()), '99.0.0', base_url='http://127.0.0.1:1').resolve() -except Exception as e: print('offline no cache:', type(e).__name__, e.code) -try: TemplateRepository(HttpClient(), cache, '98.0.0', base_url='http://127.0.0.1:8765').resolve() -except Exception as e: print('missing version:', type(e).__name__, e.code)" -``` -Expected: `... remote ['agent.yml.j2', 'dashboard.yml.j2', 'engines', 'manifest.json']`, `second: remote` (manifest re-fetché, fichiers en cache non re-téléchargés), `offline: cache`, `offline no cache: TemplateError E_TEMPLATE`, `missing version: TemplateError E_TEMPLATE`. Arrêter le serveur. - -- [ ] **Step 4: Commit** - -```bash -git add services/templates.py -git commit -m "feat(services): add TemplateRepository with manifest verification and cache" -``` - ---- - -### Task 7 : `scripts/render_check.py` - -**Files:** -- Create: `scripts/render_check.py` - -**Interfaces:** -- Consumes: `TemplateRepository` (mode local), `engines.registry`, `FixedPortAllocator`. -- Produces: script exécutable, exit 0 si tous les rendus sont du YAML valide (et `docker compose config` valide si Docker disponible), exit 1 sinon. Réutilisé par la CI (Task 8) et remplacé par un appel à `ComposeRenderer` au Plan 4. - -- [ ] **Step 1: Écrire le script** - -```python -#!/usr/bin/env python3 -"""Render every template with fixture contexts and validate the output. - -Usage: uv run python scripts/render_check.py [--templates DIR] [--no-compose] -Exit 0 on success. Prints one line per rendered case. -""" - -from __future__ import annotations - -import argparse -import os -import shutil -import subprocess -import sys -import tempfile -from pathlib import Path - -import yaml - -sys.path.insert(0, str(Path(__file__).resolve().parent.parent)) - -from core.config import GlobalConfig # noqa: E402 -from engines import registry # noqa: E402 -from services.http import HttpClient # noqa: E402 -from services.ports import FixedPortAllocator # noqa: E402 -from services.templates import TemplateRepository # noqa: E402 - -AGENT_GLOBALS = { - "tz_var": "${TZ}", - "edge_key_var": "${EDGE_KEY}", - "log_level_var": "${LOG_LEVEL}", - "polling_var": "${POLLING}", -} -AGENT_ENV = 'TZ="UTC"\nEDGE_KEY="x"\nLOG_LEVEL="info"\nPOLLING="5"\n' -DASHBOARD_VARS = { - "project_name_var": "pb", - "host_port_var": "${HOST_PORT}", - "tz_var": "${TZ}", - "log_level_var": "${LOG_LEVEL}", - "project_secret_var": "${PROJECT_SECRET}", - "project_url_var": "${PROJECT_URL}", - "pg_port_var": "${PG_PORT}", - "postgres_db_var": "${POSTGRES_DB}", - "postgres_user_var": "${POSTGRES_USER}", - "postgres_password_var": "${POSTGRES_PASSWORD}", -} -DASHBOARD_ENV = ( - 'HOST_PORT="8887"\nTZ="UTC"\nLOG_LEVEL="info"\nPROJECT_SECRET="s"\nPROJECT_URL="http://localhost"\n' - 'PG_PORT="5433"\nPOSTGRES_DB="pb"\nPOSTGRES_USER="pb"\nPOSTGRES_PASSWORD="p"\n' -) - - -class Failure(Exception): - pass - - -def validate(label: str, compose: str, env_text: str, use_compose: bool) -> None: - try: - doc = yaml.safe_load(compose) - except yaml.YAMLError as e: - raise Failure(f"{label}: invalid YAML: {e}\n{compose}") from e - if not isinstance(doc, dict) or "services" not in doc: - raise Failure(f"{label}: no services key\n{compose}") - if use_compose: - with tempfile.TemporaryDirectory() as tmp: - Path(tmp, "docker-compose.yml").write_text(compose, encoding="utf-8") - Path(tmp, ".env").write_text(env_text, encoding="utf-8") - Path(tmp, "databases.json").write_text('{"databases": []}', encoding="utf-8") - proc = subprocess.run( - ["docker", "compose", "-p", "rendercheck", "config", "--quiet"], - cwd=tmp, - capture_output=True, - text=True, - check=False, - ) - if proc.returncode != 0: - raise Failure(f"{label}: docker compose config failed:\n{proc.stderr}\n{compose}") - print(f"ok {label}") - - -def agent_cases(repo: TemplateRepository) -> list[tuple[str, str, str]]: - ports = FixedPortAllocator() - cases = [] - # 1. empty agent, all toggles off - cases.append(("agent/empty", render_agent(repo, [], False, False, []), AGENT_ENV)) - # 2. toggles on, sqlite mount - cases.append( - ( - "agent/toggles", - render_agent(repo, [], True, True, [{"host": "./x.sqlite", "container": "/config/x.sqlite"}]), - AGENT_ENV, - ) - ) - # 3. one case per engine/variant - all_services, all_env = [], AGENT_ENV - for engine in registry: - if engine.template is None: - continue - for auth in (True, False) if engine.auth_variants else (True,): - spec = engine.generate(auth=auth, ports=ports, answers={}) - env = engine.env_vars(spec) - env_text = AGENT_ENV + "".join(f'{k}="{v}"\n' for k, v in env.items()) - body = repo.engine_template(engine.key).render(**engine.template_ctx(spec)) - service = {"name": spec.host, "volume": f"{spec.host}-data", "body": body} - cases.append((f"agent/{engine.key}{'/auth' if auth else '/noauth' if engine.auth_variants else ''}", - render_agent(repo, [service], False, False, []), env_text)) - all_services.append(service) - all_env += "".join(f'{k}="{v}"\n' for k, v in env.items()) - # 4. everything at once - cases.append(("agent/all", render_agent(repo, all_services, True, True, []), all_env)) - return cases - - -def render_agent(repo, services, host_gateway, docker_socket, mounts) -> str: - return repo.get("agent.yml.j2").render( - services=services, host_gateway=host_gateway, docker_socket=docker_socket, mounts=mounts, **AGENT_GLOBALS - ) - - -def dashboard_cases(repo: TemplateRepository) -> list[tuple[str, str, str]]: - return [ - (f"dashboard/{mode}", repo.get("dashboard.yml.j2").render(db_mode=mode, **DASHBOARD_VARS), DASHBOARD_ENV) - for mode in ("external", "internal", "custom") - ] - - -def engines_check(repo: TemplateRepository) -> None: - mapped = repo.manifest.engines - for engine in registry: - if engine.template is None: - continue - if mapped.get(engine.key) != engine.template: - raise Failure(f"engines.map.json: {engine.key} -> {mapped.get(engine.key)} but code says {engine.template}") - if engine.template not in repo.manifest.files: - raise Failure(f"{engine.key}: template {engine.template} not found") - for key in mapped: - if key not in registry: - raise Failure(f"engines.map.json maps unknown engine '{key}'") - print(f"ok engines-check ({len(mapped)} mapped)") - - -def main() -> int: - parser = argparse.ArgumentParser() - parser.add_argument("--templates", default=os.environ.get("PORTABASE_TEMPLATES_DIR", "templates")) - parser.add_argument("--no-compose", action="store_true", help="Skip docker compose config validation") - args = parser.parse_args() - - use_compose = not args.no_compose and shutil.which("docker") is not None - if not use_compose: - print("note: docker not available, YAML validation only") - repo = TemplateRepository(HttpClient(), GlobalConfig().cache_dir, "local", local_dir=Path(args.templates)) - try: - engines_check(repo) - for label, compose, env_text in agent_cases(repo) + dashboard_cases(repo): - validate(label, compose, env_text, use_compose) - except Failure as e: - print(f"FAIL {e}", file=sys.stderr) - return 1 - return 0 - - -if __name__ == "__main__": - raise SystemExit(main()) -``` - -- [ ] **Step 2: Exécuter** - -Run: `uv run python scripts/render_check.py` -Expected: `ok engines-check (9 mapped)` puis une ligne `ok` par cas : `agent/empty`, `agent/toggles`, `agent/postgresql`, `agent/postgresql-cluster`, `agent/mysql`, `agent/mariadb`, `agent/firebird`, `agent/mongodb/auth`, `agent/mongodb/noauth`, `agent/redis/auth`, `agent/redis/noauth`, `agent/valkey/auth`, `agent/valkey/noauth`, `agent/mssql`, `agent/all`, `dashboard/external`, `dashboard/internal`, `dashboard/custom`. Exit 0. - -Si `docker compose config` échoue sur un cas : lire l'erreur, corriger le template (pas le script). - -- [ ] **Step 3: Ruff sur le script** - -Run: `uv run ruff check scripts/ && uv run ruff format scripts/` - -- [ ] **Step 4: Commit** - -```bash -git add scripts/render_check.py -git commit -m "ci: add render_check script validating every template with fixtures" -``` - ---- - -### Task 8 : CI — `render-check`, `engines-check`, manifest à l'upload, hotfix - -**Files:** -- Modify: `.github/workflows/ci.yml` -- Modify: `.github/workflows/templates-upload.yml` -- Create: `.github/workflows/templates-hotfix.yml` - -- [ ] **Step 1: Ajouter le job `render-check` à `ci.yml`** (après `test`) - -```yaml - render-check: - name: render-check - runs-on: ubuntu-latest - permissions: - contents: read - steps: - - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4 - - uses: astral-sh/setup-uv@caf0cab7a618c569241d31dcd442f54681755d39 # v3 - - name: Install - run: uv sync --frozen --all-groups - - name: Render and validate templates (YAML + docker compose config) - run: uv run python scripts/render_check.py --templates templates -``` - -Le job `engines-check` de la spec est couvert par la fonction `engines_check()` du même script (une seule exécution, deux vérifications). Pas de job séparé. - -- [ ] **Step 2: `templates-upload.yml` — source et manifest** - -Remplacer les deux étapes d'upload par : - -```yaml - - name: Generate manifest - env: - VERSION: ${{ inputs.version }} - run: | - set -euo pipefail - CLEAN_VERSION="${VERSION#v}" - cd templates - FILES=$(find . -name '*.j2' -type f | sort | while read -r f; do - rel="${f#./}" - printf '{"%s":{"sha256":"%s","size":%s}}\n' "$rel" "$(sha256sum "$f" | cut -d' ' -f1)" "$(stat -c%s "$f")" - done | jq -s 'add') - jq -n \ - --arg version "$CLEAN_VERSION" \ - --arg commit "$GITHUB_SHA" \ - --arg date "$(date -u +%FT%TZ)" \ - --argjson files "$FILES" \ - --argjson engines "$(cat engines.map.json)" \ - '{schema:1, version:$version, generated_at:$date, commit:$commit, files:$files, engines:$engines}' \ - > manifest.json - cat manifest.json - - - name: Upload versioned templates - env: - VERSION: ${{ inputs.version }} - run: | - CLEAN_VERSION="${VERSION#v}" - s3cmd $S3CMD_ARGS sync templates/ \ - "s3://${{ secrets.S3_BUCKET }}/cli/public/templates/${CLEAN_VERSION}/" --acl-public --delete-removed - - - name: Upload latest templates (stable only, legacy fallback) - if: ${{ !inputs.is_prerelease }} - run: | - s3cmd $S3CMD_ARGS sync templates/ \ - "s3://${{ secrets.S3_BUCKET }}/cli/public/templates/latest/" --acl-public -``` - -`latest/` reste alimenté pour les vieux binaires (fallback legacy). Le nouveau code ne le lit jamais. À retirer quand plus aucune version legacy n'est supportée. - -- [ ] **Step 3: `templates-hotfix.yml`** - -```yaml -name: Templates hotfix - -on: - workflow_dispatch: - inputs: - version: - description: "Existing CLI version to re-publish templates for (e.g. 26.09.0). Templates must stay compatible with that version's code." - required: true - type: string - -permissions: {} - -jobs: - hotfix: - uses: ./.github/workflows/templates-upload.yml - with: - version: ${{ inputs.version }} - is_prerelease: true # never touch latest/ from a hotfix - secrets: - S3_ENDPOINT: ${{ secrets.S3_ENDPOINT }} - S3_ACCESS_KEY: ${{ secrets.S3_ACCESS_KEY }} - S3_SECRET_KEY: ${{ secrets.S3_SECRET_KEY }} - S3_BUCKET: ${{ secrets.S3_BUCKET }} -``` - -- [ ] **Step 4: Valider les YAML** - -Run: `for f in .github/workflows/ci.yml .github/workflows/templates-upload.yml .github/workflows/templates-hotfix.yml; do uv run python -c "import yaml,sys; yaml.safe_load(open('$f')); print('ok $f')"; done` - -- [ ] **Step 5: Tester la génération du manifest en local** - -Run: `cd templates && FILES=$(find . -name '*.j2' -type f | sort | while read -r f; do rel="${f#./}"; printf '{"%s":{"sha256":"%s","size":%s}}\n' "$rel" "$(sha256sum "$f" | cut -d' ' -f1)" "$(stat -c%s "$f")"; done | jq -s 'add') && jq -n --arg version 0.0.0 --arg commit x --arg date now --argjson files "$FILES" --argjson engines "$(cat engines.map.json)" '{schema:1, version:$version, generated_at:$date, commit:$commit, files:$files, engines:$engines}' | uv run python -c "import json,sys; from services.templates import Manifest; m = Manifest.from_json(json.load(sys.stdin)); print(len(m.files), 'files,', len(m.engines), 'engines')"; cd ..` -Expected: `9 files, 9 engines`. - -- [ ] **Step 6: Commit** - -```bash -git add .github/workflows/ -git commit -m "ci: render-check job, manifest generation on template upload, hotfix workflow" -``` - ---- - -### Task 9 : PR et release candidate - -- [ ] **Step 1: Lint complet et PR** - -Run: `uv run ruff check . && uv run ruff format --check . && uv run python scripts/render_check.py --no-compose` -Puis : -```bash -git checkout -b refactor/templates-engines -git push -u origin refactor/templates-engines -``` -PR « feat: Jinja2 templates, engine registry, template repository ». Checks attendus verts : `lint`, `test`, `render-check`, `gitleaks`, `plumber`, `build-smoke`. - -- [ ] **Step 2: Release candidate** - -Après merge : Bump version `26.08.0rc2` (ou suivant), channel `rc`. Vérifier sur S3 que `cli/public/templates/26.08.0rc2/` contient `manifest.json`, `agent.yml.j2`, `engines/`, **et** `agent.yml` / `dashboard.yml` legacy. Installer le binaire rc et lancer `portabase agent test-rc` (code legacy) : doit fonctionner comme avant (fetch `agent.yml` sous la version exacte). - ---- - -## Self-review - -**Spec coverage :** -- §5.4 `TemplateRepository` : résolution version/env/dev ✔, cache ✔, manifest sha256+size ✔, suppression fichiers obsolètes ✔, pas de `latest` côté client ✔, Jinja2 `StrictUndefined` ✔, schéma inconnu → `TemplateError` ✔, version ≠ → `TemplateError` ✔. -- §5.6 templates : `agent.yml.j2` avec `mounts`, `docker_socket`, `host_gateway`, `services`, `volumes` ✔ ; moteurs avec `{% if auth %}` ✔ ; `dashboard.yml.j2` avec `db_mode` ✔. -- §6 moteurs : hiérarchie, hooks, registre imports explicites ✔ ; `agent_database` hook ✔ ; Redis/Valkey séparés ✔ ; `describe` pour `db list` (Plan 4). -- §6.1 options : `option_fields`, `non_default_options`, projection ✔ ; parsing `-o` et prompts → Plan 4 (flow). -- §9.1 `render-check` ✔, `engines-check` (fusionné dans le script) ✔. §9.3 manifest ✔, hotfix ✔. -- §10 D : shippable en rc, legacy intact ✔. - -**Placeholders :** aucun. - -**Cohérence :** `DbEngine.template_ctx` produit `name/volume/auth/*_var` = variables consommées par tous les `.j2` ✔ ; `FirebirdEngine.template_ctx` ajoute `root_password_var` consommé par `firebird.yml.j2` ✔ ; `render_check.render_agent` passe `AGENT_GLOBALS` = variables `*_var` de `agent.yml.j2` ✔ ; `Manifest.engines` clé → `TemplateRepository.engine_template` ✔ ; `FixedPortAllocator` défini Task 1, utilisé Task 7 ✔. - -**Écarts connus :** -- `DatabaseSpec.host_port` est `None` pour les specs chargées depuis une install legacy tant que Plan 4 ne lit pas `.env` ; sans effet ici. diff --git a/docs/superpowers/plans/2026-09-11-plan-4-render-commands.md b/docs/superpowers/plans/2026-09-11-plan-4-render-commands.md deleted file mode 100644 index 1856538..0000000 --- a/docs/superpowers/plans/2026-09-11-plan-4-render-commands.md +++ /dev/null @@ -1,1935 +0,0 @@ -# Plan 4 — Rendu déclaratif et commandes agent / dashboard / db / build (chantiers E + F) - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** Remplacer le code legacy (`agent.py`, `db.py`, `dashboard.py`, chirurgie texte du compose) par un rendu complet depuis l'état (`.env` + `databases.json` + faits du compose), un flux d'ajout de base partagé, des commandes entièrement pilotables par flags, et la commande `build`. Fin de la refonte : plus aucun `LegacyCommand`, plus de `console` global, plus de fallback `latest`. - -**Architecture:** `AgentProject`/`DashboardProject` chargent l'état depuis le dossier ; `ComposeRenderer` produit `docker-compose.yml` (+ `databases.json`) via Jinja2 ; `RenderResult.write()` valide, sauvegarde un compose legacy en `.legacy.yml`, écrit atomiquement. `AddDatabaseFlow` collecte un `DatabaseSpec` (flags → prompts → défauts) et mute le projet ; `AgentCommand` et `DbAddCommand` l'utilisent tous deux. - -**Tech Stack:** Python 3.12, Typer, Jinja2, PyYAML, questionary/Rich via `ui/`. - -**Spec:** `docs/superpowers/specs/2026-09-11-cli-refactor-design.md` — sections 4.2, 4.3, 5.1–5.3, 5.5, 5.7, 6.1, 7.2 (Summary, DataTable, Diff), 10 (E, F). - -## Global Constraints - -- Prérequis : Plans 1–3 exécutés. -- Règle de dépendance descendante (`commands → services, engines, ui, core` ; `services → engines, core` ; `flows` dans `commands/`). -- `.env` ne contient que des variables consommées par les conteneurs. Aucune clé `PORTABASE_*`. -- `databases.json` conserve exactement le format legacy (projection par `DbEngine.agent_entry`). -- Détection `managed` : `.env` contient `{PREFIX}_PORT` pour le `host` de l'entrée. -- Le compose généré porte l'en-tête `# Generated by Portabase CLI . Do not edit — use docker-compose.override.yml.` ; un compose sans cet en-tête est sauvegardé en `docker-compose.legacy.yml` avant la première réécriture (une seule fois). -- Toute commande mutante : `templates.resolve()` **avant** de muter quoi que ce soit. -- Pas de tests unitaires. Vérifications exécutables par commande, en interactif et en `--non-interactive`, sur une install neuve et sur une install legacy générée avec le binaire 26.07.6. -- Ce plan supprime : `commands/common.py` (déjà), `core/network.py`, `core/docker.py`, `templates/compose.py`, `templates/__init__.py`, `templates/agent.yml`, `templates/dashboard.yml`, `LegacyCommand`, les fonctions legacy de `core/config.py`, `console`/`print_banner`/`HINTS`/`check_system`/`start_docker`/`validate_work_dir`/`get_free_port`/`get_random_hint` de `core/utils.py`, toutes les `per-file-ignores` ruff. - ---- - -## File Structure - -| Fichier | Action | Responsabilité | -|---|---|---| -| `engines/base.py` | modifier | `label_default` | -| `services/envfile.py` | créer | `EnvFile` | -| `services/compose_facts.py` | créer | `ComposeFacts` | -| `services/project.py` | créer | `AgentProject`, `DashboardProject`, `ProjectKind`, `detect_kind`, `spec_from_entry` | -| `services/renderer.py` | créer | `ComposeRenderer`, `RenderResult`, `WriteReport` | -| `services/docker.py` | modifier | `remove_volume` | -| `ui/components/summary.py`, `table.py`, `diff.py` | créer | composants | -| `ui/__init__.py` | modifier | `summary`, `table`, `diff` | -| `commands/flows/__init__.py`, `add_database.py` | créer | `AddDatabaseFlow` | -| `commands/agent.py` | réécrire | `AgentCommand` | -| `commands/dashboard.py` | réécrire | `DashboardCommand` | -| `commands/db.py` | réécrire | `DbCommands` (`add`, `remove`, `list`) | -| `commands/build.py` | créer | `BuildCommand` | -| `commands/decrypt.py` | réécrire | `DecryptCommand` | -| `core/crypto.py` | modifier | `DecryptionError(PortabaseError)` | -| `commands/base.py` | modifier | retirer `LegacyCommand` | -| `main.py` | modifier | câblage final | -| `core/config.py`, `core/utils.py` | modifier | retirer le legacy | -| `scripts/render_check.py` | modifier | utiliser `ComposeRenderer` | -| `.github/workflows/ci.yml` | modifier | `build-smoke` avec `agent --non-interactive` | -| `pyproject.toml` | modifier | retirer `per-file-ignores` | -| `README.md` | modifier | note migration | - ---- - -### Task 1 : `EnvFile` - -**Files:** -- Create: `services/envfile.py` - -**Interfaces:** -- Produces: `EnvFile(path)` : `load() -> EnvFile` (classmethod `EnvFile.load(path)`), `get(key, default=None)`, `set(key, value)`, `merge(mapping)`, `remove(key)`, `remove_prefix(prefix)`, `as_dict() -> dict[str, str]`, `save()`, `exists`. Préserve ordre, commentaires, lignes vides. - -- [ ] **Step 1: Écrire le module** - -```python -"""Dotenv file kept as a list of lines so comments and order survive rewrites. - -Only container runtime variables live here. Values are always written double-quoted. -""" - -from __future__ import annotations - -import os -import re -from collections.abc import Mapping -from dataclasses import dataclass, field -from pathlib import Path - -_LINE = re.compile(r"""^\s*(?:export\s+)?([A-Za-z_][A-Za-z0-9_]*)\s*=\s*(.*?)\s*$""") - - -def _unquote(raw: str) -> str: - if len(raw) >= 2 and raw[0] == raw[-1] and raw[0] in "\"'": - inner = raw[1:-1] - if raw[0] == '"': - return inner.replace('\\"', '"').replace("\\\\", "\\") - return inner - # unquoted: strip trailing comment - return raw.split(" #", 1)[0].rstrip() - - -def _quote(value: str) -> str: - return '"' + value.replace("\\", "\\\\").replace('"', '\\"') + '"' - - -@dataclass -class EnvFile: - path: Path - _lines: list[str] = field(default_factory=list) # raw lines, without newline - _index: dict[str, int] = field(default_factory=dict) # key -> line number - - @classmethod - def load(cls, path: Path) -> EnvFile: - env = cls(path) - if path.exists(): - text = path.read_text(encoding="utf-8") - env._lines = text.splitlines() - for i, line in enumerate(env._lines): - m = _LINE.match(line) - if m and not line.lstrip().startswith("#"): - env._index[m.group(1)] = i - return env - - @property - def exists(self) -> bool: - return self.path.exists() - - def get(self, key: str, default: str | None = None) -> str | None: - i = self._index.get(key) - if i is None: - return default - m = _LINE.match(self._lines[i]) - return _unquote(m.group(2)) if m else default - - def as_dict(self) -> dict[str, str]: - return {k: self.get(k) or "" for k in self._index} - - def set(self, key: str, value: str) -> None: - line = f"{key}={_quote(str(value))}" - i = self._index.get(key) - if i is None: - self._lines.append(line) - self._index[key] = len(self._lines) - 1 - else: - self._lines[i] = line - - def merge(self, mapping: Mapping[str, str]) -> None: - for k, v in mapping.items(): - self.set(k, v) - - def remove(self, key: str) -> None: - i = self._index.pop(key, None) - if i is None: - return - del self._lines[i] - self._index = {k: (n - 1 if n > i else n) for k, n in self._index.items()} - - def remove_prefix(self, prefix: str) -> None: - for key in [k for k in self._index if k.startswith(prefix + "_")]: - self.remove(key) - - def save(self) -> None: - self.path.parent.mkdir(parents=True, exist_ok=True) - tmp = self.path.with_suffix(".env.tmp") - tmp.write_text("\n".join(self._lines) + "\n", encoding="utf-8") - os.replace(tmp, self.path) -``` - -- [ ] **Step 2: Vérifier** - -Run: `uv run python -c " -import tempfile; from pathlib import Path -from services.envfile import EnvFile -p = Path(tempfile.mkdtemp())/'.env' -p.write_text('# header\nTZ=\"UTC\"\nEDGE_KEY=\"a=b\"\n\nDB_PG_A1_PORT=\"5433\"\nDB_PG_A1_PASS=\"p\\\"q\"\nCUSTOM=plain # note\n') -e = EnvFile.load(p); print(e.get('TZ'), e.get('EDGE_KEY'), e.get('DB_PG_A1_PASS'), e.get('CUSTOM')) -e.set('TZ','Europe/Paris'); e.merge({'NEW':'x'}); e.remove_prefix('DB_PG_A1'); e.save() -print(p.read_text())"` -Expected: `UTC a=b p"q plain` puis le fichier avec `# header`, `TZ="Europe/Paris"`, `EDGE_KEY`, ligne vide conservée, `CUSTOM` réécrit tel quel, `NEW="x"` en fin, plus aucune `DB_PG_A1_*`. - -- [ ] **Step 3: Commit** - -```bash -git add services/envfile.py -git commit -m "feat(services): add EnvFile preserving order and comments" -``` - ---- - -### Task 2 : `ComposeFacts`, `project.py`, `label_default` - -**Files:** -- Create: `services/compose_facts.py` -- Create: `services/project.py` -- Modify: `engines/base.py` (ajouter `label_default = "External DB"` ; `DockerVolumeEngine.label_default = "Docker Volume"` dans `engines/docker_volume.py`) - -**Interfaces:** -- Produces: - - `ComposeFacts(path)` : `exists`, `is_generated` (en-tête présent), `host_gateway -> bool`, `raw -> dict`. - - `ProjectKind = Literal["agent", "dashboard"]`, `detect_kind(path) -> ProjectKind` (`ConfigError` sinon). - - `spec_from_entry(entry: dict, env: EnvFile) -> DatabaseSpec`. - - `AgentProject(path, env, databases, host_gateway)` : `load(path)`, `create(path, env_vars: dict, host_gateway)`, `managed`, `needs_docker_socket`, `sqlite_mounts`, `add(spec, engine)`, `remove(spec, engine)`, `find(id_or_name) -> DatabaseSpec`, `save_state()` (écrit `.env` seulement ; `databases.json` est rendu). - - `DashboardProject(path, env)` : `load(path)`, `create(path, env_vars)`, `db_mode`, `save_state()`. - -- [ ] **Step 1: `services/compose_facts.py`** - -```python -"""Read-only structural facts from an existing docker-compose.yml. Never writes.""" - -from __future__ import annotations - -from pathlib import Path - -import yaml - -GENERATED_MARKER = "# Generated by Portabase CLI" - - -class ComposeFacts: - def __init__(self, path: Path) -> None: - self.path = path - self.raw: dict = {} - self.text = "" - if path.exists(): - try: - self.text = path.read_text(encoding="utf-8") - loaded = yaml.safe_load(self.text) - self.raw = loaded if isinstance(loaded, dict) else {} - except (OSError, yaml.YAMLError): - self.raw = {} - - @property - def exists(self) -> bool: - return self.path.exists() - - @property - def is_generated(self) -> bool: - return self.text.startswith(GENERATED_MARKER) - - def _service(self, name: str) -> dict: - services = self.raw.get("services") or {} - svc = services.get(name) if isinstance(services, dict) else None - return svc if isinstance(svc, dict) else {} - - @property - def host_gateway(self) -> bool: - extra = self._service("agent").get("extra_hosts") - if isinstance(extra, list): - return any("host-gateway" in str(x) for x in extra) - if isinstance(extra, dict): - return any("host-gateway" in str(v) for v in extra.values()) - return False -``` - -- [ ] **Step 2: `services/project.py`** - -```python -"""Project state loaded from .env + databases.json + compose facts. Nothing else is stored.""" - -from __future__ import annotations - -import json -from dataclasses import dataclass, field -from pathlib import Path -from typing import Any, Literal - -from core.errors import ConfigError, ValidationError -from core.specs import DatabaseSpec -from engines.base import DbEngine -from engines.sqlite import SqliteEngine -from services.compose_facts import ComposeFacts -from services.envfile import EnvFile - -ProjectKind = Literal["agent", "dashboard"] -DATABASES_FILE = "databases.json" -COMPOSE_FILE = "docker-compose.yml" -ENV_FILE = ".env" - - -def detect_kind(path: Path) -> ProjectKind: - if (path / DATABASES_FILE).exists(): - return "agent" - env = EnvFile.load(path / ENV_FILE) - if env.get("PROJECT_SECRET") is not None: - return "dashboard" - raise ConfigError( - f"{path} is not a Portabase agent or dashboard folder.", - hint="Expected databases.json (agent) or a .env with PROJECT_SECRET (dashboard).", - ) - - -def spec_from_entry(entry: dict[str, Any], env: EnvFile) -> DatabaseSpec: - engine = str(entry.get("type", "")) - host = entry.get("host") - managed, host_port, root_password = False, None, None - if host: - prefix = str(host).upper().replace("-", "_") - raw_port = env.get(f"{prefix}_PORT") - if raw_port and raw_port.isdigit(): - managed, host_port = True, int(raw_port) - root_password = env.get(f"{prefix}_ROOT_PASS") - return DatabaseSpec( - id=str(entry.get("generated_id") or DbEngine.new_id()), - engine=engine, - name=str(entry.get("name", "")), - managed=managed, - host=str(host) if host else None, - port=int(entry["port"]) if entry.get("port") not in (None, "") else None, - host_port=host_port, - database=str(entry["database"]) if entry.get("database") is not None else None, - username=str(entry["username"]) if entry.get("username") is not None else None, - password=str(entry["password"]) if entry.get("password") not in (None, "") else None, - root_password=root_password, - path=str(entry["database"]) if engine == "sqlite" and entry.get("database") else None, - volume=str(entry["volume_name"]) if entry.get("volume_name") else None, - container=str(entry["container_name"]) if entry.get("container_name") else None, - options=dict(entry.get("options") or {}), - ) - - -@dataclass -class AgentProject: - path: Path - env: EnvFile - databases: list[DatabaseSpec] = field(default_factory=list) - host_gateway: bool = False - - # ---- construction ----------------------------------------------------- - - @classmethod - def load(cls, path: Path) -> AgentProject: - path = path.resolve() - env_path, db_path = path / ENV_FILE, path / DATABASES_FILE - if not env_path.exists() or not db_path.exists(): - raise ConfigError( - f"Not a Portabase agent folder: {path}", - hint=f"Expected {ENV_FILE} and {DATABASES_FILE}.", - ) - env = EnvFile.load(env_path) - try: - data = json.loads(db_path.read_text(encoding="utf-8")) - except (OSError, ValueError) as e: - raise ConfigError(f"{db_path} is not valid JSON.", cause=e) from e - entries = data.get("databases", []) if isinstance(data, dict) else [] - databases = [spec_from_entry(e, env) for e in entries if isinstance(e, dict)] - project = cls(path, env, databases, ComposeFacts(path / COMPOSE_FILE).host_gateway) - project.validate() - return project - - @classmethod - def create(cls, path: Path, env_vars: dict[str, str], *, host_gateway: bool) -> AgentProject: - path.mkdir(parents=True, exist_ok=True) - env = EnvFile.load(path / ENV_FILE) - env.merge(env_vars) - return cls(path, env, [], host_gateway) - - # ---- derived facts ---------------------------------------------------- - - @property - def managed(self) -> list[DatabaseSpec]: - return [d for d in self.databases if d.managed] - - @property - def needs_docker_socket(self) -> bool: - return any(d.engine == "docker-volume" for d in self.databases) - - @property - def sqlite_mounts(self) -> list[tuple[str, str]]: - mounts = [] - for d in self.databases: - if d.engine == "sqlite": - m = SqliteEngine.mount_for(d) - if m and m not in mounts: - mounts.append(m) - return mounts - - def validate(self) -> None: - seen: set[str] = set() - for d in self.managed: - if d.host in seen: - raise ConfigError(f"Two managed databases share the service name '{d.host}'.") - seen.add(d.host or "") - - # ---- mutation --------------------------------------------------------- - - def add(self, spec: DatabaseSpec, engine: DbEngine) -> None: - if spec.managed: - self.env.merge(engine.env_vars(spec)) - self.databases.append(spec) - self.validate() - - def remove(self, spec: DatabaseSpec, engine: DbEngine) -> None: - self.databases = [d for d in self.databases if d.id != spec.id] - if spec.managed and spec.host: - self.env.remove_prefix(spec.env_prefix) - - def find(self, id_or_name: str) -> DatabaseSpec: - matches = [d for d in self.databases if d.id == id_or_name or d.id.startswith(id_or_name) or d.name == id_or_name] - if not matches: - raise ValidationError(f"No database matching '{id_or_name}'.", hint="See: portabase db list") - if len(matches) > 1: - raise ValidationError(f"'{id_or_name}' matches several databases; use the id.") - return matches[0] - - def save_state(self) -> None: - self.env.save() - - -@dataclass -class DashboardProject: - path: Path - env: EnvFile - - @classmethod - def load(cls, path: Path) -> DashboardProject: - path = path.resolve() - env = EnvFile.load(path / ENV_FILE) - if env.get("PROJECT_SECRET") is None: - raise ConfigError(f"Not a Portabase dashboard folder: {path}", hint="Expected a .env with PROJECT_SECRET.") - return cls(path, env) - - @classmethod - def create(cls, path: Path, env_vars: dict[str, str]) -> DashboardProject: - path.mkdir(parents=True, exist_ok=True) - env = EnvFile.load(path / ENV_FILE) - env.merge(env_vars) - return cls(path, env) - - @property - def db_mode(self) -> Literal["external", "internal", "custom"]: - host = self.env.get("POSTGRES_HOST") - if host is None: - return "internal" - return "external" if host == "db" else "custom" - - @property - def project_name(self) -> str: - return self.env.get("PROJECT_NAME") or self.path.name - - def save_state(self) -> None: - self.env.save() -``` - -- [ ] **Step 3: `label_default`** - -Dans `engines/base.py`, après `has_modes: bool = True` : `label_default: str = "External DB"`. Dans `engines/docker_volume.py`, après `has_modes = False` : `label_default = "Docker Volume"`. Remplacer dans `from_existing` de `base.py` et `sqlite.py` `or "External DB"` par `or self.label_default`, et dans `docker_volume.py` `or "Docker Volume"` par `or self.label_default`. - -- [ ] **Step 4: Vérifier sur une install legacy réelle** - -Générer une install avec le binaire 26.07.6 (télécharger depuis la release GitHub) ou avec `git stash`/checkout du tag : - -```bash -cd /tmp && rm -rf legacy-agent && git -C /home/soluce/Documents/PROJETS/Portabase/cli stash -u -q; git -C /home/soluce/Documents/PROJETS/Portabase/cli checkout -q 26.07.6 -uv --directory /home/soluce/Documents/PROJETS/Portabase/cli run python /home/soluce/Documents/PROJETS/Portabase/cli/main.py agent legacy-agent --key "$(printf '{"serverUrl":"http://x","agentId":"a","masterKeyB64":"k"}' | base64 -w0)" -``` -Au wizard : tz `UTC`, polling `5`, extra_hosts `y`, puis `database` → `new` → `postgresql` (ownership `n`, clean `clean`), puis `database` → `new` → `redis` → `with-auth`, puis `database` → `existing` → `sqlite` → `Display` / path `ext.sqlite`, puis `docker-volume` (`Vol`, `myvol`, container vide), puis `done`, ne pas démarrer. - -```bash -git -C /home/soluce/Documents/PROJETS/Portabase/cli checkout -q main; git -C /home/soluce/Documents/PROJETS/Portabase/cli stash pop -q -uv --directory /home/soluce/Documents/PROJETS/Portabase/cli run python -c " -from pathlib import Path -from services.project import AgentProject, detect_kind -p = AgentProject.load(Path('/tmp/legacy-agent')) -print(detect_kind(p.path), 'gateway:', p.host_gateway, 'socket:', p.needs_docker_socket, 'mounts:', p.sqlite_mounts) -for d in p.databases: print(f'{d.engine:14} managed={d.managed!s:5} host={d.host} host_port={d.host_port} db={d.database} opts={d.options}')" -``` -Expected: `agent gateway: True socket: True mounts: [('./ext.sqlite', '/config/ext.sqlite')]` ; postgresql `managed=True host=db-pg-xxxx host_port=` ; redis `managed=True host=db-redis-auth-xxxx` ; sqlite `managed=False` ; docker-volume `managed=False`. - -- [ ] **Step 5: Commit** - -```bash -git add services/compose_facts.py services/project.py engines/base.py engines/sqlite.py engines/docker_volume.py -git commit -m "feat(services): add ComposeFacts and project state loaded from .env, databases.json and compose" -``` - ---- - -### Task 3 : `ComposeRenderer` et `RenderResult` - -**Files:** -- Create: `services/renderer.py` -- Modify: `services/docker.py` (ajouter `remove_volume`) - -**Interfaces:** -- Consumes: `TemplateRepository`, `EngineRegistry`, `AgentProject`, `DashboardProject`, `SqliteEngine.mount_for`. -- Produces: - - `ComposeRenderer(templates, engines, cli_version)` : `render_agent(project, *, inline=False) -> RenderResult`, `render_dashboard(project, *, inline=False) -> RenderResult`. - - `RenderResult(compose: str, databases: list[dict] | None)` : `validate()` (`TemplateError`), `write(path) -> WriteReport`, `diff_against(path) -> str`. - - `WriteReport(backed_up: Path | None, wrote: list[Path])`. - - `DockerRunner.remove_volume(name) -> None`. - -- [ ] **Step 1: `services/renderer.py`** - -```python -"""State → docker-compose.yml (+ databases.json). The only writer of those files.""" - -from __future__ import annotations - -import difflib -import json -import os -import shutil -from dataclasses import dataclass, field -from pathlib import Path -from typing import Any - -import jinja2 -import yaml - -from core.errors import TemplateError -from core.specs import DatabaseSpec -from engines.registry import EngineRegistry -from services.compose_facts import GENERATED_MARKER, ComposeFacts -from services.envfile import EnvFile -from services.project import COMPOSE_FILE, DATABASES_FILE, AgentProject, DashboardProject -from services.templates import TemplateRepository - -LEGACY_BACKUP = "docker-compose.legacy.yml" - - -@dataclass -class WriteReport: - backed_up: Path | None = None - wrote: list[Path] = field(default_factory=list) - - -@dataclass -class RenderResult: - compose: str - databases: list[dict[str, Any]] | None = None - - def validate(self) -> None: - try: - doc = yaml.safe_load(self.compose) - except yaml.YAMLError as e: - raise TemplateError("Rendered compose is not valid YAML; templates are broken.", cause=e) from e - if not isinstance(doc, dict) or "services" not in doc: - raise TemplateError("Rendered compose has no 'services' section; templates are broken.") - - def write(self, path: Path) -> WriteReport: - self.validate() - report = WriteReport() - compose_path = path / COMPOSE_FILE - facts = ComposeFacts(compose_path) - if facts.exists and not facts.is_generated: - backup = path / LEGACY_BACKUP - if not backup.exists(): - shutil.copy2(compose_path, backup) - report.backed_up = backup - _atomic_write(compose_path, self.compose) - report.wrote.append(compose_path) - if self.databases is not None: - db_path = path / DATABASES_FILE - _atomic_write(db_path, json.dumps({"databases": self.databases}, indent=2) + "\n") - try: - os.chmod(db_path, 0o666) # agent container may run as another uid (legacy behaviour) - except OSError: - pass - report.wrote.append(db_path) - return report - - def diff_against(self, path: Path) -> str: - current = (path / COMPOSE_FILE).read_text(encoding="utf-8") if (path / COMPOSE_FILE).exists() else "" - return "".join( - difflib.unified_diff( - current.splitlines(keepends=True), - self.compose.splitlines(keepends=True), - fromfile=f"{COMPOSE_FILE} (current)", - tofile=f"{COMPOSE_FILE} (rendered)", - ) - ) - - -def _atomic_write(path: Path, content: str) -> None: - path.parent.mkdir(parents=True, exist_ok=True) - tmp = path.with_suffix(path.suffix + ".tmp") - tmp.write_text(content, encoding="utf-8") - os.replace(tmp, path) - - -class ComposeRenderer: - def __init__(self, templates: TemplateRepository, engines: EngineRegistry, cli_version: str) -> None: - self.templates = templates - self.engines = engines - self.cli_version = cli_version - - def header(self) -> str: - return f"{GENERATED_MARKER} {self.cli_version}. Do not edit — use docker-compose.override.yml.\n" - - # ---- agent ------------------------------------------------------------ - - def render_agent(self, project: AgentProject, *, inline: bool = False) -> RenderResult: - env = project.env - ctx = { - "host_gateway": project.host_gateway, - "docker_socket": project.needs_docker_socket, - "mounts": [{"host": h, "container": c} for h, c in project.sqlite_mounts], - "services": [self._service(spec, inline) for spec in project.managed], - "tz_var": _var(env, "TZ", inline), - "edge_key_var": _var(env, "EDGE_KEY", inline), - "log_level_var": _var(env, "LOG_LEVEL", inline), - "polling_var": _var(env, "POLLING", inline), - } - compose = self.header() + self._render("agent.yml.j2", ctx) - databases = [self.engines.get(d.engine).agent_entry(d) for d in project.databases] - return RenderResult(compose=compose, databases=databases) - - def _service(self, spec: DatabaseSpec, inline: bool) -> dict[str, str]: - engine = self.engines.get(spec.engine) - body = self._render_template(self.templates.engine_template(spec.engine), engine.template_ctx(spec, inline=inline)) - return {"name": spec.host or "", "volume": f"{spec.host}-data", "body": body} - - # ---- dashboard -------------------------------------------------------- - - def render_dashboard(self, project: DashboardProject, *, inline: bool = False) -> RenderResult: - env = project.env - ctx = { - "db_mode": project.db_mode, - "project_name_var": project.project_name, # literal, as the legacy CLI did - "host_port_var": _var(env, "HOST_PORT", inline), - "tz_var": _var(env, "TZ", inline), - "log_level_var": _var(env, "LOG_LEVEL", inline), - "project_secret_var": _var(env, "PROJECT_SECRET", inline), - "project_url_var": _var(env, "PROJECT_URL", inline), - "pg_port_var": _var(env, "PG_PORT", inline), - "postgres_db_var": _var(env, "POSTGRES_DB", inline), - "postgres_user_var": _var(env, "POSTGRES_USER", inline), - "postgres_password_var": _var(env, "POSTGRES_PASSWORD", inline), - } - return RenderResult(compose=self.header() + self._render("dashboard.yml.j2", ctx), databases=None) - - # ---- internals -------------------------------------------------------- - - def _render(self, name: str, ctx: dict[str, Any]) -> str: - return self._render_template(self.templates.get(name), ctx) - - @staticmethod - def _render_template(template: jinja2.Template, ctx: dict[str, Any]) -> str: - try: - return template.render(**ctx) - except jinja2.TemplateError as e: - raise TemplateError(f"Template rendering failed: {e}", cause=e) from e - - -def _var(env: EnvFile, key: str, inline: bool) -> str: - return (env.get(key) or "") if inline else f"${{{key}}}" -``` - -- [ ] **Step 2: `DockerRunner.remove_volume`** (dans `services/docker.py`, après `ensure_network`) - -```python - def remove_volume(self, name: str) -> bool: - """True if removed, False if it did not exist. Raises on other failures.""" - proc = subprocess.run([self.binary, "volume", "rm", name], capture_output=True, text=True, check=False) - if proc.returncode == 0: - return True - if "no such volume" in (proc.stderr or "").lower(): - return False - raise DockerError(f"Could not remove volume '{name}': {proc.stderr.strip()}") -``` - -- [ ] **Step 3: Vérifier le rendu sur l'install legacy et le `--diff`** - -Run: `uv run python -c " -from pathlib import Path -from services.project import AgentProject -from services.renderer import ComposeRenderer -from services.templates import TemplateRepository -from services.http import HttpClient -from core.config import GlobalConfig -from engines import registry -import yaml -repo = TemplateRepository.from_environment(HttpClient(), GlobalConfig()) -r = ComposeRenderer(repo, registry, '0.0.0-dev') -p = AgentProject.load(Path('/tmp/legacy-agent')) -res = r.render_agent(p); res.validate() -doc = yaml.safe_load(res.compose) -print(sorted(doc['services']), doc['services']['agent']['volumes'], doc['services']['agent'].get('extra_hosts')) -print(res.diff_against(p.path)[:1200]) -print(len(res.databases), [d['type'] for d in res.databases])"` -Expected: services = `agent` + le service postgres + le service redis (mêmes noms que le compose legacy) ; volumes agent = `databases.json`, `./ext.sqlite:/config/ext.sqlite`, socket ; `extra_hosts` présent ; diff limité à l'en-tête, l'ordre des lignes, `restart: unless-stopped` sur redis ; `4 ['postgresql', 'redis', 'sqlite', 'docker-volume']`. - -Comparer aussi `res.databases` à `/tmp/legacy-agent/databases.json` : mêmes clés et valeurs par entrée (à l'ordre des clés près). - -- [ ] **Step 4: Commit** - -```bash -git add services/renderer.py services/docker.py -git commit -m "feat(services): add ComposeRenderer with validation, atomic write and legacy backup" -``` - ---- - -### Task 4 : Composants `Summary`, `DataTable`, `Diff` - -**Files:** -- Create: `ui/components/summary.py`, `ui/components/table.py`, `ui/components/diff.py` -- Modify: `ui/__init__.py` - -**Interfaces:** -- Produces: `UI.summary(rows: list[tuple[str, str]], *, title: str | None = None)`, `UI.table(columns: list[str], rows: list[list[str]], *, title: str | None = None)`, `UI.diff(text: str)`. - -- [ ] **Step 1: `ui/components/summary.py`** - -```python -from __future__ import annotations - -import re - -from rich.panel import Panel -from rich.table import Table - -from ui.components.base import Component - -_SENSITIVE = re.compile(r"(password|secret|key|token)", re.I) -_URL_CREDS = re.compile(r"://([^:/@]+):([^@/]+)@") - - -def mask(label: str, value: str) -> str: - if _SENSITIVE.search(label): - return "••••••••" - return _URL_CREDS.sub(r"://\1:****@", value) - - -class Summary(Component): - def __call__(self, rows: list[tuple[str, str]], *, title: str | None = None) -> None: - table = Table(show_header=False, box=None, padding=(0, 2)) - table.add_column("Property", style="bold cyan") - table.add_column("Value", style="white") - for label, value in rows: - table.add_row(label, mask(label, str(value))) - self.console.print("") - self.console.print(Panel(table, title=f"[bold white]{title}[/bold white]" if title else None, border_style="bold blue", expand=False)) -``` - -- [ ] **Step 2: `ui/components/table.py`** - -```python -from __future__ import annotations - -from rich.table import Table - -from ui.components.base import Component - -_STYLES = ["cyan", "blue", "magenta", "green", "white", "dim"] - - -class DataTable(Component): - def __call__(self, columns: list[str], rows: list[list[str]], *, title: str | None = None) -> None: - table = Table(title=title) - for i, col in enumerate(columns): - table.add_column(col, style=_STYLES[i % len(_STYLES)]) - for row in rows: - table.add_row(*[str(c) for c in row]) - self.console.print(table) -``` - -- [ ] **Step 3: `ui/components/diff.py`** - -```python -from __future__ import annotations - -from rich.syntax import Syntax - -from ui.components.base import Component - - -class Diff(Component): - def __call__(self, text: str) -> None: - if not text.strip(): - self.console.print("[info]ℹ No changes.[/info]") - return - self.console.print(Syntax(text, "diff", theme="ansi_dark", word_wrap=False)) -``` - -- [ ] **Step 4: Façade** — ajouter à `ui/__init__.py` les imports et méthodes : - -```python -from ui.components.diff import Diff -from ui.components.summary import Summary -from ui.components.table import DataTable - - def summary(self, rows: list[tuple[str, str]], *, title: str | None = None) -> None: - Summary(self.console)(rows, title=title) - - def table(self, columns: list[str], rows: list[list[str]], *, title: str | None = None) -> None: - DataTable(self.console)(columns, rows, title=title) - - def diff(self, text: str) -> None: - Diff(self.console)(text) -``` - -- [ ] **Step 5: Vérifier** - -Run: `uv run python -c " -from ui import UI -ui = UI() -ui.summary([('Name','x'),('Password','hunter2'),('Connection URL','postgresql://u:p@h:5432/d')], title='PROPOSED') -ui.table(['A','B'], [['1','2']], title='T') -ui.diff('--- a\n+++ b\n@@ -1 +1 @@\n-old\n+new\n'); ui.diff('')"` -Expected: panneau avec `••••••••` et `:****@`, table, diff colorisé, `ℹ No changes.`. - -- [ ] **Step 6: Commit** - -```bash -git add ui/ -git commit -m "feat(ui): add Summary, DataTable and Diff components" -``` - ---- - -### Task 5 : `AddDatabaseFlow` - -**Files:** -- Create: `commands/flows/__init__.py` (vide) -- Create: `commands/flows/add_database.py` - -**Interfaces:** -- Consumes: `UI`, `EngineRegistry`, `PortAllocator`, `Form`, `Field`, `DatabaseSpec`, `AgentProject`. -- Produces: `AddDatabaseFlow(ui, engines, ports)` : `collect(values: dict) -> tuple[DatabaseSpec, DbEngine]`, `apply(project, spec, engine) -> None`, `parse_options(items: list[str]) -> dict[str, str]` (static). - -- [ ] **Step 1: Écrire le module** - -```python -"""Shared 'add a database' wizard. Flags fill `values`; anything missing is prompted or errors.""" - -from __future__ import annotations - -from typing import Any - -from core.errors import ValidationError -from core.fields import Field -from core.specs import DatabaseSpec -from engines.base import DbEngine -from engines.registry import EngineRegistry -from services.ports import PortAllocator -from services.project import AgentProject -from ui import UI - -FLOW_KEYS = {"engine", "mode", "auth", "label", "options"} - - -class AddDatabaseFlow: - def __init__(self, ui: UI, engines: EngineRegistry, ports: PortAllocator) -> None: - self.ui = ui - self.engines = engines - self.ports = ports - - # ---- public ----------------------------------------------------------- - - @staticmethod - def parse_options(items: list[str] | None) -> dict[str, str]: - out: dict[str, str] = {} - for item in items or []: - if "=" not in item: - raise ValidationError(f"Invalid option '{item}'.", hint="Use -o KEY=VALUE") - key, value = item.split("=", 1) - out[key.strip()] = value.strip() - return out - - def collect(self, values: dict[str, Any]) -> tuple[DatabaseSpec, DbEngine]: - form = self.ui.form() - engine = self.engines.get( - form.choice("Select Database Engine", self.engines.choices(), value=values.get("engine"), name="engine") - ) - if engine.warning: - self.ui.warning(engine.warning) - - mode = "new" - if engine.has_modes: - mode = form.choice("Configuration Mode", ["new", "existing"], value=values.get("mode"), default="new", name="mode") - - auth = True - if mode == "new" and engine.auth_variants: - raw = values.get("auth") - if raw is None: - auth = form.choice("Variant", ["with-auth", "no-auth"], name="auth") == "with-auth" - else: - auth = bool(raw) - - fields = list(engine.fields_new() if mode == "new" else engine.fields_existing()) - if mode == "existing" or not engine.has_modes: - fields.insert(0, Field("label", "Display Name", "text", default=engine.label_default)) - - self._reject_irrelevant(values, fields, engine, mode) - - if mode == "existing": - self.ui.info(f"{engine.display} — existing database") - answers = form.collect(fields, values) - answers["options"] = self._collect_options(form, engine, values.get("options") or {}) - - if mode == "new": - spec = engine.generate(auth=auth, ports=self.ports, answers=answers) - else: - spec = engine.from_existing(answers) - return spec.with_options(answers["options"]), engine - - def apply(self, project: AgentProject, spec: DatabaseSpec, engine: DbEngine) -> None: - project.add(spec, engine) - - # ---- internals -------------------------------------------------------- - - def _collect_options(self, form, engine: DbEngine, provided: dict[str, str]) -> dict[str, Any]: - option_fields = engine.option_fields() - known = {f.name for f in option_fields} - unknown = set(provided) - known - if unknown: - raise ValidationError( - f"Unknown option(s) for {engine.key}: {', '.join(sorted(unknown))}.", - hint=("Valid options: " + ", ".join(sorted(known))) if known else f"{engine.key} has no options.", - ) - if not option_fields: - return {} - return form.collect(option_fields, provided) - - @staticmethod - def _reject_irrelevant(values: dict[str, Any], fields: list[Field], engine: DbEngine, mode: str) -> None: - relevant = {f.name for f in fields} | FLOW_KEYS - extra = sorted(k for k, v in values.items() if v is not None and k not in relevant) - if extra: - raise ValidationError( - f"Option(s) not applicable to {engine.key} in '{mode}' mode: {', '.join('--' + k.replace('_', '-') for k in extra)}.", - hint="Applicable: " + ", ".join(f"--{f.name.replace('_', '-')}" for f in fields) if fields else "No extra input needed.", - ) -``` - -- [ ] **Step 2: Vérifier en non-interactif** - -Run: `uv run python -c " -from ui import UI -from engines import registry -from services.ports import FixedPortAllocator -from commands.flows.add_database import AddDatabaseFlow -from core.errors import ValidationError -f = AddDatabaseFlow(UI(non_interactive=True), registry, FixedPortAllocator()) -s, e = f.collect({'engine':'postgresql','mode':'new','options': f.parse_options(['clean_mode=none'])}); print(e.key, s.managed, s.options) -s, e = f.collect({'engine':'redis','mode':'new','auth':False}); print(s.host[:9], s.auth) -s, e = f.collect({'engine':'sqlite','mode':'existing','path':'x.sqlite'}); print(s.name, s.database) -s, e = f.collect({'engine':'docker-volume','volume':'v'}); print(s.name, s.volume, s.container) -for bad in ({'engine':'postgresql','mode':'existing'}, {'engine':'redis','mode':'new','host':'h'}, {'engine':'mysql','mode':'new','options':{'clean_mode':'x'}}, {'engine':'nope'}): - try: f.collect(bad) - except ValidationError as err: print('ERR', err.message)"` -Expected: -``` -postgresql True {'clean_mode': 'none', 'keep_ownership': False} -db-redis- False -External DB /config/x.sqlite -Docker Volume v None -ERR Missing --host -ERR Option(s) not applicable to redis in 'new' mode: --host. -ERR Unknown option(s) for mysql: clean_mode. -ERR Unknown engine 'nope'. -``` - -- [ ] **Step 3: Commit** - -```bash -git add commands/flows/ -git commit -m "feat(commands): add AddDatabaseFlow shared by agent and db add" -``` - ---- - -### Task 6 : `commands/db.py` — `add`, `remove`, `list` - -**Files:** -- Modify: `commands/db.py` (réécriture complète) - -**Interfaces:** -- Produces: `DbCommands(ui, telemetry, engines, ports, templates, renderer, docker)` groupe `db` avec `DbAddCommand`, `DbRemoveCommand`, `DbListCommand`. - -- [ ] **Step 1: Réécrire le module** - -```python -"""db add / remove / list.""" - -from __future__ import annotations - -from pathlib import Path -from typing import Annotated - -import typer - -from commands.base import Command, CommandGroup -from commands.flows.add_database import AddDatabaseFlow -from core.errors import ValidationError -from engines.registry import EngineRegistry -from services.docker import DockerRunner -from services.ports import PortAllocator -from services.project import AgentProject -from services.renderer import ComposeRenderer, WriteReport -from services.telemetry import Telemetry -from services.templates import TemplateRepository -from ui import UI - -NameArg = Annotated[Path, typer.Argument(help="Agent folder")] - - -def report_write(ui: UI, report: WriteReport) -> None: - if report.backed_up: - ui.warning(f"Legacy compose backed up to {report.backed_up.name}. Manual edits belong in docker-compose.override.yml.") - - -class _DbCommand(Command): - panel = "Configuration" - no_args_is_help = True - - def __init__(self, ui, telemetry, engines: EngineRegistry, ports: PortAllocator, templates: TemplateRepository, renderer: ComposeRenderer, docker: DockerRunner) -> None: - super().__init__(ui, telemetry) - self.engines, self.ports, self.templates, self.renderer, self.docker = engines, ports, templates, renderer, docker - - def render_and_write(self, project: AgentProject) -> None: - with self.ui.status("Rendering configuration..."): - result = self.renderer.render_agent(project) - project.save_state() - report = result.write(project.path) - report_write(self.ui, report) - - -class DbAddCommand(_DbCommand): - name, help = "add", "Add a database to an agent." - - def run( - self, - name: NameArg, - engine: Annotated[str | None, typer.Option("--engine", "-e", help="Database engine")] = None, - mode: Annotated[str | None, typer.Option("--mode", help="new (container) or existing")] = None, - auth: Annotated[bool | None, typer.Option("--auth/--no-auth", help="Auth variant for mongodb/redis/valkey")] = None, - label: Annotated[str | None, typer.Option("--label", help="Display name")] = None, - host: Annotated[str | None, typer.Option("--host")] = None, - port: Annotated[int | None, typer.Option("--port")] = None, - database: Annotated[str | None, typer.Option("--database")] = None, - user: Annotated[str | None, typer.Option("--user")] = None, - password: Annotated[str | None, typer.Option("--password", help="Prefer --password-stdin")] = None, - password_stdin: Annotated[bool, typer.Option("--password-stdin", help="Read password from stdin")] = False, - path: Annotated[str | None, typer.Option("--path", help="SQLite file path (existing)")] = None, - db_name: Annotated[str | None, typer.Option("--name", help="SQLite file name (new)")] = None, - volume: Annotated[str | None, typer.Option("--volume", help="Docker volume name")] = None, - container: Annotated[str | None, typer.Option("--container", help="Container to restart after restore")] = None, - option: Annotated[list[str] | None, typer.Option("--option", "-o", help="Engine option KEY=VALUE (repeatable)")] = None, - ) -> None: - if password_stdin: - import sys - - password = sys.stdin.readline().rstrip("\n") - elif password is not None: - self.ui.warning("--password is visible in shell history; prefer --password-stdin.") - - project_path = self.require_project_dir(name) - self.templates.resolve() - project = AgentProject.load(project_path) - - flow = AddDatabaseFlow(self.ui, self.engines, self.ports) - values = { - "engine": engine, "mode": mode, "auth": auth, "label": label, "host": host, "port": port, - "database": database, "username": user, "password": password, "path": path, "name": db_name, - "volume": volume, "container": container, "options": flow.parse_options(option), - } - spec, eng = flow.collect(values) - flow.apply(project, spec, eng) - self.render_and_write(project) - - self.ui.success(f"Added {eng.display} database '{spec.name}' ({eng.describe(spec)}).") - self.ui.info(f"Restart the agent to apply changes: portabase restart {project_path.name}") - - -class DbRemoveCommand(_DbCommand): - name, help = "remove", "Remove a database from an agent." - - def run( - self, - name: NameArg, - target: Annotated[str | None, typer.Option("--id", "--name", "-i", help="Database id (or prefix) or display name")] = None, - purge_volume: Annotated[bool, typer.Option("--purge-volume", help="Also delete the Docker volume of a managed database")] = False, - yes: Annotated[bool, typer.Option("--yes", "-y", help="Skip confirmation")] = False, - ) -> None: - project_path = self.require_project_dir(name) - self.templates.resolve() - project = AgentProject.load(project_path) - if not project.databases: - self.ui.warning("No databases to remove.") - return - - if target is None: - choices = [f"{d.name} ({d.engine}) [{d.id[:8]}]" for d in project.databases] - picked = self.ui.form().choice("Which database to remove?", choices, name="id") - spec = project.databases[choices.index(picked)] - else: - spec = project.find(target) - engine = self.engines.get(spec.engine) - - if not yes: - extra = " and its Docker volume" if (purge_volume and spec.managed) else "" - self.confirm_or_abort(f"Remove '{spec.name}' ({engine.describe(spec)}){extra}?", default=False) - - project.remove(spec, engine) - self.render_and_write(project) - self.ui.success(f"Removed {spec.name}") - - if spec.managed: - volume_name = f"{self.docker.project_name(project_path)}_{spec.host}-data" - if purge_volume: - self.require_docker(self.docker) - removed = self.docker.remove_volume(volume_name) - self.ui.success(f"Deleted volume {volume_name}" if removed else f"Volume {volume_name} did not exist") - else: - self.ui.info(f"Data volume kept: {volume_name}. Delete it with: docker volume rm {volume_name}") - self.ui.info(f"Restart the agent to apply changes: portabase restart {project_path.name}") - - -class DbListCommand(_DbCommand): - name, help = "list", "List the databases of an agent." - - def run(self, name: NameArg) -> None: - project = AgentProject.load(self.require_project_dir(name)) - if not project.databases: - self.ui.warning("No databases configured.") - return - rows = [] - for d in project.databases: - engine = self.engines.get(d.engine) - opts = ", ".join(f"{k}={v}" for k, v in engine.non_default_options(d).items()) - rows.append([d.name, d.database or "", d.engine, engine.describe(d), d.username or "" if d.engine not in ("sqlite", "docker-volume") else "N/A", opts, d.id[:8] + "..."]) - self.ui.table(["Display Name", "Database", "Type", "Host:Port", "User", "Options", "ID"], rows, title=f"Databases for {project.path.name}") - - -class DbCommands(CommandGroup): - name, help, panel = "db", "Manage the databases of an agent.", "Configuration" - - def __init__(self, ui: UI, telemetry: Telemetry, engines: EngineRegistry, ports: PortAllocator, templates: TemplateRepository, renderer: ComposeRenderer, docker: DockerRunner) -> None: - super().__init__(ui, telemetry) - self._deps = (ui, telemetry, engines, ports, templates, renderer, docker) - - @property - def commands(self) -> list[Command]: - return [DbAddCommand(*self._deps), DbRemoveCommand(*self._deps), DbListCommand(*self._deps)] -``` - -Note : `sys` importé localement dans `run` pour `--password-stdin` ; déplacer en tête de module si ruff le demande. - -- [ ] **Step 2: Commit** (vérification à Task 9, une fois `main.py` câblé) - -```bash -git add commands/db.py -git commit -m "feat(commands): rewrite db add/remove/list on the declarative renderer" -``` - ---- - -### Task 7 : `commands/agent.py` et `commands/dashboard.py` - -**Files:** -- Modify: `commands/agent.py` (réécriture complète) -- Modify: `commands/dashboard.py` (réécriture complète) - -**Interfaces:** -- Produces: `AgentCommand(ui, telemetry, docker, templates, renderer, engines, ports)` ; `DashboardCommand(ui, telemetry, docker, templates, renderer, ports)`. - -- [ ] **Step 1: `commands/agent.py`** - -```python -"""portabase agent NAME — create an agent folder. Databases are added by db add (or the interactive loop).""" - -from __future__ import annotations - -from pathlib import Path -from typing import Annotated - -import typer - -from commands.base import Command -from commands.db import report_write -from commands.flows.add_database import AddDatabaseFlow -from core.errors import ValidationError -from core.utils import validate_edge_key -from engines.registry import EngineRegistry -from services.docker import DockerRunner -from services.ports import PortAllocator -from services.project import AgentProject -from services.renderer import ComposeRenderer -from services.telemetry import Telemetry -from services.templates import TemplateRepository -from ui import UI - -NETWORK = "portabase_network" - - -def _edge_key(value: str) -> str: - if not validate_edge_key(value): - raise ValidationError("Invalid Edge Key.", hint="Expected Base64 or JSON with serverUrl, agentId, masterKeyB64.") - return value - - -class AgentCommand(Command): - name, help, panel = "agent", "Create a new Portabase Agent instance.", "Creation" - no_args_is_help = True - - def __init__(self, ui: UI, telemetry: Telemetry, docker: DockerRunner, templates: TemplateRepository, renderer: ComposeRenderer, engines: EngineRegistry, ports: PortAllocator) -> None: - super().__init__(ui, telemetry) - self.docker, self.templates, self.renderer, self.engines, self.ports = docker, templates, renderer, engines, ports - - def run( - self, - name: Annotated[str, typer.Argument(help="Agent name (creates a folder)")], - key: Annotated[str | None, typer.Option("--key", "-k", help="Edge Key")] = None, - tz: Annotated[str | None, typer.Option("--tz", help="Timezone")] = None, - polling: Annotated[int | None, typer.Option("--polling", help="Polling frequency in seconds")] = None, - host_gateway: Annotated[bool | None, typer.Option("--host-gateway/--no-host-gateway", help="Map localhost to host-gateway")] = None, - start: Annotated[bool, typer.Option("--start", "-s", help="Start immediately")] = False, - force: Annotated[bool, typer.Option("--force", "-f", help="Overwrite an existing folder")] = False, - ) -> None: - self.ui.banner() - self.require_docker(self.docker) - self.docker.ensure_network(NETWORK) - self.templates.resolve() - - path = Path(name).resolve() - if path.exists() and not force: - self.ui.warning(f"Directory '{name}' already exists.") - self.confirm_or_abort("Overwrite?", default=False) - - form = self.ui.form() - env_vars = { - "EDGE_KEY": form.text("Edge Key", value=key, validator=_edge_key, name="key"), - "TZ": form.text("Timezone", value=tz, default="UTC", name="tz"), - "POLLING": str(form.integer("Polling frequency (seconds)", value=polling, default=5, name="polling")), - "LOG_LEVEL": "info", - } - gateway = form.confirm("Add extra_hosts mapping (localhost -> host-gateway)?", value=host_gateway, default=False, name="host_gateway") - - project = AgentProject.create(path, env_vars, host_gateway=gateway) - self._write(project) - self.ui.success(f"Agent '{name}' created in {path}") - - if not self.ui.non_interactive: - self.ui.section("Database Setup") - flow = AddDatabaseFlow(self.ui, self.engines, self.ports) - while self.ui.confirm("Add a database?", default=True): - spec, engine = flow.collect({}) - flow.apply(project, spec, engine) - self._write(project) - self.ui.success(f"Added {engine.display} '{spec.name}' ({engine.describe(spec)})") - else: - self.ui.hint(f"Add databases with: portabase db add {name} --engine postgresql --mode new") - - if start or (not self.ui.non_interactive and self.ui.confirm("Start agent now?", default=False)): - with self.ui.status("Starting agent..."): - self.docker.compose(path, ["up", "-d"]) - self.ui.success("Agent started.") - else: - self.ui.info(f"Run: portabase start {name}") - - def _write(self, project: AgentProject) -> None: - with self.ui.status("Rendering configuration..."): - result = self.renderer.render_agent(project) - project.save_state() - report = result.write(project.path) - report_write(self.ui, report) -``` - -- [ ] **Step 2: `commands/dashboard.py`** - -```python -"""portabase dashboard NAME — create a dashboard folder.""" - -from __future__ import annotations - -import secrets -from pathlib import Path -from typing import Annotated -from urllib.parse import quote - -import typer - -from commands.base import Command -from commands.db import report_write -from core.utils import generate_password, slugify_project_name -from services.docker import DockerRunner -from services.ports import PortAllocator -from services.project import DashboardProject -from services.renderer import ComposeRenderer -from services.telemetry import Telemetry -from services.templates import TemplateRepository -from ui import UI - -DB_MODES = ("external", "internal", "custom") -MODE_LABELS = { - "external": "Dedicated Docker Container (Recommended)", - "internal": "Embedded Database (In-container)", - "custom": "Custom/Existing Database", -} - - -class DashboardCommand(Command): - name, help, panel = "dashboard", "Create a new Portabase Dashboard instance.", "Creation" - no_args_is_help = True - - def __init__(self, ui: UI, telemetry: Telemetry, docker: DockerRunner, templates: TemplateRepository, renderer: ComposeRenderer, ports: PortAllocator) -> None: - super().__init__(ui, telemetry) - self.docker, self.templates, self.renderer, self.ports = docker, templates, renderer, ports - - def run( - self, - name: Annotated[str, typer.Argument(help="Dashboard name (creates a folder)")], - port: Annotated[int | None, typer.Option("--port", help="Web port")] = None, - db_mode: Annotated[str | None, typer.Option("--db-mode", help="external | internal | custom")] = None, - db_host: Annotated[str | None, typer.Option("--db-host")] = None, - db_port: Annotated[int | None, typer.Option("--db-port")] = None, - db_name: Annotated[str | None, typer.Option("--db-name")] = None, - db_user: Annotated[str | None, typer.Option("--db-user")] = None, - db_password_stdin: Annotated[bool, typer.Option("--db-password-stdin", help="Read the custom DB password from stdin")] = False, - tz: Annotated[str | None, typer.Option("--tz")] = None, - start: Annotated[bool, typer.Option("--start", "-s")] = False, - force: Annotated[bool, typer.Option("--force", "-f")] = False, - yes: Annotated[bool, typer.Option("--yes", "-y", help="Skip the configuration confirmation")] = False, - ) -> None: - self.ui.banner() - self.require_docker(self.docker) - self.templates.resolve() - - path = Path(name).resolve() - if path.exists() and not force: - self.ui.warning(f"Directory '{name}' already exists.") - self.confirm_or_abort("Overwrite?", default=False) - - form = self.ui.form() - web_port = form.integer("Web Port", value=port, default=8887, name="port") - mode = form.choice("Database Setup", list(DB_MODES), value=db_mode, default="external", name="db_mode") - project_name = slugify_project_name(path.name) - - env_vars = { - "HOST_PORT": str(web_port), - "PROJECT_SECRET": secrets.token_hex(32), - "PROJECT_URL": f"http://localhost:{web_port}", - "PROJECT_NAME": project_name, - "TZ": form.text("Timezone", value=tz, default="Europe/Paris", name="tz"), - "LOG_LEVEL": "info", - } - rows = [("Dashboard Name", name), ("Path", str(path)), ("Access URL", env_vars["PROJECT_URL"]), ("Database Setup", MODE_LABELS[mode])] - - if mode == "external": - pg_pass, pg_port = generate_password(16), self.ports.free() - env_vars.update(self._pg_env("portabase", "portabase", pg_pass, "db", 5432, pg_port)) - rows.append(("Internal Port", str(pg_port))) - elif mode == "custom": - self.ui.info("External Database Configuration") - host = form.text("Host", value=db_host, default="localhost", name="db_host") - dport = form.integer("Port", value=db_port, default=5432, name="db_port") - dbname = form.text("Database Name", value=db_name, default="portabase", name="db_name") - user = form.text("Username", value=db_user, name="db_user") - if db_password_stdin: - import sys - - password = sys.stdin.readline().rstrip("\n") - else: - password = form.secret("Password", name="db_password") - env_vars.update(self._pg_env(dbname, user, password, host, dport, dport)) - rows += [("DB Host", host), ("DB Name", dbname), ("Connection URL", env_vars["DATABASE_URL"])] - - rows.append(("Files to Create", "docker-compose.yml, .env")) - self.ui.summary(rows, title="SUMMARY") - if not yes: - self.confirm_or_abort("Apply this configuration and generate files?", default=True) - - project = DashboardProject.create(path, env_vars) - with self.ui.status("Rendering configuration..."): - result = self.renderer.render_dashboard(project) - project.save_state() - report = result.write(path) - report_write(self.ui, report) - self.ui.success(f"Dashboard '{name}' created in {path}") - - if start or (not self.ui.non_interactive and self.ui.confirm("Start dashboard now?", default=False)): - with self.ui.status("Starting..."): - self.docker.compose(path, ["up", "-d"]) - self.ui.success(f"Live at: {env_vars['PROJECT_URL']}") - else: - self.ui.info(f"Run: portabase start {name}") - - @staticmethod - def _pg_env(db: str, user: str, password: str, host: str, port: int, host_port: int) -> dict[str, str]: - return { - "POSTGRES_DB": db, - "POSTGRES_USER": user, - "POSTGRES_PASSWORD": password, - "POSTGRES_HOST": host, - "DATABASE_URL": f"postgresql://{quote(user, safe='')}:{quote(password, safe='')}@{host}:{port}/{db}?schema=public", - "PG_PORT": str(host_port), - } -``` - -`--yes` en non-interactif : `confirm_or_abort(default=True)` renvoie `True` sans prompt, donc `--yes` n'est nécessaire que pour sauter l'affichage ; conservé pour la lisibilité des scripts. - -- [ ] **Step 3: Commit** - -```bash -git add commands/agent.py commands/dashboard.py -git commit -m "feat(commands): rewrite agent and dashboard on the declarative renderer" -``` - ---- - -### Task 8 : `commands/build.py` - -**Files:** -- Create: `commands/build.py` - -**Interfaces:** -- Produces: `BuildCommand(ui, telemetry, templates, renderer)`. - -- [ ] **Step 1: Écrire le module** - -```python -"""portabase build PATH — re-render compose from state. Also the legacy migration entry point.""" - -from __future__ import annotations - -from pathlib import Path -from typing import Annotated - -import typer - -from commands.base import Command -from commands.db import report_write -from core.errors import ValidationError -from services.project import COMPOSE_FILE, DATABASES_FILE, ENV_FILE, AgentProject, DashboardProject, detect_kind -from services.renderer import ComposeRenderer, RenderResult -from services.telemetry import Telemetry -from services.templates import TemplateRepository -from ui import UI - - -class BuildCommand(Command): - name, help, panel = "build", "Re-render docker-compose.yml from the component's configuration.", "Configuration" - no_args_is_help = True - - def __init__(self, ui: UI, telemetry: Telemetry, templates: TemplateRepository, renderer: ComposeRenderer) -> None: - super().__init__(ui, telemetry) - self.templates, self.renderer = templates, renderer - - def run( - self, - path: Annotated[Path, typer.Argument(help="Component folder")], - diff: Annotated[bool, typer.Option("--diff", help="Show the diff, write nothing")] = False, - stdout: Annotated[bool, typer.Option("--stdout", help="Print the compose, write nothing")] = False, - inline_env: Annotated[bool, typer.Option("--inline-env", help="Substitute values instead of ${VAR} references")] = False, - output: Annotated[Path | None, typer.Option("--output", "-o", help="Write files to another directory")] = None, - ) -> None: - if sum([diff, stdout, output is not None]) > 1: - raise ValidationError("Use only one of --diff, --stdout, --output.") - path = self.require_project_dir(path) - self.templates.resolve() - kind = detect_kind(path) - - if kind == "agent": - project = AgentProject.load(path) - result: RenderResult = self.renderer.render_agent(project, inline=inline_env) - else: - project = DashboardProject.load(path) - result = self.renderer.render_dashboard(project, inline=inline_env) - result.validate() - - if inline_env and not stdout: - self.ui.warning("--inline-env writes secrets in clear text into the compose file.") - - if stdout: - self.ui.console.print(result.compose, end="", markup=False, highlight=False) - return - if diff: - self.ui.diff(result.diff_against(path)) - return - - target = (output or path).resolve() - if output is not None: - target.mkdir(parents=True, exist_ok=True) - (target / ENV_FILE).write_text((path / ENV_FILE).read_text(encoding="utf-8"), encoding="utf-8") - report = result.write(target) - report_write(self.ui, report) - self.ui.success(f"Rendered {', '.join(p.name for p in report.wrote)} in {target}") - if kind == "agent" and output is None: - self.ui.info(f"Restart to apply: portabase restart {path.name}") -``` - -`--stdout` imprime via `console.print(markup=False)` pour qu'aucun `[x]` du compose ne soit interprété comme balise Rich. Pour un pipe propre, `main.py` doit **ne pas** afficher la notification de mise à jour quand `--stdout` est présent (déjà géré : non-interactif ou stdin non-TTY ; ajouter `"--stdout" in sys.argv` à `_notify_update` par sécurité, Task 9). - -- [ ] **Step 2: Commit** - -```bash -git add commands/build.py -git commit -m "feat(commands): add build command (re-render, --diff, --stdout, --inline-env, --output)" -``` - ---- - -### Task 8b : `commands/decrypt.py` en classe - -**Files:** -- Modify: `core/crypto.py` (`DecryptionError` hérite de `PortabaseError`) -- Modify: `commands/decrypt.py` (réécriture) - -**Interfaces:** -- Produces: `DecryptCommand(ui, telemetry)` ; `core.crypto.DecryptionError(PortabaseError)` avec `code = "E_CRYPTO"`, `exit_code = 8`. - -- [ ] **Step 1: `core/crypto.py`** - -Remplacer la définition de `DecryptionError` par : - -```python -from core.errors import PortabaseError - - -class DecryptionError(PortabaseError): - """Raised when a ``.enc`` file cannot be decrypted.""" - - code = "E_CRYPTO" - exit_code = 8 -``` - -Le reste du module (fonctions pures de déchiffrement) est inchangé. - -- [ ] **Step 2: `commands/decrypt.py`** - -```python -"""portabase decrypt INPUT [OUTPUT] — decrypt .enc backups (file or folder).""" - -from __future__ import annotations - -from pathlib import Path -from typing import Annotated - -import typer - -from commands.base import Command -from core.crypto import ( - ENC_SUFFIX, - DecryptionError, - decrypt_enc_file, - default_output_for, - load_master_key, -) -from core.errors import ConfigError, ValidationError - - -def _looks_like_dir(path: Path) -> bool: - if path.exists(): - return path.is_dir() - return str(path).endswith(("/", "\\")) or path.suffix == "" - - -class DecryptCommand(Command): - name, help, panel = "decrypt", "Decrypt Portabase .enc backup files (single file or folder).", "Configuration" - no_args_is_help = True - - def run( - self, - input_path: Annotated[Path, typer.Argument(help="A .enc file, or a folder containing .enc files.")], - output_path: Annotated[Path | None, typer.Argument(help="Output file or folder (must match the input type). Defaults to the input directory.")] = None, - key: Annotated[Path | None, typer.Option("--key", "-k", help="Master key file. Defaults to ./master_key.bin")] = None, - ) -> None: - input_path = input_path.resolve() - if not input_path.exists(): - raise ConfigError(f"Input path not found: {input_path}") - master_key = load_master_key(key.resolve() if key else None) # raises DecryptionError - if input_path.is_dir(): - self._folder(input_path, output_path, master_key) - else: - self._single(input_path, output_path, master_key) - - def _single(self, enc_path: Path, output_path: Path | None, master_key: bytes) -> None: - if enc_path.suffix != ENC_SUFFIX: - self.ui.warning(f"{enc_path.name} does not end with {ENC_SUFFIX}; decrypting anyway.") - if output_path is None: - out = enc_path.parent / default_output_for(enc_path) - elif _looks_like_dir(output_path): - out = output_path.resolve() / default_output_for(enc_path) - else: - out = output_path.resolve() - try: - decrypt_enc_file(enc_path, out, master_key) - except OSError as e: - raise DecryptionError(f"I/O error on {enc_path.name}: {e}", cause=e) from e - self.ui.success(f"Decrypted {enc_path.name} → {out}") - - def _folder(self, in_dir: Path, output_path: Path | None, master_key: bytes) -> None: - enc_files = sorted(p for p in in_dir.iterdir() if p.is_file() and p.suffix == ENC_SUFFIX) - if not enc_files: - self.ui.warning(f"No {ENC_SUFFIX} files found in {in_dir}.") - return - if output_path is None: - out_dir = in_dir - elif _looks_like_dir(output_path): - out_dir = output_path.resolve() - else: - raise ValidationError("Input is a folder, so the output must be a folder too.") - out_dir.mkdir(parents=True, exist_ok=True) - - failures: list[tuple[str, str]] = [] - with self.ui.status(f"Decrypting {len(enc_files)} file(s)..."): - for enc_path in enc_files: - out = out_dir / default_output_for(enc_path) - try: - decrypt_enc_file(enc_path, out, master_key) - except (DecryptionError, OSError) as e: - failures.append((enc_path.name, str(e))) - continue - succeeded = len(enc_files) - len(failures) - self.ui.info(f"Done: {succeeded} succeeded, {len(failures)} failed of {len(enc_files)} file(s).") - if failures: - for name, reason in failures: - self.ui.print(f" [danger]•[/danger] {name}: {reason}") - raise DecryptionError(f"{len(failures)} file(s) failed to decrypt.") -``` - -Différence assumée : les succès individuels ne sont plus imprimés un par un sous le spinner (un `console.print` sous `status` est autorisé mais bruyant) ; le résumé final reste. `main.py` remplace le `LegacyCommand` decrypt par `DecryptCommand(ui, telemetry)` (Task 9). - -- [ ] **Step 3: Vérifier** - -Run: `uv run python main.py decrypt /tmp/nope; echo "exit=$?"` → `E_CONFIG`, exit 3. `uv run python main.py decrypt . ; echo "exit=$?"` sans `master_key.bin` → `E_CRYPTO`, exit 8. Avec un vrai `.enc` et sa clé si disponible : déchiffrement identique à 26.08.12. - -- [ ] **Step 4: Commit** - -```bash -git add core/crypto.py commands/decrypt.py -git commit -m "refactor(commands): rewrite decrypt as a Command class" -``` - ---- - -### Task 9 : Câblage final, suppression du legacy - -**Files:** -- Modify: `main.py` -- Modify: `commands/base.py` (retirer `LegacyCommand`) -- Delete: `core/network.py`, `core/docker.py`, `templates/compose.py`, `templates/__init__.py`, `templates/agent.yml`, `templates/dashboard.yml` -- Modify: `core/config.py` (retirer les fonctions legacy, garder `GlobalConfig`, `TEMPLATE_BASE_URL`, `GLOBAL_CONFIG_DIR/FILE`) -- Modify: `core/utils.py` (garder uniquement `generate_password`, `slugify_project_name`, `validate_edge_key`, et l'import `current_version` re-exporté peut disparaître) -- Modify: `pyproject.toml` (retirer `per-file-ignores`) -- Modify: `.github/workflows/templates-upload.yml` (retirer l'étape `latest`) -- Modify: `scripts/render_check.py` - -- [ ] **Step 1: `main.py` — remplacer les `LegacyCommand` et `legacy_db`** - -Imports à remplacer : - -```python -from commands.agent import AgentCommand -from commands.build import BuildCommand -from commands.dashboard import DashboardCommand -from commands.db import DbCommands -from commands.decrypt import DecryptCommand -from engines import registry as engine_registry -from services.ports import PortAllocator -from services.renderer import ComposeRenderer -from services.templates import TemplateRepository -``` - -(supprimer `from commands import agent as legacy_agent`, `dashboard as legacy_dashboard`, `db as legacy_db`, `decrypt as legacy_decrypt`, `from commands.base import LegacyCommand`.) - -Dans `build_app`, après `updater = Updater(http, version)` : - -```python - templates = TemplateRepository.from_environment(http, config) - ports = PortAllocator() - renderer = ComposeRenderer(templates, engine_registry, version) -``` - -Liste `commands` : - -```python - commands = [ - AgentCommand(ui, telemetry, docker, templates, renderer, engine_registry, ports), - DashboardCommand(ui, telemetry, docker, templates, renderer, ports), - StartCommand(ui, telemetry, docker), - StopCommand(ui, telemetry, docker), - RestartCommand(ui, telemetry, docker), - LogsCommand(ui, telemetry, docker), - UninstallCommand(ui, telemetry, docker), - BuildCommand(ui, telemetry, templates, renderer), - DecryptCommand(ui, telemetry), - UpdateCommand(ui, telemetry, checker, updater), - ] - for cmd in commands: - cmd.register(app) - DbCommands(ui, telemetry, engine_registry, ports, templates, renderer, docker).register(app) - ConfigCommands(ui, telemetry, config).register(app) -``` - -Dans `_notify_update`, condition étendue : `or "--stdout" in sys.argv`. - -Retirer du catcher les branches `click.exceptions.Exit` et `click.exceptions.Abort` ? **Non** : `--help` lève toujours `Exit(0)` et `version_callback` lève `typer.Exit()`. Garder `Exit` ; retirer `Abort` (plus de `typer.confirm`). - -- [ ] **Step 2: Nettoyage** - -```bash -git rm core/network.py core/docker.py templates/compose.py templates/__init__.py templates/agent.yml templates/dashboard.yml -``` - -`commands/base.py` : supprimer la classe `LegacyCommand` et l'import `Callable` s'il devient inutilisé. - -`core/config.py` : ne garder que les constantes (`TEMPLATE_BASE_URL`, `GLOBAL_CONFIG_DIR`, `GLOBAL_CONFIG_FILE`), les imports nécessaires et `GlobalConfig`. Supprimer `write_file`, `write_env_file`, `load_global_config`, `save_global_config`, `get_config_value`, `set_config_value`, `load_db_config`, `save_db_config`, `add_db_to_json`. - -`core/utils.py` : ne garder que `generate_password`, `slugify_project_name`, `validate_edge_key` et leurs imports (`base64`, `binascii`, `json`, `re`, `secrets`, `string`). Supprimer `questionary_style`, `custom_theme`, `HINTS`, `get_random_hint`, `console`, `BANNER`, `print_banner`, `get_free_port`, `start_docker`, `check_system`, `validate_work_dir`, le re-export `current_version`. - -Vérifier qu'aucune référence ne subsiste : -Run: `grep -rn "core.network\|core.docker\|templates.compose\|LegacyCommand\|get_random_hint\|print_banner\|check_system\|validate_work_dir\|get_free_port\|load_db_config\|add_db_to_json\|write_env_file\|get_config_value" --include=*.py . | grep -v ".venv"` -Expected: aucune sortie. - -`pyproject.toml` : supprimer entièrement `[tool.ruff.lint.per-file-ignores]` ; retirer `"templates"` de `known-first-party`. - -`.github/workflows/templates-upload.yml` : supprimer l'étape `Upload latest templates (stable only, legacy fallback)` — **seulement si** plus aucune version legacy n'est supportée. Sinon la garder ; par défaut la garder et ouvrir une issue « retirer latest/ ». Décision utilisateur. - -- [ ] **Step 3: `scripts/render_check.py` via `ComposeRenderer`** - -Remplacer `agent_cases`, `render_agent`, `dashboard_cases` par une construction de projets en mémoire : - -```python -from core.specs import DatabaseSpec # noqa: E402 -from services.envfile import EnvFile # noqa: E402 -from services.project import AgentProject, DashboardProject # noqa: E402 -from services.renderer import ComposeRenderer # noqa: E402 - - -def agent_project(tmp: Path, specs: list, engines_for, host_gateway=False, sqlite=False) -> AgentProject: - env = EnvFile(tmp / ".env") - env.merge({"TZ": "UTC", "EDGE_KEY": "x", "LOG_LEVEL": "info", "POLLING": "5"}) - project = AgentProject(tmp, env, [], host_gateway) - for spec, engine in zip(specs, engines_for): - project.add(spec, engine) - if sqlite: - sq = registry.get("sqlite") - project.add(sq.generate(auth=False, ports=FixedPortAllocator(), answers={"name": "x"}), sq) - return project - - -def agent_cases(renderer: ComposeRenderer) -> list[tuple[str, str, str]]: - ports = FixedPortAllocator() - tmp = Path(tempfile.mkdtemp()) - cases = [] - empty = agent_project(tmp, [], []) - cases.append(("agent/empty", renderer.render_agent(empty).compose, env_text(empty))) - toggles = agent_project(tmp, [], [], host_gateway=True, sqlite=True) - dv = registry.get("docker-volume") - toggles.add(dv.from_existing({"volume": "v"}), dv) - cases.append(("agent/toggles", renderer.render_agent(toggles).compose, env_text(toggles))) - all_specs, all_engines = [], [] - for engine in registry: - if engine.template is None: - continue - for auth in (True, False) if engine.auth_variants else (True,): - spec = engine.generate(auth=auth, ports=ports, answers={}) - one = agent_project(tmp, [spec], [engine]) - label = f"agent/{engine.key}" + ("/auth" if auth else "/noauth" if engine.auth_variants else "") - cases.append((label, renderer.render_agent(one).compose, env_text(one))) - all_specs.append(spec); all_engines.append(engine) - everything = agent_project(tmp, all_specs, all_engines, host_gateway=True, sqlite=True) - cases.append(("agent/all", renderer.render_agent(everything).compose, env_text(everything))) - return cases - - -def env_text(project) -> str: - return "".join(f'{k}="{v}"\n' for k, v in project.env.as_dict().items()) - - -def dashboard_cases(renderer: ComposeRenderer) -> list[tuple[str, str, str]]: - tmp = Path(tempfile.mkdtemp()) - base = {"HOST_PORT": "8887", "PROJECT_SECRET": "s", "PROJECT_URL": "http://localhost:8887", "PROJECT_NAME": "pb", "TZ": "UTC", "LOG_LEVEL": "info"} - pg = {"POSTGRES_DB": "pb", "POSTGRES_USER": "pb", "POSTGRES_PASSWORD": "p", "PG_PORT": "5433", "DATABASE_URL": "postgresql://pb:p@db:5432/pb"} - variants = {"external": {**base, **pg, "POSTGRES_HOST": "db"}, "internal": base, "custom": {**base, **pg, "POSTGRES_HOST": "remote"}} - cases = [] - for mode, vars_ in variants.items(): - env = EnvFile(tmp / f".env.{mode}"); env.merge(vars_) - project = DashboardProject(tmp, env) - assert project.db_mode == mode, (project.db_mode, mode) - cases.append((f"dashboard/{mode}", renderer.render_dashboard(project).compose, env_text(project))) - return cases -``` - -et dans `main()` : - -```python - repo = TemplateRepository(HttpClient(), GlobalConfig().cache_dir, "local", local_dir=Path(args.templates)) - renderer = ComposeRenderer(repo, registry, "render-check") - try: - engines_check(repo) - for label, compose, env_text_ in agent_cases(renderer) + dashboard_cases(renderer): - validate(label, compose, env_text_, use_compose) -``` - -Supprimer `AGENT_GLOBALS`, `AGENT_ENV`, `DASHBOARD_VARS`, `DASHBOARD_ENV`, `render_agent` devenus inutiles. Ajouter `import tempfile` déjà présent. - -Run: `uv run python scripts/render_check.py` -Expected: mêmes cas qu'au Plan 3, tous `ok`, dont `agent/toggles` avec socket + mount + extra_hosts. - -- [ ] **Step 4: Lint** - -Run: `uv run ruff check . && uv run ruff format --check .` -Expected: passe **sans aucune** exception par fichier. - -- [ ] **Step 5: Commit** - -```bash -git add -A -git commit -m "refactor: remove legacy commands and helpers; wire agent, dashboard, db, build on the renderer" -``` - ---- - -### Task 10 : Vérification de bout en bout - -Docker requis. `K` = edge key de test : `K=$(printf '{"serverUrl":"http://x","agentId":"a","masterKeyB64":"k"}' | base64 -w0)`. - -- [ ] **Step 1: Agent non-interactif + db add** - -```bash -cd /tmp && rm -rf ni-agent -M="uv --directory /home/soluce/Documents/PROJETS/Portabase/cli run python /home/soluce/Documents/PROJETS/Portabase/cli/main.py" -$M --non-interactive agent ni-agent --key "$K" --tz Europe/Paris --polling 7 --host-gateway; echo "exit=$?" -$M --non-interactive db add ni-agent --engine postgresql --mode new -o clean_mode=drop_schemas; echo "exit=$?" -$M --non-interactive db add ni-agent --engine redis --mode new --no-auth; echo "exit=$?" -printf 'secret\n' | $M --non-interactive db add ni-agent --engine mysql --mode existing --label Prod --host db.example --port 3306 --database app --user app --password-stdin; echo "exit=$?" -$M --non-interactive db add ni-agent --engine sqlite --mode new --name local; echo "exit=$?" -$M --non-interactive db add ni-agent --engine docker-volume --volume some_vol; echo "exit=$?" -$M db list ni-agent -cat ni-agent/.env; head -3 ni-agent/docker-compose.yml; python3 -c "import json; print([ (d['type'], d.get('options')) for d in json.load(open('ni-agent/databases.json'))['databases']])" -(cd ni-agent && docker compose config --quiet && echo "compose config OK") -``` -Expected: 6 × `exit=0` ; table à 6 lignes ; `.env` avec `TZ="Europe/Paris"`, `POLLING="7"`, `DB_PG_*` (4 vars), `DB_REDIS_*_PORT` seul ; en-tête `# Generated by Portabase CLI` ; options `{'clean_mode': 'drop_schemas'}` sur postgres, `None` ailleurs ; `compose config OK`. - -- [ ] **Step 2: Erreurs non-interactives** - -```bash -$M --non-interactive db add ni-agent --engine postgresql --mode existing; echo "exit=$?" -$M --non-interactive db add ni-agent --engine redis --mode new --host x; echo "exit=$?" -$M --non-interactive db add ni-agent --engine postgresql --mode new -o nope=1; echo "exit=$?" -$M --non-interactive agent ni-agent --key "$K"; echo "exit=$?" -``` -Expected: `Missing --host` exit 2 ; `not applicable` exit 2 ; `Unknown option(s)` exit 2 ; `Directory 'ni-agent' already exists` → confirm par défaut False → `Cancelled.` exit 130. - -- [ ] **Step 3: db remove, build** - -```bash -ID=$(python3 -c "import json; print([d for d in json.load(open('ni-agent/databases.json'))['databases'] if d['type']=='redis'][0]['generated_id'])") -$M db remove ni-agent --id "$ID" --yes; echo "exit=$?" -grep -c "db-redis" ni-agent/docker-compose.yml ni-agent/.env -$M build ni-agent --diff -$M build ni-agent --stdout --inline-env | head -20 -$M build ni-agent --output /tmp/ni-export && ls /tmp/ni-export -``` -Expected: `Removed`, `0` occurrences de `db-redis` dans les deux fichiers, diff « No changes. », compose inline avec valeurs littérales (pas de `${`), export contenant `docker-compose.yml`, `.env`, `databases.json`. - -- [ ] **Step 4: Lifecycle réel** - -```bash -$M start ni-agent && sleep 5 && $M logs ni-agent --no-follow | tail -5 && $M stop ni-agent && $M uninstall ni-agent --force -``` -Expected: services `agent` et `db-pg-*` démarrent (`docker compose ps` pendant le `sleep` si besoin), puis arrêt et suppression. - -- [ ] **Step 5: Install legacy** - -Réutiliser `/tmp/legacy-agent` (Task 2 step 4 ; le recréer sinon). - -```bash -cp -r /tmp/legacy-agent /tmp/legacy-copy -$M build /tmp/legacy-copy --diff -$M db add /tmp/legacy-copy --engine valkey --mode new --auth --non-interactive; echo "exit=$?" -ls /tmp/legacy-copy; head -1 /tmp/legacy-copy/docker-compose.yml -(cd /tmp/legacy-copy && docker compose config --quiet && echo "compose config OK") -diff <(python3 -c "import json; print(sorted((d['type'], d.get('host')) for d in json.load(open('/tmp/legacy-agent/databases.json'))['databases']))") <(python3 -c "import json; print(sorted((d['type'], d.get('host')) for d in json.load(open('/tmp/legacy-copy/databases.json'))['databases'] if d['type']!='valkey'))") && echo "entries preserved" -$M start /tmp/legacy-copy && $M stop /tmp/legacy-copy && $M uninstall /tmp/legacy-copy --force -``` -Expected: diff montre en-tête + `restart:` sur redis ; `db add` exit 0 avec le warning `Legacy compose backed up to docker-compose.legacy.yml` ; `docker-compose.legacy.yml` présent ; en-tête `# Generated` ; `compose config OK` ; `entries preserved` ; les anciens services démarrent avec leurs volumes existants (noms de service inchangés). - -- [ ] **Step 6: Dashboard** - -```bash -cd /tmp && rm -rf ni-dash -$M --non-interactive dashboard ni-dash --port 8899 --db-mode external --yes; echo "exit=$?" -(cd ni-dash && docker compose config --quiet && echo OK && grep -c "db:" docker-compose.yml) -$M --non-interactive dashboard ni-dash2 --port 8898 --db-mode internal --yes && (cd ni-dash2 && grep -c "postgres" docker-compose.yml) -printf 'pw\n' | $M --non-interactive dashboard ni-dash3 --port 8897 --db-mode custom --db-host pg.example --db-user u --db-password-stdin --yes && grep DATABASE_URL ni-dash3/.env -$M build ni-dash3 --diff -``` -Expected: external `OK 1` ; internal `0` ; custom `.env` avec `DATABASE_URL="postgresql://u:pw@pg.example:5432/portabase?schema=public"` ; diff vide. - -- [ ] **Step 7: Interactif** - -`$M agent int-agent` sans flags : bannière, prompts Edge Key / Timezone / Polling / extra_hosts, création, boucle « Add a database? » → ajouter un postgres (prompts options avec aide affichée) puis un redis (variante), répondre non, ne pas démarrer. Puis `$M db remove int-agent` avec sélection interactive. Ctrl-C au milieu d'un prompt → `Cancelled.` exit 130 sans traceback, fichiers cohérents (`docker compose config` passe). - -- [ ] **Step 8: CI `build-smoke`** - -Dans `ci.yml`, étape `Smoke` du job `build-smoke` : - -```yaml - - name: Smoke - env: - PORTABASE_TEMPLATES_DIR: ${{ github.workspace }}/templates - run: | - ./dist/portabase_smoke --version - cd "$(mktemp -d)" - K=$(printf '{"serverUrl":"http://x","agentId":"a","masterKeyB64":"k"}' | base64 -w0) - "$GITHUB_WORKSPACE/dist/portabase_smoke" --non-interactive agent smoke --key "$K" - "$GITHUB_WORKSPACE/dist/portabase_smoke" --non-interactive db add smoke --engine postgresql --mode new - "$GITHUB_WORKSPACE/dist/portabase_smoke" --non-interactive build smoke --diff - cd smoke && docker compose config --quiet -``` - -Le runner GitHub a Docker ; `agent` appelle `require_docker` + `ensure_network` → fonctionne. Si le daemon n'est pas disponible sur un runner donné, l'étape échoue explicitement (`E_DOCKER`) — c'est voulu. - -- [ ] **Step 9: README** - -Ajouter une section « Upgrading from 26.07 or earlier » : le premier `db add`/`build` re-génère `docker-compose.yml` (sauvegarde `docker-compose.legacy.yml`), les personnalisations vont dans `docker-compose.override.yml`, `portabase build --diff` montre les changements avant. - -- [ ] **Step 10: Commit, PR, rc** - -```bash -git add -A -git commit -m "ci: smoke-test agent creation and rendering with the built binary; document migration" -git checkout -b refactor/render-commands && git push -u origin refactor/render-commands -``` -PR « refactor: declarative rendering, flag-driven commands, build » → merge → Bump `26.09.0rc1` → tester le binaire rc sur une vraie install legacy → Bump `26.09.0` stable. - ---- - -## Self-review - -**Spec coverage :** -- §4.2 inventaire commandes : `agent` (flags, boucle interactive, aucune DB en non-interactif) ✔ ; `dashboard` (flags, modes, `--yes`) ✔ ; `db add` (tous flags + `-o`, rejet des flags non pertinents) ✔ ; `db remove` (`--id/--name`, `--purge-volume`, volume conservé par défaut) ✔ ; `db list` (options non-défaut) ✔ ; `build` (`--diff`, `--stdout`, `--inline-env`, `--output`) ✔ ; détection `kind` ✔ ; `back` supprimé ✔. -- §4.3 `AddDatabaseFlow.collect/apply`, séquence, usage par les deux commandes, rendu après chaque ajout ✔. -- §5.1 `DatabaseSpec.from_json` → `spec_from_entry` ; `AgentProject` (managed, socket, mounts, validate doublons) ✔ ; `DashboardProject.db_mode` ✔ ; `host` managé sans `_PORT` → traité externe (warning non émis : ajouter `ui.warning` dans `DbListCommand` si souhaité — non bloquant). -- §5.2 `EnvFile` ✔ (ordre, commentaires, quotes, merge, remove_prefix, atomique). -- §5.3 `ComposeFacts.host_gateway` liste/dict, jamais d'erreur ✔. -- §5.5 renderer, contexte, `*_var`, validation avant écriture, en-tête, `templates.resolve()` avant mutation ✔. -- §5.7 legacy : backup `.legacy.yml` une fois, `start/stop/logs` sans migration, `build --diff` ✔. -- §6.1 options : `-o`, validation clés, prompts avec `help`, projection non-défaut ✔. -- §7.2 `Summary` (masque), `DataTable`, `Diff` ✔. -- §10 F : `build-smoke` étendu, rc obligatoire ✔. - -**Placeholders :** aucun. - -**Cohérence :** `report_write` défini dans `commands/db.py`, importé par `agent`, `dashboard`, `build` ✔ ; `DbEngine.describe/label_default/non_default_options` utilisés ✔ ; `SqliteEngine.mount_for` (Plan 3) utilisé par `AgentProject.sqlite_mounts` ✔ ; `RenderResult.write/diff_against/validate` utilisés par les 4 commandes ✔ ; `DockerRunner.remove_volume` ajouté et utilisé ✔ ; `UI.summary/table/diff` ajoutés et utilisés ✔ ; `TemplateRepository.engine_template` (Plan 3) utilisé par `_service` ✔. - -**Écarts connus :** -- `templates-upload.yml` `latest/` : décision utilisateur (Task 9 step 2). -- Import local de `sys` dans deux `run` : ruff peut préférer un import de module ; déplacer. diff --git a/docs/superpowers/specs/2026-09-11-cli-refactor-design.md b/docs/superpowers/specs/2026-09-11-cli-refactor-design.md deleted file mode 100644 index cb62728..0000000 --- a/docs/superpowers/specs/2026-09-11-cli-refactor-design.md +++ /dev/null @@ -1,592 +0,0 @@ -# Refonte du CLI Portabase — Design - -Date : 2026-09-11 -Statut : validé en brainstorming, en attente de relecture avant plan d'implémentation. - -## 1. Objectifs et périmètre - -Refonte structurelle du CLI sans changement de dépendances majeures (Typer, Rich, questionary, requests, PyYAML conservés ; Jinja2 ajouté). - -Objectifs : - -- Supprimer la duplication entre `commands/agent.py` et `commands/db.py` (~500 lignes du wizard "ajouter une base" copiées). -- Remplacer la génération de `docker-compose.yml` par chirurgie texte (`.replace`, regex, ancres) par un rendu Jinja2 complet et déterministe. -- POO sur toute l'application : commandes, services, moteurs, composants UI en classes. Les utilitaires purs (`slugify`, `generate_password`, `validate_edge_key`) restent des fonctions. -- Chaque commande configurable intégralement par paramètres, sans mode interactif. -- Bibliothèque de composants `ui/` (approche shadcn : tokens, composants stateless, façade unique). -- Catcher d'erreurs unique avec hiérarchie d'exceptions et codes de sortie stables. -- Couche télémétrie prête pour OpenTelemetry, opt-in, sans dépendance immédiate. -- Plus d'auto-update : notification seulement, mise à jour manuelle. -- CI de PR (lint, Gitleaks, Plumber, validation des templates, build smoke), suppression du script `./release`. - -Hors périmètre de cette spec : - -- Tests automatisés (spec suivante ; la structure en tient compte : injection de dépendances, services sans I/O terminal, job `test` vide en CI). -- Sortie machine `--json`. -- Exporter OTel réel (le contrat est posé, l'implémentation viendra avec un endpoint). -- Fichier de spec déclaratif `build -f spec.yml`. -- Support Windows (inchangé : code présent, hors matrice de build). - -## 2. Décisions structurantes - -| Sujet | Décision | -|---|---| -| Layout | Plat, conservé (`main.py` racine, `--paths=.`). Nouveaux dossiers `services/`, `engines/`, `ui/`. | -| État d'une install | Aucun fichier d'état ajouté. Source de vérité = `.env` (variables runtime des conteneurs uniquement) + `databases.json` (contrat agent, inchangé) + lecture structurelle du compose existant pour le seul fait non dérivable (`host_gateway`). | -| Compose | Artefact dérivé, propriété du CLI, re-rendu intégralement à chaque commande mutante. Personnalisations utilisateur via `docker-compose.override.yml` (mécanisme Compose natif). | -| Templates | 100 % remote (S3), versionnés par version CLI exacte, manifest avec sha256, cache disque. Suppression du fallback `latest`. Source dans `templates/` à la racine du dépôt. | -| Création multi-DB | En non-interactif, `portabase agent` crée un agent sans base ; les bases s'ajoutent par `portabase db add` (un appel par base). En interactif, `agent` enchaîne sur une boucle « Add a database? » qui réutilise le même flux que `db add`. Pas de DSL `--db engine:opts`. | -| Options moteur | Flag générique répétable `-o/--option KEY=VALUE`, validé contre `DbEngine.option_fields()`. Pas de flag Typer par option. | -| Moteurs DB | Classes Python (`engines/`), registre à imports explicites. Pas de manifeste data-driven. | -| Input UI | questionary uniquement. `rich.prompt` et `typer.prompt` bannis (ruff). | -| Non-interactif | Flag `--non-interactive`, env `PORTABASE_NON_INTERACTIVE`, ou `stdin` non-TTY. Géré par `ui.Form`, pas par les commandes. | -| Erreurs | `PortabaseError` + sous-classes, codes de sortie distincts, un seul `try` dans `main.py`. `except:` nus interdits. | -| Télémétrie | Interface `Telemetry`, `NoopTelemetry` par défaut, opt-in via config globale, jamais de prompt. | -| Updater | Notification après la commande (cache 24 h, silencieux si hors ligne), `portabase update` manuel avec vérification de checksum. | -| Release | `bump.yml` (`workflow_dispatch`) remplace `./release`. Workflows de release sur tag inchangés. | -| Mot de passe | `generate_password` retire `$` et `` ` `` des symboles (cassent `--requirepass "${PASSWORD}"` via shell). Ne s'applique qu'aux nouvelles bases. | - -## 3. Structure des fichiers - -``` -cli/ -├── main.py # build_app(), catcher d'erreurs, codes de sortie -├── pyproject.toml # + jinja2 ; pyinstaller/ruff/pytest en groupe dev -│ -├── commands/ -│ ├── base.py # Command ABC, CommandGroup -│ ├── agent.py # AgentCommand -│ ├── dashboard.py # DashboardCommand -│ ├── build.py # BuildCommand -│ ├── lifecycle.py # Start/Stop/Restart/Logs/Uninstall (ex-common.py) -│ ├── db.py # DbCommands : add / remove / list -│ ├── config.py # ConfigCommands : get / set -│ ├── update.py # UpdateCommand -│ └── flows/ -│ └── add_database.py # AddDatabaseFlow : collecte + application, partagé par agent et db add -│ -├── services/ -│ ├── project.py # AgentProject, DashboardProject, DatabaseSpec, detect_kind() -│ ├── envfile.py # EnvFile -│ ├── compose_facts.py # ComposeFacts (lecture structurelle, jamais d'écriture) -│ ├── renderer.py # ComposeRenderer, RenderResult -│ ├── templates.py # TemplateRepository, Manifest -│ ├── docker.py # DockerRunner -│ ├── ports.py # PortAllocator -│ ├── http.py # HttpClient -│ ├── updater.py # UpdateChecker, Updater -│ └── telemetry.py # Telemetry ABC, NoopTelemetry, ConsoleTelemetry, TelemetryFactory -│ -├── engines/ -│ ├── __init__.py # registry = EngineRegistry([...]) — imports explicites -│ ├── base.py # DbEngine ABC, Field -│ ├── registry.py # EngineRegistry -│ ├── sql.py # StandardSqlEngine + Postgres/PostgresCluster/MySQL/MariaDB/MSSQL/Firebird -│ ├── redis.py # RedisEngine -│ ├── valkey.py # ValkeyEngine -│ ├── mongo.py # MongoEngine -│ ├── sqlite.py # SqliteEngine -│ └── docker_volume.py # DockerVolumeEngine -│ -├── ui/ -│ ├── __init__.py # façade UI -│ ├── theme.py # PALETTE → RICH_THEME + QUESTIONARY_STYLE -│ ├── form.py # Form (flag → prompt → défaut → erreur) -│ └── components/ -│ ├── base.py # Component(console) -│ ├── banner.py message.py section.py summary.py table.py -│ ├── status.py hints.py diff.py prompt.py -│ -├── core/ -│ ├── errors.py # PortabaseError + sous-classes -│ ├── config.py # GlobalConfig (~/.portabase/config.json) -│ ├── version.py # current_version() -│ └── utils.py # slugify, generate_password, validate_edge_key — fonctions pures -│ -├── templates/ # source des templates remote (assets, pas un package Python) -│ ├── agent.yml.j2 -│ ├── dashboard.yml.j2 -│ ├── engines.map.json # clé moteur → template (pour engines-check et manifest) -│ └── engines/ -│ ├── postgresql.yml.j2 mysql.yml.j2 mariadb.yml.j2 mssql.yml.j2 -│ ├── firebird.yml.j2 mongodb.yml.j2 redis.yml.j2 valkey.yml.j2 -│ -├── scripts/ -│ └── render_check.py # rend tous les templates avec fixtures, valide YAML + compose config -│ -├── .github/workflows/ -│ ├── ci.yml # PR : lint, render-check, engines-check, gitleaks, plumber, build-smoke, test -│ ├── bump.yml # workflow_dispatch : bump version + tag -│ ├── templates-hotfix.yml # workflow_dispatch : re-upload templates vers une version existante -│ ├── release.yml, release-candidate.yml, python.yml, github.yml # inchangés (hors durcissement) -│ └── templates-upload.yml # + génération manifest.json, source templates/ -│ -├── .gitleaks.toml -└── supprimés : release, templates/compose.py, templates/__init__.py, commands/common.py, - core/network.py, core/docker.py, .github/assets/templates/ -``` - -Règle de dépendance, descendante uniquement : - -- `commands` → `services`, `engines`, `ui`, `core` -- `services` → `engines`, `core` (jamais `ui` : un service lève, n'affiche rien) -- `engines` → `core` -- `ui` → `core` - -## 4. Commandes - -### 4.1 `Command` - -```python -class Command(ABC): - name: str - help: str - panel: str = "General" - - def __init__(self, ui: UI, telemetry: Telemetry): ... - def register(self, app: typer.Typer) -> None: - app.command(self.name, help=self.help, rich_help_panel=self.panel)(self.run) - - @abstractmethod - def run(self, *args, **kwargs) -> None: ... -``` - -`base.py` wrappe `run` dans `telemetry.span(f"command.{name}")`. Les dépendances (`DockerRunner`, `TemplateRepository`, `EngineRegistry`, `PortAllocator`) sont injectées par constructeur dans `main.build_app()`. - -Signatures Typer en `Annotated[...]`. Chaque option qui correspond à une question du wizard a une valeur par défaut `None` : présente → utilisée, absente → prompt (interactif) ou défaut/erreur (non-interactif). Une seule méthode `_collect()` par commande, aucun `if non_interactive` dans la logique métier. - -### 4.2 Inventaire - -| Commande | Options notables | Effet | -|---|---|---| -| `agent NAME` | `--key`, `--tz`, `--polling`, `--host-gateway/--no-host-gateway`, `--start`, `--force`, `--non-interactive` | crée le dossier, `.env`, `databases.json` vide, rend le compose. En interactif, enchaîne sur une boucle « Add a database? » (`AddDatabaseFlow`, rendu après chaque ajout). En non-interactif, ne crée aucune base. | -| `dashboard NAME` | `--port`, `--db-mode external\|internal\|custom`, `--db-host/--db-port/--db-name/--db-user/--db-password-stdin`, `--start`, `--force` | crée `.env`, rend le compose. | -| `db add NAME` | `--engine`, `--mode new\|existing`, `--auth/--no-auth`, `--name`, `--host`, `--port`, `--database`, `--user`, `--password`, `--password-stdin`, `--path`, `--volume`, `--container`, `--label`, `-o/--option KEY=VALUE` (répétable) | collecte via `AddDatabaseFlow` selon le moteur, mute `.env` + `databases.json`, re-rend. Flag ou option fourni mais non pertinent pour le moteur/mode → `ValidationError`. | -| `db remove NAME` | `--id` ou `--name`, `--purge-volume` | retire l'entrée, retire les variables `.env` du service, re-rend. Le volume Docker n'est supprimé que sur `--purge-volume`. | -| `db list NAME` | — | lecture seule. | -| `build PATH` | `--diff`, `--stdout`, `--inline-env`, `--output DIR` | re-rend depuis l'état. Sans option : écrit en place (= migration legacy). `--inline-env` substitue les valeurs au lieu de `${VAR}` avec avertissement secrets en clair. | -| `start/stop/restart/logs/uninstall PATH` | inchangées (`uninstall --force`) | n'utilisent pas le renderer, fonctionnent sur toute install. | -| `config get/set` | inchangées + clés `telemetry`, `telemetry_endpoint`, `channel` | config globale. | -| `update` | — | mise à jour manuelle avec vérification checksum. | -| `decrypt INPUT [OUTPUT]` | `--key` | déchiffre des sauvegardes `.enc` (ajouté en 26.08.12) ; `DecryptCommand`, `DecryptionError(PortabaseError)` code `E_CRYPTO` exit 8. `core/crypto.py` reste un module de fonctions pures. | - -Options globales : `--verbose`, `--debug`, `--no-color`, `--non-interactive`. Détection `kind` d'un dossier : `databases.json` présent → agent ; `PROJECT_SECRET` dans `.env` → dashboard. - -Le choix `back` dans les selects disparaît : interactif = Ctrl-C (`UserAbort`) ou entrée "cancel" en fin de liste. - -### 4.3 `AddDatabaseFlow` (`commands/flows/add_database.py`) - -Le wizard d'ajout de base est un objet réutilisable, pas une commande. C'est la duplication actuelle entre `agent.py` et `db.py` qui disparaît. - -```python -class AddDatabaseFlow: - def __init__(self, ui: UI, engines: EngineRegistry, ports: PortAllocator): ... - - def collect(self, values: dict) -> DatabaseSpec: - """values = flags parsés (engine, mode, auth, host…, options). - Champ manquant → prompt (interactif) / défaut / ValidationError (non-interactif).""" - - def apply(self, project: AgentProject, spec: DatabaseSpec) -> None: - """Mute project.env (variables du service si managed) et project.databases, en mémoire.""" -``` - -Séquence de `collect` : moteur (`--engine` ou select) → affiche `engine.warning` s'il existe → mode (`--mode` ou select ; sqlite et docker-volume n'ont pas de mode `existing`/`new` au sens service : sqlite distingue fichier créé vs chemin existant, docker-volume n'a qu'un mode) → variante auth si `engine.auth_variants` → `Form.collect(engine.fields_new() | fields_existing(), values)` → `Form.collect(engine.option_fields(), values["options"])` → `engine.generate(...)` ou construction depuis les réponses. - -Utilisation : - -- `DbAddCommand.run` : `AgentProject.load` → `templates.ensure()` → `flow.collect(flags)` → `flow.apply` → `renderer.render_agent` → `write`. -- `AgentCommand.run` (interactif seulement) : après le premier rendu, `while ui.confirm("Add a database?", default=True)` : `flow.collect({})` → `flow.apply` → rendu + écriture. Rendu après chaque ajout : un Ctrl-C au milieu laisse un état cohérent sur disque. - -`flows/` vit dans `commands/` parce qu'il prompte via `ui` ; il ne fait aucune I/O fichier (c'est `RenderResult.write` qui écrit). - -## 5. État, templates, rendu - -### 5.1 Modèle de données (`services/project.py`) - -Objets en mémoire construits depuis le disque, jamais persistés tels quels. - -```python -@dataclass(frozen=True) -class DatabaseSpec: - id: str; engine: str; name: str; managed: bool - host: str | None; port: int | None; database: str | None - username: str | None; password: str | None - path: str | None # sqlite - volume: str | None; container: str | None # docker-volume - options: dict - - @classmethod - def from_json(cls, raw: dict, env: EnvFile) -> "DatabaseSpec": ... - def to_json(self) -> dict: ... # format databases.json actuel, inchangé - @property - def env_prefix(self) -> str: ... # "db-pg-a1f2" → "DB_PG_A1F2" - - -@dataclass -class AgentProject: - path: Path; env: EnvFile; databases: list[DatabaseSpec]; host_gateway: bool - - @property - def needs_docker_socket(self) -> bool # une entrée docker-volume - @property - def managed(self) -> list[DatabaseSpec] - @property - def sqlite_mounts(self) -> list[tuple[str, str]] # database commence par /config/ → ./x:/config/x - - -@dataclass -class DashboardProject: - path: Path; env: EnvFile - @property - def db_mode(self) -> Literal["external", "custom", "internal"] - # POSTGRES_HOST absent → internal ; == "db" → external ; sinon custom -``` - -Détection `managed` : `.env` contient `{PREFIX}_PORT` pour ce `host` (toutes les bases `new` l'écrivent, aucune `existing`). Si l'agent tolère les clés inconnues dans `databases.json`, une clé explicite `managed: true` sera ajoutée et la détection deviendra le fallback — à vérifier côté agent. - -Cas limites : - -- `host` managé sans `{PREFIX}_PORT` dans `.env` → `ui.warning`, la base est traitée comme externe. -- Deux entrées avec le même `host` → `ConfigError` avant tout rendu. -- `.env` ou `databases.json` absent → `ConfigError("Not a Portabase agent folder")`. - -### 5.2 `EnvFile` (`services/envfile.py`) - -Remplace `write_env_file`. Parse `KEY="v"`, `KEY='v'`, `KEY=v`, `export KEY=`, commentaires, lignes vides. Conserve l'ordre et les commentaires (liste de lignes typées). `merge()` met à jour en place et ajoute en fin ; `remove(prefix)` retire les `PREFIX_*`. Écriture toujours quotée `"…"`, `"` et `\` échappés. Sauvegarde atomique (tmp + `os.replace`). - -`.env` ne contient que des variables consommées par les conteneurs. Aucune métadonnée CLI. - -### 5.3 `ComposeFacts` (`services/compose_facts.py`) - -`yaml.safe_load` du compose existant, lecture seule, jamais réécrit. Expose `host_gateway` (présence de `extra_hosts` sur `services.agent`, forme liste ou dict tolérée). Compose absent ou invalide → valeurs par défaut + `ui.warning`, jamais d'erreur. - -### 5.4 `TemplateRepository` (`services/templates.py`) - -- URL : `{TEMPLATE_BASE_URL}/{version}/manifest.json` puis fichiers listés. -- Résolution de version : `current_version()` ; sinon `PORTABASE_TEMPLATES_VERSION` ; sinon `PORTABASE_TEMPLATES_DIR` (court-circuite S3) ; sinon `TemplateError`. En dev non-frozen, `./templates` à côté de `main.py` est utilisé automatiquement s'il existe. -- Cache `~/.portabase/cache/templates//`. Séquence `ensure()` : GET manifest (10 s) → pour chaque fichier, sha256 identique en cache → skip, sinon GET + vérification sha256 et taille → écriture. Fichiers en cache absents du manifest supprimés. Manifest injoignable avec cache complet → warning et cache ; sans cache → `TemplateError` avec hint. -- Jinja2 : `Environment(undefined=StrictUndefined, keep_trailing_newline=True, autoescape=False)`. `{{ }}` ne collisionne pas avec `${}` Compose. - -Manifest : - -```json -{ - "schema": 1, - "version": "26.09.0", - "generated_at": "2026-09-11T14:02:17Z", - "commit": "858d4926…", - "files": { - "agent.yml.j2": { "sha256": "…", "size": 612 }, - "engines/postgresql.yml.j2": { "sha256": "…", "size": 498 } - }, - "engines": { - "postgresql": "engines/postgresql.yml.j2", - "postgresql-cluster": "engines/postgresql.yml.j2" - } -} -``` - -`schema` inconnu → `TemplateError`. `version` ≠ version demandée → `TemplateError`. `engines` sert à `engines-check` en CI et à `get_engine(key)`. - -### 5.5 `ComposeRenderer` (`services/renderer.py`) - -```python -class ComposeRenderer: - def __init__(self, templates: TemplateRepository, engines: EngineRegistry): ... - def render_agent(self, project: AgentProject, inline: bool = False) -> RenderResult: ... - def render_dashboard(self, project: DashboardProject, inline: bool = False) -> RenderResult: ... -``` - -Contexte `agent.yml.j2` : `host_gateway`, `docker_socket`, `mounts` (sqlite), `services` (liste de `{name, volume, body}` où `body` est le rendu du template moteur). Le renderer passe aux templates moteurs des variables **déjà formées** (`port_var = "${DB_PG_A1F2_PORT}"` ou valeur littérale si `inline`) : la logique de nommage reste en Python, les templates restent lisibles. - -`RenderResult` : `compose: str`, `databases: list[dict]`. `write(path)` valide d'abord (`yaml.safe_load` du compose → sinon `TemplateError`, un template remote cassé ne corrompt jamais une install), puis écrit `docker-compose.yml` et `databases.json` atomiquement. Le compose porte un en-tête `# Generated by Portabase CLI . Do not edit — use docker-compose.override.yml.` - -Ordre dans une commande mutante : collecte → `templates.ensure()` → mutation `.env`/`databases.json` en mémoire → rendu → validation → écriture. Le manifest est vérifié avant toute mutation. - -### 5.6 Templates - -`agent.yml.j2` : - -```jinja -services: - agent: - restart: unless-stopped - image: portabase/agent:latest - volumes: - - ./databases.json:/config/config.json -{%- for m in mounts %} - - {{ m.host }}:{{ m.container }} -{%- endfor %} -{%- if docker_socket %} - - /var/run/docker.sock:/var/run/docker.sock -{%- endif %} -{%- if host_gateway %} - extra_hosts: - - "localhost:host-gateway" -{%- endif %} - environment: - TZ: "${TZ}" - EDGE_KEY: "${EDGE_KEY}" - LOG_LEVEL: "${LOG_LEVEL}" - POLLING: "${POLLING}" - networks: - - portabase -{% for s in services %} -{{ s.body }} -{%- endfor %} -{% if services %} -volumes: -{%- for s in services %} - {{ s.volume }}: -{%- endfor %} -{% endif %} -networks: - portabase: - name: portabase_network - external: true -``` - -Templates moteurs : un par moteur, variante auth par `{% if auth %}` (10 snippets actuels → 8 templates ; `postgresql-cluster` réutilise `postgresql.yml.j2`). `dashboard.yml.j2` : `{% if db_mode == "external" %}` autour du service `db`, de `depends_on` et du volume — remplace les trois `re.sub` de `dashboard.py`. - -### 5.7 Installs legacy - -Aucun marqueur de version nécessaire. `AgentProject.load()` fonctionne sur toute install (`.env` + `databases.json` existent déjà). Au premier `RenderResult.write()` sur un compose sans l'en-tête `# Generated by Portabase CLI`, le fichier est copié en `docker-compose.legacy.yml` et un avertissement est affiché. `portabase build PATH --diff` permet de voir le diff avant. Les commandes `start/stop/logs` ne déclenchent rien. - -Différences attendues au premier rendu d'une install ancienne : `restart: unless-stopped` ajouté sur redis/valkey (absent des snippets actuels) ; à mentionner dans le changelog rc. - -## 6. Moteurs DB (`engines/`) - -```python -@dataclass(frozen=True) -class Field: - name: str; prompt: str - kind: Literal["text", "int", "secret", "bool", "choice"] - default: Any = None; choices: tuple[str, ...] = (); help: str | None = None - validator: Callable[[Any], Any] | None = None - - -class DbEngine(ABC): - key: str; display: str; default_port: int - template: str | None # None = aucun service Compose (sqlite, docker-volume, existing) - auth_variants: bool = False - warning: str | None = None - - def fields_existing(self) -> list[Field]: ... # défaut : host, port, database, username, password - def fields_new(self) -> list[Field]: ... # défaut : [] (tout généré) - def option_fields(self) -> list[Field]: ... # défaut : [] - def generate(self, service: str, auth: bool, ports: PortAllocator) -> DatabaseSpec: ... - def env_vars(self, spec: DatabaseSpec) -> dict[str, str]: ... - def template_ctx(self, spec: DatabaseSpec, inline: bool) -> dict: ... - def agent_entry(self, spec: DatabaseSpec) -> dict: ... # projection databases.json - def agent_database(self, spec: DatabaseSpec) -> str: ... # défaut : spec.database ; hook pour "0", chemin… -``` - -Hiérarchie : `StandardSqlEngine` (postgresql, postgresql-cluster, mysql, mariadb, mssql, firebird), `RedisEngine`, `ValkeyEngine`, `MongoEngine`, `SqliteEngine`, `DockerVolumeEngine`. Redis et Valkey sont deux classes indépendantes dans deux fichiers, sans base commune (images, commandes et healthchecks divergent ; ce qu'elles partagent — `agent_database = "0"`, `auth_variants` — passe par les hooks de `DbEngine`). Les sous-classes ne surchargent que leurs particularités : - -| Moteur | Particularité | -|---|---| -| postgresql | `option_fields` : `keep_ownership` (bool, défaut False), `clean_mode` (choice clean/none/drop_schemas/drop_database, défaut clean) | -| postgresql-cluster | `warning` superuser ; pas d'options | -| firebird | `agent_entry.name = "mirror.fdb"` ; var `_ROOT_PASS` | -| mssql | `agent_entry.username = "sa"` | -| redis | `agent_database = "0"` ; `auth_variants = True` ; no-auth → `_PORT` seul | -| valkey | idem redis, classe et template distincts | -| mongodb | `auth_variants = True` | -| sqlite | `fields_new` : nom de fichier ; `fields_existing` : chemin ; pas de template ; mount si chemin relatif | -| docker-volume | `fields` : volume, container (optionnel), label ; `warning` socket ; pas de template | - -`EngineRegistry` : dict `key → instance`, imports explicites (compatible PyInstaller). `get(key)` inconnu → `ValidationError` avec la liste des clés. - -### 6.1 Options moteur - -Certains moteurs exposent des options que l'agent lit dans `databases.json` (`options` : aujourd'hui `keep_ownership` et `clean_mode` pour PostgreSQL). Le système doit accepter de nouvelles options sans toucher à la signature Typer. - -- Déclaration : `DbEngine.option_fields() -> list[Field]`. Un `Field` comme les autres : nom, prompt, type, défaut, choix, validateur. -- Saisie non-interactive : flag générique répétable `-o KEY=VALUE` / `--option KEY=VALUE` sur `db add`. Parsé en `dict[str, str]`, converti selon `Field.kind` (`bool` : `true/false/1/0/yes/no`, `int`, `choice` validé contre `choices`). Clé inconnue pour ce moteur → `ValidationError` listant les options valides. -- Saisie interactive : `Form.collect(engine.option_fields(), values["options"])`, un prompt par option non fournie, avec le texte d'aide actuel (par exemple l'explication de `--no-owner` / `pg_restore --clean`) porté par `Field.help`. -- Stockage : `DatabaseSpec.options: dict` (valeurs typées). -- Projection : `agent_entry()` n'écrit dans `options` que les valeurs différentes du défaut. Comportement actuel conservé : `keep_ownership` absent si False, `clean_mode` absent si `clean`. Aucune clé `options` si vide. -- Affichage : `db list` montre les options non-défaut ; `Summary` les inclut lors de l'ajout. - -Ajouter une option = une ligne dans `option_fields()` du moteur concerné. - -## 7. `ui/` - -### 7.1 Principes - -- Tokens uniques (`ui/theme.py`) : `PALETTE` → `RICH_THEME` et `QUESTIONARY_STYLE`. -- Composants stateless, un par fichier, `Component(console)`. -- Façade `UI` : seule chose importée par `commands/`. Rich et questionary ne sont jamais importés hors de `ui/`. -- Le markup Rich est autorisé dans les arguments texte (`ui.success("Added [bold]x[/bold]")`). -- `NO_COLOR` / `--no-color` → `Console(no_color=True)`, style questionary vide. -- Jamais de prompt à l'intérieur d'un `ui.status()` (structurellement garanti : les services ne promptent pas). - -### 7.2 Composants - -| Composant | Remplace | -|---|---| -| `Banner` | `print_banner` | -| `Message` (`success/info/warning/error`) | ~60 `console.print("[success]✔ …")` | -| `Section` | `Panel("[bold]Database Setup[/bold]")` | -| `Summary` (masque auto des clés `password/secret/key`) | `Table(show_header=False)` de dashboard | -| `DataTable` | `Table` de `db list` | -| `Status` (context manager, hint injecté) | `console.status(msg + hint)` | -| `Hint` | `get_random_hint` | -| `Diff` | nouveau, pour `build --diff` | -| `Prompt` (`text/integer/secret/confirm/select/path`) | `rich.prompt.*`, `questionary.*` épars | - -Règle anti-dérive : un composant n'existe que s'il a un appelant. L'inventaire ci-dessus est un plafond. - -### 7.3 `Form` - -```python -class Form: - def ask(self, field: Field, value: Any | None) -> Any: - # 1. valeur du flag → validée - # 2. non-interactif : défaut, sinon ValidationError("Missing --") - # 3. interactif : prompt selon field.kind (dispatch dict), None (Ctrl-C) → UserAbort - # validation en boucle jusqu'à valeur acceptée - def collect(self, fields: list[Field], values: dict) -> dict: ... - def text(...), integer(...), confirm(...), choice(...) # raccourcis -``` - -`non_interactive` résolu une fois dans `main.py`. `ui.confirm()` en non-interactif renvoie le défaut ; les confirmations destructives ont `default=False` et un flag `--force`. - -## 8. Erreurs, télémétrie, updater - -### 8.1 Hiérarchie (`core/errors.py`) - -| Classe | `code` | exit | -|---|---|---| -| `PortabaseError` | `E_GENERIC` | 1 | -| `UserAbort` | `E_ABORT` | 130 | -| `ValidationError` | `E_VALIDATION` | 2 | -| `ConfigError` | `E_CONFIG` | 3 | -| `DockerError` | `E_DOCKER` | 4 | -| `TemplateError` | `E_TEMPLATE` | 5 | -| `NetworkError` | `E_NETWORK` | 6 | -| `UpdateError` | `E_UPDATE` | 7 | -| `DecryptionError` (`core/crypto.py`) | `E_CRYPTO` | 8 | - -Constructeur : `(message, *, hint=None, cause=None)`. Les exceptions tierces (`requests`, `subprocess`, `yaml`, `jinja2`) sont wrappées à la frontière du service. `typer.Exit` n'est plus levé hors de `main.py`. Ruff : `E722`, `BLE001`, `S110`, `TID251`. - -### 8.2 Catcher (`main.py`) - -`app(standalone_mode=False)` dans un seul `try` : `UserAbort` → "Cancelled." exit 130 ; `PortabaseError` → `ui.error(e)` (message, hint, code ; `--verbose` ajoute cause et traceback), `telemetry.error(e)`, exit `e.exit_code` ; `click.UsageError` → mappé en `ValidationError` ; `KeyboardInterrupt` → exit 130 ; `Exception` → "Unexpected error", télémétrie `unexpected=True`, exit 1. `finally: telemetry.flush()`. - -### 8.3 Télémétrie (`services/telemetry.py`) - -```python -class Telemetry(ABC): - def session(self, **attrs) -> ContextManager # span racine par invocation - def span(self, name: str, **attrs) -> ContextManager - def event(self, name: str, **attrs) -> None - def error(self, exc: Exception, unexpected: bool = False) -> None - def flush(self) -> None -``` - -Implémentations : `NoopTelemetry` (défaut), `ConsoleTelemetry` (`--debug`, stderr), `OtelTelemetry` (futur, import lazy, construit seulement si `telemetry=true` et `telemetry_endpoint` défini). Spans : `Command.run`, `TemplateRepository.ensure`, `ComposeRenderer.render`, `DockerRunner.compose`. Attributs : commande, moteur, mode, durée, code de sortie, `error.code`, version CLI, OS. Jamais : nom d'agent, chemin, clé, credentials, contenu de fichier. - -Opt-in : `portabase config set telemetry true` ou `PORTABASE_TELEMETRY=1`. Une ligne d'information à la première exécution, aucun prompt. - -### 8.4 Updater (`services/updater.py`) - -`UpdateChecker.notify(ui)` appelé après la commande, cache 24 h (`~/.portabase/cache/release.json`), silencieux si hors ligne, `--stdout` ou non-interactif. `Updater.apply()` vérifie le sha256 via `checksums.txt` de la release avant remplacement du binaire. Canal `beta` conservé. - -## 9. CI, sécurité, release - -### 9.1 `ci.yml` (`pull_request`, `push: main`) - -| Job | Contenu | -|---|---| -| `lint` | `ruff check`, `ruff format --check` | -| `render-check` | `scripts/render_check.py` : rend `agent.yml.j2` (0 base, chaque moteur auth/no-auth, socket, host_gateway, mounts sqlite) et `dashboard.yml.j2` × 3 modes via le vrai `ComposeRenderer` (`PORTABASE_TEMPLATES_DIR=./templates`), puis `yaml.safe_load` et `docker compose config` avec `.env` fixture | -| `engines-check` | chaque `DbEngine.template` existe dans `templates/`, chaque template a un moteur, `engines.map.json` cohérent | -| `gitleaks` | action pinnée ; `.gitleaks.toml` allowlist `templates/**` et `HINTS` | -| `plumber` | action drop-in, `verify-attestation: true` | -| `build-smoke` | PyInstaller linux/amd64, `./dist/portabase --version`, `agent smoke --key --non-interactive` avec templates locaux | -| `test` | `pytest` — vide, réservé à la spec tests | - -### 9.2 Durcissement - -- Toutes les actions pinnées par SHA avec commentaire de version ; Dependabot `github-actions` et `uv` hebdomadaires. -- `permissions: {}` au top de chaque workflow, permissions explicites par job. `packages: write` retiré (inutilisé). -- `templates-upload.yml` : plus de `~/.s3cfg` par heredoc ; credentials par variables d'environnement. -- `actions/attest-build-provenance` sur les binaires. - -### 9.3 Release - -`bump.yml` (`workflow_dispatch`, inputs `version`, `channel: stable|rc`) : validation regex, `stable` uniquement depuis `main`, `sed` `pyproject.toml` + `CITATION.cff`, commit `chore(release): X`, tag, push. Les workflows sur tag restent inchangés. `./release` supprimé. Pas de release-please (historique non conventional). Si `main` exige une PR, le workflow ouvre une PR au lieu de pousser — à régler selon la protection de branche. - -`templates-hotfix.yml` (`workflow_dispatch`, input `version`) : re-sync `templates/` vers `templates//` et régénère le manifest. Réservé aux corrections compatibles avec le code de cette version. - -`templates-upload.yml` : source `templates/`, génération de `manifest.json` (sha256, taille, version, commit, date, mapping moteurs depuis `engines.map.json`) avant `s3cmd sync`. - -### 9.4 `pyproject.toml` - -```toml -dependencies = ["typer", "rich", "questionary", "requests", "pyyaml", "jinja2"] -[dependency-groups] -dev = ["pyinstaller", "ruff", "pytest"] -``` - -## 10. Ordre des chantiers - -Graphe de dépendances, pas un calendrier. - -``` -[A] Hygiène CI ─────────────────────────────────────────────┐ indépendant - ci.yml, pin SHA, permissions, bump.yml, pyproject │ - │ -[B] Fondations │ - core/errors, ui/, services/{envfile,docker,http,ports}, │ - main.py catcher, Command ABC │ - │ │ - ├──► [C] Lifecycle en POO (start/stop/…/config/update) - │ (ancien code agent/db/dashboard via LegacyCommand) - │ - └──► [D] Templates .j2 + TemplateRepository + engines/ + render-check - │ - ▼ - [E] Rendu : project, compose_facts, renderer, build - │ - ▼ - [F] agent / dashboard / db réécrits, ancien code supprimé - │ - ▼ - [G] OTel réel, --json, spec tests -``` - -Contraintes : - -- B avant tout code métier. -- D avant E (le renderer se construit contre des templates réels). -- E avant F. -- C et F ne touchent pas les mêmes fichiers ; C peut aller avant ou après D/E. -- A avant F de préférence : `render-check` et `build-smoke` sont le seul filet avant la spec tests. - -Points de livraison : - -| Après | État | Canal | -|---|---|---| -| A | fonctionnellement identique, CI verte | stable | -| B + C | lifecycle en POO, ui/ et erreurs neuves ; `agent`/`db`/`dashboard` = ancien code via `LegacyCommand` | stable | -| D | templates `.j2` uploadés sous la nouvelle version ; l'ancien code lit `agent.yml`, coexistence sur S3 | rc | -| E + F | bascule complète | rc obligatoire, puis stable | - -Risques et parades : - -| Étape | Risque | Parade | -|---|---|---| -| A | mauvais SHA casse un workflow | tag rc jetable | -| B | sur-conception de `ui/` | un composant = un appelant | -| D | template `.j2` diverge d'un snippet actuel | diff manuel des rendus contre l'ancien CLI, une fois | -| E | `ComposeFacts` lit mal un vieux compose | tolérance, warning, jamais de crash | -| F | install legacy cassée après `db add` | `.legacy.yml`, `build --diff`, changelog rc | -| F | mots de passe existants avec `$` | ne pas régénérer ; correction pour les nouvelles bases seulement | - -## 11. Questions ouvertes - -- L'agent tolère-t-il des clés inconnues dans `databases.json` ? Si oui : clé `managed: true` explicite. -- Protection de la branche `main` : `bump.yml` pousse directement ou ouvre une PR ? -- Garder la clé `engines` dans le manifest (double source de vérité avec le code) ou s'en tenir au registre Python ? diff --git a/docs/superpowers/specs/2026-09-11-dashboard-settings-design.md b/docs/superpowers/specs/2026-09-11-dashboard-settings-design.md deleted file mode 100644 index 97096eb..0000000 --- a/docs/superpowers/specs/2026-09-11-dashboard-settings-design.md +++ /dev/null @@ -1,198 +0,0 @@ -# Dashboard settings and authentication — Design - -Date : 2026-09-11 -Dépend de : `2026-09-11-cli-refactor-design.md` (état déclaratif, `EnvFile`, `Field`, `Form`, `CommandGroup`). - -## 1. Objectif - -Configurer depuis le CLI ce que le dashboard lit dans son environnement : API et MCP, onboarding, authentification par mot de passe, providers OIDC et OAuth2. À la création et après coup, en interactif et par flags. - -Au passage, une rupture de surface décidée pour être cohérente : un namespace par composant. - -## 2. Surface CLI - -### 2.1 Avant / après - -| Avant | Après | -|---|---| -| `agent NAME` | `agent create NAME` | -| `db add\|remove\|list NAME` | `agent db add\|remove\|list NAME` | -| `dashboard NAME` | `dashboard create NAME` | -| — | `dashboard show NAME` | -| — | `dashboard set NAME KEY VALUE [KEY VALUE…]` | -| — | `dashboard auth add\|remove\|list NAME` | -| `start\|stop\|restart\|logs\|uninstall\|build PATH` | inchangés | -| `config`, `update`, `decrypt` | inchangés | - -Un groupe Typer ne peut pas porter à la fois un argument positionnel et des sous-commandes (vérifié : `dashboard auth` créerait un dashboard nommé `auth`). D'où `create`. - -### 2.2 Compatibilité - -- `db` reste à la racine **une version**, alias de `agent db`, avec un avertissement à chaque appel : `'portabase db' is deprecated, use 'portabase agent db'`. Retiré à la version suivante. -- `portabase agent my-agent` et `portabase dashboard my-dash` produisent une `UsageError` « No such command ». Le catcher de `main.py` la reconnaît (token précédent = `agent` ou `dashboard`) et ajoute le hint `Did you mean: portabase agent create my-agent?`. -- README, `CONTRIBUTING.md`, doc portabase.io et script d'installation à mettre à jour dans la même release. - -### 2.3 Implémentation - -`CommandGroup` gagne `groups: list[CommandGroup]` pour imbriquer (`agent` contient `db`). `AgentCommand` devient `AgentCreateCommand` dans un `AgentCommands(CommandGroup)` ; idem `DashboardCreateCommand` dans `DashboardCommands`. - -## 3. Registre de settings - -Le cœur : une liste déclarative, un endroit à toucher pour ajouter un réglage. - -```python -@dataclass(frozen=True) -class Setting: - field: Field # name, prompt, kind, default, choices, help, validator - env: str # variable écrite dans .env - section: str # "network" | "api" | "onboarding" | "auth" - secret: bool = False # jamais en flag visible sans avertissement, masqué à l'affichage -``` - -`services/dashboard_settings.py` : - -| Section | `field.name` | `env` | kind | défaut dashboard | -|---|---|---|---|---| -| network | `url` | `PROJECT_URL` | text | `http://localhost:` | -| network | `behind_proxy` | `TUSD_BEHIND_PROXY` | bool | false | -| network | `trusted_domains` | `TRUSTED_DOMAINS` | text | — | -| api | `api` | `API_ENABLED` | bool | false | -| api | `openapi` | `OPENAPI_ENABLED` | bool | false | -| api | `mcp` | `MCP_ENABLED` | bool | false | -| onboarding | `skip_onboarding` | `SKIP_ONBOARDING` | bool | false | -| onboarding | `admin_name` | `AUTH_DEFAULT_USER_NAME` | text | — | -| onboarding | `admin_email` | `AUTH_DEFAULT_USER` | text | — | -| onboarding | `admin_password` | `AUTH_DEFAULT_PASSWORD` | secret | — | -| auth | `password_auth` | `AUTH_EMAIL_PASSWORD_ENABLED` | bool | true | -| auth | `signup` | `AUTH_SIGNUP_ENABLED` | bool | — | -| auth | `passkey` | `AUTH_PASSKEY_ENABLED` | bool | — | -| auth | `account_linking` | `AUTH_ALLOW_LINKING` | bool | — | -| auth | `account_unlinking` | `AUTH_ALLOW_UNLINKING` | bool | — | -| auth | `sync_oidc_roles` | `AUTH_SYNC_OIDC_ROLES_ON_LOGIN` | bool | — | -| auth | `role_map` | `AUTH_ROLE_MAP` | text | — | -| auth | `allowed_group` | `ALLOWED_GROUP` | text | — | - -Ce que le registre produit, sans code par réglage : - -- les flags de `dashboard create` : `--api/--no-api` pour un bool, `--url` pour un texte, `--admin-password-stdin` pour un secret ; -- les prompts interactifs, groupés par section ; -- la validation de `dashboard set` : `KEY` doit être un `field.name` du registre, `VALUE` est coercé par `Form` (bool `true/false/yes/no/1/0`, choix, validateur) ; -- l'affichage de `dashboard show`, section par section, secrets masqués. - -Écriture dans `.env` : booléens en `true`/`false`. À la création, seuls les réglages fournis ou différents du défaut sont écrits (le `.env` reste lisible). `set` écrit toujours la valeur demandée. - -`admin_password` porte le validateur documenté : 8 caractères, majuscule, minuscule, chiffre, spécial. - -## 4. Providers d'authentification - -Répétables, donc en sous-commande, comme `agent db`. - -### 4.1 Modèle - -```python -@dataclass(frozen=True) -class AuthProvider: - kind: Literal["oidc", "oauth"] - id: str # providerId : "keycloak", "github" - values: dict[str, str] # champs → valeurs, sans le préfixe -``` - -Stockage dans `.env` par préfixe, ce qui est exactement le mécanisme des bases managées : - -| kind | préfixe | champs | -|---|---|---| -| oidc | `AUTH_OIDC__` | `ID`, `TITLE`, `DESC`, `ICON`, `ISSUER_URL`, `CLIENT`, `SECRET`, `SCOPES`, `PKCE`, `HOST` | -| oauth | `AUTH_SOCIAL__` | `CLIENT`, `SECRET`, `TITLE` | - -`` est l'identifiant en majuscules avec `-` → `_` ; `AUTH_OIDC__ID` reçoit l'identifiant tel que saisi (c'est le `providerId` du callback). Les providers OAuth sont limités aux noms connus du dashboard : `google`, `github`, `discord`, `apple`, `linkedin`, `x`, `reddit`. - -Lecture : `DashboardProject.providers` scanne les clés du `.env` par préfixe et reconstruit la liste. Aucun autre état. - -### 4.2 Commandes - -``` -dashboard auth add NAME oidc ID --issuer URL --client CLIENT (--secret S | --secret-stdin) - [--title T] [--scopes "openid profile email"] [--pkce] [--host H] -dashboard auth add NAME oauth PROVIDER --client CLIENT (--secret S | --secret-stdin) [--title T] -dashboard auth list NAME -dashboard auth remove NAME ID [--yes] -``` - -- `add` sur un `ID` existant → `ValidationError`, hint « remove it first ». -- `remove` fait `env.remove_prefix(...)` puis re-rend. -- `list` affiche kind, id, titre, issuer/provider, et le callback à déclarer chez le fournisseur : `/api/auth/sso/callback/`. -- `--secret` visible accepté avec avertissement, `--secret-stdin` recommandé — même règle que `agent db add`. - -En interactif, `add` sans flags pose les champs du kind via `Form`. - -## 5. Validations croisées - -Dans `DashboardProject.validate()`, appelé avant toute écriture. Ce sont des refus, pas des avertissements : chacune laisse une instance inaccessible. - -| Condition | Erreur | -|---|---| -| `skip_onboarding` sans `admin_email` **et** `admin_password` | « Skipping onboarding needs an initial account: set admin_email and admin_password. » | -| `password_auth = false` et aucun provider | « Disabling password login with no OIDC or OAuth provider would lock everyone out. » | -| au moins un provider et `url` sur `localhost` | « Providers need a public URL for their callback; set url (currently http://localhost:8887). » | -| `auth remove` du dernier provider alors que `password_auth = false` | même refus que la ligne 2 | - -`admin_password` faible → refus par le validateur du champ. - -## 6. Interactif — `dashboard create` - -Le flux actuel reste : port, mode base, timezone, résumé, confirmation. Entre le résumé et la confirmation, une question : - -``` -Configure API, MCP and authentication now? [y/N] -``` - -Non par défaut. Si oui, trois sections courtes, chacune précédée de `ui.section(...)` : - -1. **API** — `api`, `openapi`, `mcp` -2. **Onboarding** — `skip_onboarding` ; si oui, `admin_name`, `admin_email`, `admin_password` (masqué) -3. **Authentication** — `password_auth`, `signup`, `passkey` - -Les providers ne sont pas dans le wizard : hint `Add a login provider with: portabase dashboard auth add NAME oidc …`, comme `agent create` renvoie vers `db add`. - -Le résumé inclut les réglages non-défaut. `--yes` saute la confirmation, `--non-interactive` prend les défauts et ne pose pas la question des sections. - -## 7. Application des changements - -Toute commande mutante (`create`, `set`, `auth add`, `auth remove`) termine par `project.save_state()` puis `renderer.render_dashboard(project).write(path)`. Le template a `env_file: .env`, donc le compose ne change pas — mais `write` est appelé quand même pour garder un seul chemin. - -Puis le message : `Apply with: portabase start NAME`. - -**`restart` est corrigé dans cette spec** : `docker compose restart` ne relit pas `env_file` ni ne crée un service ajouté (bug déjà constaté sur `agent db add`). `RestartCommand` fait `up -d` puis `restart`, pour converger vers l'état déclaré. Le message des commandes mutantes peut alors dire `portabase restart NAME` sans mentir. - -## 8. Ce qui ne change pas - -- `dashboard.yml.j2` : aucune modification, `env_file` suffit. -- `EnvFile`, `Form`, `Field`, `Summary`, `render_dashboard` : réutilisés tels quels. -- Le mode `custom`/`external`/`internal` de la base : inchangé. - -## 9. Hors périmètre - -- SMTP (`SMTP_*`), `RETENTION_CRON`, `STALE_BACKUP_THRESHOLD_HOURS`, `BACKUP_FOLDER_NAME`, `TELEMETRY` du dashboard. Le registre les accepte en une ligne chacun le jour venu. -- Provider OAuth2 générique (endpoints libres) : le dashboard ne le documente pas. -- Un `agent set` : l'agent n'a que `TZ`, `POLLING`, `LOG_LEVEL` ; à ajouter si le besoin apparaît, avec le même registre. -- Tests automatisés : spec séparée. Vérification ici par `render_check` et les parcours non-interactifs. - -## 10. Fichiers - -| Fichier | Action | -|---|---| -| `commands/base.py` | `CommandGroup.groups` pour l'imbrication | -| `commands/agent.py` | `AgentCommands` (groupe) + `AgentCreateCommand` ; `db` devient `agent db` | -| `commands/db.py` | inchangé, enregistré sous `agent` ; alias racine déprécié | -| `commands/dashboard.py` | `DashboardCommands` : `create`, `show`, `set` | -| `commands/auth.py` | `add`, `list`, `remove` | -| `commands/lifecycle.py` | `RestartCommand` → `up -d` puis `restart` | -| `services/dashboard_settings.py` | `Setting`, `SETTINGS`, `OAUTH_PROVIDERS`, `OIDC_FIELDS` | -| `services/project.py` | `DashboardProject` : `settings`, `providers`, `validate()`, `set()`, `add_provider()`, `remove_provider()` | -| `main.py` | enregistrement des groupes, hint « did you mean » | -| `README.md`, `.github/CONTRIBUTING.md` | nouvelle surface | - -## 11. Questions ouvertes - -- Le `providerId` OIDC : imposer le même que `` en minuscules, ou le laisser libre via `--id` ? Défaut retenu : identique, pas de flag. -- `dashboard set` accepte plusieurs paires en une commande ; faut-il aussi `dashboard unset KEY` pour revenir au défaut du dashboard (retirer la variable) ? Défaut retenu : oui, trivial avec `env.remove`. diff --git a/engines/base.py b/engines/base.py index 5107c10..444a8d7 100644 --- a/engines/base.py +++ b/engines/base.py @@ -7,7 +7,7 @@ from core.fields import Field from core.specs import DatabaseSpec -from core.utils import generate_password +from core.utils import escape_yaml_double_quoted, generate_password from services.ports import PortAllocator STANDARD_EXISTING_FIELDS = ( @@ -53,9 +53,9 @@ def __init_subclass__(cls, **kwargs: Any) -> None: def fields_existing(self) -> list[Field]: return [ Field("port", "Port", "int", default=self.default_port) - if f.name == "port" - else f - for f in STANDARD_EXISTING_FIELDS + if field.name == "port" + else field + for field in STANDARD_EXISTING_FIELDS ] def fields_new(self) -> list[Field]: @@ -83,12 +83,12 @@ def from_existing(self, answers: dict[str, Any]) -> DatabaseSpec: ) def env_vars(self, spec: DatabaseSpec) -> dict[str, str]: - p = spec.env_prefix + prefix = spec.env_prefix return { - f"{p}_PORT": str(spec.host_port), - f"{p}_DB": spec.database or "", - f"{p}_USER": spec.username or "", - f"{p}_PASS": spec.password or "", + f"{prefix}_PORT": str(spec.host_port), + f"{prefix}_DB": spec.database or "", + f"{prefix}_USER": spec.username or "", + f"{prefix}_PASS": spec.password or "", } def template_ctx( @@ -127,9 +127,11 @@ def describe(self, spec: DatabaseSpec) -> str: return f"{spec.host}:{spec.port}" def non_default_options(self, spec: DatabaseSpec) -> dict[str, Any]: - defaults = {f.name: f.default for f in self.option_fields()} + defaults = {field.name: field.default for field in self.option_fields()} return { - k: v for k, v in spec.options.items() if k in defaults and v != defaults[k] + key: value + for key, value in spec.options.items() + if key in defaults and value != defaults[key] } @staticmethod @@ -144,7 +146,7 @@ def service_name(slug: str, auth: bool = False) -> str: @staticmethod def var(spec: DatabaseSpec, suffix: str, value: Any, inline: bool) -> str: if inline: - return str(value if value is not None else "") + return escape_yaml_double_quoted(str(value if value is not None else "")) return f"${{{spec.env_prefix}_{suffix}}}" diff --git a/engines/mongodb.py b/engines/mongodb.py index fca6a4b..09232ba 100644 --- a/engines/mongodb.py +++ b/engines/mongodb.py @@ -32,9 +32,12 @@ def generate( ) def env_vars(self, spec: DatabaseSpec) -> dict[str, str]: - p = spec.env_prefix - out = {f"{p}_PORT": str(spec.host_port), f"{p}_DB": spec.database or ""} + prefix = spec.env_prefix + out = { + f"{prefix}_PORT": str(spec.host_port), + f"{prefix}_DB": spec.database or "", + } if spec.auth: - out[f"{p}_USER"] = spec.username or "" - out[f"{p}_PASS"] = spec.password or "" + out[f"{prefix}_USER"] = spec.username or "" + out[f"{prefix}_PASS"] = spec.password or "" return out diff --git a/engines/mssql.py b/engines/mssql.py index 2d110bc..ae56b62 100644 --- a/engines/mssql.py +++ b/engines/mssql.py @@ -29,5 +29,8 @@ def generate( ) def env_vars(self, spec: DatabaseSpec) -> dict[str, str]: - p = spec.env_prefix - return {f"{p}_PORT": str(spec.host_port), f"{p}_PASS": spec.password or ""} + prefix = spec.env_prefix + return { + f"{prefix}_PORT": str(spec.host_port), + f"{prefix}_PASS": spec.password or "", + } diff --git a/engines/redis.py b/engines/redis.py index 0a872a8..45f1eec 100644 --- a/engines/redis.py +++ b/engines/redis.py @@ -41,10 +41,10 @@ def generate( ) def env_vars(self, spec: DatabaseSpec) -> dict[str, str]: - p = spec.env_prefix - out = {f"{p}_PORT": str(spec.host_port)} + prefix = spec.env_prefix + out = {f"{prefix}_PORT": str(spec.host_port)} if spec.auth: - out[f"{p}_PASS"] = spec.password or "" + out[f"{prefix}_PASS"] = spec.password or "" return out def agent_database(self, spec: DatabaseSpec) -> str: diff --git a/engines/valkey.py b/engines/valkey.py index c9a95c6..376c81f 100644 --- a/engines/valkey.py +++ b/engines/valkey.py @@ -41,10 +41,10 @@ def generate( ) def env_vars(self, spec: DatabaseSpec) -> dict[str, str]: - p = spec.env_prefix - out = {f"{p}_PORT": str(spec.host_port)} + prefix = spec.env_prefix + out = {f"{prefix}_PORT": str(spec.host_port)} if spec.auth: - out[f"{p}_PASS"] = spec.password or "" + out[f"{prefix}_PASS"] = spec.password or "" return out def agent_database(self, spec: DatabaseSpec) -> str: diff --git a/main.py b/main.py index 1ed4388..22ce500 100644 --- a/main.py +++ b/main.py @@ -9,7 +9,6 @@ import typer from commands.agent import AgentCommands -from commands.base import DeprecatedAlias from commands.build import BuildCommand from commands.config import ConfigCommands from commands.dashboard import DashboardCommands @@ -136,7 +135,6 @@ def root( DashboardCommands(ui, telemetry, docker, templates, renderer, ports).register(app) for cmd in commands: cmd.register(app) - DeprecatedAlias(ui, telemetry, agent.db, name="db", use="agent db").register(app) ConfigCommands(ui, telemetry, config).register(app) return app, checker @@ -144,6 +142,8 @@ def root( def _usage_hint(error: click.UsageError) -> str: group = error.ctx.command.name if error.ctx and error.ctx.command else None match = re.match(r"No such command '(.+)'", error.format_message()) + if match and match.group(1) == "db": + return "Database commands belong to the agent: portabase agent db ..." if group in ("agent", "dashboard") and match: return f"Did you mean: portabase {group} create {match.group(1)}?" return "Run 'portabase --help' for usage." @@ -172,7 +172,9 @@ def main() -> None: ui = UI(non_interactive=settings.non_interactive, no_color=settings.no_color) telemetry = NoopTelemetry() app, checker = build_app(ui, telemetry, config, settings) - invoked = next((a for a in sys.argv[1:] if not a.startswith("-")), None) + invoked = next( + (argument for argument in sys.argv[1:] if not argument.startswith("-")), None + ) exit_code = 0 try: @@ -180,20 +182,20 @@ def main() -> None: result = app(standalone_mode=False) if isinstance(result, int): exit_code = result - except UserAbort as e: - ui.warning(e.message) + except UserAbort as error: + ui.warning(error.message) telemetry.event("abort") - exit_code = e.exit_code - except PortabaseError as e: - ui.error(e) - telemetry.error(e) - exit_code = e.exit_code + exit_code = error.exit_code + except PortabaseError as error: + ui.error(error) + telemetry.error(error) + exit_code = error.exit_code except click.exceptions.NoArgsIsHelpError: exit_code = 0 - except click.exceptions.Exit as e: - exit_code = e.exit_code - except click.UsageError as e: - err = ValidationError(e.format_message(), hint=_usage_hint(e)) + except click.exceptions.Exit as error: + exit_code = error.exit_code + except click.UsageError as error: + err = ValidationError(error.format_message(), hint=_usage_hint(error)) ui.error(err) telemetry.error(err) exit_code = err.exit_code @@ -201,10 +203,10 @@ def main() -> None: ui.print("") ui.warning("Canceled.") exit_code = 130 - except Exception as e: # noqa: BLE001 — last resort: a bug, not an expected error - wrapped = PortabaseError("Unexpected error: " + str(e), cause=e) + except Exception as error: # noqa: BLE001 — last resort: a bug, not an expected error + wrapped = PortabaseError("Unexpected error: " + str(error), cause=error) ui.error(wrapped, unexpected=True) - telemetry.error(e, unexpected=True) + telemetry.error(error, unexpected=True) exit_code = 1 finally: telemetry.flush() diff --git a/pyproject.toml b/pyproject.toml index 5ae4b14..3c41f35 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -60,11 +60,14 @@ known-first-party = ["commands", "core", "engines", "services", "ui"] [tool.pytest.ini_options] testpaths = ["tests"] -addopts = "-q" +python_files = ["*.py"] +python_functions = ["[!_]*"] +pythonpath = ["."] +addopts = "-q --import-mode=importlib" [tool.mypy] python_version = "3.12" -files = ["commands", "core", "engines", "services", "ui", "scripts", "main.py"] +files = ["commands", "core", "engines", "services", "ui", "tests", "main.py"] ignore_missing_imports = true warn_unused_ignores = true warn_redundant_casts = true diff --git a/scripts/render_check.py b/scripts/render_check.py deleted file mode 100644 index 68c7952..0000000 --- a/scripts/render_check.py +++ /dev/null @@ -1,215 +0,0 @@ -#!/usr/bin/env python3 - -from __future__ import annotations - -import argparse -import os -import shutil -import subprocess -import sys -import tempfile -from pathlib import Path - -import yaml - -sys.path.insert(0, str(Path(__file__).resolve().parent.parent)) - -from core.specs import DatabaseSpec # noqa: E402 -from engines import registry # noqa: E402 -from engines.base import DbEngine # noqa: E402 -from services.envfile import EnvFile # noqa: E402 -from services.ports import FixedPortAllocator # noqa: E402 -from services.project import AgentProject, DashboardProject # noqa: E402 -from services.renderer import ComposeRenderer # noqa: E402 -from services.templates import TemplateRepository # noqa: E402 - -AGENT_ENV = {"TZ": "UTC", "EDGE_KEY": "x", "LOG_LEVEL": "info", "POLLING": "5"} -DASHBOARD_BASE = { - "HOST_PORT": "8887", - "PROJECT_SECRET": "s", - "PROJECT_URL": "http://localhost:8887", - "PROJECT_NAME": "pb", - "TZ": "UTC", - "LOG_LEVEL": "info", -} -DASHBOARD_PG = { - "POSTGRES_DB": "pb", - "POSTGRES_USER": "pb", - "POSTGRES_PASSWORD": "p", - "PG_PORT": "5433", - "DATABASE_URL": "postgresql://pb:p@db:5432/pb", -} - - -class Failure(Exception): - pass - - -def env_text(project: AgentProject | DashboardProject) -> str: - return "".join(f'{k}="{v}"\n' for k, v in project.env.as_dict().items()) - - -def validate(label: str, compose: str, env: str, use_compose: bool) -> None: - try: - doc = yaml.safe_load(compose) - except yaml.YAMLError as e: - raise Failure(f"{label}: invalid YAML: {e}\n{compose}") from e - if not isinstance(doc, dict) or "services" not in doc: - raise Failure(f"{label}: no services key\n{compose}") - if use_compose: - with tempfile.TemporaryDirectory() as tmp: - Path(tmp, "docker-compose.yml").write_text(compose, encoding="utf-8") - Path(tmp, ".env").write_text(env, encoding="utf-8") - Path(tmp, "databases.json").write_text( - '{"databases": []}', encoding="utf-8" - ) - proc = subprocess.run( - ["docker", "compose", "-p", "rendercheck", "config", "--quiet"], - cwd=tmp, - capture_output=True, - text=True, - check=False, - ) - if proc.returncode != 0: - raise Failure( - f"{label}: docker compose config failed:\n{proc.stderr}\n{compose}" - ) - print(f"ok {label}") - - -def agent_project( - tmp: Path, - pairs: list[tuple[DatabaseSpec, DbEngine]], - *, - host_gateway: bool = False, - sqlite: bool = False, - docker_volume: bool = False, -) -> AgentProject: - env = EnvFile(tmp / ".env") - env.merge(AGENT_ENV) - project = AgentProject(tmp, env, [], host_gateway) - for spec, engine in pairs: - project.add(spec, engine) - if sqlite: - sq = registry.get("sqlite") - project.add( - sq.generate(auth=False, ports=FixedPortAllocator(), answers={"name": "x"}), - sq, - ) - if docker_volume: - dv = registry.get("docker-volume") - project.add(dv.from_existing({"volume": "v"}), dv) - return project - - -def agent_cases(renderer: ComposeRenderer) -> list[tuple[str, str, str]]: - ports = FixedPortAllocator() - tmp = Path(tempfile.mkdtemp()) - cases: list[tuple[str, str, str]] = [] - - empty = agent_project(tmp, []) - cases.append(("agent/empty", renderer.render_agent(empty).compose, env_text(empty))) - - toggles = agent_project(tmp, [], host_gateway=True, sqlite=True, docker_volume=True) - cases.append( - ("agent/toggles", renderer.render_agent(toggles).compose, env_text(toggles)) - ) - - everything: list[tuple[DatabaseSpec, DbEngine]] = [] - for engine in registry: - if engine.template is None: - continue - for auth in (True, False) if engine.auth_variants else (True,): - spec = engine.generate(auth=auth, ports=ports, answers={}) - one = agent_project(tmp, [(spec, engine)]) - variant = ("/auth" if auth else "/noauth") if engine.auth_variants else "" - cases.append( - ( - f"agent/{engine.key}{variant}", - renderer.render_agent(one).compose, - env_text(one), - ) - ) - everything.append((spec, engine)) - - combined = agent_project(tmp, everything, host_gateway=True, sqlite=True) - cases.append( - ("agent/all", renderer.render_agent(combined).compose, env_text(combined)) - ) - return cases - - -def dashboard_cases(renderer: ComposeRenderer) -> list[tuple[str, str, str]]: - tmp = Path(tempfile.mkdtemp()) - variants = { - "external": {**DASHBOARD_BASE, **DASHBOARD_PG, "POSTGRES_HOST": "db"}, - "internal": DASHBOARD_BASE, - "custom": {**DASHBOARD_BASE, **DASHBOARD_PG, "POSTGRES_HOST": "remote"}, - } - cases = [] - for mode, values in variants.items(): - env = EnvFile(tmp / f".env.{mode}") - env.merge(values) - project = DashboardProject(tmp, env) - if project.db_mode != mode: - raise Failure(f"db_mode mismatch: {project.db_mode} != {mode}") - cases.append( - ( - f"dashboard/{mode}", - renderer.render_dashboard(project).compose, - env_text(project), - ) - ) - return cases - - -def engines_check(repo: TemplateRepository) -> None: - shipped = set(repo.names()) - used = set() - for engine in registry: - if engine.template is None: - continue - if engine.template not in shipped: - raise Failure( - f"{engine.key}: template {engine.template} not found in {repo.root}" - ) - used.add(engine.template) - orphans = { - name for name in shipped if name.startswith("engines/") and name not in used - } - if orphans: - raise Failure( - f"template(s) not used by any engine: {', '.join(sorted(orphans))}" - ) - print(f"ok engines-check ({len(used)} templates, {len(registry.keys())} engines)") - - -def main() -> int: - parser = argparse.ArgumentParser() - parser.add_argument( - "--templates", default=os.environ.get("PORTABASE_TEMPLATES_DIR", "templates") - ) - parser.add_argument( - "--no-compose", - action="store_true", - help="Skip docker compose config validation", - ) - args = parser.parse_args() - - use_compose = not args.no_compose and shutil.which("docker") is not None - if not use_compose: - print("note: docker not available, YAML validation only") - repo = TemplateRepository(Path(args.templates)) - renderer = ComposeRenderer(repo, registry, "render-check") - try: - engines_check(repo) - for label, compose, env in agent_cases(renderer) + dashboard_cases(renderer): - validate(label, compose, env, use_compose) - except Failure as e: - print(f"FAIL {e}", file=sys.stderr) - return 1 - return 0 - - -if __name__ == "__main__": - raise SystemExit(main()) diff --git a/services/compose_facts.py b/services/compose_facts.py index 6b3568a..b3cae80 100644 --- a/services/compose_facts.py +++ b/services/compose_facts.py @@ -38,9 +38,9 @@ def _service(self, name: str) -> dict: def host_gateway(self) -> bool: extra = self._service("agent").get("extra_hosts") if isinstance(extra, list): - return any("host-gateway" in str(x) for x in extra) + return any("host-gateway" in str(entry) for entry in extra) if isinstance(extra, dict): - return any("host-gateway" in str(v) for v in extra.values()) + return any("host-gateway" in str(value) for value in extra.values()) return False @property diff --git a/services/docker.py b/services/docker.py index 8dd3491..8f60b3d 100644 --- a/services/docker.py +++ b/services/docker.py @@ -53,8 +53,10 @@ def start_daemon(self, *, wait_seconds: int = 20) -> bool: return False try: subprocess.run(cmd, check=True) - except (subprocess.CalledProcessError, OSError) as e: - raise DockerError(f"Failed to start Docker: {e}", cause=e) from e + except (subprocess.CalledProcessError, OSError) as error: + raise DockerError( + f"Failed to start Docker: {error}", cause=error + ) from error deadline = time.monotonic() + wait_seconds while time.monotonic() < deadline: if self.daemon_running(): @@ -77,10 +79,10 @@ def ensure_network(self, name: str) -> None: stdout=subprocess.DEVNULL, check=True, ) - except subprocess.CalledProcessError as e: + except subprocess.CalledProcessError as error: raise DockerError( - f"Could not create Docker network '{name}'.", cause=e - ) from e + f"Could not create Docker network '{name}'.", cause=error + ) from error def remove_volume(self, name: str) -> bool: proc = subprocess.run( @@ -116,11 +118,11 @@ def compose( capture_output=capture, text=capture, ) - except subprocess.CalledProcessError as e: + except subprocess.CalledProcessError as error: raise DockerError( - f"docker compose {' '.join(args)} failed (exit {e.returncode}).", + f"docker compose {' '.join(args)} failed (exit {error.returncode}).", hint=f"Run it manually in {cwd} to see the full output.", - cause=e, - ) from e - except OSError as e: - raise DockerError(f"Could not run docker: {e}", cause=e) from e + cause=error, + ) from error + except OSError as error: + raise DockerError(f"Could not run docker: {error}", cause=error) from error diff --git a/services/envfile.py b/services/envfile.py index 2cc9f3e..ceb4dfb 100644 --- a/services/envfile.py +++ b/services/envfile.py @@ -34,10 +34,10 @@ def load(cls, path: Path) -> EnvFile: if path.exists(): text = path.read_text(encoding="utf-8") env._lines = text.splitlines() - for i, line in enumerate(env._lines): - m = _LINE.match(line) - if m and not line.lstrip().startswith("#"): - env._index[m.group(1)] = i + for index, line in enumerate(env._lines): + match = _LINE.match(line) + if match and not line.lstrip().startswith("#"): + env._index[match.group(1)] = index return env @property @@ -45,37 +45,40 @@ def exists(self) -> bool: return self.path.exists() def get(self, key: str, default: str | None = None) -> str | None: - i = self._index.get(key) - if i is None: + index = self._index.get(key) + if index is None: return default - m = _LINE.match(self._lines[i]) - return _unquote(m.group(2)) if m else default + match = _LINE.match(self._lines[index]) + return _unquote(match.group(2)) if match else default def as_dict(self) -> dict[str, str]: - return {k: self.get(k) or "" for k in self._index} + return {key: self.get(key) or "" for key in self._index} def set(self, key: str, value: str) -> None: line = f"{key}={_quote(str(value))}" - i = self._index.get(key) - if i is None: + index = self._index.get(key) + if index is None: self._lines.append(line) self._index[key] = len(self._lines) - 1 else: - self._lines[i] = line + self._lines[index] = line def merge(self, mapping: Mapping[str, str]) -> None: - for k, v in mapping.items(): - self.set(k, v) + for key, value in mapping.items(): + self.set(key, value) def remove(self, key: str) -> None: - i = self._index.pop(key, None) - if i is None: + index = self._index.pop(key, None) + if index is None: return - del self._lines[i] - self._index = {k: (n - 1 if n > i else n) for k, n in self._index.items()} + del self._lines[index] + self._index = { + name: (position - 1 if position > index else position) + for name, position in self._index.items() + } def remove_prefix(self, prefix: str) -> None: - for key in [k for k in self._index if k.startswith(prefix + "_")]: + for key in [name for name in self._index if name.startswith(prefix + "_")]: self.remove(key) def save(self) -> None: diff --git a/services/http.py b/services/http.py index 03661e4..14d083b 100644 --- a/services/http.py +++ b/services/http.py @@ -21,27 +21,35 @@ def __init__( def get_json(self, url: str) -> Any: try: - r = self.session.get(url, timeout=self.timeout) - r.raise_for_status() - return r.json() - except requests.RequestException as e: - raise NetworkError(f"GET {url} failed: {e}", hint=_HINT, cause=e) from e - except ValueError as e: - raise NetworkError(f"GET {url}: response is not JSON", cause=e) from e + response = self.session.get(url, timeout=self.timeout) + response.raise_for_status() + return response.json() + except requests.RequestException as error: + raise NetworkError( + f"GET {url} failed: {error}", hint=_HINT, cause=error + ) from error + except ValueError as error: + raise NetworkError( + f"GET {url}: response is not JSON", cause=error + ) from error def get_text(self, url: str) -> str: try: - r = self.session.get(url, timeout=self.timeout) - r.raise_for_status() - return r.text - except requests.RequestException as e: - raise NetworkError(f"GET {url} failed: {e}", hint=_HINT, cause=e) from e + response = self.session.get(url, timeout=self.timeout) + response.raise_for_status() + return response.text + except requests.RequestException as error: + raise NetworkError( + f"GET {url} failed: {error}", hint=_HINT, cause=error + ) from error def status(self, url: str) -> int: try: return self.session.get(url, timeout=self.timeout, stream=True).status_code - except requests.RequestException as e: - raise NetworkError(f"GET {url} failed: {e}", hint=_HINT, cause=e) from e + except requests.RequestException as error: + raise NetworkError( + f"GET {url} failed: {error}", hint=_HINT, cause=error + ) from error def download( self, @@ -53,27 +61,29 @@ def download( ) -> int: written = 0 try: - with self.session.get(url, stream=True, timeout=timeout) as r: - r.raise_for_status() - with open(dest, "wb") as f: - for chunk in r.iter_content(chunk_size=64 * 1024): + with self.session.get(url, stream=True, timeout=timeout) as response: + response.raise_for_status() + with open(dest, "wb") as file: + for chunk in response.iter_content(chunk_size=64 * 1024): if not chunk: continue - f.write(chunk) + file.write(chunk) written += len(chunk) if on_progress: on_progress(len(chunk)) - except requests.RequestException as e: + except requests.RequestException as error: dest.unlink(missing_ok=True) raise NetworkError( - f"Download of {url} failed: {e}", hint=_HINT, cause=e - ) from e + f"Download of {url} failed: {error}", hint=_HINT, cause=error + ) from error return written def content_length(self, url: str) -> int | None: try: - r = self.session.head(url, timeout=self.timeout, allow_redirects=True) - value = r.headers.get("content-length") + response = self.session.head( + url, timeout=self.timeout, allow_redirects=True + ) + value = response.headers.get("content-length") return int(value) if value else None except (requests.RequestException, ValueError): return None diff --git a/services/ports.py b/services/ports.py index 7bff3fa..f95ee29 100644 --- a/services/ports.py +++ b/services/ports.py @@ -9,9 +9,9 @@ def __init__(self) -> None: def free(self) -> int: for _ in range(50): - with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s: - s.bind(("", 0)) - port = s.getsockname()[1] + with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock: + sock.bind(("", 0)) + port = sock.getsockname()[1] if port not in self._given: self._given.add(port) return port diff --git a/services/project.py b/services/project.py index 77b430f..c59989d 100644 --- a/services/project.py +++ b/services/project.py @@ -87,10 +87,12 @@ def load(cls, path: Path) -> AgentProject: env = EnvFile.load(env_path) try: data = json.loads(db_path.read_text(encoding="utf-8")) - except (OSError, ValueError) as e: - raise ConfigError(f"{db_path} is not valid JSON.", cause=e) from e + except (OSError, ValueError) as error: + raise ConfigError(f"{db_path} is not valid JSON.", cause=error) from error entries = data.get("databases", []) if isinstance(data, dict) else [] - databases = [spec_from_entry(e, env) for e in entries if isinstance(e, dict)] + databases = [ + spec_from_entry(entry, env) for entry in entries if isinstance(entry, dict) + ] facts = ComposeFacts(path / COMPOSE_FILE) project = cls(path, env, databases, facts.host_gateway) project._ca_bundle = facts.ca_bundle @@ -108,20 +110,20 @@ def create( @property def managed(self) -> list[DatabaseSpec]: - return [d for d in self.databases if d.managed] + return [database for database in self.databases if database.managed] @property def needs_docker_socket(self) -> bool: - return any(d.engine == "docker-volume" for d in self.databases) + return any(database.engine == "docker-volume" for database in self.databases) @property def sqlite_mounts(self) -> list[tuple[str, str]]: mounts: list[tuple[str, str]] = [] - for d in self.databases: - if d.engine == "sqlite": - m = SqliteEngine.mount_for(d) - if m and m not in mounts: - mounts.append(m) + for database in self.databases: + if database.engine == "sqlite": + mount = SqliteEngine.mount_for(database) + if mount and mount not in mounts: + mounts.append(mount) return mounts def validate(self) -> None: @@ -132,12 +134,12 @@ def validate(self) -> None: hint=f"Path is resolved from {self.path}; use an absolute path otherwise.", ) seen: set[str] = set() - for d in self.managed: - if d.host in seen: + for database in self.managed: + if database.host in seen: raise ConfigError( - f"Two managed databases share the service name '{d.host}'." + f"Two managed databases share the service name '{database.host}'." ) - seen.add(d.host or "") + seen.add(database.host or "") def add(self, spec: DatabaseSpec, engine: DbEngine) -> None: if spec.managed: @@ -146,7 +148,9 @@ def add(self, spec: DatabaseSpec, engine: DbEngine) -> None: self.validate() def remove(self, spec: DatabaseSpec, engine: DbEngine) -> None: - self.databases = [d for d in self.databases if d.id != spec.id] + self.databases = [ + database for database in self.databases if database.id != spec.id + ] if spec.managed and spec.host: self.env.remove_prefix(spec.env_prefix) @@ -159,7 +163,7 @@ def setting(self, name: str) -> Any: return setting.from_env(self.env.get(setting.env)) def settings(self) -> dict[str, Any]: - return {s.name: self.setting(s.name) for s in self.registry} + return {setting.name: self.setting(setting.name) for setting in self.registry} def set(self, name: str, value: Any) -> None: setting = self.registry.get(name) @@ -180,9 +184,11 @@ def unset(self, name: str) -> None: @property def extra_env(self) -> list[str]: return [ - s.env - for s in self.registry - if s.env and not s.core and self.env.get(s.env) is not None + setting.env + for setting in self.registry + if setting.env + and not setting.core + and self.env.get(setting.env) is not None ] _ca_bundle: str | None = None @@ -201,9 +207,11 @@ def ca_bundle(self, host_path: str | None) -> None: def find(self, id_or_name: str) -> DatabaseSpec: matches = [ - d - for d in self.databases - if d.id == id_or_name or d.id.startswith(id_or_name) or d.name == id_or_name + database + for database in self.databases + if database.id == id_or_name + or database.id.startswith(id_or_name) + or database.name == id_or_name ] if not matches: raise ValidationError( @@ -262,7 +270,7 @@ def setting(self, name: str) -> Any: return setting.from_env(self.env.get(setting.env or "")) def settings(self) -> dict[str, Any]: - return {s.name: self.setting(s.name) for s in self.registry} + return {setting.name: self.setting(setting.name) for setting in self.registry} def set(self, name: str, value: Any) -> None: setting = self.registry.get(name) @@ -299,7 +307,7 @@ def providers(self) -> list[AuthProvider]: return providers def add_provider(self, provider: AuthProvider) -> None: - if any(p.id == provider.id for p in self.providers): + if any(existing.id == provider.id for existing in self.providers): raise ValidationError( f"A provider named '{provider.id}' already exists.", hint="Remove it first: portabase dashboard auth remove", @@ -315,7 +323,10 @@ def add_provider(self, provider: AuthProvider) -> None: self.env.set(f"{prefix}_{env_map[field_name]}", raw) def remove_provider(self, provider_id: str) -> AuthProvider: - match = next((p for p in self.providers if p.id == provider_id), None) + match = next( + (provider for provider in self.providers if provider.id == provider_id), + None, + ) if match is None: raise ValidationError( f"No provider named '{provider_id}'.", diff --git a/services/renderer.py b/services/renderer.py index ac0043a..8c69ae3 100644 --- a/services/renderer.py +++ b/services/renderer.py @@ -14,6 +14,7 @@ from core.errors import TemplateError from core.specs import DatabaseSpec +from core.utils import escape_yaml_double_quoted from engines import EngineRegistry from services.compose_facts import ( CA_BUNDLE_IN_CONTAINER, @@ -46,10 +47,10 @@ class RenderResult: def validate(self) -> None: try: doc = yaml.safe_load(self.compose) - except yaml.YAMLError as e: + except yaml.YAMLError as error: raise TemplateError( - "Rendered compose is not valid YAML; templates are broken.", cause=e - ) from e + "Rendered compose is not valid YAML; templates are broken.", cause=error + ) from error if not isinstance(doc, dict) or "services" not in doc: raise TemplateError( "Rendered compose has no 'services' section; templates are broken." @@ -100,7 +101,9 @@ def _atomic_write(path: Path, content: str) -> None: def _var(env: EnvFile, key: str, inline: bool) -> str: - return (env.get(key) or "") if inline else f"${{{key}}}" + if inline: + return escape_yaml_double_quoted(env.get(key) or "") + return f"${{{key}}}" class ComposeRenderer: @@ -121,7 +124,10 @@ def render_agent( ctx = { "host_gateway": project.host_gateway, "docker_socket": project.needs_docker_socket, - "mounts": [{"host": h, "container": c} for h, c in project.sqlite_mounts], + "mounts": [ + {"host": host, "container": container} + for host, container in project.sqlite_mounts + ], "services": [self._service(spec, inline) for spec in project.managed], "tz_var": _var(env, "TZ", inline), "edge_key_var": _var(env, "EDGE_KEY", inline), @@ -136,7 +142,8 @@ def render_agent( } compose = self.header() + self._render("agent.yml.j2", ctx) databases = [ - self.engines.get(d.engine).agent_entry(d) for d in project.databases + self.engines.get(database.engine).agent_entry(database) + for database in project.databases ] return RenderResult(compose=compose, databases=databases) @@ -173,5 +180,7 @@ def _render(self, name: str, ctx: dict[str, Any]) -> str: def _render_template(template: jinja2.Template, ctx: dict[str, Any]) -> str: try: return template.render(**ctx) - except jinja2.TemplateError as e: - raise TemplateError(f"Template rendering failed: {e}", cause=e) from e + except jinja2.TemplateError as error: + raise TemplateError( + f"Template rendering failed: {error}", cause=error + ) from error diff --git a/services/settings.py b/services/settings.py index d7a3002..fba5f65 100644 --- a/services/settings.py +++ b/services/settings.py @@ -79,7 +79,7 @@ def from_env(self, raw: str | None) -> Any: class Registry: def __init__(self, settings: tuple[Setting, ...], sections: dict[str, str]) -> None: self._settings = settings - self._by_name = {s.name: s for s in settings} + self._by_name = {setting.name: setting for setting in settings} self.sections = sections def __iter__(self) -> Iterator[Setting]: @@ -97,7 +97,7 @@ def get(self, name: str) -> Setting: ) from None def in_section(self, section: str) -> list[Setting]: - return [s for s in self._settings if s.section == section] + return [setting for setting in self._settings if setting.section == section] AGENT = Registry( diff --git a/services/templates.py b/services/templates.py index dbd5b6c..8607980 100644 --- a/services/templates.py +++ b/services/templates.py @@ -39,7 +39,7 @@ def resolve(self) -> Path: def names(self) -> list[str]: return sorted( - p.relative_to(self.root).as_posix() for p in self.root.rglob("*.j2") + path.relative_to(self.root).as_posix() for path in self.root.rglob("*.j2") ) def get(self, name: str) -> jinja2.Template: @@ -53,11 +53,11 @@ def get(self, name: str) -> jinja2.Template: ) try: return self._env.get_template(name) - except jinja2.TemplateNotFound as e: + except jinja2.TemplateNotFound as error: raise TemplateError( - f"Template '{name}' is missing from {root}.", cause=e - ) from e - except jinja2.TemplateError as e: + f"Template '{name}' is missing from {root}.", cause=error + ) from error + except jinja2.TemplateError as error: raise TemplateError( - f"Template '{name}' failed to load: {e}", cause=e - ) from e + f"Template '{name}' failed to load: {error}", cause=error + ) from error diff --git a/services/updater.py b/services/updater.py index 708ec6b..c18bc4e 100644 --- a/services/updater.py +++ b/services/updater.py @@ -34,7 +34,8 @@ def from_api(cls, data: dict) -> Release: return cls( tag=str(data.get("tag_name", "")).lstrip("v"), assets={ - a["name"]: a["browser_download_url"] for a in data.get("assets", []) + asset["name"]: asset["browser_download_url"] + for asset in data.get("assets", []) }, prerelease=bool(data.get("prerelease", False)), ) @@ -98,8 +99,8 @@ def available(self, *, force: bool = False) -> str | None: def _read_cache(self) -> Release | None: try: - with open(self.cache_file, encoding="utf-8") as f: - data = json.load(f) + with open(self.cache_file, encoding="utf-8") as file: + data = json.load(file) if time.time() - float(data.get("checked_at", 0)) > CACHE_TTL: return None if data.get("channel_pre") != self.include_prerelease: @@ -115,7 +116,7 @@ def _read_cache(self) -> Release | None: def _write_cache(self, release: Release) -> None: try: self.cache_file.parent.mkdir(parents=True, exist_ok=True) - with open(self.cache_file, "w", encoding="utf-8") as f: + with open(self.cache_file, "w", encoding="utf-8") as file: json.dump( { "checked_at": time.time(), @@ -124,7 +125,7 @@ def _write_cache(self, release: Release) -> None: "assets": release.assets, "prerelease": release.prerelease, }, - f, + file, ) except OSError: pass @@ -190,8 +191,8 @@ def _verify(self, release: Release, name: str, path: Path) -> None: f"{name} not listed in {self.CHECKSUMS_ASSET}; refusing to install." ) digest = hashlib.sha256() - with open(path, "rb") as f: - for chunk in iter(lambda: f.read(1 << 20), b""): + with open(path, "rb") as file: + for chunk in iter(lambda: file.read(1 << 20), b""): digest.update(chunk) if digest.hexdigest() != expected: raise UpdateError( @@ -218,5 +219,7 @@ def install(self, tmp: Path, target: Path) -> None: subprocess.run(["sudo", "mv", str(target), str(backup)], check=True) subprocess.run(["sudo", "mv", str(tmp), str(target)], check=True) subprocess.run(["sudo", "chmod", "+x", str(target)], check=True) - except (OSError, subprocess.CalledProcessError) as e: - raise UpdateError(f"Could not install to {target}: {e}", cause=e) from e + except (OSError, subprocess.CalledProcessError) as error: + raise UpdateError( + f"Could not install to {target}: {error}", cause=error + ) from error diff --git a/templates/dashboard.yml.j2 b/templates/dashboard.yml.j2 index aa20be5..75efd94 100644 --- a/templates/dashboard.yml.j2 +++ b/templates/dashboard.yml.j2 @@ -9,10 +9,10 @@ services: ports: - "{{ host_port_var }}:80" environment: - - TZ={{ tz_var }} - - LOG_LEVEL={{ log_level_var }} - - PROJECT_SECRET={{ project_secret_var }} - - PROJECT_URL={{ project_url_var }} + - "TZ={{ tz_var }}" + - "LOG_LEVEL={{ log_level_var }}" + - "PROJECT_SECRET={{ project_secret_var }}" + - "PROJECT_URL={{ project_url_var }}" volumes: - portabase-data:/data {%- if db_mode == "external" %} @@ -36,9 +36,9 @@ services: volumes: - postgres-data:/var/lib/postgresql/data environment: - - POSTGRES_DB={{ postgres_db_var }} - - POSTGRES_USER={{ postgres_user_var }} - - POSTGRES_PASSWORD={{ postgres_password_var }} + - "POSTGRES_DB={{ postgres_db_var }}" + - "POSTGRES_USER={{ postgres_user_var }}" + - "POSTGRES_PASSWORD={{ postgres_password_var }}" healthcheck: test: ["CMD-SHELL", "pg_isready -U {{ postgres_user_var }} -d {{ postgres_db_var }}"] interval: 10s diff --git a/templates/engines/firebird.yml.j2 b/templates/engines/firebird.yml.j2 index f11f488..cfaac88 100644 --- a/templates/engines/firebird.yml.j2 +++ b/templates/engines/firebird.yml.j2 @@ -8,11 +8,11 @@ volumes: - {{ volume }}:/var/lib/firebird/data environment: - - FIREBIRD_DATABASE={{ db_var }} - - FIREBIRD_USER={{ user_var }} - - FIREBIRD_PASSWORD={{ password_var }} - - FIREBIRD_ROOT_PASSWORD={{ root_password_var }} - - FIREBIRD_DATABASE_DEFAULT_CHARSET=UTF8 + - "FIREBIRD_DATABASE={{ db_var }}" + - "FIREBIRD_USER={{ user_var }}" + - "FIREBIRD_PASSWORD={{ password_var }}" + - "FIREBIRD_ROOT_PASSWORD={{ root_password_var }}" + - "FIREBIRD_DATABASE_DEFAULT_CHARSET=UTF8" healthcheck: test: ["CMD-SHELL", "nc -z localhost 3050"] interval: 10s diff --git a/templates/engines/mariadb.yml.j2 b/templates/engines/mariadb.yml.j2 index 31c6c22..de59a3c 100644 --- a/templates/engines/mariadb.yml.j2 +++ b/templates/engines/mariadb.yml.j2 @@ -6,10 +6,10 @@ ports: - "{{ port_var }}:3306" environment: - - MYSQL_DATABASE={{ db_var }} - - MYSQL_USER={{ user_var }} - - MYSQL_PASSWORD={{ password_var }} - - MYSQL_RANDOM_ROOT_PASSWORD=yes + - "MYSQL_DATABASE={{ db_var }}" + - "MYSQL_USER={{ user_var }}" + - "MYSQL_PASSWORD={{ password_var }}" + - "MYSQL_RANDOM_ROOT_PASSWORD=yes" volumes: - {{ volume }}:/var/lib/mysql healthcheck: diff --git a/templates/engines/mongodb.yml.j2 b/templates/engines/mongodb.yml.j2 index c1e0e6e..0cd5023 100644 --- a/templates/engines/mongodb.yml.j2 +++ b/templates/engines/mongodb.yml.j2 @@ -7,10 +7,10 @@ - "{{ port_var }}:27017" environment: {%- if auth %} - - MONGO_INITDB_ROOT_USERNAME={{ user_var }} - - MONGO_INITDB_ROOT_PASSWORD={{ password_var }} + - "MONGO_INITDB_ROOT_USERNAME={{ user_var }}" + - "MONGO_INITDB_ROOT_PASSWORD={{ password_var }}" {%- endif %} - - MONGO_INITDB_DATABASE={{ db_var }} + - "MONGO_INITDB_DATABASE={{ db_var }}" {%- if auth %} command: mongod --auth {%- endif %} diff --git a/templates/engines/mssql.yml.j2 b/templates/engines/mssql.yml.j2 index 2855ea7..535623e 100644 --- a/templates/engines/mssql.yml.j2 +++ b/templates/engines/mssql.yml.j2 @@ -6,8 +6,8 @@ ports: - "{{ port_var }}:1433" environment: - - ACCEPT_EULA=Y - - MSSQL_SA_PASSWORD={{ password_var }} + - "ACCEPT_EULA=Y" + - "MSSQL_SA_PASSWORD={{ password_var }}" volumes: - {{ volume }}:/var/opt/mssql healthcheck: diff --git a/templates/engines/mysql.yml.j2 b/templates/engines/mysql.yml.j2 index 31c6c22..de59a3c 100644 --- a/templates/engines/mysql.yml.j2 +++ b/templates/engines/mysql.yml.j2 @@ -6,10 +6,10 @@ ports: - "{{ port_var }}:3306" environment: - - MYSQL_DATABASE={{ db_var }} - - MYSQL_USER={{ user_var }} - - MYSQL_PASSWORD={{ password_var }} - - MYSQL_RANDOM_ROOT_PASSWORD=yes + - "MYSQL_DATABASE={{ db_var }}" + - "MYSQL_USER={{ user_var }}" + - "MYSQL_PASSWORD={{ password_var }}" + - "MYSQL_RANDOM_ROOT_PASSWORD=yes" volumes: - {{ volume }}:/var/lib/mysql healthcheck: diff --git a/templates/engines/postgresql-cluster.yml.j2 b/templates/engines/postgresql-cluster.yml.j2 index 6dcf870..7006e9b 100644 --- a/templates/engines/postgresql-cluster.yml.j2 +++ b/templates/engines/postgresql-cluster.yml.j2 @@ -8,9 +8,9 @@ volumes: - {{ volume }}:/var/lib/postgresql/data environment: - - POSTGRES_DB={{ db_var }} - - POSTGRES_USER={{ user_var }} - - POSTGRES_PASSWORD={{ password_var }} + - "POSTGRES_DB={{ db_var }}" + - "POSTGRES_USER={{ user_var }}" + - "POSTGRES_PASSWORD={{ password_var }}" healthcheck: test: ["CMD-SHELL", "pg_isready -U {{ user_var }} -d {{ db_var }}"] interval: 10s diff --git a/templates/engines/postgresql.yml.j2 b/templates/engines/postgresql.yml.j2 index 6dcf870..7006e9b 100644 --- a/templates/engines/postgresql.yml.j2 +++ b/templates/engines/postgresql.yml.j2 @@ -8,9 +8,9 @@ volumes: - {{ volume }}:/var/lib/postgresql/data environment: - - POSTGRES_DB={{ db_var }} - - POSTGRES_USER={{ user_var }} - - POSTGRES_PASSWORD={{ password_var }} + - "POSTGRES_DB={{ db_var }}" + - "POSTGRES_USER={{ user_var }}" + - "POSTGRES_PASSWORD={{ password_var }}" healthcheck: test: ["CMD-SHELL", "pg_isready -U {{ user_var }} -d {{ db_var }}"] interval: 10s diff --git a/templates/engines/redis.yml.j2 b/templates/engines/redis.yml.j2 index 52a18ec..d7e55c8 100644 --- a/templates/engines/redis.yml.j2 +++ b/templates/engines/redis.yml.j2 @@ -7,7 +7,7 @@ - {{ volume }}:/data {%- if auth %} environment: - - REDIS_PASSWORD={{ password_var }} + - "REDIS_PASSWORD={{ password_var }}" command: ["redis-server", "--requirepass", "{{ password_var }}", "--appendonly", "yes"] {%- else %} command: ["redis-server", "--appendonly", "yes"] diff --git a/templates/engines/valkey.yml.j2 b/templates/engines/valkey.yml.j2 index 885d123..3867e35 100644 --- a/templates/engines/valkey.yml.j2 +++ b/templates/engines/valkey.yml.j2 @@ -2,10 +2,10 @@ image: valkey/valkey:latest restart: unless-stopped {%- if auth %} - command: --requirepass "{{ password_var }}" + command: ["valkey-server", "--requirepass", "{{ password_var }}"] {%- else %} environment: - - ALLOW_EMPTY_PASSWORD=yes + - "ALLOW_EMPTY_PASSWORD=yes" {%- endif %} ports: - "{{ port_var }}:6379" diff --git a/tests/__init__.py b/tests/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/tests/conftest.py b/tests/conftest.py new file mode 100644 index 0000000..f533a18 --- /dev/null +++ b/tests/conftest.py @@ -0,0 +1,90 @@ +from __future__ import annotations + +from collections.abc import Callable +from pathlib import Path +from typing import Any + +import pytest +import yaml + +from engines import registry +from services.ports import FixedPortAllocator +from services.project import AgentProject, DashboardProject +from services.renderer import ComposeRenderer +from services.templates import TemplateRepository +from tests.support import AGENT_ENV, DASHBOARD_MODES, ROOT, Rendered + +collect_ignore = ["conftest.py", "support.py"] + + +def pytest_pycollect_makeitem(collector, name, obj): + # Tests have no test_ prefix (python_functions = "[!_]*"): never collect a + # callable that a test module only imports (functions, lru_cache wrappers, + # pytest.mark decorators kept in constants). + if callable(obj) and getattr(obj, "__module__", None) != collector.module.__name__: + return [] + return None + + +@pytest.fixture +def ports() -> FixedPortAllocator: + return FixedPortAllocator() + + +@pytest.fixture +def templates() -> TemplateRepository: + return TemplateRepository(ROOT / "templates") + + +@pytest.fixture +def renderer(templates: TemplateRepository) -> ComposeRenderer: + return ComposeRenderer(templates, registry, "test") + + +@pytest.fixture +def agent(tmp_path: Path) -> AgentProject: + return AgentProject.create(tmp_path / "agent", dict(AGENT_ENV), host_gateway=False) + + +@pytest.fixture +def dashboard_for(tmp_path: Path) -> Callable[..., DashboardProject]: + def build(mode: str = "internal", **env: str) -> DashboardProject: + folder = tmp_path / f"dashboard-{len(list(tmp_path.iterdir()))}" + return DashboardProject.create(folder, {**DASHBOARD_MODES[mode], **env}) + + return build + + +@pytest.fixture +def dashboard(dashboard_for: Callable[..., DashboardProject]) -> DashboardProject: + return dashboard_for(PROJECT_URL="https://d.example") + + +@pytest.fixture +def render_engine( + tmp_path: Path, renderer: ComposeRenderer, ports: FixedPortAllocator +) -> Callable[..., Rendered]: + """Generate one database of an engine, add it to a fresh agent, render it.""" + + def render( + key: str, + *, + auth: bool = True, + inline: bool = False, + answers: dict[str, Any] | None = None, + ) -> Rendered: + engine = registry.get(key) + spec = engine.generate(auth=auth, ports=ports, answers=answers or {}) + folder = tmp_path / f"{key}-{len(list(tmp_path.iterdir()))}" + project = AgentProject.create(folder, dict(AGENT_ENV), host_gateway=False) + project.add(spec, engine) + result = renderer.render_agent(project, inline=inline) + result.validate() + return Rendered( + spec, + yaml.safe_load(result.compose), + project.env.as_dict(), + result.databases or [], + ) + + return render diff --git a/tests/core/__init__.py b/tests/core/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/tests/core/config.py b/tests/core/config.py new file mode 100644 index 0000000..002d660 --- /dev/null +++ b/tests/core/config.py @@ -0,0 +1,38 @@ +import json + +import pytest + +from core.config import GlobalConfig + + +@pytest.fixture +def config(tmp_path): + return GlobalConfig(tmp_path / "home" / ".portabase" / "config.json") + + +def missing_file_reads_as_empty(config): + assert config.all() == {} + assert config.get("update_channel", "stable") == "stable" + assert config.update_channel is None + + +@pytest.mark.parametrize("content", ["{not json", "[1, 2]", ""]) +def unreadable_file_reads_as_empty(config, content): + config.path.parent.mkdir(parents=True) + config.path.write_text(content, encoding="utf-8") + assert config.all() == {} + + +def set_creates_the_file_and_keeps_other_keys(config): + config.set("update_channel", "beta") + config.set("other", 1) + assert json.loads(config.path.read_text(encoding="utf-8")) == { + "update_channel": "beta", + "other": 1, + } + assert config.update_channel == "beta" + assert not config.path.with_suffix(".json.tmp").exists() + + +def cache_dir_is_next_to_the_file(config): + assert config.cache_dir == config.path.parent / "cache" diff --git a/tests/core/crypto.py b/tests/core/crypto.py new file mode 100644 index 0000000..5e8f8e4 --- /dev/null +++ b/tests/core/crypto.py @@ -0,0 +1,135 @@ +import base64 +import json +import os +import struct + +import pytest + +from core.crypto import ( + DecryptionError, + decrypt_enc_file, + default_output_for, + load_master_key, +) +from tests.support import encrypt + +KEY = bytes(range(32)) +HEADER = {"cipher": "AES-256-GCM", "base_nonce": list(range(8)), "chunk_size": 4} + + +def _header(**overrides) -> bytes: + return json.dumps({**HEADER, **overrides}).encode() + b"\n" + + +@pytest.fixture +def paths(tmp_path): + return tmp_path / "backup.sql.enc", tmp_path / "out" / "backup.sql" + + +def load_master_key_raw(tmp_path): + path = tmp_path / "master_key.bin" + path.write_bytes(KEY) + assert load_master_key(path) == KEY + + +def load_master_key_base64(tmp_path): + path = tmp_path / "master_key.bin" + path.write_bytes(base64.b64encode(KEY) + b"\n") + assert load_master_key(path) == KEY + + +def load_master_key_defaults_to_cwd(tmp_path, monkeypatch): + (tmp_path / "master_key.bin").write_bytes(KEY) + monkeypatch.chdir(tmp_path) + assert load_master_key(None) == KEY + + +@pytest.mark.parametrize( + "content", [b"short", b"not base64 !!", base64.b64encode(b"x" * 16)] +) +def load_master_key_rejects_wrong_length(tmp_path, content): + path = tmp_path / "master_key.bin" + path.write_bytes(content) + with pytest.raises(DecryptionError, match="32-byte"): + load_master_key(path) + + +def load_master_key_missing(tmp_path): + with pytest.raises(DecryptionError, match="not found"): + load_master_key(tmp_path / "nope.bin") + + +def load_master_key_directory(tmp_path): + with pytest.raises(DecryptionError, match="not a file"): + load_master_key(tmp_path) + + +@pytest.mark.parametrize("plain", [b"", b"abc", b"exactly8", os.urandom(1000)]) +def decrypt_round_trip(paths, plain): + enc, out = paths + enc.write_bytes(encrypt(plain, KEY, chunk_size=4)) + decrypt_enc_file(enc, out, KEY) + assert out.read_bytes() == plain + assert not out.with_name(out.name + ".part").exists() + + +def decrypt_wrong_key_keeps_previous_output(paths): + enc, out = paths + enc.write_bytes(encrypt(b"secret data", KEY)) + out.parent.mkdir() + out.write_bytes(b"previous") + with pytest.raises(DecryptionError, match="Authentication failed on chunk 0"): + decrypt_enc_file(enc, out, bytes(32)) + assert out.read_bytes() == b"previous" + assert not out.with_name(out.name + ".part").exists() + + +def decrypt_detects_reordered_chunks(paths): + enc, out = paths + head, body = encrypt(b"aaaabbbb", KEY, chunk_size=4).split(b"\n", 1) + size = 4 + 4 + 16 + enc.write_bytes(head + b"\n" + body[size : 2 * size] + body[:size]) + with pytest.raises(DecryptionError, match="Authentication failed"): + decrypt_enc_file(enc, out, KEY) + assert not out.exists() + + +@pytest.mark.parametrize( + ("content", "message"), + [ + (b"", "missing header"), + (b"nope\n", "Invalid or missing JSON header"), + (_header(cipher="AES-128-CBC"), "Unsupported cipher"), + (_header(base_nonce=[1, 2]), "Invalid base_nonce length"), + (_header() + b"\x00\x00", "Truncated chunk length prefix"), + (_header() + struct.pack(">I", 4) + b"1234", "smaller than the 16-byte tag"), + (_header() + struct.pack(">I", 1000), "exceeds the maximum 20 bytes"), + (_header() + struct.pack(">I", 20) + b"short", "Truncated chunk 0"), + ], +) +def decrypt_rejects_corrupt_files(paths, content, message): + enc, out = paths + enc.write_bytes(content) + with pytest.raises(DecryptionError, match=message): + decrypt_enc_file(enc, out, KEY) + assert not out.exists() + assert not out.with_name(out.name + ".part").exists() + + +def decrypt_truncated_last_chunk(paths): + enc, out = paths + enc.write_bytes(encrypt(b"abcdefgh", KEY)[:-1]) + with pytest.raises(DecryptionError, match="Truncated chunk 1"): + decrypt_enc_file(enc, out, KEY) + + +@pytest.mark.parametrize( + ("name", "expected"), + [("dump.sql.enc", "dump.sql"), ("dump.enc", "dump"), ("dump.sql", "dump.sql.dec")], +) +def default_output_name(tmp_path, name, expected): + assert default_output_for(tmp_path / name) == expected + + +def decryption_error_exit_code(): + assert (DecryptionError.code, DecryptionError.exit_code) == ("E_CRYPTO", 8) diff --git a/tests/core/errors.py b/tests/core/errors.py new file mode 100644 index 0000000..3aee3eb --- /dev/null +++ b/tests/core/errors.py @@ -0,0 +1,53 @@ +import pytest + +from core import errors +from core.crypto import DecryptionError + +CODES = [ + (errors.PortabaseError, "E_GENERIC", 1), + (errors.ValidationError, "E_VALIDATION", 2), + (errors.ConfigError, "E_CONFIG", 3), + (errors.DockerError, "E_DOCKER", 4), + (errors.TemplateError, "E_TEMPLATE", 5), + (errors.NetworkError, "E_NETWORK", 6), + (errors.UpdateError, "E_UPDATE", 7), + (DecryptionError, "E_CRYPTO", 8), + (errors.UserAbort, "E_ABORT", 130), +] + + +@pytest.mark.parametrize( + ("cls", "code", "exit_code"), CODES, ids=[case[0].__name__ for case in CODES] +) +def codes_and_exit_codes(cls, code, exit_code): + assert issubclass(cls, errors.PortabaseError) + assert (cls.code, cls.exit_code) == (code, exit_code) + + +def exit_codes_are_unique(): + exit_codes = [exit_code for _, _, exit_code in CODES] + assert len(exit_codes) == len(set(exit_codes)) + + +def message_hint_and_cause(): + cause = ValueError("boom") + error = errors.ValidationError("Bad input.", hint="Try again.", cause=cause) + assert str(error) == "Bad input." + assert (error.message, error.hint, error.cause) == ( + "Bad input.", + "Try again.", + cause, + ) + assert error.__cause__ is cause + + +def no_hint_and_no_cause_by_default(): + error = errors.ConfigError("Broken.") + assert error.hint is None + assert error.cause is None + assert error.__cause__ is None + + +def user_abort_default_message(): + assert str(errors.UserAbort()) == "Canceled." + assert errors.UserAbort(hint="Run it again.").hint == "Run it again." diff --git a/tests/core/fields.py b/tests/core/fields.py new file mode 100644 index 0000000..2f53258 --- /dev/null +++ b/tests/core/fields.py @@ -0,0 +1,29 @@ +import dataclasses + +import pytest + +from core.fields import Field + + +def defaults(): + field = Field("name", "Prompt") + assert (field.kind, field.default, field.choices, field.help, field.validator) == ( + "text", + None, + (), + None, + None, + ) + + +@pytest.mark.parametrize( + ("name", "flag"), + [("key", "--key"), ("retry_attempts", "--retry-attempts"), ("a_b_c", "--a-b-c")], +) +def flag_uses_dashes(name, flag): + assert Field(name, "Prompt").flag == flag + + +def is_frozen(): + with pytest.raises(dataclasses.FrozenInstanceError): + Field("a", "A").name = "b" diff --git a/tests/core/specs.py b/tests/core/specs.py new file mode 100644 index 0000000..858e9aa --- /dev/null +++ b/tests/core/specs.py @@ -0,0 +1,33 @@ +import pytest + +from core.specs import DatabaseSpec + + +def _spec(**kwargs): + return DatabaseSpec(id="id-1", engine="postgresql", name="db", **kwargs) + + +def env_prefix_uppercases_and_replaces_dashes(): + assert _spec(host="db-pg-ab12").env_prefix == "DB_PG_AB12" + + +def env_prefix_requires_host(): + with pytest.raises(ValueError, match="host"): + _ = _spec().env_prefix + + +@pytest.mark.parametrize( + ("password", "expected"), [("s3cret", True), ("", False), (None, False)] +) +def auth_follows_password(password, expected): + assert _spec(password=password).auth is expected + + +def with_options_returns_a_copy(): + original = _spec(options={"a": 1}) + options = {"b": 2} + updated = original.with_options(options) + options["c"] = 3 + assert updated.options == {"b": 2} + assert original.options == {"a": 1} + assert updated.id == original.id diff --git a/tests/core/utils.py b/tests/core/utils.py new file mode 100644 index 0000000..224b271 --- /dev/null +++ b/tests/core/utils.py @@ -0,0 +1,99 @@ +import base64 +import json +import string + +import pytest + +from core.utils import ( + escape_yaml_double_quoted, + generate_password, + slugify_project_name, + validate_edge_key, +) +from tests.support import EDGE_KEY_PAYLOAD + +SYMBOLS = "!@#%^&*()-_=+[]{}|;:,.<>?" + + +@pytest.mark.parametrize( + ("length", "expected"), [(16, 16), (8, 8), (40, 40), (7, 8), (0, 8)] +) +def generate_password_length(length, expected): + assert len(generate_password(length)) == expected + + +def generate_password_has_every_character_class(): + for _ in range(100): + password = generate_password(8) + assert any(char in string.ascii_lowercase for char in password) + assert any(char in string.ascii_uppercase for char in password) + assert any(char in string.digits for char in password) + assert any(char in SYMBOLS for char in password) + + +def generate_password_is_random(): + assert len({generate_password() for _ in range(20)}) == 20 + + +@pytest.mark.parametrize( + ("value", "expected"), + [ + ("my-agent", "my-agent"), + ("My Agent", "my-agent"), + ("--Prod_DB--", "prod_db"), + ("_-x", "x"), + ("été 2026", "t-2026"), + ("a b", "a-b"), + ("!!!", "portabase"), + ("", "portabase"), + ], +) +def slugify_project_name_cases(value, expected): + assert slugify_project_name(value) == expected + + +def slugify_project_name_fallback(): + assert slugify_project_name("***", fallback="agent") == "agent" + + +def validate_edge_key_accepts_base64_json(): + key = base64.b64encode(json.dumps(EDGE_KEY_PAYLOAD).encode()).decode() + assert validate_edge_key(key) + + +def validate_edge_key_accepts_raw_json(): + assert validate_edge_key(json.dumps(EDGE_KEY_PAYLOAD)) + + +@pytest.mark.parametrize("missing", ["serverUrl", "agentId", "masterKeyB64"]) +def validate_edge_key_rejects_missing_field(missing): + payload = {key: value for key, value in EDGE_KEY_PAYLOAD.items() if key != missing} + key = base64.b64encode(json.dumps(payload).encode()).decode() + assert not validate_edge_key(key) + + +@pytest.mark.parametrize( + "key", + ["", "not a key", "1234", base64.b64encode(b"hello").decode(), "{broken json"], +) +def validate_edge_key_rejects_garbage(key): + assert not validate_edge_key(key) + + +@pytest.mark.parametrize( + "data", + [list(EDGE_KEY_PAYLOAD), "serverUrl agentId masterKeyB64", None, 42], + ids=["list", "string", "null", "number"], +) +def validate_edge_key_requires_an_object(data): + raw = json.dumps(data) + assert not validate_edge_key(raw) + assert not validate_edge_key(base64.b64encode(raw.encode()).decode()) + + +@pytest.mark.parametrize( + ("value", "expected"), + [("plain", "plain"), ('a"b', 'a\\"b'), ("a\\b", "a\\\\b"), ('\\"', '\\\\\\"')], +) +def escape_yaml_double_quoted_cases(value, expected): + assert escape_yaml_double_quoted(value) == expected diff --git a/tests/core/version.py b/tests/core/version.py new file mode 100644 index 0000000..4e4d019 --- /dev/null +++ b/tests/core/version.py @@ -0,0 +1,99 @@ +import re +import tomllib + +import pytest + +from core.version import current_version, is_prerelease, parse_version +from tests.support import ROOT + +ZERO = (0, 0, 0, 0, 0, 0) + + +def current_version_reads_pyproject(): + with open(ROOT / "pyproject.toml", "rb") as file: + expected = tomllib.load(file)["project"]["version"] + assert current_version() == expected + + +@pytest.mark.parametrize( + ("version", "expected"), + [ + ("26.08.12", (26, 8, 12, 3, 0, 0)), + ("v1.2.3", (1, 2, 3, 3, 0, 0)), + (" 1.2.3 ", (1, 2, 3, 3, 0, 0)), + ("1.2.3rc1", (1, 2, 3, 2, 1, 0)), + ("1.2.3-rc2", (1, 2, 3, 2, 2, 0)), + ("1.2.3.beta4", (1, 2, 3, 1, 4, 0)), + ("1.2.3b", (1, 2, 3, 1, 0, 0)), + ("1.2.3-alpha", (1, 2, 3, 0, 0, 0)), + ("1.2.3a7", (1, 2, 3, 0, 7, 0)), + ("1.2.3RC1", (1, 2, 3, 2, 1, 0)), + ("1.2.3-beta.2", (1, 2, 3, 1, 2, 0)), + ("26.09.0rc1.2", (26, 9, 0, 2, 1, 2)), + ("unknown", ZERO), + ("1.2", ZERO), + ("1.2.3.4", ZERO), + ], +) +def parse_version_cases(version, expected): + assert parse_version(version) == expected + + +@pytest.mark.parametrize( + ("older", "newer"), + [ + ("1.0.0-alpha1", "1.0.0-beta1"), + ("1.0.0-beta9", "1.0.0rc1"), + ("1.0.0-beta.1", "1.0.0-beta.2"), + ("1.0.0-beta.3", "1.0.0rc1"), + ("1.0.0rc1", "1.0.0rc1.1"), + ("1.0.0rc1.9", "1.0.0rc2"), + ("1.0.0rc9", "1.0.0"), + ("1.0.9", "1.0.10"), + ("26.08.12", "26.09.0"), + ], +) +def parse_version_ordering(older, newer): + assert parse_version(older) < parse_version(newer) + + +@pytest.mark.parametrize( + ("version", "expected"), + [ + ("1.0.0", False), + ("v1.0.0", False), + ("1.0.0rc1", True), + ("1.0.0-beta", True), + ("1.0.0-beta.1", True), + ("26.09.0rc1.2", True), + ("garbage", False), + ], +) +def is_prerelease_cases(version, expected): + assert is_prerelease(version) is expected + + +def _bump_patterns(): + workflow = (ROOT / ".github" / "workflows" / "bump.yml").read_text(encoding="utf-8") + patterns = re.findall(r"=~ (\^\S+\$) \]\]", workflow) + assert len(patterns) == 2, "bump.yml version checks changed; update this test" + return patterns + + +@pytest.mark.parametrize( + "version", + [ + "26.09.0", + "26.09.0rc1", + "26.09.0rc1.2", + "26.09.0-beta.1", + "26.09.0.alpha3", + "26.09.0b", + "26.09", + "26.09.0-dev1", + ], +) +def bump_versions_are_understood(version): + if any(re.fullmatch(pattern, version) for pattern in _bump_patterns()): + assert parse_version(version) != ZERO + assert is_prerelease(version) is not re.fullmatch(r"\d+\.\d+\.\d+", version) diff --git a/tests/engines/__init__.py b/tests/engines/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/tests/engines/base.py b/tests/engines/base.py new file mode 100644 index 0000000..83a38bb --- /dev/null +++ b/tests/engines/base.py @@ -0,0 +1,106 @@ +import json +import re + +import pytest + +from core.specs import DatabaseSpec +from engines import registry +from engines.base import DbEngine, StandardSqlEngine +from services.envfile import EnvFile +from services.project import spec_from_entry + +MANAGED = [engine for engine in registry if engine.template is not None] +VARIANTS = [ + pytest.param(engine, auth, id=f"{engine.key}-{'auth' if auth else 'noauth'}") + for engine in MANAGED + for auth in ((True, False) if engine.auth_variants else (True,)) +] + + +@pytest.mark.parametrize(("engine", "auth"), VARIANTS) +def generate_is_unique(engine, auth, ports): + first = engine.generate(auth=auth, ports=ports, answers={}) + second = engine.generate(auth=auth, ports=ports, answers={}) + assert first.id != second.id + assert first.host != second.host + assert first.host_port != second.host_port + + +@pytest.mark.parametrize(("engine", "auth"), VARIANTS) +def env_vars_are_prefixed_strings(engine, auth, ports): + spec = engine.generate(auth=auth, ports=ports, answers={}) + env = engine.env_vars(spec) + assert env[f"{spec.env_prefix}_PORT"] == str(spec.host_port) + assert all(key.startswith(spec.env_prefix + "_") for key in env) + assert all(isinstance(value, str) for value in env.values()) + + +@pytest.mark.parametrize(("engine", "auth"), VARIANTS) +def template_ctx_points_at_env_vars(engine, auth, ports): + spec = engine.generate(auth=auth, ports=ports, answers={}) + ctx = engine.template_ctx(spec) + assert (ctx["name"], ctx["volume"], ctx["auth"]) == ( + spec.host, + f"{spec.host}-data", + auth, + ) + for key in ("port_var", "db_var", "user_var", "password_var"): + assert re.fullmatch(rf"\$\{{{spec.env_prefix}_[A-Z_]+\}}", ctx[key]) + + +@pytest.mark.parametrize(("engine", "auth"), VARIANTS) +def agent_entry_survives_databases_json(engine, auth, ports, tmp_path): + spec = engine.generate(auth=auth, ports=ports, answers={}) + entry = json.loads(json.dumps(engine.agent_entry(spec))) + env = EnvFile(tmp_path / ".env") + env.merge(engine.env_vars(spec)) + loaded = spec_from_entry(entry, env) + assert loaded.managed + assert (loaded.id, loaded.engine, loaded.host, loaded.port, loaded.host_port) == ( + spec.id, + spec.engine, + spec.host, + spec.port, + spec.host_port, + ) + assert (loaded.password, loaded.root_password) == ( + spec.password, + spec.root_password, + ) + + +def service_name_format(): + assert re.fullmatch(r"db-pg-[0-9a-f]{4}", DbEngine.service_name("pg")) + assert re.fullmatch( + r"db-pg-auth-[0-9a-f]{4}", DbEngine.service_name("pg", auth=True) + ) + + +def var_references_or_escapes(): + spec = DatabaseSpec(id="1", engine="postgresql", name="n", host="db-pg-ab12") + assert DbEngine.var(spec, "PASS", "x", inline=False) == "${DB_PG_AB12_PASS}" + assert DbEngine.var(spec, "PASS", 'a"b\\c', inline=True) == 'a\\"b\\\\c' + assert DbEngine.var(spec, "PASS", None, inline=True) == "" + assert DbEngine.var(spec, "PORT", 5432, inline=True) == "5432" + + +def engine_without_required_attributes_is_refused(): + with pytest.raises(TypeError, match="missing required engine attribute"): + + class Broken(StandardSqlEngine): + key, display = "broken", "Broken" + + +def abstract_engine_may_be_incomplete(): + class Base(DbEngine): + abstract = True + + assert Base.abstract + + +def template_must_live_under_engines(): + with pytest.raises(TypeError, match="must be a path under 'engines/'"): + + class Misplaced(StandardSqlEngine): + key, display, default_port = "misplaced", "Misplaced", 1 + template, slug, db_prefix = "misplaced.yml.j2", "m", "m" diff --git a/tests/engines/docker_volume.py b/tests/engines/docker_volume.py new file mode 100644 index 0000000..f2123eb --- /dev/null +++ b/tests/engines/docker_volume.py @@ -0,0 +1,84 @@ +from core.specs import DatabaseSpec +from engines import registry +from engines.docker_volume import DockerVolumeEngine +from tests.support import agent_service, field_specs + +VOLUME = registry.get("docker-volume") +SOCKET = "/var/run/docker.sock:/var/run/docker.sock" + + +def attributes(): + assert type(VOLUME) is DockerVolumeEngine + assert (VOLUME.key, VOLUME.display, VOLUME.default_port) == ( + "docker-volume", + "Docker Volume", + None, + ) + assert VOLUME.template is None + assert (VOLUME.auth_variants, VOLUME.has_modes) == (False, False) + assert "/var/run/docker.sock" in (VOLUME.warning or "") + + +def generate(ports): + answers = {"volume": " data ", "container": "app", "label": "Files"} + spec = VOLUME.generate(auth=False, ports=ports, answers=answers) + assert spec == DatabaseSpec( + id=spec.id, + engine="docker-volume", + name="Files", + volume="data", + container="app", + ) + assert VOLUME.describe(spec) == "volume: data" + + +def env_vars(): + assert VOLUME.env_vars(VOLUME.from_existing({"volume": "v"})) == {} + + +def agent_entry(): + bare = VOLUME.from_existing({"volume": "v"}) + with_container = VOLUME.from_existing({"volume": "v", "container": "app"}) + assert VOLUME.agent_entry(bare) == { + "name": "Docker Volume", + "type": "docker-volume", + "volume_name": "v", + "generated_id": bare.id, + } + assert VOLUME.agent_entry(with_container) == { + "name": "Docker Volume", + "type": "docker-volume", + "volume_name": "v", + "container_name": "app", + "generated_id": with_container.id, + } + + +def fields(): + assert field_specs(VOLUME.fields_existing()) == [ + ("volume", "text", None), + ("container", "text", ""), + ] + assert VOLUME.fields_new() == VOLUME.fields_existing() + assert VOLUME.option_fields() == [] + + +def from_existing(): + spec = VOLUME.from_existing({"volume": " data ", "container": " "}) + assert spec == DatabaseSpec( + id=spec.id, engine="docker-volume", name="Docker Volume", volume="data" + ) + + +def compose_service(render_engine): + rendered = render_engine( + "docker-volume", answers={"volume": "v", "container": "app"} + ) + assert rendered.doc["services"] == {"agent": agent_service(SOCKET)} + assert "volumes" not in rendered.doc + assert rendered.databases == [VOLUME.agent_entry(rendered.spec)] + + +def compose_service_inline(render_engine): + rendered = render_engine("docker-volume", inline=True, answers={"volume": "v"}) + assert rendered.doc["services"] == {"agent": agent_service(SOCKET, inline=True)} diff --git a/tests/engines/firebird.py b/tests/engines/firebird.py new file mode 100644 index 0000000..6431bee --- /dev/null +++ b/tests/engines/firebird.py @@ -0,0 +1,145 @@ +import re + +from core.specs import DatabaseSpec +from engines import registry +from engines.firebird import FirebirdEngine +from tests.support import EXISTING_ANSWERS, field_specs + +FIREBIRD = registry.get("firebird") +HEALTH = {"interval": "10s", "timeout": "5s", "retries": 5} + + +def attributes(): + assert type(FIREBIRD) is FirebirdEngine + assert (FIREBIRD.key, FIREBIRD.display, FIREBIRD.default_port) == ( + "firebird", + "Firebird", + 3050, + ) + assert FIREBIRD.template == "engines/firebird.yml.j2" + assert (FIREBIRD.auth_variants, FIREBIRD.has_modes, FIREBIRD.warning) == ( + False, + True, + None, + ) + + +def generate(ports): + spec = FIREBIRD.generate(auth=True, ports=ports, answers={}) + assert re.fullmatch(r"db-firebird-[0-9a-f]{4}", spec.host or "") + assert len(spec.password or "") == 16 + assert len(spec.root_password or "") == 16 + assert spec.password != spec.root_password + assert spec == DatabaseSpec( + id=spec.id, + engine="firebird", + name="mirror.fdb", + managed=True, + host=spec.host, + port=3050, + host_port=40000, + database="/var/lib/firebird/data/mirror.fdb", + username="alice", + password=spec.password, + root_password=spec.root_password, + ) + assert FIREBIRD.describe(spec) == f"{spec.host}:3050" + + +def env_vars(ports): + spec = FIREBIRD.generate(auth=True, ports=ports, answers={}) + prefix = spec.env_prefix + assert FIREBIRD.env_vars(spec) == { + f"{prefix}_PORT": "40000", + f"{prefix}_DB": "mirror.fdb", + f"{prefix}_USER": "alice", + f"{prefix}_PASS": spec.password, + f"{prefix}_ROOT_PASS": spec.root_password, + } + + +def agent_entry(ports): + spec = FIREBIRD.generate(auth=True, ports=ports, answers={}) + assert FIREBIRD.agent_entry(spec) == { + "name": "mirror.fdb", + "database": "/var/lib/firebird/data/mirror.fdb", + "type": "firebird", + "username": "alice", + "password": spec.password, + "port": 3050, + "host": spec.host, + "generated_id": spec.id, + } + + +def fields(): + assert field_specs(FIREBIRD.fields_existing()) == [ + ("host", "text", "localhost"), + ("port", "int", 3050), + ("database", "text", None), + ("username", "text", None), + ("password", "secret", None), + ] + assert FIREBIRD.fields_new() == [] + assert FIREBIRD.option_fields() == [] + + +def from_existing(): + spec = FIREBIRD.from_existing({**EXISTING_ANSWERS, "label": "Prod"}) + assert spec == DatabaseSpec( + id=spec.id, + engine="firebird", + name="Prod", + host="db.example", + port=1234, + database="app", + username="u", + password="p", + ) + assert FIREBIRD.from_existing(EXISTING_ANSWERS).name == "External DB" + + +def compose_service(render_engine): + rendered = render_engine("firebird") + assert rendered.service == { + "image": "firebirdsql/firebird", + "restart": "unless-stopped", + "networks": ["portabase"], + "ports": [f"{rendered.var('PORT')}:3050"], + "volumes": [f"{rendered.spec.host}-data:/var/lib/firebird/data"], + "environment": [ + f"FIREBIRD_DATABASE={rendered.var('DB')}", + f"FIREBIRD_USER={rendered.var('USER')}", + f"FIREBIRD_PASSWORD={rendered.var('PASS')}", + f"FIREBIRD_ROOT_PASSWORD={rendered.var('ROOT_PASS')}", + "FIREBIRD_DATABASE_DEFAULT_CHARSET=UTF8", + ], + "healthcheck": {"test": ["CMD-SHELL", "nc -z localhost 3050"], **HEALTH}, + } + + +def compose_service_inline(render_engine): + rendered = render_engine("firebird", inline=True) + assert rendered.service["ports"] == ["40000:3050"] + assert rendered.service["environment"] == [ + "FIREBIRD_DATABASE=mirror.fdb", + "FIREBIRD_USER=alice", + f"FIREBIRD_PASSWORD={rendered.spec.password}", + f"FIREBIRD_ROOT_PASSWORD={rendered.spec.root_password}", + "FIREBIRD_DATABASE_DEFAULT_CHARSET=UTF8", + ] + + +def template_ctx_uses_the_file_name(ports): + spec = FIREBIRD.generate(auth=True, ports=ports, answers={}) + prefix = spec.env_prefix + ctx = FIREBIRD.template_ctx(spec) + inline = FIREBIRD.template_ctx(spec, inline=True) + assert (ctx["db_var"], ctx["root_password_var"]) == ( + f"${{{prefix}_DB}}", + f"${{{prefix}_ROOT_PASS}}", + ) + assert (inline["db_var"], inline["root_password_var"]) == ( + "mirror.fdb", + spec.root_password, + ) diff --git a/tests/engines/mariadb.py b/tests/engines/mariadb.py new file mode 100644 index 0000000..a3e53a2 --- /dev/null +++ b/tests/engines/mariadb.py @@ -0,0 +1,131 @@ +import re + +from core.specs import DatabaseSpec +from engines import registry +from engines.mariadb import MariaDbEngine +from tests.support import EXISTING_ANSWERS, field_specs + +MARIADB = registry.get("mariadb") +HEALTH = {"interval": "10s", "timeout": "5s", "retries": 5} + + +def attributes(): + assert type(MARIADB) is MariaDbEngine + assert (MARIADB.key, MARIADB.display, MARIADB.default_port) == ( + "mariadb", + "MariaDB", + 3306, + ) + assert MARIADB.template == "engines/mariadb.yml.j2" + assert (MARIADB.auth_variants, MARIADB.has_modes, MARIADB.warning) == ( + False, + True, + None, + ) + + +def generate(ports): + spec = MARIADB.generate(auth=True, ports=ports, answers={}) + assert re.fullmatch(r"db-mariadb-[0-9a-f]{4}", spec.host or "") + assert re.fullmatch(r"mysql_[0-9a-f]{8}", spec.database or "") + assert len(spec.password or "") == 16 + assert spec == DatabaseSpec( + id=spec.id, + engine="mariadb", + name=spec.database or "", + managed=True, + host=spec.host, + port=3306, + host_port=40000, + database=spec.database, + username="admin", + password=spec.password, + ) + assert MARIADB.describe(spec) == f"{spec.host}:3306" + + +def env_vars(ports): + spec = MARIADB.generate(auth=True, ports=ports, answers={}) + prefix = spec.env_prefix + assert MARIADB.env_vars(spec) == { + f"{prefix}_PORT": "40000", + f"{prefix}_DB": spec.database, + f"{prefix}_USER": "admin", + f"{prefix}_PASS": spec.password, + } + + +def agent_entry(ports): + spec = MARIADB.generate(auth=True, ports=ports, answers={}) + assert MARIADB.agent_entry(spec) == { + "name": spec.name, + "database": spec.database, + "type": "mariadb", + "username": "admin", + "password": spec.password, + "port": 3306, + "host": spec.host, + "generated_id": spec.id, + } + + +def fields(): + assert field_specs(MARIADB.fields_existing()) == [ + ("host", "text", "localhost"), + ("port", "int", 3306), + ("database", "text", None), + ("username", "text", None), + ("password", "secret", None), + ] + assert MARIADB.fields_new() == [] + assert MARIADB.option_fields() == [] + + +def from_existing(): + spec = MARIADB.from_existing({**EXISTING_ANSWERS, "label": "Prod"}) + assert spec == DatabaseSpec( + id=spec.id, + engine="mariadb", + name="Prod", + host="db.example", + port=1234, + database="app", + username="u", + password="p", + ) + assert MARIADB.from_existing(EXISTING_ANSWERS).name == "External DB" + + +def compose_service(render_engine): + rendered = render_engine("mariadb") + assert rendered.service == { + "image": "mariadb:latest", + "restart": "unless-stopped", + "networks": ["portabase"], + "ports": [f"{rendered.var('PORT')}:3306"], + "environment": [ + f"MYSQL_DATABASE={rendered.var('DB')}", + f"MYSQL_USER={rendered.var('USER')}", + f"MYSQL_PASSWORD={rendered.var('PASS')}", + "MYSQL_RANDOM_ROOT_PASSWORD=yes", + ], + "volumes": [f"{rendered.spec.host}-data:/var/lib/mysql"], + "healthcheck": { + "test": [ + "CMD-SHELL", + f"mariadb-admin ping -h localhost -u {rendered.var('USER')} -p{rendered.var('PASS')}", + ], + **HEALTH, + }, + } + + +def compose_service_inline(render_engine): + rendered = render_engine("mariadb", inline=True) + assert rendered.service["ports"] == ["40000:3306"] + assert rendered.service["environment"] == [ + f"MYSQL_DATABASE={rendered.spec.database}", + "MYSQL_USER=admin", + f"MYSQL_PASSWORD={rendered.spec.password}", + "MYSQL_RANDOM_ROOT_PASSWORD=yes", + ] diff --git a/tests/engines/mongodb.py b/tests/engines/mongodb.py new file mode 100644 index 0000000..dce63a4 --- /dev/null +++ b/tests/engines/mongodb.py @@ -0,0 +1,132 @@ +import re + +import pytest + +from core.specs import DatabaseSpec +from engines import registry +from engines.mongodb import MongoEngine +from tests.support import EXISTING_ANSWERS, field_specs + +MONGO = registry.get("mongodb") +HEALTH = {"interval": "10s", "timeout": "5s", "retries": 5} +AUTH = pytest.mark.parametrize("auth", [True, False], ids=["auth", "noauth"]) + + +def attributes(): + assert type(MONGO) is MongoEngine + assert (MONGO.key, MONGO.display, MONGO.default_port) == ( + "mongodb", + "MongoDB", + 27017, + ) + assert MONGO.template == "engines/mongodb.yml.j2" + assert (MONGO.auth_variants, MONGO.has_modes, MONGO.warning) == (True, True, None) + + +@AUTH +def generate(ports, auth): + spec = MONGO.generate(auth=auth, ports=ports, answers={}) + suffix = "auth-" if auth else "" + assert re.fullmatch(rf"db-mongo-{suffix}[0-9a-f]{{4}}", spec.host or "") + assert re.fullmatch(r"mongo_[0-9a-f]{8}", spec.database or "") + assert len(spec.password or "") == (16 if auth else 0) + assert spec == DatabaseSpec( + id=spec.id, + engine="mongodb", + name=spec.database or "", + managed=True, + host=spec.host, + port=27017, + host_port=40000, + database=spec.database, + username="admin" if auth else "", + password=spec.password, + ) + assert MONGO.describe(spec) == f"{spec.host}:27017" + + +@AUTH +def env_vars(ports, auth): + spec = MONGO.generate(auth=auth, ports=ports, answers={}) + prefix = spec.env_prefix + expected = {f"{prefix}_PORT": "40000", f"{prefix}_DB": spec.database} + if auth: + expected |= {f"{prefix}_USER": "admin", f"{prefix}_PASS": spec.password} + assert MONGO.env_vars(spec) == expected + + +@AUTH +def agent_entry(ports, auth): + spec = MONGO.generate(auth=auth, ports=ports, answers={}) + assert MONGO.agent_entry(spec) == { + "name": spec.name, + "database": spec.database, + "type": "mongodb", + "username": "admin" if auth else "", + "password": spec.password or "", + "port": 27017, + "host": spec.host, + "generated_id": spec.id, + } + + +def fields(): + assert field_specs(MONGO.fields_existing()) == [ + ("host", "text", "localhost"), + ("port", "int", 27017), + ("database", "text", None), + ("username", "text", None), + ("password", "secret", None), + ] + assert MONGO.fields_new() == [] + assert MONGO.option_fields() == [] + + +def from_existing(): + spec = MONGO.from_existing({**EXISTING_ANSWERS, "label": "Prod"}) + assert spec == DatabaseSpec( + id=spec.id, + engine="mongodb", + name="Prod", + host="db.example", + port=1234, + database="app", + username="u", + password="p", + ) + assert MONGO.from_existing(EXISTING_ANSWERS).name == "External DB" + + +@AUTH +def compose_service(render_engine, auth): + rendered = render_engine("mongodb", auth=auth) + expected = { + "image": "mongo:latest", + "restart": "unless-stopped", + "networks": ["portabase"], + "ports": [f"{rendered.var('PORT')}:27017"], + "environment": [f"MONGO_INITDB_DATABASE={rendered.var('DB')}"], + "volumes": [f"{rendered.spec.host}-data:/data/db"], + "healthcheck": { + "test": ["CMD-SHELL", "mongosh --eval 'db.runCommand({ping:1})' --quiet"], + **HEALTH, + }, + } + if auth: + expected["environment"] = [ + f"MONGO_INITDB_ROOT_USERNAME={rendered.var('USER')}", + f"MONGO_INITDB_ROOT_PASSWORD={rendered.var('PASS')}", + f"MONGO_INITDB_DATABASE={rendered.var('DB')}", + ] + expected["command"] = "mongod --auth" + assert rendered.service == expected + + +def compose_service_inline(render_engine): + rendered = render_engine("mongodb", auth=True, inline=True) + assert rendered.service["ports"] == ["40000:27017"] + assert rendered.service["environment"] == [ + "MONGO_INITDB_ROOT_USERNAME=admin", + f"MONGO_INITDB_ROOT_PASSWORD={rendered.spec.password}", + f"MONGO_INITDB_DATABASE={rendered.spec.database}", + ] diff --git a/tests/engines/mssql.py b/tests/engines/mssql.py new file mode 100644 index 0000000..f749e82 --- /dev/null +++ b/tests/engines/mssql.py @@ -0,0 +1,120 @@ +import re + +from core.specs import DatabaseSpec +from engines import registry +from engines.mssql import MssqlEngine +from tests.support import EXISTING_ANSWERS, field_specs + +MSSQL = registry.get("mssql") + + +def attributes(): + assert type(MSSQL) is MssqlEngine + assert (MSSQL.key, MSSQL.display, MSSQL.default_port) == ( + "mssql", + "Microsoft SQL Server", + 1433, + ) + assert MSSQL.template == "engines/mssql.yml.j2" + assert (MSSQL.auth_variants, MSSQL.has_modes, MSSQL.warning) == (False, True, None) + + +def generate(ports): + spec = MSSQL.generate(auth=True, ports=ports, answers={}) + assert re.fullmatch(r"db-mssql-[0-9a-f]{4}", spec.host or "") + assert len(spec.password or "") == 16 + assert spec == DatabaseSpec( + id=spec.id, + engine="mssql", + name="MSSQL", + managed=True, + host=spec.host, + port=1433, + host_port=40000, + database="master", + username="sa", + password=spec.password, + ) + assert MSSQL.describe(spec) == f"{spec.host}:1433" + + +def env_vars(ports): + spec = MSSQL.generate(auth=True, ports=ports, answers={}) + prefix = spec.env_prefix + assert MSSQL.env_vars(spec) == { + f"{prefix}_PORT": "40000", + f"{prefix}_PASS": spec.password, + } + + +def agent_entry(ports): + spec = MSSQL.generate(auth=True, ports=ports, answers={}) + assert MSSQL.agent_entry(spec) == { + "name": "MSSQL", + "database": "master", + "type": "mssql", + "username": "sa", + "password": spec.password, + "port": 1433, + "host": spec.host, + "generated_id": spec.id, + } + + +def fields(): + assert field_specs(MSSQL.fields_existing()) == [ + ("host", "text", "localhost"), + ("port", "int", 1433), + ("database", "text", None), + ("username", "text", None), + ("password", "secret", None), + ] + assert MSSQL.fields_new() == [] + assert MSSQL.option_fields() == [] + + +def from_existing(): + spec = MSSQL.from_existing({**EXISTING_ANSWERS, "label": "Prod"}) + assert spec == DatabaseSpec( + id=spec.id, + engine="mssql", + name="Prod", + host="db.example", + port=1234, + database="app", + username="u", + password="p", + ) + assert MSSQL.from_existing(EXISTING_ANSWERS).name == "External DB" + + +def compose_service(render_engine): + rendered = render_engine("mssql") + assert rendered.service == { + "image": "mcr.microsoft.com/azure-sql-edge:latest", + "restart": "unless-stopped", + "networks": ["portabase"], + "ports": [f"{rendered.var('PORT')}:1433"], + "environment": ["ACCEPT_EULA=Y", f"MSSQL_SA_PASSWORD={rendered.var('PASS')}"], + "volumes": [f"{rendered.spec.host}-data:/var/opt/mssql"], + "healthcheck": { + "test": ["CMD-SHELL", "cat /proc/net/tcp6 | grep -q '059901' || exit 1"], + "interval": "10s", + "timeout": "5s", + "retries": 20, + }, + } + + +def compose_service_inline(render_engine): + rendered = render_engine("mssql", inline=True) + assert rendered.service["ports"] == ["40000:1433"] + assert rendered.service["environment"] == [ + "ACCEPT_EULA=Y", + f"MSSQL_SA_PASSWORD={rendered.spec.password}", + ] + + +def options_are_ignored(ports): + spec = MSSQL.generate(auth=True, ports=ports, answers={"options": {"x": 1}}) + assert spec.options == {} diff --git a/tests/engines/mysql.py b/tests/engines/mysql.py new file mode 100644 index 0000000..ace0c9c --- /dev/null +++ b/tests/engines/mysql.py @@ -0,0 +1,126 @@ +import re + +from core.specs import DatabaseSpec +from engines import registry +from engines.mariadb import MariaDbEngine +from engines.mysql import MySqlEngine +from tests.support import EXISTING_ANSWERS, field_specs + +MYSQL = registry.get("mysql") +HEALTH = {"interval": "10s", "timeout": "5s", "retries": 5} + + +def attributes(): + assert type(MYSQL) is MySqlEngine + assert isinstance(MYSQL, MariaDbEngine) + assert (MYSQL.key, MYSQL.display, MYSQL.default_port) == ("mysql", "MySQL", 3306) + assert MYSQL.template == "engines/mysql.yml.j2" + assert (MYSQL.auth_variants, MYSQL.has_modes, MYSQL.warning) == (False, True, None) + + +def generate(ports): + spec = MYSQL.generate(auth=True, ports=ports, answers={}) + assert re.fullmatch(r"db-mariadb-[0-9a-f]{4}", spec.host or "") + assert re.fullmatch(r"mysql_[0-9a-f]{8}", spec.database or "") + assert len(spec.password or "") == 16 + assert spec == DatabaseSpec( + id=spec.id, + engine="mysql", + name=spec.database or "", + managed=True, + host=spec.host, + port=3306, + host_port=40000, + database=spec.database, + username="admin", + password=spec.password, + ) + assert MYSQL.describe(spec) == f"{spec.host}:3306" + + +def env_vars(ports): + spec = MYSQL.generate(auth=True, ports=ports, answers={}) + prefix = spec.env_prefix + assert MYSQL.env_vars(spec) == { + f"{prefix}_PORT": "40000", + f"{prefix}_DB": spec.database, + f"{prefix}_USER": "admin", + f"{prefix}_PASS": spec.password, + } + + +def agent_entry(ports): + spec = MYSQL.generate(auth=True, ports=ports, answers={}) + assert MYSQL.agent_entry(spec) == { + "name": spec.name, + "database": spec.database, + "type": "mysql", + "username": "admin", + "password": spec.password, + "port": 3306, + "host": spec.host, + "generated_id": spec.id, + } + + +def fields(): + assert field_specs(MYSQL.fields_existing()) == [ + ("host", "text", "localhost"), + ("port", "int", 3306), + ("database", "text", None), + ("username", "text", None), + ("password", "secret", None), + ] + assert MYSQL.fields_new() == [] + assert MYSQL.option_fields() == [] + + +def from_existing(): + spec = MYSQL.from_existing({**EXISTING_ANSWERS, "label": "Prod"}) + assert spec == DatabaseSpec( + id=spec.id, + engine="mysql", + name="Prod", + host="db.example", + port=1234, + database="app", + username="u", + password="p", + ) + assert MYSQL.from_existing(EXISTING_ANSWERS).name == "External DB" + + +def compose_service(render_engine): + rendered = render_engine("mysql") + assert rendered.service == { + # MySQL databases have always run on the MariaDB image (wire compatible). + "image": "mariadb:latest", + "restart": "unless-stopped", + "networks": ["portabase"], + "ports": [f"{rendered.var('PORT')}:3306"], + "environment": [ + f"MYSQL_DATABASE={rendered.var('DB')}", + f"MYSQL_USER={rendered.var('USER')}", + f"MYSQL_PASSWORD={rendered.var('PASS')}", + "MYSQL_RANDOM_ROOT_PASSWORD=yes", + ], + "volumes": [f"{rendered.spec.host}-data:/var/lib/mysql"], + "healthcheck": { + "test": [ + "CMD-SHELL", + f"mariadb-admin ping -h localhost -u {rendered.var('USER')} -p{rendered.var('PASS')}", + ], + **HEALTH, + }, + } + + +def compose_service_inline(render_engine): + rendered = render_engine("mysql", inline=True) + assert rendered.service["ports"] == ["40000:3306"] + assert rendered.service["environment"] == [ + f"MYSQL_DATABASE={rendered.spec.database}", + "MYSQL_USER=admin", + f"MYSQL_PASSWORD={rendered.spec.password}", + "MYSQL_RANDOM_ROOT_PASSWORD=yes", + ] diff --git a/tests/engines/postgresql.py b/tests/engines/postgresql.py new file mode 100644 index 0000000..40900b0 --- /dev/null +++ b/tests/engines/postgresql.py @@ -0,0 +1,139 @@ +import re + +from core.specs import DatabaseSpec +from engines import registry +from engines.postgresql import PostgresEngine +from tests.support import EXISTING_ANSWERS, field_specs + +PG = registry.get("postgresql") +HEALTH = {"interval": "10s", "timeout": "5s", "retries": 5} + + +def attributes(): + assert type(PG) is PostgresEngine + assert (PG.key, PG.display, PG.default_port) == ("postgresql", "PostgreSQL", 5432) + assert PG.template == "engines/postgresql.yml.j2" + assert (PG.auth_variants, PG.has_modes, PG.warning) == (False, True, None) + + +def generate(ports): + spec = PG.generate(auth=True, ports=ports, answers={}) + assert re.fullmatch(r"db-pg-[0-9a-f]{4}", spec.host or "") + assert re.fullmatch(r"pg_[0-9a-f]{8}", spec.database or "") + assert len(spec.password or "") == 16 + assert spec == DatabaseSpec( + id=spec.id, + engine="postgresql", + name=spec.database or "", + managed=True, + host=spec.host, + port=5432, + host_port=40000, + database=spec.database, + username="admin", + password=spec.password, + ) + assert PG.describe(spec) == f"{spec.host}:5432" + + +def env_vars(ports): + spec = PG.generate(auth=True, ports=ports, answers={}) + prefix = spec.env_prefix + assert PG.env_vars(spec) == { + f"{prefix}_PORT": "40000", + f"{prefix}_DB": spec.database, + f"{prefix}_USER": "admin", + f"{prefix}_PASS": spec.password, + } + + +def agent_entry(ports): + spec = PG.generate(auth=True, ports=ports, answers={}) + assert PG.agent_entry(spec) == { + "name": spec.name, + "database": spec.database, + "type": "postgresql", + "username": "admin", + "password": spec.password, + "port": 5432, + "host": spec.host, + "generated_id": spec.id, + } + + +def fields(): + assert field_specs(PG.fields_existing()) == [ + ("host", "text", "localhost"), + ("port", "int", 5432), + ("database", "text", None), + ("username", "text", None), + ("password", "secret", None), + ] + assert PG.fields_new() == [] + assert field_specs(PG.option_fields()) == [ + ("keep_ownership", "bool", False), + ("clean_mode", "choice", "clean"), + ] + assert PG.option_fields()[1].choices == ( + "clean", + "none", + "drop_schemas", + "drop_database", + ) + + +def from_existing(): + spec = PG.from_existing({**EXISTING_ANSWERS, "label": "Prod"}) + assert spec == DatabaseSpec( + id=spec.id, + engine="postgresql", + name="Prod", + host="db.example", + port=1234, + database="app", + username="u", + password="p", + ) + assert PG.from_existing(EXISTING_ANSWERS).name == "External DB" + + +def compose_service(render_engine): + rendered = render_engine("postgresql") + assert rendered.service == { + "image": "postgres:17-alpine", + "restart": "unless-stopped", + "networks": ["portabase"], + "ports": [f"{rendered.var('PORT')}:5432"], + "volumes": [f"{rendered.spec.host}-data:/var/lib/postgresql/data"], + "environment": [ + f"POSTGRES_DB={rendered.var('DB')}", + f"POSTGRES_USER={rendered.var('USER')}", + f"POSTGRES_PASSWORD={rendered.var('PASS')}", + ], + "healthcheck": { + "test": [ + "CMD-SHELL", + f"pg_isready -U {rendered.var('USER')} -d {rendered.var('DB')}", + ], + **HEALTH, + }, + } + + +def compose_service_inline(render_engine): + rendered = render_engine("postgresql", inline=True) + assert rendered.service["ports"] == ["40000:5432"] + assert rendered.service["environment"] == [ + f"POSTGRES_DB={rendered.spec.database}", + "POSTGRES_USER=admin", + f"POSTGRES_PASSWORD={rendered.spec.password}", + ] + + +def only_non_default_options_reach_the_agent(ports): + options = {"keep_ownership": True, "clean_mode": "clean", "unknown": 1} + spec = PG.generate(auth=True, ports=ports, answers={"options": options}) + assert spec.options == options + assert PG.non_default_options(spec) == {"keep_ownership": True} + assert PG.agent_entry(spec)["options"] == {"keep_ownership": True} + assert "options" not in PG.agent_entry(spec.with_options({"clean_mode": "clean"})) diff --git a/tests/engines/postgresql_cluster.py b/tests/engines/postgresql_cluster.py new file mode 100644 index 0000000..930616f --- /dev/null +++ b/tests/engines/postgresql_cluster.py @@ -0,0 +1,135 @@ +import re + +from core.specs import DatabaseSpec +from engines import registry +from engines.postgresql import PostgresClusterEngine +from tests.support import EXISTING_ANSWERS, field_specs + +CLUSTER = registry.get("postgresql-cluster") +HEALTH = {"interval": "10s", "timeout": "5s", "retries": 5} + + +def attributes(): + assert type(CLUSTER) is PostgresClusterEngine + assert (CLUSTER.key, CLUSTER.display, CLUSTER.default_port) == ( + "postgresql-cluster", + "PostgreSQL Cluster", + 5432, + ) + assert CLUSTER.template == "engines/postgresql-cluster.yml.j2" + assert (CLUSTER.auth_variants, CLUSTER.has_modes) == (False, True) + assert "superuser" in (CLUSTER.warning or "") + assert "pg_dumpall" in (CLUSTER.warning or "") + + +def generate(ports): + spec = CLUSTER.generate(auth=True, ports=ports, answers={}) + assert re.fullmatch(r"db-pg-[0-9a-f]{4}", spec.host or "") + assert re.fullmatch(r"pg_[0-9a-f]{8}", spec.database or "") + assert len(spec.password or "") == 16 + assert spec == DatabaseSpec( + id=spec.id, + engine="postgresql-cluster", + name=spec.database or "", + managed=True, + host=spec.host, + port=5432, + host_port=40000, + database=spec.database, + username="admin", + password=spec.password, + ) + assert CLUSTER.describe(spec) == f"{spec.host}:5432" + + +def env_vars(ports): + spec = CLUSTER.generate(auth=True, ports=ports, answers={}) + prefix = spec.env_prefix + assert CLUSTER.env_vars(spec) == { + f"{prefix}_PORT": "40000", + f"{prefix}_DB": spec.database, + f"{prefix}_USER": "admin", + f"{prefix}_PASS": spec.password, + } + + +def agent_entry(ports): + spec = CLUSTER.generate(auth=True, ports=ports, answers={}) + assert CLUSTER.agent_entry(spec) == { + "name": spec.name, + "database": spec.database, + "type": "postgresql-cluster", + "username": "admin", + "password": spec.password, + "port": 5432, + "host": spec.host, + "generated_id": spec.id, + } + + +def fields(): + assert field_specs(CLUSTER.fields_existing()) == [ + ("host", "text", "localhost"), + ("port", "int", 5432), + ("database", "text", None), + ("username", "text", None), + ("password", "secret", None), + ] + assert CLUSTER.fields_new() == [] + assert CLUSTER.option_fields() == [] + + +def from_existing(): + spec = CLUSTER.from_existing({**EXISTING_ANSWERS, "label": "Prod"}) + assert spec == DatabaseSpec( + id=spec.id, + engine="postgresql-cluster", + name="Prod", + host="db.example", + port=1234, + database="app", + username="u", + password="p", + ) + assert CLUSTER.from_existing(EXISTING_ANSWERS).name == "External DB" + + +def compose_service(render_engine): + rendered = render_engine("postgresql-cluster") + assert rendered.service == { + "image": "postgres:17-alpine", + "restart": "unless-stopped", + "networks": ["portabase"], + "ports": [f"{rendered.var('PORT')}:5432"], + "volumes": [f"{rendered.spec.host}-data:/var/lib/postgresql/data"], + "environment": [ + f"POSTGRES_DB={rendered.var('DB')}", + f"POSTGRES_USER={rendered.var('USER')}", + f"POSTGRES_PASSWORD={rendered.var('PASS')}", + ], + "healthcheck": { + "test": [ + "CMD-SHELL", + f"pg_isready -U {rendered.var('USER')} -d {rendered.var('DB')}", + ], + **HEALTH, + }, + } + + +def compose_service_inline(render_engine): + rendered = render_engine("postgresql-cluster", inline=True) + assert rendered.service["ports"] == ["40000:5432"] + assert rendered.service["environment"] == [ + f"POSTGRES_DB={rendered.spec.database}", + "POSTGRES_USER=admin", + f"POSTGRES_PASSWORD={rendered.spec.password}", + ] + + +def options_are_dropped(ports): + spec = CLUSTER.generate( + auth=True, ports=ports, answers={"options": {"keep_ownership": True}} + ) + assert CLUSTER.non_default_options(spec) == {} + assert "options" not in CLUSTER.agent_entry(spec) diff --git a/tests/engines/redis.py b/tests/engines/redis.py new file mode 100644 index 0000000..a8a053a --- /dev/null +++ b/tests/engines/redis.py @@ -0,0 +1,143 @@ +import dataclasses +import re + +import pytest + +from core.specs import DatabaseSpec +from engines import registry +from engines.redis import RedisEngine +from tests.support import EXISTING_ANSWERS, field_specs + +REDIS = registry.get("redis") +HEALTH = {"interval": "10s", "timeout": "5s", "retries": 5} +AUTH = pytest.mark.parametrize("auth", [True, False], ids=["auth", "noauth"]) + + +def attributes(): + assert type(REDIS) is RedisEngine + assert (REDIS.key, REDIS.display, REDIS.default_port) == ("redis", "Redis", 6379) + assert REDIS.template == "engines/redis.yml.j2" + assert (REDIS.auth_variants, REDIS.has_modes, REDIS.warning) == (True, True, None) + + +@AUTH +def generate(ports, auth): + spec = REDIS.generate(auth=auth, ports=ports, answers={}) + suffix = "auth-" if auth else "" + assert re.fullmatch(rf"db-redis-{suffix}[0-9a-f]{{4}}", spec.host or "") + assert re.fullmatch(r"redis_[0-9a-f]{8}", spec.name) + assert len(spec.password or "") == (16 if auth else 0) + assert spec == DatabaseSpec( + id=spec.id, + engine="redis", + name=spec.name, + managed=True, + host=spec.host, + port=6379, + host_port=40000, + database="0", + username="", + password=spec.password, + ) + assert REDIS.describe(spec) == f"{spec.host}:6379" + + +@AUTH +def env_vars(ports, auth): + spec = REDIS.generate(auth=auth, ports=ports, answers={}) + prefix = spec.env_prefix + expected = {f"{prefix}_PORT": "40000"} + if auth: + expected[f"{prefix}_PASS"] = spec.password or "" + assert REDIS.env_vars(spec) == expected + + +@AUTH +def agent_entry(ports, auth): + spec = REDIS.generate(auth=auth, ports=ports, answers={}) + assert REDIS.agent_entry(spec) == { + "name": spec.name, + "database": "0", + "type": "redis", + "username": "", + "password": spec.password or "", + "port": 6379, + "host": spec.host, + "generated_id": spec.id, + } + + +def fields(): + assert field_specs(REDIS.fields_existing()) == [ + ("host", "text", "localhost"), + ("port", "int", 6379), + ("database", "text", "0"), + ("username", "text", ""), + ("password", "text", ""), + ] + assert REDIS.fields_new() == [] + assert REDIS.option_fields() == [] + + +def from_existing(): + spec = REDIS.from_existing({**EXISTING_ANSWERS, "label": "Prod"}) + assert spec == DatabaseSpec( + id=spec.id, + engine="redis", + name="Prod", + host="db.example", + port=1234, + database="app", + username="u", + password="p", + ) + assert REDIS.from_existing(EXISTING_ANSWERS).name == "External DB" + + +@AUTH +def compose_service(render_engine, auth): + rendered = render_engine("redis", auth=auth) + password = rendered.var("PASS") + expected = { + "image": "redis:latest", + "restart": "unless-stopped", + "ports": [f"{rendered.var('PORT')}:6379"], + "volumes": [f"{rendered.spec.host}-data:/data"], + "command": ["redis-server", "--appendonly", "yes"], + "networks": ["portabase", "default"], + "healthcheck": {"test": ["CMD-SHELL", "redis-cli ping | grep PONG"], **HEALTH}, + } + if auth: + expected["environment"] = [f"REDIS_PASSWORD={password}"] + expected["command"] = [ + "redis-server", + "--requirepass", + password, + "--appendonly", + "yes", + ] + expected["healthcheck"] = { + "test": ["CMD-SHELL", f"redis-cli -a {password} ping | grep PONG"], + **HEALTH, + } + assert rendered.service == expected + + +def compose_service_inline(render_engine): + rendered = render_engine("redis", auth=True, inline=True) + password = rendered.spec.password + assert rendered.service["ports"] == ["40000:6379"] + assert rendered.service["environment"] == [f"REDIS_PASSWORD={password}"] + assert rendered.service["command"] == [ + "redis-server", + "--requirepass", + password, + "--appendonly", + "yes", + ] + + +def agent_database_defaults_to_index_zero(ports): + spec = REDIS.generate(auth=False, ports=ports, answers={}) + assert REDIS.agent_database(dataclasses.replace(spec, database=None)) == "0" + assert REDIS.agent_database(dataclasses.replace(spec, database="3")) == "3" diff --git a/tests/engines/registry.py b/tests/engines/registry.py new file mode 100644 index 0000000..0a27e17 --- /dev/null +++ b/tests/engines/registry.py @@ -0,0 +1,47 @@ +import pytest + +from core.errors import ValidationError +from engines import ALL, EngineRegistry, registry +from engines.postgresql import PostgresEngine + +EXPECTED = [ + "postgresql", + "postgresql-cluster", + "mysql", + "mariadb", + "sqlite", + "firebird", + "mongodb", + "redis", + "valkey", + "mssql", + "docker-volume", +] + + +def keys_in_order(): + assert registry.keys() == EXPECTED + assert registry.choices() == EXPECTED + assert [engine.key for engine in registry] == EXPECTED + assert "redis" in registry + assert "nope" not in registry + + +def get_engine(): + assert isinstance(registry.get("postgresql"), PostgresEngine) + with pytest.raises(ValidationError, match="Unknown engine 'nope'") as exc: + registry.get("nope") + assert "postgresql" in (exc.value.hint or "") + + +def duplicate_keys_are_refused(): + with pytest.raises(ValueError, match="Duplicate engine key: postgresql"): + EngineRegistry([*ALL, PostgresEngine()]) + + +def templates_match_engines(templates): + shipped = {name for name in templates.names() if name.startswith("engines/")} + used = {engine.template for engine in registry if engine.template is not None} + assert used == shipped + for name in used: + templates.get(name) diff --git a/tests/engines/sqlite.py b/tests/engines/sqlite.py new file mode 100644 index 0000000..4abc8aa --- /dev/null +++ b/tests/engines/sqlite.py @@ -0,0 +1,108 @@ +import pytest + +from core.specs import DatabaseSpec +from engines import registry +from engines.sqlite import SqliteEngine +from tests.support import agent_service, field_specs + +SQLITE = registry.get("sqlite") + + +def attributes(): + assert type(SQLITE) is SqliteEngine + assert (SQLITE.key, SQLITE.display, SQLITE.default_port) == ( + "sqlite", + "SQLite", + None, + ) + assert SQLITE.template is None + assert (SQLITE.auth_variants, SQLITE.has_modes, SQLITE.warning) == ( + False, + True, + None, + ) + + +@pytest.mark.parametrize( + ("name", "expected"), + [ + (None, "local.sqlite"), + ("", "local.sqlite"), + ("app", "app.sqlite"), + ("app.sqlite", "app.sqlite"), + ], +) +def generate(ports, name, expected): + spec = SQLITE.generate(auth=False, ports=ports, answers={"name": name}) + assert spec == DatabaseSpec( + id=spec.id, + engine="sqlite", + name=expected, + path=expected, + database=f"/config/{expected}", + ) + assert SQLITE.describe(spec) == "Local File" + + +def env_vars(ports): + spec = SQLITE.generate(auth=False, ports=ports, answers={"name": "app"}) + assert SQLITE.env_vars(spec) == {} + + +def agent_entry(ports): + spec = SQLITE.generate(auth=False, ports=ports, answers={"name": "app"}) + assert SQLITE.agent_entry(spec) == { + "name": "app.sqlite", + "database": "/config/app.sqlite", + "type": "sqlite", + "generated_id": spec.id, + } + + +def fields(): + assert field_specs(SQLITE.fields_existing()) == [("path", "text", None)] + assert field_specs(SQLITE.fields_new()) == [("name", "text", "local")] + assert SQLITE.option_fields() == [] + + +@pytest.mark.parametrize( + ("path", "database"), + [("data/app.db", "/config/data/app.db"), ("/srv/app.db", "/srv/app.db")], + ids=["relative", "absolute"], +) +def from_existing(path, database): + spec = SQLITE.from_existing({"path": path, "label": "Prod"}) + assert spec == DatabaseSpec( + id=spec.id, engine="sqlite", name="Prod", path=path, database=database + ) + assert SQLITE.from_existing({"path": path}).name == "External DB" + + +def compose_service(render_engine): + rendered = render_engine("sqlite", answers={"name": "app"}) + assert rendered.doc["services"] == { + "agent": agent_service("./app.sqlite:/config/app.sqlite") + } + assert "volumes" not in rendered.doc + assert rendered.databases == [SQLITE.agent_entry(rendered.spec)] + + +def compose_service_inline(render_engine): + rendered = render_engine("sqlite", inline=True, answers={"name": "app"}) + assert rendered.doc["services"] == { + "agent": agent_service("./app.sqlite:/config/app.sqlite", inline=True) + } + + +@pytest.mark.parametrize( + ("database", "mount"), + [ + ("/config/app.sqlite", ("./app.sqlite", "/config/app.sqlite")), + ("/config/data/app.db", ("./data/app.db", "/config/data/app.db")), + ("/srv/app.db", None), + (None, None), + ], +) +def mount_for(database, mount): + spec = DatabaseSpec(id="1", engine="sqlite", name="n", database=database) + assert SqliteEngine.mount_for(spec) == mount diff --git a/tests/engines/valkey.py b/tests/engines/valkey.py new file mode 100644 index 0000000..cbdec5c --- /dev/null +++ b/tests/engines/valkey.py @@ -0,0 +1,141 @@ +import dataclasses +import re + +import pytest + +from core.specs import DatabaseSpec +from engines import registry +from engines.valkey import ValkeyEngine +from tests.support import EXISTING_ANSWERS, field_specs + +VALKEY = registry.get("valkey") +HEALTH = {"interval": "10s", "timeout": "5s", "retries": 5} +AUTH = pytest.mark.parametrize("auth", [True, False], ids=["auth", "noauth"]) + + +def attributes(): + assert type(VALKEY) is ValkeyEngine + assert (VALKEY.key, VALKEY.display, VALKEY.default_port) == ( + "valkey", + "Valkey", + 6379, + ) + assert VALKEY.template == "engines/valkey.yml.j2" + assert (VALKEY.auth_variants, VALKEY.has_modes, VALKEY.warning) == ( + True, + True, + None, + ) + + +@AUTH +def generate(ports, auth): + spec = VALKEY.generate(auth=auth, ports=ports, answers={}) + suffix = "auth-" if auth else "" + assert re.fullmatch(rf"db-valkey-{suffix}[0-9a-f]{{4}}", spec.host or "") + assert re.fullmatch(r"valkey_[0-9a-f]{8}", spec.name) + assert len(spec.password or "") == (16 if auth else 0) + assert spec == DatabaseSpec( + id=spec.id, + engine="valkey", + name=spec.name, + managed=True, + host=spec.host, + port=6379, + host_port=40000, + database="0", + username="", + password=spec.password, + ) + assert VALKEY.describe(spec) == f"{spec.host}:6379" + + +@AUTH +def env_vars(ports, auth): + spec = VALKEY.generate(auth=auth, ports=ports, answers={}) + prefix = spec.env_prefix + expected = {f"{prefix}_PORT": "40000"} + if auth: + expected[f"{prefix}_PASS"] = spec.password or "" + assert VALKEY.env_vars(spec) == expected + + +@AUTH +def agent_entry(ports, auth): + spec = VALKEY.generate(auth=auth, ports=ports, answers={}) + assert VALKEY.agent_entry(spec) == { + "name": spec.name, + "database": "0", + "type": "valkey", + "username": "", + "password": spec.password or "", + "port": 6379, + "host": spec.host, + "generated_id": spec.id, + } + + +def fields(): + assert field_specs(VALKEY.fields_existing()) == [ + ("host", "text", "localhost"), + ("port", "int", 6379), + ("database", "text", "0"), + ("username", "text", ""), + ("password", "text", ""), + ] + assert VALKEY.fields_new() == [] + assert VALKEY.option_fields() == [] + + +def from_existing(): + spec = VALKEY.from_existing({**EXISTING_ANSWERS, "label": "Prod"}) + assert spec == DatabaseSpec( + id=spec.id, + engine="valkey", + name="Prod", + host="db.example", + port=1234, + database="app", + username="u", + password="p", + ) + assert VALKEY.from_existing(EXISTING_ANSWERS).name == "External DB" + + +@AUTH +def compose_service(render_engine, auth): + rendered = render_engine("valkey", auth=auth) + password = rendered.var("PASS") + expected = { + "image": "valkey/valkey:latest", + "restart": "unless-stopped", + "environment": ["ALLOW_EMPTY_PASSWORD=yes"], + "ports": [f"{rendered.var('PORT')}:6379"], + "volumes": [f"{rendered.spec.host}-data:/data"], + "networks": ["portabase", "default"], + "healthcheck": {"test": ["CMD-SHELL", "valkey-cli ping | grep PONG"], **HEALTH}, + } + if auth: + del expected["environment"] + expected["command"] = ["valkey-server", "--requirepass", password] + expected["healthcheck"] = { + "test": ["CMD-SHELL", f"valkey-cli -a {password} ping | grep PONG"], + **HEALTH, + } + assert rendered.service == expected + + +def compose_service_inline(render_engine): + rendered = render_engine("valkey", auth=True, inline=True) + assert rendered.service["ports"] == ["40000:6379"] + assert rendered.service["command"] == [ + "valkey-server", + "--requirepass", + rendered.spec.password, + ] + + +def agent_database_defaults_to_index_zero(ports): + spec = VALKEY.generate(auth=False, ports=ports, answers={}) + assert VALKEY.agent_database(dataclasses.replace(spec, database=None)) == "0" + assert VALKEY.agent_database(dataclasses.replace(spec, database="3")) == "3" diff --git a/tests/services/__init__.py b/tests/services/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/tests/services/auth_providers.py b/tests/services/auth_providers.py new file mode 100644 index 0000000..8e0d8ed --- /dev/null +++ b/tests/services/auth_providers.py @@ -0,0 +1,49 @@ +import pytest + +from core.errors import ValidationError +from services import auth_providers as ap + + +@pytest.mark.parametrize( + ("kind", "provider_id", "expected"), + [ + ("oidc", "keycloak", "AUTH_OIDC_KEYCLOAK"), + ("oidc", "my-kc", "AUTH_OIDC_MY_KC"), + ("oauth", "github", "AUTH_SOCIAL_GITHUB"), + ], +) +def provider_prefix_cases(kind, provider_id, expected): + assert ap.provider_prefix(kind, provider_id) == expected + + +@pytest.mark.parametrize( + ("kind", "provider_id", "expected"), + [ + ("oidc", " KeyCloak ", "keycloak"), + ("oidc", "my-kc-2", "my-kc-2"), + ("oauth", "GitHub", "github"), + ], +) +def validate_provider_id_accepts(kind, provider_id, expected): + assert ap.validate_provider_id(kind, provider_id) == expected + + +@pytest.mark.parametrize("provider_id", ["", "-kc", "k_c", "k c", "kc!"]) +def validate_provider_id_rejects_bad_ids(provider_id): + with pytest.raises(ValidationError, match="Invalid provider id"): + ap.validate_provider_id("oidc", provider_id) + + +def validate_provider_id_rejects_unknown_oauth(): + with pytest.raises(ValidationError, match="Unknown OAuth provider 'gitlab'") as exc: + ap.validate_provider_id("oauth", "gitlab") + assert "github" in (exc.value.hint or "") + + +@pytest.mark.parametrize( + ("fields", "env"), + [(ap.OIDC_FIELDS, ap.OIDC_ENV), (ap.OAUTH_FIELDS, ap.OAUTH_ENV)], + ids=["oidc", "oauth"], +) +def every_field_has_an_env_suffix(fields, env): + assert [field.name for field in fields] == list(env) diff --git a/tests/services/compose_facts.py b/tests/services/compose_facts.py new file mode 100644 index 0000000..c1d23a7 --- /dev/null +++ b/tests/services/compose_facts.py @@ -0,0 +1,76 @@ +import pytest + +from services.compose_facts import ( + CA_BUNDLE_IN_CONTAINER, + GENERATED_MARKER, + ComposeFacts, +) + + +def _facts(tmp_path, text): + path = tmp_path / "docker-compose.yml" + path.write_text(text, encoding="utf-8") + return ComposeFacts(path) + + +def missing_file(tmp_path): + facts = ComposeFacts(tmp_path / "docker-compose.yml") + assert not facts.exists + assert not facts.is_generated + assert not facts.host_gateway + assert facts.ca_bundle is None + + +def generated_marker(tmp_path): + header = f"{GENERATED_MARKER} 1.0. Do not edit.\nservices: {{}}\n" + assert _facts(tmp_path, header).is_generated + assert not _facts(tmp_path, "services: {}\n").is_generated + + +@pytest.mark.parametrize( + "text", + [ + "services:\n agent:\n extra_hosts:\n - localhost:host-gateway\n", + "services:\n agent:\n extra_hosts:\n localhost: host-gateway\n", + ], + ids=["list", "dict"], +) +def host_gateway_detected(tmp_path, text): + assert _facts(tmp_path, text).host_gateway + + +@pytest.mark.parametrize( + "text", + [ + "services:\n agent:\n image: x\n", + "services:\n other:\n extra_hosts: ['localhost:host-gateway']\n", + "services: []\n", + "- not a mapping\n", + "services: [unclosed\n", + ], +) +def host_gateway_absent(tmp_path, text): + assert not _facts(tmp_path, text).host_gateway + + +def ca_bundle_short_syntax(tmp_path): + text = ( + "services:\n agent:\n volumes:\n" + " - ./databases.json:/config/config.json\n" + f" - ./ca.crt:{CA_BUNDLE_IN_CONTAINER}:ro\n" + ) + assert _facts(tmp_path, text).ca_bundle == "./ca.crt" + + +def ca_bundle_long_syntax(tmp_path): + text = ( + "services:\n agent:\n volumes:\n" + " - type: bind\n source: /etc/ca.crt\n" + f" target: {CA_BUNDLE_IN_CONTAINER}\n" + ) + assert _facts(tmp_path, text).ca_bundle == "/etc/ca.crt" + + +def ca_bundle_absent(tmp_path): + text = "services:\n agent:\n volumes:\n - ./db.json:/config/config.json\n" + assert _facts(tmp_path, text).ca_bundle is None diff --git a/tests/services/docker.py b/tests/services/docker.py new file mode 100644 index 0000000..183aa03 --- /dev/null +++ b/tests/services/docker.py @@ -0,0 +1,23 @@ +import pytest + +from core.errors import DockerError +from services import docker +from services.docker import DockerRunner + + +def project_name_is_the_slugified_folder(tmp_path): + folder = tmp_path / "My Agent" + folder.mkdir() + assert DockerRunner.project_name(folder) == "my-agent" + + +def binary_missing(monkeypatch): + monkeypatch.setattr(docker.shutil, "which", lambda _: None) + runner = DockerRunner() + assert not runner.available() + with pytest.raises(DockerError, match="Docker not found"): + _ = runner.binary + + +def binary_explicit(): + assert DockerRunner("/opt/docker").binary == "/opt/docker" diff --git a/tests/services/envfile.py b/tests/services/envfile.py new file mode 100644 index 0000000..aa9fa0c --- /dev/null +++ b/tests/services/envfile.py @@ -0,0 +1,125 @@ +import pytest + +from services.envfile import EnvFile + +SAMPLE = r"""# comment +export A=1 +B = "two words" +C='single # kept' +D=plain # trailing comment +E="esc \"q\" back\\slash" +#F=commented + +G= +""" + + +@pytest.fixture +def env(tmp_path): + path = tmp_path / ".env" + path.write_text(SAMPLE, encoding="utf-8") + return EnvFile.load(path) + + +def missing_file(tmp_path): + env = EnvFile.load(tmp_path / ".env") + assert not env.exists + assert env.as_dict() == {} + assert env.get("A", "default") == "default" + + +def parse(env): + assert env.as_dict() == { + "A": "1", + "B": "two words", + "C": "single # kept", + "D": "plain", + "E": 'esc "q" back\\slash', + "G": "", + } + assert env.get("F") is None + + +def last_duplicate_wins(tmp_path): + path = tmp_path / ".env" + path.write_text("A=1\nA=2\n", encoding="utf-8") + assert EnvFile.load(path).get("A") == "2" + + +@pytest.mark.parametrize( + "value", + [ + "", + "simple", + "with space", + 'quote"inside', + "back\\slash", + "ends\\", + '\\"', + "hash # x", + "a=b", + "'single'", + ], +) +def set_save_load_round_trip(tmp_path, value): + env = EnvFile.load(tmp_path / ".env") + env.set("KEY", value) + env.save() + assert EnvFile.load(tmp_path / ".env").get("KEY") == value + + +def set_existing_key_keeps_position_and_comments(env): + env.set("B", "new") + env.set("Z", "added") + env.save() + lines = env.path.read_text(encoding="utf-8").splitlines() + assert lines[0] == "# comment" + assert lines[2] == 'B="new"' + assert lines[-1] == 'Z="added"' + + +def merge_overrides_and_adds(env): + env.merge({"A": "10", "NEW": "x"}) + assert env.get("A") == "10" + assert env.get("NEW") == "x" + + +def remove_keeps_index_consistent(env): + env.remove("B") + env.remove("missing") + env.set("D", "changed") + assert env.get("B") is None + assert env.get("A") == "1" + assert env.get("C") == "single # kept" + assert env.get("D") == "changed" + env.save() + assert "B =" not in env.path.read_text(encoding="utf-8") + assert EnvFile.load(env.path).get("D") == "changed" + + +def remove_prefix_only_removes_that_prefix(tmp_path): + env = EnvFile.load(tmp_path / ".env") + env.merge( + { + "AUTH_OIDC_KC": "kept", + "AUTH_OIDC_KC_ID": "kc", + "AUTH_OIDC_KC_SECRET": "s", + "AUTH_OIDC_KCX_ID": "kept", + "OTHER": "kept", + } + ) + env.remove_prefix("AUTH_OIDC_KC") + assert env.as_dict() == { + "AUTH_OIDC_KC": "kept", + "AUTH_OIDC_KCX_ID": "kept", + "OTHER": "kept", + } + + +def save_creates_parent_and_ends_with_newline(tmp_path): + env = EnvFile.load(tmp_path / "nested" / ".env") + env.set("A", "1") + env.save() + assert env.exists + assert env.path.read_text(encoding="utf-8") == 'A="1"\n' + assert not (tmp_path / "nested" / ".env.tmp").exists() diff --git a/tests/services/http.py b/tests/services/http.py new file mode 100644 index 0000000..c001919 --- /dev/null +++ b/tests/services/http.py @@ -0,0 +1,147 @@ +import pytest +import requests + +from core.errors import NetworkError +from services.http import HttpClient + + +class _Response: + def __init__(self, *, status=200, text="", json=None, chunks=(), headers=None): + self.status_code = status + self.text = text + self.headers = headers or {} + self._json = json + self._chunks = chunks + + def raise_for_status(self): + if self.status_code >= 400: + raise requests.HTTPError(f"{self.status_code} Error") + + def json(self): + if self._json is None: + raise ValueError("Expecting value") + return self._json + + def iter_content(self, chunk_size): + for chunk in self._chunks: + if isinstance(chunk, Exception): + raise chunk + yield chunk + + def __enter__(self): + return self + + def __exit__(self, *exc): + return False + + +class _Session: + def __init__(self, response=None, error=None): + self.response = response + self.error = error + self.calls = [] + + def _call(self, method, url, kwargs): + self.calls.append((method, url, kwargs)) + if self.error is not None: + raise self.error + return self.response + + def get(self, url, **kwargs): + return self._call("GET", url, kwargs) + + def head(self, url, **kwargs): + return self._call("HEAD", url, kwargs) + + +def _client(**session): + client = HttpClient(timeout=3) + client.session = _Session(**session) + return client + + +def user_agent_header(): + assert HttpClient(user_agent="ua/1").session.headers["User-Agent"] == "ua/1" + + +def get_json_returns_the_payload(): + client = _client(response=_Response(json={"a": 1})) + assert client.get_json("https://x") == {"a": 1} + assert client.session.calls == [("GET", "https://x", {"timeout": 3})] + + +@pytest.mark.parametrize( + ("session", "message"), + [ + ({"response": _Response(status=404)}, "GET https://x failed: 404"), + ({"error": requests.ConnectionError("down")}, "GET https://x failed: down"), + ({"response": _Response(json=None)}, "response is not JSON"), + ], + ids=["http-error", "connection-error", "not-json"], +) +def get_json_failures(session, message): + with pytest.raises(NetworkError, match=message): + _client(**session).get_json("https://x") + + +def get_text_returns_the_body(): + assert _client(response=_Response(text="hello")).get_text("https://x") == "hello" + + +def get_text_failure_has_a_hint(): + with pytest.raises(NetworkError) as exc: + _client(response=_Response(status=500)).get_text("https://x") + assert "internet connection" in (exc.value.hint or "") + + +def status_returns_the_code_without_raising(): + client = _client(response=_Response(status=503)) + assert client.status("https://x") == 503 + assert client.session.calls[0][2] == {"timeout": 3, "stream": True} + + +def status_connection_error(): + with pytest.raises(NetworkError): + _client(error=requests.Timeout("slow")).status("https://x") + + +def download_writes_every_chunk(tmp_path): + client = _client(response=_Response(chunks=[b"ab", b"", b"cde"])) + dest = tmp_path / "file" + progress = [] + assert client.download("https://x", dest, progress.append) == 5 + assert dest.read_bytes() == b"abcde" + assert progress == [2, 3] + assert client.session.calls[0][2] == {"stream": True, "timeout": 30.0} + + +def download_failure_removes_the_partial_file(tmp_path): + chunks = [b"ab", requests.ConnectionError("cut")] + dest = tmp_path / "file" + with pytest.raises(NetworkError, match="Download of https://x failed: cut"): + _client(response=_Response(chunks=chunks)).download("https://x", dest) + assert not dest.exists() + + +def download_http_error(tmp_path): + dest = tmp_path / "file" + with pytest.raises(NetworkError): + _client(response=_Response(status=404)).download("https://x", dest, timeout=5) + assert not dest.exists() + + +@pytest.mark.parametrize( + ("session", "expected"), + [ + ({"response": _Response(headers={"content-length": "42"})}, 42), + ({"response": _Response()}, None), + ({"response": _Response(headers={"content-length": "abc"})}, None), + ({"error": requests.ConnectionError("down")}, None), + ], + ids=["header", "no-header", "bad-header", "error"], +) +def content_length_cases(session, expected): + client = _client(**session) + assert client.content_length("https://x") == expected + assert client.session.calls[0][:2] == ("HEAD", "https://x") + assert client.session.calls[0][2]["allow_redirects"] is True diff --git a/tests/services/ports.py b/tests/services/ports.py new file mode 100644 index 0000000..018835b --- /dev/null +++ b/tests/services/ports.py @@ -0,0 +1,41 @@ +import pytest + +from services import ports +from services.ports import FixedPortAllocator, PortAllocator + + +def port_allocator_returns_distinct_free_ports(): + allocator = PortAllocator() + given = [allocator.free() for _ in range(5)] + assert len(set(given)) == 5 + assert all(1024 <= port <= 65535 for port in given) + + +def port_allocator_gives_up_when_the_os_repeats_a_port(monkeypatch): + class _Socket: + def __init__(self, *args): + pass + + def __enter__(self): + return self + + def __exit__(self, *exc): + return False + + def bind(self, address): + pass + + def getsockname(self): + return ("0.0.0.0", 45678) + + monkeypatch.setattr(ports.socket, "socket", _Socket) + allocator = PortAllocator() + assert allocator.free() == 45678 + with pytest.raises(RuntimeError, match="Could not allocate a free port"): + allocator.free() + + +def fixed_port_allocator_counts_up(): + allocator = FixedPortAllocator() + assert [allocator.free() for _ in range(3)] == [40000, 40001, 40002] + assert FixedPortAllocator(start=5000).free() == 5000 diff --git a/tests/services/project.py b/tests/services/project.py new file mode 100644 index 0000000..5a7c063 --- /dev/null +++ b/tests/services/project.py @@ -0,0 +1,382 @@ +import dataclasses +import json + +import pytest + +from core.errors import ConfigError, ValidationError +from engines import registry +from services.compose_facts import CA_BUNDLE_IN_CONTAINER +from services.envfile import EnvFile +from services.project import ( + AgentProject, + AuthProvider, + DashboardProject, + detect_kind, + spec_from_entry, +) +from tests.support import EXISTING_ANSWERS + +PG = registry.get("postgresql") +FIREBIRD = registry.get("firebird") +SQLITE = registry.get("sqlite") +VOLUME = registry.get("docker-volume") +KEYCLOAK = AuthProvider( + kind="oidc", + id="my-kc", + values={ + "issuer": "https://kc.example", + "client": "c", + "secret": "s", + "title": "", + "pkce": True, + "host": "", + }, +) +GITHUB = AuthProvider( + kind="oauth", + id="github", + values={"client": "gc", "secret": "gs", "title": "GitHub"}, +) + + +def detect_kind_cases(tmp_path): + agent, dashboard, empty = (tmp_path / name for name in ("a", "d", "e")) + for folder in (agent, dashboard, empty): + folder.mkdir() + (agent / "databases.json").write_text("{}", encoding="utf-8") + (dashboard / ".env").write_text("PROJECT_SECRET=s\n", encoding="utf-8") + assert detect_kind(agent) == "agent" + assert detect_kind(dashboard) == "dashboard" + with pytest.raises(ConfigError, match="not a Portabase agent or dashboard"): + detect_kind(empty) + + +def spec_from_entry_managed(tmp_path): + env = EnvFile(tmp_path / ".env") + env.merge({"DB_FB_AB12_PORT": "40001", "DB_FB_AB12_ROOT_PASS": "root"}) + entry = { + "name": "fb", + "database": "/data/mirror.fdb", + "type": "firebird", + "username": "alice", + "password": "pw", + "port": 3050, + "host": "db-fb-ab12", + "generated_id": "id-1", + "options": {"x": 1}, + } + spec = spec_from_entry(entry, env) + assert spec.managed + assert (spec.id, spec.engine, spec.host, spec.port, spec.host_port) == ( + "id-1", + "firebird", + "db-fb-ab12", + 3050, + 40001, + ) + assert (spec.password, spec.root_password) == ("pw", "root") + assert spec.options == {"x": 1} + assert spec.path is None + + +def spec_from_entry_external(tmp_path): + entry = {**EXISTING_ANSWERS, "type": "postgresql", "port": "5432", "password": ""} + spec = spec_from_entry(entry, EnvFile(tmp_path / ".env")) + assert not spec.managed + assert spec.host_port is None + assert spec.port == 5432 + assert spec.password is None + assert spec.id + + +def spec_from_entry_sqlite_and_volume(tmp_path): + env = EnvFile(tmp_path / ".env") + sqlite = spec_from_entry({"type": "sqlite", "database": "/config/app.sqlite"}, env) + volume = spec_from_entry( + {"type": "docker-volume", "volume_name": "v", "container_name": ""}, env + ) + assert (sqlite.path, sqlite.host) == ("/config/app.sqlite", None) + assert (volume.volume, volume.container) == ("v", None) + + +def agent_create_does_not_write_env(agent): + assert agent.path.is_dir() + assert not agent.env.exists + assert agent.env.get("POLLING") == "5" + + +def agent_add_managed_merges_env(agent, ports): + spec = PG.generate(auth=True, ports=ports, answers={}) + agent.add(spec, PG) + assert agent.managed == [spec] + assert agent.env.get(f"{spec.env_prefix}_PORT") == "40000" + assert agent.env.get(f"{spec.env_prefix}_PASS") == spec.password + + +def agent_add_external_leaves_env(agent): + before = agent.env.as_dict() + agent.add(PG.from_existing(EXISTING_ANSWERS), PG) + assert agent.env.as_dict() == before + assert agent.managed == [] + + +def agent_add_refuses_a_duplicate_service_name(agent, ports): + first = PG.generate(auth=True, ports=ports, answers={}) + agent.add(first, PG) + clash = dataclasses.replace( + PG.generate(auth=True, ports=ports, answers={}), host=first.host + ) + with pytest.raises(ConfigError, match="share the service name"): + agent.add(clash, PG) + + +def agent_remove_purges_its_env_prefix(agent, ports): + keep = PG.generate(auth=True, ports=ports, answers={}) + gone = PG.generate(auth=True, ports=ports, answers={}) + agent.add(keep, PG) + agent.add(gone, PG) + agent.remove(gone, PG) + assert agent.databases == [keep] + assert not any(key.startswith(gone.env_prefix) for key in agent.env.as_dict()) + assert agent.env.get(f"{keep.env_prefix}_PORT") == "40000" + assert agent.env.get("EDGE_KEY") == "x" + + +def agent_find(agent, ports): + first = dataclasses.replace( + PG.generate(auth=True, ports=ports, answers={}), id="aaaa-1111", name="main" + ) + second = dataclasses.replace( + PG.generate(auth=True, ports=ports, answers={}), id="aaab-2222", name="other" + ) + agent.add(first, PG) + agent.add(second, PG) + assert agent.find("aaaa-1111") is first + assert agent.find("aaab") is second + assert agent.find("main") is first + with pytest.raises(ValidationError, match="No database matching"): + agent.find("zzz") + with pytest.raises(ValidationError, match="matches several"): + agent.find("aaa") + + +def agent_settings(agent): + assert agent.setting("polling") == 5 + assert agent.setting("host_gateway") is False + assert agent.setting("retry_attempts") is None + agent.set("polling", 30) + agent.set("host_gateway", True) + agent.set("retry_attempts", 3) + assert agent.env.get("POLLING") == "30" + assert agent.host_gateway is True + assert agent.extra_env == ["RETRY_ATTEMPTS"] + assert set(agent.settings()) == set(agent.registry.names()) + agent.unset("retry_attempts") + assert agent.env.get("RETRY_ATTEMPTS") is None + assert agent.extra_env == [] + + +@pytest.mark.parametrize("name", ["key", "tz", "polling", "log_level", "host_gateway"]) +def agent_core_settings_cannot_be_unset(agent, name): + with pytest.raises(ValidationError, match="cannot be unset"): + agent.unset(name) + + +def agent_ca_bundle(agent): + (agent.path / "ca.crt").write_text("pem", encoding="utf-8") + agent.ca_bundle = "ca.crt" + agent.validate() + assert agent.env.get("SSL_CERT_FILE") == CA_BUNDLE_IN_CONTAINER + agent.ca_bundle = None + assert agent.ca_bundle is None + assert agent.env.get("SSL_CERT_FILE") is None + + +def agent_missing_ca_bundle(agent): + agent.ca_bundle = "missing.crt" + with pytest.raises(ValidationError, match="CA bundle not found"): + agent.save_state() + assert not agent.env.exists + + +def agent_docker_socket_and_sqlite_mounts(agent, ports): + assert not agent.needs_docker_socket + agent.add(VOLUME.from_existing({"volume": "v"}), VOLUME) + for _ in range(2): + agent.add( + SQLITE.generate(auth=False, ports=ports, answers={"name": "app"}), SQLITE + ) + agent.add(SQLITE.from_existing({"path": "/abs/app.db"}), SQLITE) + assert agent.needs_docker_socket + assert agent.sqlite_mounts == [("./app.sqlite", "/config/app.sqlite")] + + +def agent_load_round_trip(agent, ports, renderer): + spec = FIREBIRD.generate(auth=True, ports=ports, answers={}) + agent.add(spec, FIREBIRD) + agent.add(VOLUME.from_existing({"volume": "v", "container": "c"}), VOLUME) + agent.host_gateway = True + (agent.path / "ca.crt").write_text("pem", encoding="utf-8") + agent.ca_bundle = "./ca.crt" + renderer.render_agent(agent).write(agent.path) + agent.save_state() + + loaded = AgentProject.load(agent.path) + assert loaded.host_gateway + assert loaded.ca_bundle == "./ca.crt" + assert loaded.env.as_dict() == agent.env.as_dict() + firebird, volume = loaded.databases + assert firebird.managed + assert (firebird.id, firebird.host, firebird.host_port) == ( + spec.id, + spec.host, + spec.host_port, + ) + assert (firebird.password, firebird.root_password) == ( + spec.password, + spec.root_password, + ) + assert (volume.volume, volume.container) == ("v", "c") + + +def agent_load_rejects_non_agent_folders(tmp_path): + with pytest.raises(ConfigError, match="Not a Portabase agent folder"): + AgentProject.load(tmp_path) + (tmp_path / ".env").write_text("", encoding="utf-8") + (tmp_path / "databases.json").write_text("{oops", encoding="utf-8") + with pytest.raises(ConfigError, match="not valid JSON"): + AgentProject.load(tmp_path) + + +def agent_load_skips_odd_entries(tmp_path): + (tmp_path / ".env").write_text("", encoding="utf-8") + entries = {"databases": ["junk", {"type": "sqlite", "database": "/config/a"}]} + (tmp_path / "databases.json").write_text(json.dumps(entries), encoding="utf-8") + assert [database.engine for database in AgentProject.load(tmp_path).databases] == [ + "sqlite" + ] + + +def dashboard_load(dashboard, tmp_path): + dashboard.env.save() + loaded = DashboardProject.load(dashboard.path) + assert loaded.env.as_dict() == dashboard.env.as_dict() + with pytest.raises(ConfigError, match="Not a Portabase dashboard folder"): + DashboardProject.load(tmp_path) + + +@pytest.mark.parametrize( + ("host", "mode"), [(None, "internal"), ("db", "external"), ("pg.example", "custom")] +) +def dashboard_db_mode(dashboard, host, mode): + if host: + dashboard.env.set("POSTGRES_HOST", host) + assert dashboard.db_mode == mode + + +def dashboard_project_name(dashboard): + assert dashboard.project_name == "pb" + dashboard.env.remove("PROJECT_NAME") + assert dashboard.project_name == dashboard.path.name + + +def dashboard_settings(dashboard): + assert dashboard.setting("password_auth") is True + assert dashboard.setting("api") is False + assert dashboard.setting("url") == "https://d.example" + dashboard.set("api", True) + assert dashboard.env.get("API_ENABLED") == "true" + dashboard.unset("api") + assert dashboard.env.get("API_ENABLED") is None + + +def dashboard_add_providers(dashboard): + dashboard.add_provider(KEYCLOAK) + dashboard.add_provider(GITHUB) + env = dashboard.env.as_dict() + assert {key: value for key, value in env.items() if key.startswith("AUTH_")} == { + "AUTH_OIDC_MY_KC_ID": "my-kc", + "AUTH_OIDC_MY_KC_ISSUER_URL": "https://kc.example", + "AUTH_OIDC_MY_KC_CLIENT": "c", + "AUTH_OIDC_MY_KC_SECRET": "s", + "AUTH_OIDC_MY_KC_PKCE": "true", + "AUTH_SOCIAL_GITHUB_CLIENT": "gc", + "AUTH_SOCIAL_GITHUB_SECRET": "gs", + "AUTH_SOCIAL_GITHUB_TITLE": "GitHub", + } + assert dashboard.providers == [ + AuthProvider( + "oauth", "github", {"client": "gc", "secret": "gs", "title": "GitHub"} + ), + AuthProvider( + "oidc", + "my-kc", + { + "issuer": "https://kc.example", + "client": "c", + "secret": "s", + "pkce": "true", + }, + ), + ] + + +def dashboard_recovers_oidc_id_without_id_variable(dashboard): + dashboard.env.merge( + {"AUTH_OIDC_AZURE_AD_CLIENT": "c", "AUTH_OIDC_AZURE_AD_SECRET": "s"} + ) + assert [provider.id for provider in dashboard.providers] == ["azure-ad"] + + +def dashboard_add_duplicate_provider(dashboard): + dashboard.add_provider(GITHUB) + with pytest.raises(ValidationError, match="already exists"): + dashboard.add_provider(GITHUB) + + +def dashboard_remove_provider(dashboard): + dashboard.add_provider(KEYCLOAK) + dashboard.add_provider(GITHUB) + assert dashboard.remove_provider("my-kc").kind == "oidc" + assert [provider.id for provider in dashboard.providers] == ["github"] + assert not any(key.startswith("AUTH_OIDC_") for key in dashboard.env.as_dict()) + with pytest.raises(ValidationError, match="No provider named"): + dashboard.remove_provider("my-kc") + + +def dashboard_callback_url(dashboard): + assert ( + dashboard.callback_url("my-kc") + == "https://d.example/api/auth/sso/callback/my-kc" + ) + + +def dashboard_defaults_are_valid(dashboard): + dashboard.validate() + + +def dashboard_skip_onboarding_needs_an_account(dashboard): + dashboard.set("skip_onboarding", True) + dashboard.set("admin_email", "a@b.c") + with pytest.raises(ValidationError, match="needs an initial account"): + dashboard.validate() + dashboard.set("admin_password", "Abcdef1!") + dashboard.validate() + + +def dashboard_password_auth_off_needs_a_provider(dashboard): + dashboard.set("password_auth", False) + with pytest.raises(ValidationError, match="lock everyone out"): + dashboard.save_state() + assert not dashboard.env.exists + dashboard.add_provider(GITHUB) + dashboard.save_state() + assert dashboard.env.exists + + +@pytest.mark.parametrize("url", ["http://localhost:8887", "http://127.0.0.1"]) +def dashboard_providers_need_a_public_url(dashboard, url): + dashboard.add_provider(GITHUB) + dashboard.set("url", url) + with pytest.raises(ValidationError, match="need a public URL"): + dashboard.validate() diff --git a/tests/services/renderer.py b/tests/services/renderer.py new file mode 100644 index 0000000..8ebc363 --- /dev/null +++ b/tests/services/renderer.py @@ -0,0 +1,220 @@ +import dataclasses +import json +import re + +import pytest +import yaml + +from core.errors import TemplateError +from engines import registry +from services.compose_facts import CA_BUNDLE_IN_CONTAINER, GENERATED_MARKER +from services.renderer import LEGACY_BACKUP, RenderResult +from tests.support import agent_service + +TEMPLATED = [engine for engine in registry if engine.template is not None] +VARIANTS = [ + (engine, auth) + for engine in TEMPLATED + for auth in ((True, False) if engine.auth_variants else (True,)) +] +VAR = re.compile(r"\$\{([A-Za-z0-9_]+)\}") +# Values that break an unquoted "- KEY=value" item or a naive double-quoted one. +NASTY = ["abc:", "a: b", "#start", 'q"uote', "back\\slash", "ends\\", "{x}", "[y]"] + + +def _parse(result): + result.validate() + return yaml.safe_load(result.compose) + + +def _assert_vars_defined(compose, env): + missing = set(VAR.findall(compose)) - set(env.as_dict()) + assert not missing, f"compose references undefined variables: {missing}" + + +def _password_values(service): + values = [] + for item in service.get("environment") or []: + assert isinstance(item, str), f"environment item parsed as {item!r}" + key, _, value = item.partition("=") + if key.endswith("PASSWORD") and key != "MYSQL_RANDOM_ROOT_PASSWORD": + values.append(value) + command = service.get("command") + if isinstance(command, list) and "--requirepass" in command: + values.append(command[command.index("--requirepass") + 1]) + return values + + +def header_marks_the_file_as_generated(renderer): + assert renderer.header() == f"{GENERATED_MARKER} test. Do not edit.\n" + + +def agent_without_databases(agent, renderer): + result = renderer.render_agent(agent) + assert _parse(result) == { + "services": {"agent": agent_service()}, + "networks": {"portabase": {"name": "portabase_network", "external": True}}, + } + assert result.databases == [] + + +def agent_inline(agent, renderer): + result = renderer.render_agent(agent, inline=True) + assert _parse(result)["services"] == {"agent": agent_service(inline=True)} + assert "${" not in result.compose + + +def agent_with_every_option(agent, renderer, ports): + sqlite, volume = registry.get("sqlite"), registry.get("docker-volume") + agent.host_gateway = True + agent.add(sqlite.generate(auth=False, ports=ports, answers={"name": "x"}), sqlite) + agent.add(volume.from_existing({"volume": "v"}), volume) + agent.ca_bundle = "./ca.crt" + agent.set("retry_attempts", 3) + result = renderer.render_agent(agent) + expected = agent_service( + "./x.sqlite:/config/x.sqlite", + "/var/run/docker.sock:/var/run/docker.sock", + f"./ca.crt:{CA_BUNDLE_IN_CONTAINER}:ro", + ) + expected["extra_hosts"] = ["localhost:host-gateway"] + expected["environment"]["RETRY_ATTEMPTS"] = "${RETRY_ATTEMPTS}" + expected["environment"]["SSL_CERT_FILE"] = "${SSL_CERT_FILE}" + assert _parse(result)["services"] == {"agent": expected} + assert [entry["type"] for entry in result.databases or []] == [ + "sqlite", + "docker-volume", + ] + _assert_vars_defined(result.compose, agent.env) + + +def agent_with_every_engine(agent, renderer, ports): + specs = [] + for engine, auth in VARIANTS: + spec = engine.generate(auth=auth, ports=ports, answers={}) + agent.add(spec, engine) + specs.append(spec) + result = renderer.render_agent(agent) + doc = _parse(result) + assert set(doc["services"]) == {"agent", *(spec.host for spec in specs)} + assert set(doc["volumes"]) == {f"{spec.host}-data" for spec in specs} + assert result.databases == [ + registry.get(spec.engine).agent_entry(spec) for spec in specs + ] + _assert_vars_defined(result.compose, agent.env) + + +@pytest.mark.parametrize("password", NASTY) +@pytest.mark.parametrize("engine", TEMPLATED, ids=lambda engine: engine.key) +def agent_inline_keeps_passwords_intact(engine, password, agent, renderer, ports): + spec = dataclasses.replace( + engine.generate(auth=True, ports=ports, answers={}), + password=password, + root_password=password if engine.key == "firebird" else None, + ) + agent.add(spec, engine) + doc = _parse(renderer.render_agent(agent, inline=True)) + values = _password_values(doc["services"][spec.host]) + assert values + assert all(value == password for value in values) + + +@pytest.mark.parametrize( + ("mode", "services", "volumes"), + [ + ("external", {"portabase", "db"}, {"postgres-data", "portabase-data"}), + ("internal", {"portabase"}, {"portabase-data"}), + ("custom", {"portabase"}, {"portabase-data"}), + ], +) +def dashboard_modes(dashboard_for, renderer, mode, services, volumes): + project = dashboard_for(mode) + result = renderer.render_dashboard(project) + doc = _parse(result) + assert doc["name"] == "pb" + assert set(doc["services"]) == services + assert set(doc["volumes"]) == volumes + assert ("depends_on" in doc["services"]["portabase"]) is (mode == "external") + assert result.databases is None + _assert_vars_defined(result.compose, project.env) + + +def dashboard_inline(dashboard_for, renderer): + result = renderer.render_dashboard(dashboard_for("external"), inline=True) + doc = _parse(result) + assert "${" not in result.compose + assert doc["services"]["portabase"]["ports"] == ["8887:80"] + assert "POSTGRES_PASSWORD=p" in doc["services"]["db"]["environment"] + + +@pytest.mark.parametrize("password", NASTY) +def dashboard_inline_keeps_passwords_intact(password, dashboard_for, renderer): + project = dashboard_for( + "external", POSTGRES_PASSWORD=password, PROJECT_SECRET=password + ) + doc = _parse(renderer.render_dashboard(project, inline=True)) + assert _password_values(doc["services"]["db"]) == [password] + assert f"PROJECT_SECRET={password}" in doc["services"]["portabase"]["environment"] + + +@pytest.mark.parametrize("compose", ["services: [unclosed", "name: x\n", "- a\n"]) +def validate_rejects_broken_compose(compose): + with pytest.raises(TemplateError): + RenderResult(compose).validate() + + +def write_new_folder(tmp_path): + folder = tmp_path / "new" + result = RenderResult("services: {}\n", databases=[{"type": "sqlite"}]) + report = result.write(folder) + compose, databases = folder / "docker-compose.yml", folder / "databases.json" + assert report.wrote == [compose, databases] + assert report.backed_up is None + assert compose.read_text(encoding="utf-8") == "services: {}\n" + assert json.loads(databases.read_text(encoding="utf-8")) == { + "databases": [{"type": "sqlite"}] + } + assert not list(folder.glob("*.tmp")) + + +def write_skips_databases_for_dashboards(tmp_path): + RenderResult("services: {}\n").write(tmp_path) + assert not (tmp_path / "databases.json").exists() + + +def write_backs_up_a_hand_written_compose_once(tmp_path, renderer): + compose = tmp_path / "docker-compose.yml" + compose.write_text("services: {legacy: {}}\n", encoding="utf-8") + generated = RenderResult(renderer.header() + "services: {}\n") + + report = generated.write(tmp_path) + assert report.backed_up == tmp_path / LEGACY_BACKUP + assert report.backed_up.read_text(encoding="utf-8") == "services: {legacy: {}}\n" + + compose.write_text("services: {edited: {}}\n", encoding="utf-8") + assert generated.write(tmp_path).backed_up is None + backup = (tmp_path / LEGACY_BACKUP).read_text(encoding="utf-8") + assert backup == "services: {legacy: {}}\n" + + +def write_leaves_a_generated_compose_alone(tmp_path, renderer): + generated = RenderResult(renderer.header() + "services: {}\n") + generated.write(tmp_path) + assert generated.write(tmp_path).backed_up is None + assert not (tmp_path / LEGACY_BACKUP).exists() + + +def write_refuses_an_invalid_compose(tmp_path): + with pytest.raises(TemplateError): + RenderResult("nope: 1\n").write(tmp_path) + assert not (tmp_path / "docker-compose.yml").exists() + + +def diff_against_the_current_file(tmp_path): + result = RenderResult("services:\n a: {}\n") + assert "+services:" in result.diff_against(tmp_path) + result.write(tmp_path) + assert result.diff_against(tmp_path) == "" + changed = RenderResult("services:\n b: {}\n").diff_against(tmp_path) + assert "- a: {}" in changed + assert "+ b: {}" in changed diff --git a/tests/services/settings.py b/tests/services/settings.py new file mode 100644 index 0000000..3c5c802 --- /dev/null +++ b/tests/services/settings.py @@ -0,0 +1,144 @@ +import base64 +import json + +import pytest + +from core.errors import ValidationError +from core.fields import Field +from services import settings as cfg +from tests.support import EDGE_KEY_PAYLOAD + + +def _setting(kind, default=None): + return cfg.Setting(Field("x", "X", kind, default=default), "X", "s") + + +def strong_password_accepts(): + assert cfg.strong_password("Abcdef1!") == "Abcdef1!" + + +@pytest.mark.parametrize( + ("value", "missing"), + [ + ("Ab1!", "at least 8 characters"), + ("ABCDEFG1!", "a lowercase letter"), + ("abcdefg1!", "an uppercase letter"), + ("Abcdefgh!", "a digit"), + ("Abcdefgh1", "a special character"), + ], +) +def strong_password_rejects(value, missing): + with pytest.raises(ValidationError) as exc: + cfg.strong_password(value) + assert missing in (exc.value.hint or "") + + +def strong_password_lists_every_missing_rule(): + with pytest.raises(ValidationError) as exc: + cfg.strong_password("") + assert exc.value.hint == ( + "It needs at least 8 characters, a lowercase letter, an uppercase letter, " + "a digit, a special character." + ) + + +@pytest.mark.parametrize( + ("value", "expected"), + [ + ("https://portabase.example", "https://portabase.example"), + ("https://portabase.example/", "https://portabase.example"), + ("http://localhost:8887", "http://localhost:8887"), + ("https://x.example/sub/", "https://x.example/sub"), + ], +) +def public_url_accepts(value, expected): + assert cfg.public_url(value) == expected + + +@pytest.mark.parametrize( + "value", ["", "portabase.example", "ftp://x", "https://", "https:// x"] +) +def public_url_rejects(value): + with pytest.raises(ValidationError, match="Invalid URL"): + cfg.public_url(value) + + +def edge_key_validator(): + key = base64.b64encode(json.dumps(EDGE_KEY_PAYLOAD).encode()).decode() + assert cfg.edge_key(key) == key + with pytest.raises(ValidationError, match="Invalid Edge Key"): + cfg.edge_key("bad") + + +def positive_validator(): + assert cfg.positive(1) == 1 + for value in (0, -5): + with pytest.raises(ValidationError): + cfg.positive(value) + + +@pytest.mark.parametrize( + ("raw", "expected"), + [ + ("true", True), + ("TRUE", True), + (" yes ", True), + ("1", True), + ("on", True), + ("false", False), + ("0", False), + ("", False), + ], +) +def bool_from_env(raw, expected): + assert _setting("bool").from_env(raw) is expected + + +def bool_to_env(): + assert _setting("bool").to_env(True) == "true" + assert _setting("bool").to_env(False) == "false" + + +@pytest.mark.parametrize( + ("raw", "expected"), [("42", 42), (" 7 ", 7), ("-3", -3), ("abc", "abc")] +) +def int_from_env(raw, expected): + assert _setting("int").from_env(raw) == expected + + +@pytest.mark.parametrize("kind", ["bool", "int", "text"]) +def from_env_none_is_the_default(kind): + assert _setting(kind, default="d").from_env(None) == "d" + + +def text_round_trip(): + assert _setting("text").to_env(5) == "5" + assert _setting("text").from_env(" raw ") == " raw " + + +@pytest.mark.parametrize( + "registry", [cfg.AGENT, cfg.DASHBOARD], ids=["agent", "dashboard"] +) +def registry_is_consistent(registry): + names = [setting.name for setting in registry] + envs = [setting.env for setting in registry if setting.env] + assert len(names) == len(set(names)) + assert len(envs) == len(set(envs)) + assert registry.names() == names + for setting in registry: + assert setting.section in registry.sections + assert registry.get(setting.name) is setting + assert sum(len(registry.in_section(key)) for key in registry.sections) == len(names) + + +def registry_unknown_setting(): + with pytest.raises(ValidationError, match="Unknown setting 'nope'") as exc: + cfg.AGENT.get("nope") + assert "polling" in (exc.value.hint or "") + + +def dashboard_wizard_names_exist(): + for section, names in cfg.DASHBOARD_WIZARD: + assert section in cfg.DASHBOARD.sections + for name in names: + assert cfg.DASHBOARD.get(name).section == section diff --git a/tests/services/telemetry.py b/tests/services/telemetry.py new file mode 100644 index 0000000..ae0a69c --- /dev/null +++ b/tests/services/telemetry.py @@ -0,0 +1,29 @@ +import pytest + +from services.telemetry import NoopTelemetry, Telemetry + + +def telemetry_is_abstract(): + with pytest.raises(TypeError): + Telemetry() + + +def noop_session_and_span_are_context_managers(): + telemetry = NoopTelemetry() + with ( + telemetry.session(command="agent create") as session, + telemetry.span("render", engine="postgresql") as span, + ): + assert (session, span) == (None, None) + + +def noop_span_lets_exceptions_through(): + with pytest.raises(ValueError, match="boom"), NoopTelemetry().span("render"): + raise ValueError("boom") + + +def noop_records_nothing(): + telemetry = NoopTelemetry() + assert telemetry.event("created", engine="redis") is None + assert telemetry.error(ValueError("x"), unexpected=True) is None + assert telemetry.flush() is None diff --git a/tests/services/templates.py b/tests/services/templates.py new file mode 100644 index 0000000..1ba6fd7 --- /dev/null +++ b/tests/services/templates.py @@ -0,0 +1,46 @@ +import pytest + +from core.errors import TemplateError +from services.renderer import ComposeRenderer +from services.templates import TemplateRepository +from tests.support import ROOT + + +def names_are_sorted_and_complete(templates): + names = templates.names() + assert "agent.yml.j2" in names + assert "dashboard.yml.j2" in names + assert "engines/postgresql.yml.j2" in names + assert names == sorted(names) + + +def bundled_defaults_to_repo_templates(monkeypatch): + monkeypatch.delenv("PORTABASE_TEMPLATES_DIR", raising=False) + assert TemplateRepository.bundled().root == ROOT / "templates" + + +def bundled_honours_override(monkeypatch, tmp_path): + monkeypatch.setenv("PORTABASE_TEMPLATES_DIR", str(tmp_path)) + assert TemplateRepository.bundled().root == tmp_path + + +def empty_folder(tmp_path): + with pytest.raises(TemplateError, match="No templates found"): + TemplateRepository(tmp_path).get("agent.yml.j2") + + +def missing_template(templates): + with pytest.raises(TemplateError, match="'engines/nope.yml.j2' is missing"): + templates.get("engines/nope.yml.j2") + + +def broken_template(tmp_path): + (tmp_path / "agent.yml.j2").write_text("services: {}\n", encoding="utf-8") + (tmp_path / "bad.j2").write_text("{% if %}", encoding="utf-8") + with pytest.raises(TemplateError, match="failed to load"): + TemplateRepository(tmp_path).get("bad.j2") + + +def undefined_variable_is_an_error(templates): + with pytest.raises(TemplateError, match="rendering failed"): + ComposeRenderer._render_template(templates.get("agent.yml.j2"), {}) diff --git a/tests/services/updater.py b/tests/services/updater.py new file mode 100644 index 0000000..dc6fc0e --- /dev/null +++ b/tests/services/updater.py @@ -0,0 +1,271 @@ +import hashlib +import json +import sys +import tempfile +import time +from pathlib import Path + +import pytest + +from core.config import GlobalConfig +from core.errors import UpdateError +from core.version import UNKNOWN +from services import updater +from services.updater import ( + RELEASES_URL, + Release, + UpdateChecker, + Updater, + platform_asset_name, +) +from tests.support import FakeHttp + +LATEST = f"{RELEASES_URL}/latest" +PAYLOAD = b"new binary" + + +def _api_release(tag, prerelease=False, assets=()): + return { + "tag_name": tag, + "prerelease": prerelease, + "assets": [ + {"name": name, "browser_download_url": f"https://dl/{name}"} + for name in assets + ], + } + + +def _sha(data=PAYLOAD): + return hashlib.sha256(data).hexdigest() + + +def _published(checksums=None, *, binary=True): + name = platform_asset_name() + assets = {} + http = FakeHttp() + if binary: + assets[name] = f"https://dl/{name}" + http.files[assets[name]] = PAYLOAD + if checksums is not None: + assets["checksums.txt"] = "https://dl/checksums.txt" + http.text["https://dl/checksums.txt"] = checksums + return Updater(http, "1.0.0"), Release("2.0.0", assets, False) + + +@pytest.fixture +def config(tmp_path): + return GlobalConfig(tmp_path / "config.json") + + +@pytest.fixture +def downloads(monkeypatch, tmp_path): + folder = tmp_path / "downloads" + folder.mkdir() + monkeypatch.setattr(tempfile, "tempdir", str(folder)) + return folder + + +def release_from_api(): + data = _api_release("v1.2.3", prerelease=True, assets=["a", "b"]) + assert Release.from_api(data) == Release( + "1.2.3", {"a": "https://dl/a", "b": "https://dl/b"}, True + ) + assert Release.from_api({}) == Release("", {}, False) + + +@pytest.mark.parametrize( + ("system", "machine", "expected"), + [ + ("Linux", "x86_64", "portabase_linux_amd64"), + ("Linux", "aarch64", "portabase_linux_arm64"), + ("Darwin", "arm64", "portabase_macos_arm64"), + ("Darwin", "x86_64", "portabase_macos_amd64"), + ("Windows", "AMD64", "portabase_windows_amd64.exe"), + ], +) +def asset_name_per_platform(monkeypatch, system, machine, expected): + monkeypatch.setattr(updater.platform, "system", lambda: system) + monkeypatch.setattr(updater.platform, "machine", lambda: machine) + assert platform_asset_name() == expected + + +def frozen_flag(monkeypatch): + assert not updater.is_frozen() + monkeypatch.setattr(sys, "frozen", True, raising=False) + assert updater.is_frozen() + + +@pytest.mark.parametrize( + ("channel", "current", "expected"), + [ + (None, "1.0.0", False), + (None, "1.0.0rc1", True), + ("beta", "1.0.0", True), + ("stable", "1.0.0rc1", False), + ], +) +def checker_include_prerelease(config, channel, current, expected): + if channel: + config.set("update_channel", channel) + assert UpdateChecker(FakeHttp(), config, current).include_prerelease is expected + + +def checker_stable_uses_latest(config): + http = FakeHttp(json={LATEST: _api_release("1.1.0")}) + assert UpdateChecker(http, config, "1.0.0").available() == "1.1.0" + assert http.calls == [LATEST] + + +def checker_beta_uses_the_first_release(config): + config.set("update_channel", "beta") + releases = [_api_release("1.1.0rc1", True), _api_release("1.0.0")] + http = FakeHttp(json={RELEASES_URL: releases}) + assert UpdateChecker(http, config, "1.0.0").available() == "1.1.0rc1" + + +def checker_beta_without_releases(config): + config.set("update_channel", "beta") + http = FakeHttp(json={RELEASES_URL: []}) + assert UpdateChecker(http, config, "1.0.0").available() is None + + +@pytest.mark.parametrize("tag", ["1.0.0", "0.9.0", "1.0.0rc1"]) +def checker_nothing_newer(config, tag): + http = FakeHttp(json={LATEST: _api_release(tag)}) + assert UpdateChecker(http, config, "1.0.0").available() is None + + +def checker_unknown_version_never_checks(config): + http = FakeHttp() + assert UpdateChecker(http, config, UNKNOWN).available() is None + assert http.calls == [] + + +def checker_network_error_is_silent(config): + assert UpdateChecker(FakeHttp(), config, "1.0.0").available() is None + assert not (config.cache_dir / "release.json").exists() + + +def checker_caches_the_release(config): + http = FakeHttp(json={LATEST: _api_release("1.1.0")}) + checker = UpdateChecker(http, config, "1.0.0") + assert checker.latest() == Release("1.1.0", {}, False) + assert checker.latest() == Release("1.1.0", {}, False) + assert http.calls == [LATEST] + checker.latest(force=True) + assert http.calls == [LATEST, LATEST] + + +def checker_refetches_an_expired_cache(config): + http = FakeHttp(json={LATEST: _api_release("1.1.0")}) + checker = UpdateChecker(http, config, "1.0.0") + checker.latest() + data = json.loads(checker.cache_file.read_text(encoding="utf-8")) + data["checked_at"] = time.time() - updater.CACHE_TTL - 1 + checker.cache_file.write_text(json.dumps(data), encoding="utf-8") + checker.latest() + assert http.calls == [LATEST, LATEST] + + +def checker_channel_change_invalidates_the_cache(config): + http = FakeHttp( + json={ + LATEST: _api_release("1.1.0"), + RELEASES_URL: [_api_release("1.2.0rc1", True)], + } + ) + checker = UpdateChecker(http, config, "1.0.0") + assert checker.available() == "1.1.0" + config.set("update_channel", "beta") + assert checker.available() == "1.2.0rc1" + + +def checker_survives_a_corrupt_cache(config): + http = FakeHttp(json={LATEST: _api_release("1.1.0")}) + checker = UpdateChecker(http, config, "1.0.0") + checker.cache_file.parent.mkdir(parents=True) + checker.cache_file.write_text("{oops", encoding="utf-8") + assert checker.available() == "1.1.0" + + +@pytest.mark.parametrize("line", ["{sha} {name}", "{sha} *{name}", "{SHA} {name}"]) +def updater_download_verified(downloads, line): + name = platform_asset_name() + listed = line.format(sha=_sha(), SHA=_sha().upper(), name=name) + updater, release = _published(f"deadbeef other\n{listed}\n") + progress = [] + path = updater.download(release, progress.append) + assert path.read_bytes() == PAYLOAD + assert path.parent == downloads + assert progress == [len(PAYLOAD)] + + +@pytest.mark.parametrize( + ("checksums", "message"), + [ + (None, "has no checksums.txt"), + ("abc other_asset\n", "not listed in checksums.txt"), + ("{bad} {name}\n", "Checksum mismatch"), + ], +) +def updater_download_refused(downloads, checksums, message): + if checksums is not None: + checksums = checksums.format(bad=_sha(b"tampered"), name=platform_asset_name()) + updater, release = _published(checksums) + with pytest.raises(UpdateError, match=message): + updater.download(release) + assert list(downloads.iterdir()) == [] + + +def updater_no_binary_for_this_platform(): + updater, release = _published(f"{_sha()} x\n", binary=False) + with pytest.raises(UpdateError, match="No binary for this platform") as exc: + updater.download(release) + assert exc.value.hint == "Available: checksums.txt" + updater, empty = _published(None, binary=False) + with pytest.raises(UpdateError) as exc: + updater.download(empty) + assert exc.value.hint is None + + +def updater_expected_size(): + updater, release = _published(None) + assert updater.expected_size(release) == len(PAYLOAD) + assert updater.expected_size(Release("2.0.0", {}, False)) is None + + +def updater_install_replaces_and_keeps_a_backup(tmp_path): + target = tmp_path / "bin" / "portabase" + target.parent.mkdir() + target.write_bytes(b"old") + tmp = tmp_path / "download" + tmp.write_bytes(PAYLOAD) + Updater(FakeHttp(), "1.0.0").install(tmp, target) + assert target.read_bytes() == PAYLOAD + assert (target.stat().st_mode & 0o777) == 0o755 + assert target.with_name("portabase.old").read_bytes() == b"old" + assert not tmp.exists() + + +def updater_install_fresh(tmp_path): + target = tmp_path / "new" / "bin" / "portabase" + tmp = tmp_path / "download" + tmp.write_bytes(PAYLOAD) + Updater(FakeHttp(), "1.0.0").install(tmp, target) + assert target.read_bytes() == PAYLOAD + assert not target.with_name("portabase.old").exists() + + +def updater_target_path_frozen(monkeypatch, tmp_path): + exe = tmp_path / "portabase" + exe.write_bytes(b"") + monkeypatch.setattr(sys, "frozen", True, raising=False) + monkeypatch.setattr(sys, "executable", str(exe)) + assert Updater(FakeHttp(), "1.0.0").target_path() == exe.resolve() + + +def updater_target_path_windows(monkeypatch, tmp_path): + monkeypatch.setattr(updater.platform, "system", lambda: "Windows") + monkeypatch.setenv("APPDATA", str(tmp_path)) + expected = Path(tmp_path) / "Portabase" / "portabase.exe" + assert Updater(FakeHttp(), "1.0.0").target_path() == expected diff --git a/tests/structure.py b/tests/structure.py new file mode 100644 index 0000000..c3c95fd --- /dev/null +++ b/tests/structure.py @@ -0,0 +1,89 @@ +import ast + +from engines import registry +from tests.support import ROOT + +TESTS = ROOT / "tests" +PACKAGES = ("core", "services", "engines") +# Modules whose tests live under another name. +ALIASES = {"engines/__init__.py": "engines/registry.py"} +SHARED = {"conftest.py", "support.py", "structure.py"} +ENGINE_SKELETON = [ + "attributes", + "generate", + "env_vars", + "agent_entry", + "fields", + "from_existing", + "compose_service", + "compose_service_inline", +] + + +def _functions(path): + tree = ast.parse(path.read_text(encoding="utf-8")) + return [ + node.name + for node in tree.body + if isinstance(node, ast.FunctionDef) and not node.name.startswith("_") + ] + + +def every_module_has_a_test_file(): + missing = [] + for package in PACKAGES: + for module in sorted((ROOT / package).glob("*.py")): + rel = f"{package}/{module.name}" + if module.name == "__init__.py" and rel not in ALIASES: + continue + if not (TESTS / ALIASES.get(rel, rel)).exists(): + missing.append(rel) + assert not missing, f"modules without tests: {missing}" + + +def every_engine_has_a_test_file(): + missing = [ + engine.key + for engine in registry + if not (TESTS / "engines" / f"{engine.key.replace('-', '_')}.py").exists() + ] + assert not missing, f"engines without tests/engines/.py: {missing}" + + +def engine_files_follow_the_same_skeleton(): + wrong = {} + for engine in registry: + path = TESTS / "engines" / f"{engine.key.replace('-', '_')}.py" + if not path.exists(): + continue + head = _functions(path)[: len(ENGINE_SKELETON)] + if head != ENGINE_SKELETON: + wrong[path.name] = head + assert not wrong, f"expected {ENGINE_SKELETON} first, got {wrong}" + + +def every_test_file_mirrors_a_module_or_an_engine(): + engines = {engine.key.replace("-", "_") for engine in registry} + stray = [] + for path in sorted(TESTS.rglob("*.py")): + rel = path.relative_to(TESTS).as_posix() + if path.name == "__init__.py" or rel in SHARED: + continue + if (ROOT / rel).exists() or rel in ALIASES.values(): + continue + if path.parent.name == "engines" and path.stem in engines: + continue + stray.append(rel) + assert not stray, f"test files matching no module: {stray}" + + +def tests_have_no_prefix_and_no_classes(): + offenders = [] + for path in sorted(TESTS.rglob("*.py")): + tree = ast.parse(path.read_text(encoding="utf-8")) + for node in tree.body: + if isinstance(node, ast.FunctionDef) and node.name.startswith("test_"): + offenders.append(f"{path.name}::{node.name}") + if isinstance(node, ast.ClassDef) and node.name.startswith("Test"): + offenders.append(f"{path.name}::{node.name}") + assert not offenders, f"use plain names, no test_/Test prefix: {offenders}" diff --git a/tests/support.py b/tests/support.py new file mode 100644 index 0000000..b8df74c --- /dev/null +++ b/tests/support.py @@ -0,0 +1,139 @@ +from __future__ import annotations + +import json +import os +import struct +from collections.abc import Callable, Iterable +from dataclasses import dataclass, field +from pathlib import Path +from typing import Any + +from cryptography.hazmat.primitives.ciphers.aead import AESGCM + +from core.errors import NetworkError +from core.fields import Field +from core.specs import DatabaseSpec + +ROOT = Path(__file__).resolve().parent.parent + +EDGE_KEY_PAYLOAD = {"serverUrl": "http://x", "agentId": "a", "masterKeyB64": "k"} +AGENT_ENV = {"TZ": "UTC", "EDGE_KEY": "x", "LOG_LEVEL": "info", "POLLING": "5"} +DASHBOARD_BASE = { + "HOST_PORT": "8887", + "PROJECT_SECRET": "s", + "PROJECT_URL": "http://localhost:8887", + "PROJECT_NAME": "pb", + "TZ": "UTC", + "LOG_LEVEL": "info", +} +DASHBOARD_PG = { + "POSTGRES_DB": "pb", + "POSTGRES_USER": "pb", + "POSTGRES_PASSWORD": "p", + "PG_PORT": "5433", + "DATABASE_URL": "postgresql://pb:p@db:5432/pb", +} +DASHBOARD_MODES = { + "external": {**DASHBOARD_BASE, **DASHBOARD_PG, "POSTGRES_HOST": "db"}, + "internal": DASHBOARD_BASE, + "custom": {**DASHBOARD_BASE, **DASHBOARD_PG, "POSTGRES_HOST": "remote"}, +} +EXISTING_ANSWERS = { + "host": "db.example", + "port": "1234", + "database": "app", + "username": "u", + "password": "p", +} + + +@dataclass +class FakeHttp: + json: dict[str, Any] = field(default_factory=dict) + text: dict[str, str] = field(default_factory=dict) + files: dict[str, bytes] = field(default_factory=dict) + calls: list[str] = field(default_factory=list) + + def _lookup(self, table: dict[str, Any], url: str) -> Any: + self.calls.append(url) + if url not in table: + raise NetworkError(f"GET {url} failed: 404") + return table[url] + + def get_json(self, url: str) -> Any: + return self._lookup(self.json, url) + + def get_text(self, url: str) -> str: + return self._lookup(self.text, url) + + def download( + self, + url: str, + dest: Path, + on_progress: Callable[[int], None] | None = None, + *, + timeout: float = 30.0, + ) -> int: + data = self._lookup(self.files, url) + dest.write_bytes(data) + if on_progress: + on_progress(len(data)) + return len(data) + + def content_length(self, url: str) -> int | None: + data = self.files.get(url) + return len(data) if data is not None else None + + +@dataclass +class Rendered: + spec: DatabaseSpec + doc: dict[str, Any] + env: dict[str, str] + databases: list[dict[str, Any]] + + @property + def agent(self) -> dict[str, Any]: + return self.doc["services"]["agent"] + + @property + def service(self) -> dict[str, Any]: + return self.doc["services"][self.spec.host] + + def var(self, suffix: str) -> str: + return f"${{{self.spec.env_prefix}_{suffix}}}" + + +def encrypt(plain: bytes, key: bytes, chunk_size: int = 4) -> bytes: + """Build a .enc file the way the Portabase agent writes one.""" + base_nonce = os.urandom(8) + header = { + "cipher": "AES-256-GCM", + "base_nonce": list(base_nonce), + "chunk_size": chunk_size, + } + out = json.dumps(header).encode() + b"\n" + aes = AESGCM(key) + for index, start in enumerate(range(0, len(plain), chunk_size)): + nonce = base_nonce + struct.pack(">I", index) + ciphertext = aes.encrypt(nonce, plain[start : start + chunk_size], None) + out += struct.pack(">I", len(ciphertext)) + ciphertext + return out + + +def field_specs(fields: Iterable[Field]) -> list[tuple[str, str, Any]]: + return [(field.name, field.kind, field.default) for field in fields] + + +def agent_service(*volumes: str, inline: bool = False) -> dict[str, Any]: + """The agent service as rendered with no option turned on.""" + environment = ( + dict(AGENT_ENV) if inline else {key: f"${{{key}}}" for key in AGENT_ENV} + ) + return { + "restart": "unless-stopped", + "image": "portabase/agent:latest", + "volumes": ["./databases.json:/config/config.json", *volumes], + "environment": environment, + "networks": ["portabase"], + } diff --git a/ui/components/progress.py b/ui/components/progress.py index 3082328..94f1826 100644 --- a/ui/components/progress.py +++ b/ui/components/progress.py @@ -31,4 +31,4 @@ def download(self, description: str, total: int) -> Iterator[Callable[[int], Non console=self.console, ) as progress: task = progress.add_task(description, total=total or None) - yield lambda n: progress.update(task, advance=n) + yield lambda amount: progress.update(task, advance=amount) diff --git a/ui/components/prompt.py b/ui/components/prompt.py index 28a926f..02ba91a 100644 --- a/ui/components/prompt.py +++ b/ui/components/prompt.py @@ -21,8 +21,8 @@ def integer(self, message: str, *, default: int | None = None) -> int | None: answer = questionary.text( message, default="" if default is None else str(default), - validate=lambda v: ( - v.strip().lstrip("-").isdigit() or "Enter a whole number" + validate=lambda value: ( + value.strip().lstrip("-").isdigit() or "Enter a whole number" ), style=self.style, ).ask() diff --git a/ui/components/table.py b/ui/components/table.py index 193fc91..64ac262 100644 --- a/ui/components/table.py +++ b/ui/components/table.py @@ -12,8 +12,8 @@ def __call__( self, columns: list[str], rows: list[list[str]], *, title: str | None = None ) -> None: table = Table(title=title) - for i, col in enumerate(columns): - table.add_column(col, style=_STYLES[i % len(_STYLES)]) + for index, col in enumerate(columns): + table.add_column(col, style=_STYLES[index % len(_STYLES)]) for row in rows: - table.add_row(*[str(c) for c in row]) + table.add_row(*[str(cell) for cell in row]) self.console.print(table) diff --git a/ui/form.py b/ui/form.py index 3f2003a..3727d72 100644 --- a/ui/form.py +++ b/ui/form.py @@ -16,14 +16,18 @@ def __init__(self, prompt: Prompt, non_interactive: bool) -> None: self.prompt = prompt self.non_interactive = non_interactive self._askers: dict[str, Callable[[Field], Any]] = { - "text": lambda f: self.prompt.text(f.prompt, default=f.default), - "int": lambda f: self.prompt.integer(f.prompt, default=f.default), - "secret": lambda f: self.prompt.secret(f.prompt), - "bool": lambda f: self.prompt.confirm(f.prompt, default=bool(f.default)), - "choice": lambda f: self.prompt.select( - f.prompt, f.choices, default=f.default + "text": lambda field: self.prompt.text(field.prompt, default=field.default), + "int": lambda field: self.prompt.integer( + field.prompt, default=field.default ), - "path": lambda f: self.prompt.path(f.prompt, default=f.default), + "secret": lambda field: self.prompt.secret(field.prompt), + "bool": lambda field: self.prompt.confirm( + field.prompt, default=bool(field.default) + ), + "choice": lambda field: self.prompt.select( + field.prompt, field.choices, default=field.default + ), + "path": lambda field: self.prompt.path(field.prompt, default=field.default), } def ask(self, field: Field, value: Any | None = None) -> Any: @@ -41,7 +45,7 @@ def ask(self, field: Field, value: Any | None = None) -> Any: def collect( self, fields: Sequence[Field], values: dict[str, Any] ) -> dict[str, Any]: - return {f.name: self.ask(f, values.get(f.name)) for f in fields} + return {field.name: self.ask(field, values.get(field.name)) for field in fields} def text( self, prompt: str, *, value=None, default=None, validator=None, name="value" @@ -84,8 +88,8 @@ def _ask_until_valid(self, field: Field) -> Any: raise UserAbort() try: return self._coerce_and_validate(field, answer) - except ValidationError as e: - self.prompt.console.print(f"[danger]✖ {e.message}[/danger]") + except ValidationError as error: + self.prompt.console.print(f"[danger]✖ {error.message}[/danger]") def _coerce_and_validate(self, field: Field, value: Any) -> Any: value = self._coerce(field, value) @@ -103,15 +107,15 @@ def _coerce(field: Field, value: Any) -> Any: if field.kind == "int" and not isinstance(value, int): try: return int(str(value).strip()) - except ValueError as e: + except ValueError as error: raise ValidationError( f"{field.flag} must be a whole number, got {value!r}" - ) from e + ) from error if field.kind == "bool" and not isinstance(value, bool): - s = str(value).strip().lower() - if s in _TRUE: + normalized = str(value).strip().lower() + if normalized in _TRUE: return True - if s in _FALSE: + if normalized in _FALSE: return False raise ValidationError(f"{field.flag} must be true or false, got {value!r}") if field.kind in ("text", "secret", "path", "choice"): From 788e9e569dd735f3a68b6a55cbdb3fa8d36decfc Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Th=C3=A9o=20LAGACHE?= Date: Mon, 14 Sep 2026 15:47:01 +0200 Subject: [PATCH 22/24] ci: track the build action, drop SARIF upload, pass gitleaks config via env --- .github/actions/build/action.yml | 45 ++++++++++++++++++++++++++++++++ .github/workflows/plumber.yml | 2 +- .github/workflows/security.yml | 3 +-- .gitignore | 4 +-- 4 files changed, 49 insertions(+), 5 deletions(-) create mode 100644 .github/actions/build/action.yml diff --git a/.github/actions/build/action.yml b/.github/actions/build/action.yml new file mode 100644 index 0000000..2247c6b --- /dev/null +++ b/.github/actions/build/action.yml @@ -0,0 +1,45 @@ +name: Build the CLI binary +description: >- + Build the standalone binary from portabase.spec and check that it can read + its own bundled metadata. A binary that reports no version cannot resolve + its templates either, so it is rejected here rather than shipped. + +inputs: + name: + description: Binary name, without extension. + required: false + default: portabase + +outputs: + path: + description: Path to the built binary. + value: ${{ steps.build.outputs.path }} + +runs: + using: composite + steps: + - id: build + shell: bash + env: + PORTABASE_BINARY_NAME: ${{ inputs.name }} + run: | + set -euo pipefail + rm -rf build dist + uv run pyinstaller portabase.spec + + case "$(uname -s)" in + MINGW* | MSYS* | CYGWIN*) EXT=".exe" ;; + *) EXT="" ;; + esac + BINARY="dist/${PORTABASE_BINARY_NAME}${EXT}" + test -f "$BINARY" + echo "path=$BINARY" >> "$GITHUB_OUTPUT" + + - shell: bash + run: | + set -euo pipefail + VERSION=$("${{ steps.build.outputs.path }}" --version | head -1) + echo "$VERSION" + case "$VERSION" in + *unknown*) echo "::error::binary cannot read its bundled version"; exit 1 ;; + esac diff --git a/.github/workflows/plumber.yml b/.github/workflows/plumber.yml index 5dc10b4..4aba54b 100644 --- a/.github/workflows/plumber.yml +++ b/.github/workflows/plumber.yml @@ -19,10 +19,10 @@ jobs: permissions: contents: read security-events: write + id-token: write steps: - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4 - uses: getplumber/plumber@7ad9d267ee5a00163cec9e5c749a088d5f565167 # v0.4.26 with: score-push: true - upload-sarif: true soft-fail: true diff --git a/.github/workflows/security.yml b/.github/workflows/security.yml index 7a83a05..e4aac39 100644 --- a/.github/workflows/security.yml +++ b/.github/workflows/security.yml @@ -42,5 +42,4 @@ jobs: env: GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} GITLEAKS_LICENSE: ${{ secrets.GITLEAKS_LICENSE }} - with: - config-path: .gitleaks.toml + GITLEAKS_CONFIG: .gitleaks.toml diff --git a/.gitignore b/.gitignore index e2556d1..fcb772d 100644 --- a/.gitignore +++ b/.gitignore @@ -1,5 +1,5 @@ -dist/ -build/ +/dist/ +/build/ .venv/ __pycache__/ # Keep portabase.spec: it is the build definition. From db29462eae6570eead71189d19edcbee72034648 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Th=C3=A9o=20LAGACHE?= Date: Mon, 14 Sep 2026 15:58:16 +0200 Subject: [PATCH 23/24] Create .gitleaksignore --- .gitleaksignore | 2 ++ 1 file changed, 2 insertions(+) create mode 100644 .gitleaksignore diff --git a/.gitleaksignore b/.gitleaksignore new file mode 100644 index 0000000..b311557 --- /dev/null +++ b/.gitleaksignore @@ -0,0 +1,2 @@ +# Fake EDGE_KEY +394b25d811a4267e2e2c3c668030e6b9adc68e87:.github/workflows/ci.yml:generic-api-key:56 From 1b0d326bdee6f01bc6aaa1c07735099be8602f23 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Th=C3=A9o=20LAGACHE?= Date: Mon, 14 Sep 2026 16:01:38 +0200 Subject: [PATCH 24/24] Update plumber.yml --- .github/workflows/plumber.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/plumber.yml b/.github/workflows/plumber.yml index 4aba54b..e29abf7 100644 --- a/.github/workflows/plumber.yml +++ b/.github/workflows/plumber.yml @@ -18,11 +18,11 @@ jobs: timeout-minutes: 10 permissions: contents: read - security-events: write id-token: write steps: - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4 - uses: getplumber/plumber@7ad9d267ee5a00163cec9e5c749a088d5f565167 # v0.4.26 with: score-push: true + upload-sarif: false soft-fail: true