Skip to content

CI: static lint pipeline + strict JSON Schemas for state.json and sibling artifacts #55

Description

Background

Today the PR pipeline only checks docs staleness, workflow lint (actionlint), and plugin version drift. It does not validate that:

  • Bash scripts (.github/scripts/, .github/skills/**/scripts/) are shellcheck-clean
  • YAML files conform to a consistent style
  • Markdown is consistent
  • The seven JSON artifact types Git-Ape emits per deployment match a documented shape:
    • state.json, metadata.json, requirements.json, cost-estimate.json, security-gate.json, policy-recommendations.json, parameters.json

Real-world drift exists. Inspecting an existing artifact corpus (e.g. Azure/git-ape-private) shows two distinct, incompatible shapes for security-gate.json shipping under the same claimed schemaVersion: "1.0":

  • Boolean form (older): criticalPassed: true, highPassed: true
  • Count form (newer): criticalTotal: 5, criticalPassed: 5, highTotal: 5, highPassed: 5

Same field name, different type. Only state.json carries schemaVersion at all today.

Goal

Land a tiered static-validation pipeline so every PR catches the broadest set of regressions before merge — without requiring a sandbox subscription. This issue covers Phase 1 (static lint), Phase 1A (schema introduction), and a reduced Phase 2 (fixture-driven schema validation).

Phase 1 — Static lint pipeline

New .github/workflows/git-ape-ci.yml running in parallel jobs:

  • shellcheck on every .sh (strict mode)
  • yamllint on every YAML
  • markdownlint on every .md (excluding generated website/build)
  • check-jsonschema validation of every committed JSON artifact (see Phase 1A)
  • bats tests against the fixture corpus

Reuse (do not duplicate):

  • git-ape-actionlint.yml — already covers workflow lint
  • git-ape-docs-check.yml — already covers docs staleness
  • git-ape-plugin-version-check.yml — already covers plugin.json version drift

Phase 1A — Schema introduction & versioning

Author strict JSON Schemas (draft 2020-12) under schemas/git-ape/<artifact>/v1.json:

  • _defs/v1.json — shared types (deploymentId, azureSubscription, armResourceId, iso8601DateTime, cafAbbreviation, environmentTier, deploymentStatus, softDeletableType)
  • state/v1.json — matches website/docs/deployment/state.md
  • metadata/v1.json, security-gate/v1.json, requirements/v1.json, cost-estimate/v1.json, policy-recommendations/v1.json
  • plugin/v1.json — for plugin.json

Locked-in design decisions:

  1. Versioning model: per-artifact schemaVersion, shared $defs versioned together as a release train
  2. Strictness: additionalProperties: false on top-level objects only; nested objects allow extras during transition
  3. Pre-1.0 corpus: migrate-first, quarantine fallback (migration tooling shipped in this PR)
  4. security-gate.json: adopts count form only; boolean form is rejected at v1.0
  5. $schema injection: emitters write a relative path (no network coupling)
  6. Doc generation: hybrid — narrative hand-written, field tables generated in a follow-up

Phase 2 — Reduced (parity deferred)

main does not yet have a state.json emitter; those scripts arrive in #44. Phase 2 in this PR therefore covers:

  • Fixture corpus under tests/fixtures/ with valid + invalid samples per artifact type
  • bats tests asserting good fixtures pass schema validation and bad fixtures fail
  • Negative test for security-gate.json boolean form (must be rejected by v1.0 schema)

Bash↔PowerShell parity testing is deferred to a follow-up issue once #44 lands.

Out of scope (tracked separately)

  • Real-Azure sandbox E2E (/e2e label trigger) — Phase 4
  • Agent behavioral snapshot tests — Phase 5
  • PR feedback aggregation — Phase 6
  • bash↔PowerShell parity (depends on Leverage Deployment Stacks for idempotent operations #44)
  • ARM template static checks (Checkov, ARM-TTK, PSRule for Azure, MSDO templateanalyzer)
  • Auto-generated field-reference docs

Acceptance criteria

  • git-ape-ci.yml runs all jobs in parallel and gates the PR
  • At least 7 schema files committed under schemas/git-ape/
  • Shared $defs referenced from per-artifact schemas (no duplication)
  • scripts/validate-schemas.sh invokable locally and from CI
  • Negative-test fixtures (e.g., security-gate.json boolean form, state.json missing required field) cause CI to fail when run against the strict schemas
  • plugin.json itself validates against schemas/git-ape/plugin/v1.json
  • All tools pinned to versions (no floating)
  • tests/fixtures/README.md documents how to add a new fixture

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions