mayfly

Short lived ephemeral environment infrastructure.

One YAML spec declares an environment — S3 buckets, RDS databases, ElastiCache, MSK/Kafka, DynamoDB, ALBs, and app deployments. mayfly up materializes it in an isolated Kubernetes namespace, mayfly down (or the TTL reaper) destroys it. No Docker socket, no privileged pods, no real AWS.

Website: https://mayfly.sh · Full documentation: https://docs.mayfly.sh (source in docs/, built with docs/build.sh).

How it works

• Each environment is one namespace, named deterministically from the spec's seed (merry-blonde-stoat, or env-merry-blonde-stoat with namespacePrefix: env); namespace deletion is complete teardown. Two specs with different seeds coexist; re-running up on the same spec is idempotent, and up refuses a namespace whose recorded seed differs or that mayfly didn't create (name collisions error instead of cross-contaminating or adopting someone else's namespace).
• A swappable AWS emulator (emulator.kind: ministack default, or floci) runs inside the namespace behind a single Service — apps and provisioners always use http://aws:4566. Emulator images are digest-pinned by mayfly; the spec can override image/version (latest is rejected).
• Service provisioning picks a backend per service (auto | emulator | native). With ministack, RDS goes through the real AWS API: create-db-instance spawns an actual postgres container via kubedock, describe-db-instances returns a working in-cluster endpoint (aws:15432). ElastiCache works the same way (aws:16379). MSK is hybrid: mayfly deploys a real Redpanda broker natively, then registers the cluster through the MSK control-plane API — ListClusters/ DescribeCluster answer correctly and GetBootstrapBrokers (via MINISTACK_MSK_BOOTSTRAP) returns that broker. Anything the chosen emulator can't back falls to the native backend (all container services on floci) with the identical Secret contract.
• Every service's endpoints land in a per-service Kubernetes Secret — the only contract apps consume. Endpoints are uniform across backends: always servicename:standard-port (rds-appdb:5432, elasticache-cache-a:6379, msk-events:9092). For emulator-backed services mayfly creates that Service selecting the kubedock-spawned pod directly (label selectors dbid/clusterid), so data traffic goes pod-to-pod and survives emulator restarts, while the AWS API's own aws:<published-port> answer stays valid through the reverse-proxy. App pods also get AWS_ENDPOINT_URL pointing at the emulator with test/test credentials.

Install

uv sync            # dev setup (or: pip install -e . for just the CLI)

Requires kubectl on PATH (used for port-forwarding) and a Kubernetes cluster — k3d/k3s is plenty.

Usage

mayfly up env.yaml                 # create/update the environment
mayfly status env.yaml             # pods + provisioned secrets
mayfly list                        # all mayfly environments, age + TTL
mayfly render env.yaml             # print resolved plan, touch nothing
mayfly extend env.yaml --ttl 4h    # push expiry out
mayfly down env.yaml               # teardown (namespace delete)
mayfly reap [--dry-run]            # delete every expired environment
mayfly install                     # in-cluster reaper CronJob (mayfly-system)

All cluster-touching commands take --context / --kubeconfig. down and reap refuse namespaces not labeled mayfly.dev/managed=true.

The seed is the environment's identity: same seed → same environment (idempotent update/heal), new seed → a fresh environment alongside it. --seed overrides the spec without editing it — mayfly up --seed pr-1234 in CI gives each PR its own environment from one shared spec file, and mayfly up --seed scratch-$(whoami) gives you a personal sandbox.

Spec

apiVersion: mayfly/v1alpha1
seed: pr-1234          # deterministic env name derives from this
# namespacePrefix: env # namespace becomes env-<name>; omit for bare <name>
ttl: 8h                # reaped after this
# ingressDomain: envs.example.com  # domain for generated ingress hosts
#                                  # (default localtest.me -> 127.0.0.1)

emulator:
  kind: ministack      # ministack | floci; omit for pinned default
  # image: ministackorg/ministack   # override to self-host/pin your own
  # version: "1.4.4"                # tag; 'latest' is rejected

services:
  s3:
    buckets: [assets, uploads]
  rds:
    - name: appdb
      engine: postgres # postgres | mysql | mariadb
      dbName: app
      # backend: auto  # auto | emulator | native
  elasticache:
    - name: cache-a
      engine: redis    # redis | valkey | memcached
      version: "7.2"   # engine version -> container image tag
  msk:
    - name: events
      topics: [orders]

apps:
  myapi:
    image: ghcr.io/you/myapi:sha-abc123
    port: 3000
    command: ["/bin/server"]     # optional entrypoint override
    args: ["--verbose"]
    replicas: 2
    env: {LOG_LEVEL: debug}
    secrets: [rds-appdb, elasticache-cache-a]  # env-from these secrets
    resources: {cpu: 100m, memory: 128Mi, memoryLimit: 512Mi}
    readiness: {path: /healthz}  # httpGet probe; omit for none
    imagePullSecret: regcred     # copied into the namespace at `up` from
                                 # --pull-secret-namespace (default "default")

For anything without a dedicated field, each app takes a patch: — arbitrary YAML deep-merged onto the generated Deployment as the final step (maps merge recursively; lists of named objects like containers/volumes/env merge by name; other lists replace). mayfly re-asserts its invariants afterward (selector, app label, enableServiceLinks: false), so a patch can add sidecars, volumes, tolerations, or securityContext but can't silently break the wiring the environment depends on.

examples/ has runnable specs, smallest first: env-minimal.yaml (one database + the dragonfly dashboard), env-caches.yaml (redis/valkey/ memcached side by side), env-pr.yaml (a per-PR CI template driven by --seed/--set), env.yaml (the kitchen sink), and env-alb.yaml (real AWS ALBs on EKS).

Each app becomes a Deployment + Service reachable at <name>:8080 in-namespace. App pods get AWS_ENDPOINT_URL plus whatever the listed secrets carry (DATABASE_URL, REDIS_URL, KAFKA_BROKERS, ...), and apps deploy only after all services are provisioned.

Internal ALBs

services.alb gives an environment an emulated ALB with a working data plane — created through the real elbv2 API (target groups, listeners, describe-load-balancers), with live traffic routed to the target app:

services:
  alb:
    - name: hello-alb
      targetApp: hello   # must be one of apps:

Requests to http://aws:4566/_alb/hello-alb/ (or Host header hello-alb.alb.localhost) proxy through the ALB to the app with ALB-style X-Forwarded-* and X-Amzn-Trace-Id headers; the alb-<name> secret carries ALB_URL/ALB_DNS_NAME. The data plane for instance/ip targets was contributed upstream by mayfly (ministack#1113) and ships in MiniStack ≥ 1.4.4, the pinned default. Path-pattern/host-header listener rules, redirects, and fixed-responses all work against the same data plane. For real AWS ALBs later, apps take ingress: {className: alb, annotations: {...}} (see examples/env-alb.yaml).

dragonfly — the connectivity verifier

dragonfly/ is a small companion app that proves an environment's wiring end-to-end with zero configuration: it discovers services the way a real AWS application would — describe-db-instances, describe-cache-clusters, list-clusters/get-bootstrap-brokers against the emulator's control plane (using the AWS_ENDPOINT_URL mayfly injects into every app pod) — then round-trips data through every instance found: postgres insert+select, redis SET+GET, kafka produce+consume. Declare a service in the spec and a tile appears; no secrets to mount, no lists to maintain. It serves a web interface at / (one live status tile per instance, auto-refresh every 5s), JSON at /api, and /healthz for its readiness probe — so the dragonfly pod only goes Ready once every discovered service actually works, making mayfly up's success itself a connectivity test. It's also a standing fidelity test of the emulator's discovery APIs: if a describe-* call returns an endpoint that doesn't work, dragonfly is the first to know.

Published as ghcr.io/jasondcamp/mayfly-dragonfly (multi-arch; siblings: mayfly-hello, mayfly-caddis, mayfly-caddis-frontend, mayfly-cli — scripts/publish-images.sh builds and pushes all five). Clusters pull them directly; for local iteration the e2e script builds the working tree under the same names and imports them.

mayfly up examples/env.yaml
kubectl -n <namespace> port-forward svc/dragonfly 8080:8080  # then open http://localhost:8080

The e2e harness builds and imports it automatically; the example spec wires it to all three services.

Secrets written per service: s3-buckets (BUCKETS, S3_ENDPOINT), rds-<name> (DATABASE_URL, DB_), elasticache-<name> (REDIS_URL, REDIS_), msk-<name> (KAFKA_BROKERS), dynamodb-<name> (TABLE_NAME, HASH_KEY, DYNAMODB_ENDPOINT).

Invariant: every service section the spec supports is (a) provisioned with a Secret contract and (b) verified by dragonfly — adding a service kind to mayfly means adding its dragonfly check in the same change.

Development & testing

Three tiers, cheapest first:

make test    # unit tests (spec, naming, manifests, backend resolution) — no cluster
make lint    # ruff
make e2e     # full loop on a disposable k3d cluster: create cluster ->
             # up -> smoke test -> up again (idempotency) -> down -> delete cluster

make e2e never touches your default kubeconfig. CI runs unit + e2e on every PR (.github/workflows/ci.yml).

For iterating against a long-lived local cluster instead of paying cluster create/pull each run:

k3d cluster create mayfly-dev --kubeconfig-update-default=false
export KC=$(k3d kubeconfig write mayfly-dev)
mayfly up examples/env.yaml --kubeconfig "$KC"
./examples/smoke-test.sh <namespace> "$KC"   # namespace from mayfly render/up output

A real (multi-node) k3s box is worth a second-tier pass before anything serious — scheduling, image-pull latency, and storage behave differently than single-node k3d — pointed at a dedicated kubeconfig/context.

Design notes / gotchas discovered

• kubedock's Docker volumes gap decided floci's backends. floci's container-backed services (RDS etc.) require the Docker volumes API, which kubedock doesn't implement (501) — so with emulator.kind: floci every container service uses the native backend and floci serves only in-process APIs. MiniStack's Docker calls avoid the volumes API, which is why its RDS works through kubedock.
• MiniStack + kubedock must share a pod. MiniStack's container readiness checks and port bindings assume the Docker daemon is on its own localhost; colocating kubedock in the same pod makes that literally true. Combined with MINISTACK_RDS_PUBLIC_ENDPOINT=1 and MINISTACK_HOST=aws, describe-db-instances advertises aws:<port> — an endpoint that actually works from any pod in the namespace. Note the RDS public-endpoint flag is load-bearing twice over: besides host-published ports and localhost readiness, it short-circuits Docker-network detection entirely (_get_ministack_network() returns None), which is why RDS never hits the network kubedock not found failure that sinks ElastiCache.
• DOCKER_NETWORK must stay unset for MiniStack under kubedock. Setting it forces ElastiCache down the network-attach path, which kubedock rejects (network kubedock not found) → MiniStack silently falls back to advertising its compose-sidecar default redis:6379 (and even with the network pre-created via kubedock's /networks/create, the network path advertises kubedock's fake container IP 127.0.0.1). With the variable unset, ElastiCache takes the published-port branch and advertises MINISTACK_HOST:16379+ — which resolves and works in-cluster. RDS is indifferent either way: its PUBLIC_ENDPOINT mode short-circuits network detection entirely.
• Every pod sets enableServiceLinks: false: the aws Service otherwise injects AWS_PORT=tcp://...-style env vars, and Quarkus-based emulators (floci) fatally misparse the analogous FLOCI_PORT as an int property.
• Emulators return 200 + an empty list for describe-calls on nonexistent resources where real AWS raises (e.g. DBInstanceNotFound); existence checks must test emptiness, not exceptions.
• Emulator state is in-memory: an emulator pod restart forgets AWS state while the service pods live on (kubedock reaps its orphans after 1h). mayfly status therefore reads cluster state (Secrets), not the emulator.