Skip to content

plugins: make every catalogue plugin run in the service; release v1.1.0 - #33

Merged
ZergsLaw merged 5 commits into
masterfrom
fix/plugin-bundles-run
Oct 5, 2026
Merged

ZergsLaw merged 5 commits into
masterfrom
fix/plugin-bundles-run

Conversation

@ZergsLaw

@ZergsLaw ZergsLaw commented Oct 5, 2026

Copy link
Copy Markdown
Contributor

Why

Run through the service's own path (plugarchive.Unpack + exec with an empty env), 47 of the 80 catalogue plugins never worked. Three causes, measured on one archive per plugin and then on all 1743 archives:

Cause Plugins Fix
Archive holds absolute symlinks (/etc/localtime, the loader under lib64); Unpack refused the whole archive 47 service: skip them on unpack
Binary needs GLIBC_2.38; image was bookworm (2.36) 15 runtime base → debian:trixie-slim
Wrapper execs a path that only exists in the build image (/usr/bin/java, /nodejs/bin/node, /app/…) 22 wrappers resolve from DIR=${0%/*}

Plus: plugins inherited the service's cwd (/, unwritable) — betterproto writes there.

What changed

  • internal/plugarchive — absolute symlinks are skipped (the entrypoint stays refused); the write-through-a-symlink-chain escape (x -> ., l1 -> x/.., l1/evil) is closed by resolving each entry's parent and every created link against the root.
  • Dockerfile — runtime stage trixie-slim. The service binary is static; glibc is backward compatible.
  • internal/adapters/registry — each plugin run gets a private working directory under plugins_dir/.tmp, removed afterwards.
  • plugins build — every built version is verified the way the service will run it: in-bundle absolute links made relative, archive round-tripped through plugarchive, entrypoint smoke-run in the runtime image as uid 65532 with an empty env. A failing version is emptied to its build.log. plugin.yaml gains smoke: {parameter, skip}; flags --runtime-image, --no-smoke. TestSmokeImageMatchesServiceBase keeps the smoke image equal to the Dockerfile.
  • registry/ — the 22 wrappers (JVM, Node, Python glibc/musl, native) made relocatable; bufbuild/connect-kotlin's download stage moved off a bullseye snapshot that now 404s.
  • README / AGENTS.md describe the bundle contract and the build checks.

Verification

  • Unit tests for every fix; the regression tests fail on the old code. go test -race ./... and golangci-lint clean.
  • All 333 versions of the 22 plugins rebuilt on linux/amd64 with smoke checks: 333/333. Archives uploaded to easyp-plugins and verified (size for all 1743, sha256 spot checks).
  • Full sweep of all 1743 archives in the new service image: 1733 generate code, 8 run and ask for options (grpc/web, anthropics/connect-rust), 2 start but reject the synthetic request (neoeinstein-prost v0.2.1 wants source_code_info, mercari-grpc-federation v0.12.0 an option). None fail for environment reasons. Before: 33/80 latest versions worked.

Follow-ups (not in this PR)

  • plugins pack out loses its path when the argument is spelled like the --out flag.
  • smoke.parameter for the 10 archives above, so a full rebuild does not reject working versions.
  • Registered versions of the 22 plugins need UpdatePlugin (new sha256) wherever they are registered.

🤖 Generated with Claude Code

Edgar Sipki and others added 4 commits October 3, 2026 10:02
Run through the service's own path, 47 of the 80 catalogue plugins never
worked. Three causes, all on the service side for 25 of them:

- plugarchive refused any archive holding an absolute symlink, and every image
  dump holds some (/etc/localtime, the loader under lib64). Such links are now
  skipped; the entrypoint stays refused. The write-through-a-symlink-chain
  escape (x -> ., l1 -> x/.., l1/evil) is closed by resolving each entry's
  parent and every created link against the root.
- protoc's own plugins need glibc 2.38; the image was bookworm (2.36). The
  runtime stage is trixie-slim now. glibc is backward compatible; the service
  binary is static.
- plugins inherited the service's cwd, / and unwritable; betterproto writes
  there. Each run gets a private working directory under plugins_dir/.tmp.

Checked against all 80 latest archives in the rebuilt image: 33 -> 58 work,
no regressions (same output per plugin). The other 22 exec absolute paths in
their wrappers and need a rebuild.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A version now builds only if it also survives what the service does to it:

- absolute symlinks pointing at files the bundle carries are rewritten as
  relative ones (a Debian JRE reads java.security through one), the rest are
  removed;
- the bundle is packed and unpacked with plugarchive, so an archive the service
  would refuse fails here rather than after a push;
- the entrypoint is run in the service's base image (debian:trixie-slim, kept
  equal to the Dockerfile by a test) as uid 65532, with an empty environment,
  on a synthetic CodeGeneratorRequest; a CodeGeneratorResponse — even one
  carrying an error — passes.

A version that fails is emptied down to its build.log, so push, register and
the build cache no longer see an entrypoint for it. plugin.yaml gains an
optional `smoke` block (parameter, skip); flags --runtime-image and --no-smoke.

Checked end to end: grpc/go builds and passes; bufbuild/es, whose wrapper still
execs /nodejs/bin/node, is refused with that message and leaves only its log.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Their wrappers exec'd paths that only existed inside the build image
(/usr/bin/java, /nodejs/bin/node, /app/...); the service runs the unpacked
bundle from plugins_dir on its own base, so none of them could start. Each
wrapper now resolves everything from DIR=${0%/*}:

- JVM: the bundle's own JRE, found as usr/lib/jvm/*/bin/java;
- Node: the bundle's node with the script path;
- Python: the bundle's interpreter under the bundle's own dynamic loader
  (glibc or musl), so its libraries never mix with the host's;
- native: the binary beside the wrapper.

Inside the image $0 is /plugin and DIR is empty, so the paths read as before.

bufbuild/connect-kotlin's download stage moves off a bullseye snapshot whose
security pool now 404s.

Rebuilt with smoke checks on: the latest version of all 22, and all 9 versions
of connect-kotlin across its three Dockerfiles, run in debian:trixie-slim.
README and AGENTS.md describe the bundle contract and the build checks.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Every catalogue plugin now runs in the service: the image moves to trixie,
plugarchive skips absolute symlinks and closes the symlink-chain escape, each
run gets a private working directory, plugins build verifies every bundle the
way the service runs it, and the 22 wrapped plugins resolve paths from their
own directory. Minor rather than patch because the runtime base changed.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@ZergsLaw
ZergsLaw enabled auto-merge October 5, 2026 05:18
govulncheck flagged the OTLP trace exporter (v1.43.0): its config logging can
leak endpoint URLs into info logs, and the service calls it from
telemetry.Init. Fixed upstream in v1.45.0; the otel modules move together to
v1.47.0, which is what go mod tidy settles on with otelgrpc v0.70.0.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@ZergsLaw
ZergsLaw merged commit b5e8247 into master Oct 5, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant