Skip to content

feat(js): Create tsplugingen #10

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
aabedraba merged 1 commit into from
Sep 15, 2026
Merged

feat(js): Create tsplugingen #10

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions js/tools/tsplugingen/.gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
.build/
*.tgz
169 changes: 169 additions & 0 deletions js/tools/tsplugingen/Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,169 @@
# SPDX-License-Identifier: BSD-3-Clause
# Copyright (c) 2026, Unikraft GmbH. All rights reserved.
#
# Build one npm package per plugin, @unikraft/cloud-plugin-<name>-api, from the
# plugin's OpenAPI spec. See README.md.

# The plugins repository holds <PLUGIN>/api.tsp, not <PLUGIN>/openapi.yaml, so
# compile the spec there first.
SPEC_ROOT ?= ../../../../plugins
BUILD_ROOT ?= .build
TEMPLATES ?= ./templates
STATIC ?= ./static

GO ?= go
SHA256 ?= $(shell command -v sha256sum >/dev/null 2>&1 && echo sha256sum || echo shasum -a 256)
NPM ?= npm
NPX ?= npx
NODE ?= node

# sdk-range.sh and publish-check.sh run npm and node too.
export NPM
export NODE

# One knob for four things: the OpenAPI branch, the generator branch, the
# dist-tag of the SDK to build against, and the dist-tag to publish under.
CHANNEL ?= prod-staging
DIST_TAG = $(if $(filter prod-stable,$(CHANNEL)),latest,next)

# The generator follows the channel too. A prod-stable build must not render
# its plugins with staging's generator.
OPENAPI_GEN_MOD = unikraft.com/x/tools/openapi-gen
OPENAPI_GEN_REVISION = $(eval OPENAPI_GEN_REVISION := $(shell $(GO) list -m -f '{{.Version}}' $(OPENAPI_GEN_MOD)@$(CHANNEL) 2>/dev/null))$(OPENAPI_GEN_REVISION)
OPENAPI_GEN ?= $(GO) run $(OPENAPI_GEN_MOD)@$(OPENAPI_GEN_REVISION)

# A leaf subpath, so the plugin package does not pull the SDK's idiomatic layer
# into its module graph. Needs @unikraft/cloud >=0.1.1, which exports it.
CLIENT_IMPORT ?= @unikraft/cloud/core/http

SDK_SPEC ?= @unikraft/cloud@$(DIST_TAG)

# `build` installs the SDK from $(BUILD), two directories below this one, so a
# relative local spec would be looked up from the wrong directory there. Only
# path- and tarball-shaped specs become absolute; a registry spec such as
# "@unikraft/cloud@next" has to reach npm unchanged.
SDK_SPEC_ABS = $(if $(filter /% ./% ../% %.tgz,$(SDK_SPEC)),$(abspath $(SDK_SPEC)),$(SDK_SPEC))

# Derives range from "0.1.1-next.N" turns to ">=0.1.1-0 <0.2.0"
# sdk-range.sh explains why.
SDK_RANGE ?= $(shell ./sdk-range.sh $(SDK_SPEC_ABS))

PUBLISH_FLAGS ?=

# Every plugin under SPEC_ROOT that has a compiled spec.
PLUGINS := $(notdir $(patsubst %/,%,$(dir $(wildcard $(SPEC_ROOT)/*/openapi.yaml))))

SPEC = $(SPEC_ROOT)/$(PLUGIN)/openapi.yaml
BUILD = $(BUILD_ROOT)/$(PLUGIN)
# info.version, with YAML quotes and any leading "v" stripped for semver.
VERSION = $(shell awk '/^info:/{f=1;next} /^[^[:space:]#]/{f=0} f&&/^[[:space:]]+version:/{v=$$2; gsub(/["\047]/,"",v); sub(/^v/,"",v); if (v ~ /^[0-9]+\.[0-9]+\.[0-9]+(-[0-9A-Za-z.-]+)?(\+[0-9A-Za-z.-]+)?$$/) print v; exit}' $(SPEC))

# The package version is the spec's info.version, and nothing forces that line
# to move when the sources do. publish compares these against the published
# version, so a changed source under an unchanged version fails instead of
# silently reporting success.
SPEC_HASH = $(shell $(SHA256) "$(SPEC)" 2>/dev/null | cut -c1-64)
TEMPLATES_HASH = $(shell $(SHA256) $(sort $(wildcard $(TEMPLATES)/* $(STATIC)/*)) 2>/dev/null | $(SHA256) | cut -c1-64)
Comment thread
aabedraba marked this conversation as resolved.

# The spec and the templates are not the only inputs. The peer range, the
# client import and the generator decide the output as well, and none of them
# leaves a mark in any file the two hashes above cover. Without this, staging
# opening a new patch (0.1.1-next.N -> 0.1.2-next.0) rewrites every peer range
# while publish still reports the published version as built from these
# sources.
CONFIG_HASH = $(shell printf '%s\n%s\n%s\n' "$(SDK_RANGE)" "$(CLIENT_IMPORT)" "$(OPENAPI_GEN)" | $(SHA256) | cut -c1-64)

.PHONY: all list clean generate build publish publish-all config check-plugins

check-plugins:
@test -n "$(PLUGINS)" || { \
echo "error: no plugin specs under SPEC_ROOT=$(SPEC_ROOT)"; \
echo " The plugins repository holds <PLUGIN>/api.tsp, not <PLUGIN>/openapi.yaml."; \
echo " Compile it there ('npx tsp compile <PLUGIN>/api.tsp'), or set SPEC_ROOT."; \
exit 1; }

## Build every plugin.
all: check-plugins
@for p in $(PLUGINS); do $(MAKE) --no-print-directory build PLUGIN=$$p || exit 1; done

## Build and publish every plugin whose version is not on npm yet.
publish-all: check-plugins
@for p in $(PLUGINS); do $(MAKE) --no-print-directory publish PLUGIN=$$p || exit 1; done

## List the plugins that have a compiled spec.
list:
@echo $(PLUGINS)

## Print the resolved channel configuration.
config:
@echo "channel: $(CHANNEL)"
@echo "dist-tag: $(DIST_TAG)"
@echo "spec root: $(SPEC_ROOT)"
@echo "plugins: $(PLUGINS)"
@echo "client import: $(CLIENT_IMPORT)"
@echo "sdk spec: $(SDK_SPEC_ABS)"
@echo "sdk range: $(SDK_RANGE)"
@echo "generator: $(OPENAPI_GEN)"
@echo "templates: $(TEMPLATES_HASH)"
@echo "config: $(CONFIG_HASH)"

## Build one plugin, e.g. `make plugin-sandbox`.
plugin-%:
@$(MAKE) --no-print-directory build PLUGIN=$*

## Generate one plugin's TypeScript, without compiling it.
generate:
@test -n "$(PLUGIN)" || { echo "error: set PLUGIN=<name>"; exit 1; }
@test -f "$(SPEC)" || { echo "error: no spec at $(SPEC) (compile it with 'npx tsp compile $(PLUGIN)/api.tsp')"; exit 1; }
@test -n "$(VERSION)" || { echo "error: no semver info.version in $(SPEC)"; exit 1; }
@$(if $(filter file,$(origin OPENAPI_GEN)),test -n "$(OPENAPI_GEN_REVISION)" || { echo "error: could not resolve $(OPENAPI_GEN_MOD)@$(CHANNEL) to a revision with '$(GO) list -m'"; exit 1; })
@test -n "$(SDK_RANGE)" || { echo "error: could not resolve a version from SDK_SPEC=$(SDK_SPEC), so the @unikraft/cloud peer range is unknown; set SDK_RANGE explicitly"; exit 1; }
@test -n "$(SPEC_HASH)" || { echo "error: could not hash $(SPEC) with '$(SHA256)'"; exit 1; }
@test -n "$(TEMPLATES_HASH)" || { echo "error: could not hash $(TEMPLATES)/ and $(STATIC)/ with '$(SHA256)'"; exit 1; }
@test -n "$(CONFIG_HASH)" || { echo "error: could not hash the generation config with '$(SHA256)'"; exit 1; }
rm -rf "$(BUILD)"
mkdir -p "$(BUILD)"
$(OPENAPI_GEN) \
-i "$(SPEC)" \
-o "$(BUILD)" \
-t "$(TEMPLATES)" \
-v "package=api" \
-v "pluginName=$(PLUGIN)" \
-v "version=$(VERSION)" \
-v "sdkVersion=$(SDK_RANGE)" \
-v "clientImport=$(CLIENT_IMPORT)" \
-v "specHash=$(SPEC_HASH)" \
-v "templatesHash=$(TEMPLATES_HASH)" \
-v "configHash=$(CONFIG_HASH)"
cp -R $(STATIC)/. "$(BUILD)/"
@echo "generated @unikraft/cloud-plugin-$(PLUGIN)-api@$(VERSION) (peer @unikraft/cloud $(SDK_RANGE)) -> $(BUILD)"

## Generate and compile one plugin into an ESM package.
#
# The SDK goes on the install line positionally, which satisfies the
# peerDependency from a local tarball or path, so npm never queries the registry
# for the peer while the SDK is unpublished.
build: generate
cd "$(BUILD)" && $(NPM) install --no-save --no-audit --no-fund "$(SDK_SPEC_ABS)"
-cd "$(BUILD)" && $(NPX) --no-install @biomejs/biome format --write src
cd "$(BUILD)" && $(NPM) run build
@echo "built $(BUILD)/dist"

## Build one plugin and publish it, unless npm already carries that version
## built from the same sources. Needs an authenticated npm (NPM_TOKEN in CI).
##
## publish-check.sh decides: 0 publish it, 10 skip it, anything else is a
## failure it has already explained.
publish: build
@cd "$(BUILD)" && rc=0; \
"$(CURDIR)/publish-check.sh" "$(SPEC_HASH)" "$(TEMPLATES_HASH)" "$(CONFIG_HASH)" || rc=$$?; \
case $$rc in \
0) echo "publishing (tag: $(DIST_TAG))"; \
$(NPM) publish --access public --tag "$(DIST_TAG)" $(PUBLISH_FLAGS) ;; \
10) : ;; \
*) exit $$rc ;; \
esac

## Remove all build output.
clean:
rm -rf "$(BUILD_ROOT)"
193 changes: 193 additions & 0 deletions js/tools/tsplugingen/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,193 @@
# tsplugingen

`tsplugingen` builds a TypeScript client for each Unikraft Cloud plugin. It
publishes each client to npm as `@unikraft/cloud-plugin-<PLUGIN>-api`.

Its Go counterpart is [`sdkgen`](../../../go/tools/sdkgen), which renders a tree
in the [Go module proxy][goproxy] format. The package host serves that tree
directly to `go get`. npm has no mechanism that generates a package at install
time, so this tool builds and publishes in advance. CI renders each plugin's
specification to a finished npm package, then pushes the package to the
registry.

[goproxy]: https://go.dev/ref/mod#goproxy-protocol

- **Package:** `@unikraft/cloud-plugin-<PLUGIN>-api` on npm.
- **Version:** the plugin's OpenAPI `info.version`, without a leading `v`.
- **Input:** `<PLUGIN>/openapi.yaml` in the plugins repository.
- **Generator:** [`openapi-gen`](https://github.com/unikraft-cloud/x/tree/prod-staging/tools/openapi-gen)
with the templates in [`templates/`](templates). These templates are the
platform SDK's own templates, changed to use an external transport.
- **Peer dependency:** `@unikraft/cloud`. The generated classes extend its
`ApiClient`.

> **Needs `@unikraft/cloud` >= 0.1.1.** `CLIENT_IMPORT` defaults to the
> `@unikraft/cloud/core/http` subpath, which
> [js-sdk#26](https://github.com/unikraft-cloud/js-sdk/pull/26) adds. Until
> that lands and publishes, `build` cannot resolve the peer and npm fails with
> `notarget`. Point `SDK_SPEC` at a local js-sdk checkout or tarball to build
> before then.

## Generated code only

The package holds only generated code: one class per OpenAPI tag, methods that
carry the name of their `operationId`, and responses that keep the raw
envelope.

The layer with the ergonomic API is **not** here. It lives in
[`js-sdk`](https://github.com/unikraft-cloud/js-sdk), which depends on this
package and wraps it. For the same reason, this package does not derive the base
URL from the specification. `servers` in `api.tsp` hardcodes the `sandbox`
segment, but a plugin answers under the name that you attach it with. The SDK
builds the URL in `src/core/plugin.ts` instead, and this package takes a
finished `baseUrl`.

## The specifications are not in the repository

The plugins repository holds `<PLUGIN>/api.tsp`, not `<PLUGIN>/openapi.yaml`.
Compile the specification there before you generate a client:

```sh
cd ../../../../plugins
npm ci
npx tsp compile sandbox/api.tsp --warn-as-error
```

A plugin specification does not need `--namespace-flatten=strip`. `api.tsp`
already flattens its shared schema names with `@@friendlyName`.

## Usage

```sh
make config # show the resolved channel, plugins and peer range
make list # list the plugins that have a compiled specification
make plugin-sandbox # generate and build one plugin
make all # every plugin
make publish-all # publish everything that is not on npm yet
```

The build writes its output to `.build/<PLUGIN>/dist`, which git ignores.

The generated package builds with TypeScript 7 and formats with Biome 2.

## Do not hard-code the peer range

The Makefile computes `SDK_RANGE` from the `@unikraft/cloud` version that the
channel resolves to. Do not replace it with a constant such as `^0.1.0`.

npm's semver excludes a prerelease from a range unless some comparator carries a
prerelease tag on the same `major.minor.patch`. js-sdk publishes its staging
channel as `0.1.1-next.N`. Neither `^0.1.0` nor `>=0.1.0` matches that version.

npm does not report the mismatch as a conflict. npm installs a **second copy**
of `@unikraft/cloud` under this package instead. The tree then holds two
`ApiClient` classes and two `UnikraftCloudError` classes. Every `instanceof`
check that a caller makes against the SDK's error type returns `false`. The
install prints no warning.

On a prerelease channel, the Makefile adds a `-0` lower bound, so the range
matches:

| SDK version | Derived range |
| -------------- | ------------------ |
| `0.0.3` | `>=0.0.3 <0.0.4` |
| `0.1.0` | `>=0.1.0 <0.2.0` |
| `0.1.1-next.0` | `>=0.1.1-0 <0.2.0` |
| `1.2.3` | `>=1.2.3 <2.0.0` |

### A prerelease range expires at the next stable release

npm ties the prerelease exemption to one exact `major.minor.patch`, so a
derived range covers a whole staging cycle and then stops:

| SDK version | `>=0.1.1-0 <0.2.0` |
| -------------- | ------------------ |
| `0.1.1-next.0` | matches |
| `0.1.1-next.9` | matches |
| `0.1.1` | matches |
| `0.1.2` | matches |
| `0.1.2-next.0` | **no match** |

The range survives every `next.N` bump. It expires on one event: staging
opening the next patch, which is what happens when stable ships `0.1.1`. A
plugin published before that point would quietly get a second `@unikraft/cloud`
nested under it.

`configHash` below is what catches this. The peer range is part of the
published fingerprint, so at that boundary `publish` fails instead of skipping,
and the plugin has to be regenerated and republished against the new range.

## A publish checks the sources that it came from

The package version is the OpenAPI specification's `info.version`, and nothing forces
that line to move when the specification changes. A changed specification under
an unchanged version therefore generates new code. The publish step finds the
version on npm, skips it, and reports success. js-sdk keeps the stale client,
and no step fails.

To stop that, `generate` hashes every input that decides the output and writes
the hashes into the package:

```json
"unikraft": {
"specHash": "0d07a5aa…",
"templatesHash": "ce52b1af…",
"configHash": "3ea3f657…"
}
```

| Hash | Covers |
| --------------- | ----------------------------------------------------- |
| `specHash` | `<PLUGIN>/openapi.yaml` |
| `templatesHash` | `templates/` and `static/` |
| `configHash` | `SDK_RANGE`, `CLIENT_IMPORT` and `OPENAPI_GEN` |

The specification is not the only thing that moves. A template fix changes the
generated code while the specification stands still, and so does a new peer
range, a different client import, or a new generator revision. None of these
leave a mark in a file that the first two hashes cover.

By default, `OPENAPI_GEN` names the exact revision that the `CHANNEL` branch
resolves to, so a new commit on that branch changes `configHash`. An
`OPENAPI_GEN` that you set yourself, such as a local binary, enters the hash
only as its command text.

`publish-check.sh` compares all three against the published version:

| On npm | Result |
| ------------------------- | ----------------------------------------------- |
| Version absent | Publish it. |
| Present, all hashes match | Skip. A re-run of an unchanged channel is safe. |
| Present, any hash differs | **Fail**, and name the input that changed. |

A version that npm received before these hashes existed carries none of them.
For that version, `publish` warns and skips, because there is nothing to
compare.

A registry that cannot be reached is not an answer either way, so a failed
lookup fails the step. Reading it as "not published yet" would publish over the
check, and reading it as "carries no hash" would skip the check and report
success.

## Configuration

| Variable | Default | Description |
| --------------- | ------------------------------------------------------ | --------------------------------------------------------------------- |
| `SPEC_ROOT` | `../../../../plugins` | Root that holds `<PLUGIN>/openapi.yaml`. |
| `CHANNEL` | `prod-staging` | Release channel. Sets the generator branch, the dist-tag and the SDK. |
| `OPENAPI_GEN` | `go run unikraft.com/x/tools/openapi-gen@<revision>` | The generator command. `<revision>` is the commit that `CHANNEL` resolves to. |
| `CLIENT_IMPORT` | `@unikraft/cloud/core/http` | Where the generated classes import `ApiClient` from. |
| `SDK_SPEC` | `@unikraft/cloud@<dist-tag>` | npm package specifier for the peer. A tarball or a path also works, and is resolved relative to this directory. |
| `SDK_RANGE` | _derived from_ `SDK_SPEC` | The peer range that `generate` writes into `package.json`. See above. |
| `BUILD_ROOT` | `.build` | Directory that `generate` writes each plugin into. |
| `PUBLISH_FLAGS` | _(empty)_ | Extra `npm publish` flags. CI passes `--provenance`. |

## Templates

| File | Output | Contents |
| ------------------- | ---------------------------------- | -------------------------------------- |
| `models.ts.tmpl` | `src/api/models.gen.ts` | request and response models, enums |
| `resources.tmpl` | `src/api/*.gen.ts`, `index.gen.ts` | one `…Api` class per tag, and a barrel |
| `index.ts.tmpl` | `src/index.ts` | the container class that groups them |
| `package.json.tmpl` | `package.json` | manifest, exports, peer range |
| `README.md.tmpl` | `README.md` | the per-package readme |
Loading