Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion .agents/skills/rex-plugin-authoring/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ Build for Lua 5.1 and the Rex server environment. Check the target repository's

1. Choose a unique Lua module namespace. Amber defaults `main` to the repository name, with a trailing `.rex` removed; tell users to set `main` explicitly when the entry point differs.
2. Place the entry point at `lua/<main>.lua` or `lua/<main>/init.lua`, replacing dots in `main` with directory separators. For a repository with separate Rex code, use the equivalent path under `rex/lua/`. If `rex/lua/` exists, Amber exclusively uses it and never falls back to `lua/`.
3. Return a table with a `setup(opts)` function. Amber calls it with the plugin spec's `opts` table (empty by default) on every config load, reload, and config check. Metadata is optional. Handle missing option fields, document defaults, and do not rely on setup running only once.
3. Return a table with a `setup(opts, ctx)` function. Amber calls it with the plugin spec's `opts` table (empty by default) and a context table on every config load, reload, and config check. `ctx` has `name`, `dir`, `module_dir`, `commit` (the locked commit, possibly `nil`), and `data_dir` (a state directory Amber names but does not create). Resolve paths from `ctx` rather than substituting them at install time, and tolerate `ctx` being absent in tests. Metadata is optional. Handle missing option fields, document defaults, and do not rely on setup running only once.
4. Put plugin-owned submodules beneath the main namespace, e.g. `require("example_rex.helper")` at `lua/example_rex/helper.lua`. Amber's searcher is scoped to `main` and its dotted descendants; it does not add arbitrary plugin directories to `package.path` or resolve dependencies. Lua's `package.loaded` and module names are shared, so avoid collisions.

Minimal `lua/example_rex/init.lua`:
Expand All @@ -35,6 +35,12 @@ return M

Register actions and bindings during setup; defer operations that require a live session or client to callbacks. Config checks execute setup too, so avoid runtime, network, and filesystem side effects during setup beyond configuration. Plugins are trusted code, not sandboxed. For subprocesses use `rex.exec` with an argument array, check startup errors and nonzero exit status, and respect action deadlines; `os.execute` and `io.popen` are unavailable in Rex.

## Build steps and background processes

If the plugin needs work after checkout, document a `build` option for the Amber spec (a shell command, a function, or a list; see the README) rather than an install script. It runs on install and update within Amber's action budget, so keep it short. Amber holds back `setup` until it has succeeded. Put build output in `.gitignore` or under `data_dir` (`AMBER_PLUGIN_DATA_DIR` for shell steps): untracked files make `amber_restore` and `amber_update` refuse the checkout.

Amber has no service management and does not know where the `rex` binary is. For a long-running watcher, check the current Rex Lua reference, start it from `setup` with a detached `rex.exec.start`, resolve its script path from `ctx.dir`, keep state in `ctx.data_dir`, and make it exit when another instance is already running, because `setup` runs on every config load and check. State what you verified against a live Rex.

## Document installation

Give users an Amber declaration matching the actual published repository and module name, for example:
Expand Down
58 changes: 55 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ Amber installs GitHub Lua plugins on the machine running the Rex server and load

- Plugin specs in Lua, in the style of lazy.nvim.
- Semver versions and a lockfile for reproducible installs.
- A `build` step that runs after install or update, like lazy.nvim.
- An agent skill for writing Amber plugins.

## Quick start
Expand Down Expand Up @@ -51,7 +52,8 @@ Run with `rex do <action>`. Add `plugin=owner/repo` to target one plugin. Withou
| `amber_install` | Clones missing plugins. Uses the locked commit if one exists. Records new commits in the lockfile. |
| `amber_update` | Resolves the newest commit or version, checks it out, rewrites the lock entry. Unversioned: `git pull --ff-only`. Versioned: fetch tags, check out detached. Never resets local changes. |
| `amber_restore` | Checks out the locked commit (detached). Refuses dirty checkouts. Fails if the commit is unavailable. |
| `amber_status` | Shows load status, errors, path, commit, lock match, and version match. Does not fetch. |
| `amber_build` | Runs the `build` step of plugins that declare one, even if it already ran for the locked commit. |
| `amber_status` | Shows load status, errors, path, commit, lock match, build state, and version match. Does not fetch. |

`amber_status` takes no `plugin` argument. Load failures also go to `rex.log`; see [Rex logs](https://www.superlogical.com/rex/docs/linux/shared-server#logs).

Expand All @@ -67,6 +69,7 @@ A plugin is `"owner/repo"` or a table whose first element is that name. Unknown
| `path` | none | Absolute path to a local checkout. Never installed, updated, or locked. |
| `enabled` | `true` | `false` skips loading and rejects actions for it. |
| `subdir` | none | Repository subdirectory to load the plugin from; requires `main`. Enables monorepo layouts. |
| `build` | none | Command, function, or list run after install or update; see [Build steps](#build-steps). Not allowed with `path`. |

Options for `amber.setup` itself:

Expand All @@ -84,6 +87,33 @@ For development, point `path` at a local checkout:
{ "rawnly/smart-splits-backend-rex", path = "/absolute/path/to/smart-splits-backend-rex", main = "smart-splits-rex" }
```

## Build steps

Some plugins need work after checkout, such as compiling a helper. Declare it with `build`:

```lua
{ "alice/rex-fast", build = "make release" }, -- shell command
{ "alice/rex-fast", build = { "make deps", "make release" } }, -- steps run in order
{
"alice/rex-fast",
build = function(ctx)
return ctx.exec({ "make", "release" })
end,
},
```

- A string runs as `sh -c <string>` in the plugin directory (`module_dir` for `subdir` plugins). Its environment has `AMBER_PLUGIN_NAME`, `AMBER_PLUGIN_DIR` (the checkout), `AMBER_PLUGIN_MODULE_DIR`, and `AMBER_PLUGIN_DATA_DIR`.
- A function receives `{ name, dir, module_dir, data_dir, commit, exec }`. `exec(argv, opts)` runs a command in `module_dir` within the remaining action time; `opts.env` adds to the variables above instead of replacing them. The step fails if the function raises an error, returns `false`, or returns `nil` and a message.
- `build = false` disables the step. Local `path` plugins cannot have one: Amber never runs anything for them.
- `amber_install`, `amber_update`, and `amber_restore` run the build when the locked commit differs from the one recorded as built in the lockfile (`built`). An unchanged commit is not rebuilt. `amber_build` forces it.
- A build always targets the locked commit. If the checkout has moved away from it, the build is refused; run `amber_restore` first. `amber_build` needs a lock entry, so run `amber_install` first if the lockfile was removed.
- A failed build still records the checkout in the lockfile, fails the action, and leaves the plugin unbuilt. Fix the cause and run `amber_build`.
- Keep build output out of Git's way. `amber_restore` and `amber_update` refuse checkouts with untracked or modified files, so list generated files in the plugin's `.gitignore` or write them to `data_dir`.
- Until a build has run for the locked commit, Amber does not call the plugin's `setup`; `amber_status` reports `unbuilt`. Adding `build` to an existing plugin therefore needs one `amber_build` and a reload. If the lockfile cannot be read, Amber cannot tell and does not hold plugins back.
- Steps share the roughly 23-second action budget with Git, and a bulk action shares it across all plugins. Anything longer fails with a timeout; build slow plugins one at a time with `plugin=owner/repo`, or start long-running work from the plugin itself (see [Background processes](#background-processes)).
- The build runs after the lockfile is released, so other Amber actions are not blocked by it. If the lock entry changes while it runs, the result is not recorded and `amber_build` must run again.
- Build commands are trusted code with the Rex server's privileges, like plugin Lua. Read them before installing.

## Subdirectory plugins

One repository can carry several plugins. Set `subdir` to the path each plugin loads from, relative to the repository root; `main` is required because the repository-name default cannot resolve inside a subdirectory.
Expand Down Expand Up @@ -120,7 +150,7 @@ amber.setup({
- Tags win over branches of the same name. Branches resolve to `origin/<branch>`.
- Changing `version` does not move an existing checkout. Run `amber_update`.

Amber records exact commits in the lockfile. The file is JSON and safe to commit. The lock is authoritative: `amber_install` and `amber_restore` use it until you run `amber_update`. Install never replaces an existing entry. Unrelated entries are kept and nothing is pruned.
Amber records exact commits in the lockfile, plus a `built` commit for plugins with a build step. The file is JSON and safe to commit. The lock is authoritative: `amber_install` and `amber_restore` use it until you run `amber_update`. Install never replaces an existing entry. Unrelated entries are kept and nothing is pruned.

```sh
rex do amber_restore # all enabled managed plugins
Expand Down Expand Up @@ -153,9 +183,29 @@ return M
```

- Pick a unique module name. The Lua module namespace and `package.loaded` are shared.
- `setup` also receives a second argument, `ctx`, so a plugin does not need paths filled in at install time:

| Field | Meaning |
| --- | --- |
| `ctx.name` | Plugin identity, `owner/repo` or `owner/repo@subdir`. |
| `ctx.dir` | Checkout directory. |
| `ctx.module_dir` | Directory the module tree resolves from; differs from `dir` for `subdir` plugins. |
| `ctx.commit` | Commit recorded in the lockfile, or `nil`. |
| `ctx.data_dir` | State directory `$XDG_STATE_HOME/rex/amber/<owner>/<repo>` (default `~/.local/state`). Amber names it but never creates it. |

Plugins that ignore `ctx` keep working. Amber does not tell a plugin where the `rex` binary is.
- Plugins are trusted Lua code with the Rex server's privileges. They are not sandboxed.
- Config checks run `setup` too. Avoid side effects beyond configuration.

### Background processes

Amber manages files, not processes: it has no service support and does not write launchd or systemd units. A plugin that needs a long-running watcher should start it itself from `setup`, using paths from `ctx` instead of values substituted at install time. Details to check against the current [Rex Lua reference](https://www.superlogical.com/rex/docs/reference/lua), because Amber's tests stub Rex and do not exercise them:

- `rex.on` and `rex.every` are documented as script-only, so the watcher is a Lua script run with `rex do`, not code in `setup`.
- `rex.exec.start` with `detach = true` starts a process that outlives the config load.
- `setup` runs on every config load, reload, and `rex config check`. Make the script exit when another instance is already running (for example by labeling its client connection and checking for the label), and consider whether a check should start it at all.
- Amber does not stop such a process on update or removal; the plugin should handle restarts itself.

Install the [authoring skill](.agents/skills/rex-plugin-authoring/SKILL.md) for your agent:

```sh
Expand All @@ -169,7 +219,8 @@ npx skills add rawnly/amber --skill rex-plugin-authoring
- GitHub `owner/repo` only. No other Git URLs.
- No dependency resolution.
- No lazy loading.
- No uninstall. Remove the declaration and delete the directory yourself.
- No uninstall. Remove the declaration and delete the directory yourself. Files a build created outside the plugin directory stay too.
- No service or daemon management.

## Troubleshooting

Expand All @@ -182,6 +233,7 @@ npx skills add rawnly/amber --skill rex-plugin-authoring
| Lockfile error | The file is corrupt. Fix or remove it. Amber will not overwrite it. |
| Checkout changed but lock write failed | Inspect the checkout and lockfile before retrying. |
| `checkout differs from lockfile` warning | Run `amber_restore`. It refuses dirty checkouts. |
| Plugin reports `unbuilt` | The build has not run for the locked commit. Run `amber_build` (or fix the failing step), then `rex config reload`. |

## Contributing

Expand Down
121 changes: 106 additions & 15 deletions amber.lua
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ local parse = assert(loadfile(directory .. "amber/spec.lua"))()
local git = assert(loadfile(directory .. "amber/git.lua"))(util)
local loader = assert(loadfile(directory .. "amber/loader.lua"))(util)
local lock = assert(loadfile(directory .. "amber/lock.lua"))(util)
local build = assert(loadfile(directory .. "amber/build.lua"))(util)
local register_action, load_plugin = loader.register_action, loader.load_plugin

---@param config? AmberConfig
Expand Down Expand Up @@ -42,6 +43,10 @@ function M.setup(config)
return
end

-- Per-plugin state directories are only named here; plugins create them.
local state_home = os.getenv("XDG_STATE_HOME") or (home and home .. "/.local/state")
local state_root = state_home and state_home .. "/rex/amber"

local plugins, by_name, modules, by_dir = {}, {}, {}, {}
for _, spec in ipairs(config.plugins or {}) do
local ok, plugin = pcall(parse, spec, root)
Expand All @@ -65,38 +70,95 @@ function M.setup(config)
if not plugin.local_path then
by_dir[plugin.dir] = true
end
plugin.data_dir = state_root and state_root .. "/" .. plugin.slug or nil
by_name[plugin.name] = plugin
plugins[#plugins + 1] = plugin
end
end

-- Runs the Git operation and records the checkout inside one lockfile
-- transaction, then runs any pending build outside it so a slow or killed
-- build never leaves the lockfile guard behind. A second transaction records
-- the build only if the lock entry still points at the commit that was built.
local function manage(plugin, deadline, operation)
return lock.transaction(lockfile, function(data)
local pending
local result, err = lock.transaction(lockfile, function(data)
local entry = data.plugins[plugin.name]
if operation == "restore" and not entry then
return nil, "no lockfile entry for " .. plugin.name
end
local status, err
if operation == "install" then
status, err = git.install(plugin, deadline, entry and entry.commit)
local status, op_err
if operation == "build" then
if plugin.local_path then
return nil, "local plugins are managed outside amber"
end
if not util.exists(plugin.dir .. "/.git/HEAD") then
return nil, "not installed"
end
if not plugin.build then
return "no build step"
end
if not entry then
return nil, "no lockfile entry; run amber_install"
end
status = "built"
elseif operation == "install" then
status, op_err = git.install(plugin, deadline, entry and entry.commit)
elseif operation == "restore" then
status, err = git.restore(plugin, deadline, entry.commit)
status, op_err = git.restore(plugin, deadline, entry.commit)
else
status, err = git.update(plugin, deadline)
status, op_err = git.update(plugin, deadline)
end
if not status then
return nil, err
return nil, op_err
end
if operation == "restore" or (operation == "install" and entry) then
return status
local changed = false
if operation == "update" or (operation == "install" and not entry) then
local hash, hash_err = git.commit(plugin, deadline)
if not hash then
return nil, "checkout succeeded but lockfile was not updated: " .. tostring(hash_err)
end
-- A new commit invalidates the recorded build; an unchanged one keeps it.
local kept = entry and entry.commit == hash and entry.built or nil
entry = { commit = hash, version = plugin.version, built = kept }
data.plugins[plugin.name] = entry
changed = true
end
local hash, hash_err = git.commit(plugin, deadline)
if not hash then
return nil, "checkout succeeded but lockfile was not updated: " .. tostring(hash_err)
-- The build always targets the locked commit, never whatever HEAD is.
if plugin.build and entry and (operation == "build" or entry.built ~= entry.commit) then
pending = entry.commit
end
data.plugins[plugin.name] = { commit = hash, version = plugin.version }
return status, nil, true
return status, nil, changed
end)
if not result or not pending then
return result, err
end

-- The checkout and lock entry are already recorded, so a failure below
-- leaves the plugin unbuilt and amber_build can retry.
local head, head_err = git.commit(plugin, deadline)
if not head then
return nil, "cannot read checkout commit for build: " .. tostring(head_err)
end
if head ~= pending then
return nil, "checkout differs from lockfile; run amber_restore before building"
end
local built, build_err = build.run(plugin, pending, deadline)
if not built then
return nil, build_err
end
local recorded, record_err = lock.transaction(lockfile, function(data)
local entry = data.plugins[plugin.name]
if not entry or entry.commit ~= pending then
return nil, "lockfile changed during the build; run amber_build again"
end
entry.built = pending
return result, nil, true
end)
if not recorded then
return nil, "build succeeded but lockfile was not updated: " .. tostring(record_err)
end
return result
end
register_action("amber_install", "Install Rex Plugins", function(plugin, deadline)
return manage(plugin, deadline, "install")
Expand All @@ -107,6 +169,9 @@ function M.setup(config)
register_action("amber_restore", "Restore Rex Plugins", function(plugin, deadline)
return manage(plugin, deadline, "restore")
end, by_name, plugins, true)
register_action("amber_build", "Build Rex Plugins", function(plugin, deadline)
return manage(plugin, deadline, "build")
end, by_name, plugins, true)
rex.action({
name = "amber_status",
title = "Rex Plugin Status",
Expand Down Expand Up @@ -137,6 +202,10 @@ function M.setup(config)
if locked and head then
lock_matches = locked.commit == head
end
local built
if plugin.build and locked then
built = locked.built == locked.commit
end
result[plugin.name] = {
enabled = plugin.enabled,
path = plugin.dir,
Expand All @@ -152,6 +221,7 @@ function M.setup(config)
commit = head,
locked_commit = locked and locked.commit or nil,
lock_matches = lock_matches,
built = built,
lock_error = lock_error,
}
end
Expand Down Expand Up @@ -205,7 +275,28 @@ function M.setup(config)
rex.log("warn", "amber: " .. plugin.name .. ": cannot check version: " .. err)
end
end
load_plugin(plugin)
local context = {
name = plugin.name,
dir = plugin.dir,
module_dir = plugin.module_dir,
commit = locked and locked.commit or nil,
data_dir = plugin.data_dir,
}
-- A plugin that needs a build is not set up until the build has run for
-- its locked commit, since setup typically depends on the build output.
if
plugin.enabled
and plugin.build
and lock_data
and util.exists(plugin.dir .. "/.git/HEAD")
and not (locked and locked.built == locked.commit)
then
plugin.load_status = "unbuilt"
plugin.load_error = "build has not run for the locked commit; run amber_build"
rex.log("warn", "amber: " .. plugin.name .. ": " .. plugin.load_error)
else
load_plugin(plugin, context)
end
end
end

Expand Down
Loading
Loading