Skip to content

persist: keep records for each player across sessions - #132

Draft
paralin wants to merge 41 commits into
masterfrom
persist-r1
Draft

paralin wants to merge 41 commits into
masterfrom
persist-r1

Conversation

@paralin

@paralin paralin commented Nov 15, 2026

Copy link
Copy Markdown
Owner

This is branch persist-r1 at d6b499c, opened as a draft so CI builds and tests it, including the TypeScript scriptc suite in its shards. Master c513f19 is merged into the branch.

The branch adds a persist service: a mod keeps a record for each player that outlives the session, reads it with get, and changes it with update and add. A stored value changes only by a field of the kind that wrote it, an add needs an update key, and an update that took too long can be resent with its same key. The service is defined in src/wasm/persist/persist.proto, with a Go record store that one host at a time opens and that names its holder by process ID and program name, and a library module, js/src/persist.ts, with its tests. The docs cover the service and the 256 KiB limit on a persist mod's mod.json.

Not to be merged from here until CI passes and the range is read.

paralin and others added 30 commits November 10, 2026 23:20
A mod's manifest now lists the services it calls with CallService, by name
only, each needed or optional. Before a mod starts, the sandbox logs which
provider answers each named service, refuses to start the mod when a needed
service has none, naming it, and starts it without an optional one.

modlock play's sandbox answers a named service from a fixture in the mod's
project: services/<name>.json, an object of replies by method, each sent as
its JSON.

Signed-off-by: Christian Stewart <christian@aperture.us>
StartServiceCall returns a call number at once, and a ServiceReplied event
ends the call in a later frame: with the reply, the service's refusal, a
timeout of at most 30 seconds, or unavailable when nobody provides the
service, which is then never asked. The host drops a mod's waiting calls
when it stops or reloads, so a late reply reaches nothing.

A host provides such a service with WasmHost::ProvideLater. A service
from Provide answers a started call in the next frame, and the sandbox
answers its scores and fixtures the same way, so a mod cannot tell which
provider it has. The Go, TypeScript, Python and Luau libraries gain
callServiceLater with a done callback.

Signed-off-by: Christian Stewart <christian@aperture.us>
A layout drawn from a mod's XML had no way to tell the mod anything, so a
card with buttons had to be built from Modlock's own nodes. A layout now
calls Modlock.press(action, payload), which runs the console command
modlockaction <mod> <node> <action> [payload], the same way the renderer
already sends a button's press and a text entry's input.

The engine names the slot that ran the command, so the host stamps the
player and never reads it from the layout. The host takes the press only
from a running mod, from a layout node in that player's own tree, with an
action name of at most 64 word characters and a payload of at most 180
bytes of UTF-8, and at most eight presses at once refilling four a second
for each player. A press that passes reaches the mod as a UiPressEvent on
the layout's node with the new action and payload fields, so no SDK needs
a new method; the JS SDK hands it to the layout's onAction.

Signed-off-by: Christian Stewart <christian@cjs.zip>
WasmHostObserver gains Halted, which the host sends when a mod's running
build stops for a reload, a restart, a failure or its end, after its
started service calls are dropped and before its next build starts. A
host that keeps state for those calls, such as a provider waiting on a
server's answer, forgets it there; Started comes only after the next
build's start, which may already have started calls of its own.

Signed-off-by: Christian Stewart <christian@aperture.us>
The services section said a host refuses a mod whose needed service it
cannot provide. Only the sandbox does; the game host starts the mod
either way, and the mod's call to a missing service fails.

Signed-off-by: Christian Stewart <christian@aperture.us>
Each runtime starts two calls through its library, then gets a refusal,
an answer, a second answer for the same call and a reply for a call it
never made. Each call's function runs once, with the reply or the
failure, and the extra replies are ignored.

Signed-off-by: Christian Stewart <christian@aperture.us>
The renderer writes the tree's mod, which is the name the host runs the
mod under. That is the slug unless the host loaded it under another name.

Signed-off-by: Christian Stewart <christian@cjs.zip>
Say how a layout's script presses its mod with Modlock.press and the
element's onAction, the console command it runs, what the server checks
and bounds, and that the sandbox runs no layout scripts.

Signed-off-by: Christian Stewart <christian@cjs.zip>
Signed-off-by: Christian Stewart <christian@cjs.zip>
Signed-off-by: Christian Stewart <christian@cjs.zip>
The sandbox preview drew a layout's panels and sheets but ran none of its
scripts, so a mod whose interface is a layout script showed an empty
panel, and a press the script sent never left the page.

Run each layout's script includes in order, in one scope, with a $ that
has the layout's panel as its context, cancellable timers and the
unhandled events the layout library listens on. The page serves a build's
scripts from their compiled source and the layout library from its own
copy. A script's modlockaction command becomes a press with its action
and payload, which the sandbox delivers as the game's host would, and
refuses when the game could not carry it.

Signed-off-by: Christian Stewart <christian@cjs.zip>
modlock play answered a named service from services/<name>.json only in
the sandbox, and the game host started a mod whatever services its
manifest named, so a mod that needs a service failed only at its first
call in the game.

The host now refuses to start a mod whose needed service nobody provides,
naming the service, and starts one without an optional service. modlock
play reads the projects' fixtures before a real game starts, prints who
answers each named service, and hands the host the replies in a file it
names with the new --fixtures option. Each fixture answers only its own
mod, and a method the fixture lacks is refused, naming the file.

Signed-off-by: Christian Stewart <christian@cjs.zip>
browser/dom.ts imports browser/layouts.ts since the preview began running layout scripts, so a vendored copy of the sources could not build the page without it.

Signed-off-by: Christian Stewart <christian@cjs.zip>
Signed-off-by: Christian Stewart <christian@cjs.zip>
A real game session answers a mod's named services from fixtures only, so
a mod whose service lives on a backend cannot run its own data under plain
modlock play. With the forward services flag, modlock play writes a forward
for each named service in place of its fixture, and the host sends each
call a mod starts on that service over the control link. The session posts
it to the origin the flag names with the session modlock login saved, read
at each call, and hands the answer back to the host.

The flag is off by default, never signs in, and refuses an origin that is
not https or loopback http. Calls made at once still fail on a forwarded
service, at most eight calls wait on the origin, and the host fails the
calls still waiting when the controller goes.

Signed-off-by: Christian Stewart <christian@cjs.zip>
Signed-off-by: Christian Stewart <christian@aperture.us>
Brings the 6861 SetAbsOrigin and SetAbsVelocity signature fix and the
rest of v0.1.77 onto the branch that carries the service call forwarding.

The only conflict was wasm.proto, where both sides added an rpc next to
each other; both are kept. The generated host files were rebuilt with
scripts/gen-proto.sh rather than merged by hand.

Signed-off-by: Christian Stewart <christian@aperture.us>
A mod declares record types under persist in mod.json and names the
persist service. Each type has typed fields (count, number, max, string,
tags) and a writers rule. The JS library reads a record with persist.get,
persist.getMany and changes it with persist.update, and Go mods get the
same calls. Updates are field operations applied all or none, so two
servers that update one player at once both land; an update key makes a
resend count once over the last 64 keys. Records are bounded at 8 KB,
updates at 32 changes, reads at 16 players, and calls past a bound are
refused with the record unchanged.

modlock play keeps the records in the project's .modlock/persist, one
file per player, written whole through a rename, so they survive a
restart. The sandbox answers for its stand-in player; in the game the
CLI answers forwarded persist calls itself from the players it saw join,
even with --forward-services, since the host names only a slot.

callServiceLater moves to its own module so persist can use it without
an import cycle. The prop-stats example keeps a player's wins, longest
hide and an unlock.

Signed-off-by: Christian Stewart <christian@aperture.us>
scriptc cannot compile JSON.stringify of a record whose values may be
undefined, so the persist test file fell back to QuickJS and
TestScriptcSuite failed. The test now checks the reply's field names and
roundsWon directly, and builds as WebAssembly.

Signed-off-by: Christian Stewart <christian@aperture.us>
Open takes a lock file in the store's directory and records the holder,
so a second modlock play or sandbox on the same records is refused with
the first holder named instead of both writing the same files.

Signed-off-by: Christian Stewart <christian@aperture.us>
The holder file already records the process ID and the command line of the
process keeping a store's records. The second host test now checks that a
refused Open names both, not only the process ID.

Signed-off-by: Christian Stewart <christian@aperture.us>
persist.update refuses, without sending it, an update with an add and no
key, since a resend after a lost answer would count it twice. The local
store refuses such an update too, as Hyperline does, and refuses an op on
a value an earlier build stored as another kind than mod.json now declares,
changing nothing. The guide and the Go UpdateRecord comment say so, and the
tests that added without a key now pass one.

Signed-off-by: Christian Stewart <christian@aperture.us>
The holder file recorded the full command line, readable by every user,
and a command line can carry a path or a token. It now records the process
ID and the base name of the program, never the arguments, with mode 0600,
also when an older modlock left the file; the refusal a second host gets
says the same and no more. Only the next modlock of the same user reads it.
The test now requires the process ID and program name, and that the
helper's arguments appear neither in the file nor in the refusal.

Signed-off-by: Christian Stewart <christian@aperture.us>
Hyperline refuses every persist call against a release whose mod.json is larger, though the release still publishes.

Signed-off-by: Christian Stewart <christian@aperture.us>
A number stored by a count, a number or a max was taken by a field of any of
the three, so a build that made a max field a count could add to a best
score. Each stored record now keeps the kind of the field that wrote each
value, and an op is refused unless its field is of that same kind. A value
with no recorded kind is held by no field. This matches Hyperline's rules.

Signed-off-by: Christian Stewart <christian@aperture.us>
An update the host answers as taking too long may still have been kept, so
the guide and the update method's doc say to send it again with the same
key and not to act as if nothing was written.

Signed-off-by: Christian Stewart <christian@aperture.us>
Brings v0.1.80 onto the branch that carries the service call forwarding,
layout presses and service fixtures.

Both sides took field 4 of ControlRequest and field 13 of HostEvent.
v0.1.80 keeps them for ModCall and ModCallResult, and the forwarded
service answer and call move to 5 and 14. control_link.cc keeps both the
Answer and the Call case. The README moved into docs in v0.1.80, so the
layout press text goes to docs/content/guides/layouts.mdx and the services
text and manifest row to docs/content/reference/manifest.mdx. The
generated files, the Python library among them, were rebuilt with
scripts/gen-proto.sh rather than merged by hand; that also corrects an
index the textual merge left stale in manifest.pb.h.

Signed-off-by: Christian Stewart <christian@aperture.us>
The scriptc configuration mapped @aptre/protobuf-es-lite/* to <name>.ts in
the shipped runtime source, but the runtime exports each well-known type as
google/protobuf/<name> from a file named <name>.pb.ts. An import of
google/protobuf/timestamp therefore looked for timestamp.ts, found nothing,
and scriptc reported the package as unsupported, so any mod whose messages
carry a Timestamp, Struct or wrapper fell back to QuickJS. The longer
pattern now maps those to their .pb.ts sources. The new test resolves the
written paths as TypeScript does, for the well-known types and for message
and enum.

Signed-off-by: Christian Stewart <christian@aperture.us>
This merges c13e550 into persist. The persist section of the README
moves to the docs site, where modlock-next-r1 moved the rest. The
generated manifest code is regenerated with scripts/gen-proto.sh at this
tree, so it carries the persist field (16) next to the file modifiers and
text fields from modlock-next-r1.

Signed-off-by: Christian Stewart <christian@aperture.us>
Brings Modlock master 69d241c onto modlock-next-r1. The layouts guide
keeps master's account of how data reaches a layout and the branch's press
back, and the registry keeps both sides' maps. Field numbers do not
collide. The generated code is regenerated with scripts/gen-proto.sh;
only wasm.pb.cc differed from the merged files.

Signed-off-by: Christian Stewart <christian@aperture.us>
paralin and others added 11 commits November 14, 2026 10:35
This merges 5f19949, modlock-next-r1 with Modlock master merged, into
persist. The examples table takes master's rows with the prop-stats row
added. The generated code needed no change: scripts/gen-proto.sh at this
tree reproduces every generated file as merged.

Signed-off-by: Christian Stewart <christian@aperture.us>
The library imports modlock.heroes since the hero names change, so the
test mod that starts two later service calls failed its import at Start
and trapped. Bundle heroes.py with it as the Python mod test already does.

Signed-off-by: Christian Stewart <christian@aperture.us>
This brings in b367e04, whose one change over 5f19949 is a test
line in tests/wasm_instance_test.cc. No conflicts with persist.

Signed-off-by: Christian Stewart <christian@aperture.us>
Take master at ea5897c so modlock-next-r1 carries the owner guard fix
before it is named for the tag. The merge brings only the guard header,
its source and its test; nothing here touched those files.

Signed-off-by: Christian Stewart <christian@aperture.us>
Signed-off-by: Christian Stewart <christian@aperture.us>
Player.Slot became a method when a new person in a slot became a new
Player, but GetRecord and UpdateRecord still read it as a field, so the
Go library no longer compiled for wasip1. Both requests now take
p.Slot().

Signed-off-by: Christian Stewart <christian@aperture.us>
Signed-off-by: Christian Stewart <christian@aperture.us>
Master declared the part of Panorama's $ that layout.ts calls, and this
branch's press call added $.DispatchEvent, which the declaration lacked,
so oxlint and a vendored compile saw an error typed call.

Signed-off-by: Christian Stewart <christian@aperture.us>
Brings persist-r1 onto next-r1 c904216, which merged current master. The merge had no conflicts.

Signed-off-by: Christian Stewart <christian@aperture.us>
Master's squash of #126 has the tree of next-r1 c904216, which this branch already merged, so the merge keeps this branch's tree unchanged. The conflicts in the manifest, the sandbox's services and modlock play are that same change against the persist edits made on top of it.

Signed-off-by: Christian Stewart <christian@aperture.us>
Signed-off-by: Christian Stewart <christian@aperture.us>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant