Skip to content

Add one-prompt Hermes 0.21 Substrate onboarding #31

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Pavel-Tk merged 1 commit into from
Sep 3, 2026
Merged

Add one-prompt Hermes 0.21 Substrate onboarding #31

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
291 changes: 80 additions & 211 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,235 +1,104 @@
# Substrate Memory Plugins

This public repository contains the official Substrate memory plugins. The current plugin,
`substrate_wiki` 2.0.5, connects an existing Hermes 0.20.x agent to hosted Substrate at
`https://app.trysubstrate.co`.
This is the official public repository and canonical editable source for Substrate memory integrations. The current
Hermes plugin is [`plugins/substrate`](plugins/substrate) and supports Hermes 0.21.x.

The Substrate application and server are maintained in a separate private repository. This
repository is the canonical editable source for public plugin/client code, verified installers,
tests, documentation, and immutable release artifacts.
## Install with a Hermes agent


## Retrieval MVP plugin (`substrate`)

The repository also contains the thin Substrate v5 retrieval plugin in
[`plugins/substrate`](plugins/substrate). It uses the server-side candidate pool and
associative editor, injects bounded turn context, captures completed turns, and provides
`memory_search`, `memory_expand`, and `memory_evidence`.

Install the public plugin directly from this repository:

```sh
hermes --profile developer plugins install Substrate-memory/Substrate-memory-plugins/plugins/substrate --enable
```

For production, pin the command with `--ref` and a published 40-character commit SHA.
The plugin reads `SUBSTRATE_API_URL` and `SUBSTRATE_API_KEY` from the Hermes gateway
environment. See the plugin-local README for details.

## Install with an agent

Tell your Hermes agent exactly:
The user only needs to say:

```text
Install the memory plug-in at https://github.com/Substrate-memory/Substrate-memory-plugins
```

If you are the installing agent, follow the verified procedure below. Keep the installation
command running while the user approves the browser connection. Browser approval completes the
local device-code poll and returns control to the same conversation. Immediately say that the
connection succeeded, then ask:

> Do you want to import your existing Hermes conversation history into Substrate?

Do not select, infer, or pre-authorize the answer. Do not begin history discovery or upload until
the user explicitly answers yes. A no leaves future conversation capture enabled. A blank,
interrupted, or missing answer leaves consent pending.

Release `v2.0.5` is not published yet. Do not run the download commands below until this
notice is removed and exact SHA-256 values replace the pending markers.

Security requirements:

- Install only immutable release v2.0.5 from `Substrate-memory/Substrate-memory-plugins`.
- Verify the installer and plugin archive against the exact SHA-256 values below before executing
downloaded code.
- Require `hermes --version` to report 0.20.x. Do not install or upgrade Hermes automatically.
- Use the active `HERMES_HOME` profile. Never inspect or copy another profile.
- Do not configure `HERMES_API_URL` or `HERMES_API_KEY`.
- Never print, request, or place access tokens in command arguments or logs.
- Present `verification_uri_complete` exactly when manual browser opening is needed, then keep
polling. Never approve the browser connection for the user.
- When the installer returns `action_required: history_consent`, resume this same conversation
and ask the history question above.
- Report only content-free installation, connection, and import status.

### Bash: Linux, macOS, or WSL

```bash
set -euo pipefail

hermes_version="$(hermes --version)"
case "$hermes_version" in
*0.20.*) ;;
*) printf 'Hermes 0.20.x is required; found: %s\n' "$hermes_version" >&2; exit 1 ;;
esac

install_dir="$(mktemp -d)"
chmod 700 "$install_dir"
cd "$install_dir"

base='https://github.com/Substrate-memory/Substrate-memory-plugins/releases/download/v2.0.5'
curl --fail --location --proto '=https' --tlsv1.2 --remote-name "$base/install_hermes_plugin.py"
curl --fail --location --proto '=https' --tlsv1.2 --remote-name "$base/substrate_wiki.zip"
curl --fail --location --proto '=https' --tlsv1.2 --remote-name "$base/SHA256SUMS"

python3 - <<'PY'
import hashlib
from pathlib import Path

expected = {
"install_hermes_plugin.py": "INSTALLER_SHA256_PENDING",
"substrate_wiki.zip": "PLUGIN_SHA256_PENDING",
}
for name, digest in expected.items():
actual = hashlib.sha256(Path(name).read_bytes()).hexdigest()
if actual != digest:
raise SystemExit(f"{name}: checksum mismatch")
published = Path("SHA256SUMS").read_text(encoding="utf-8")
for name, digest in expected.items():
if f"{digest} {name}" not in published.splitlines():
raise SystemExit(f"{name}: published SHA256SUMS mismatch")
print("Release checksums verified")
PY

python3 install_hermes_plugin.py \
--archive substrate_wiki.zip \
--sha256 PLUGIN_SHA256_PENDING \
--yes --json
```

Add `--headless` to the final command on a machine that cannot open a browser.

### PowerShell: Windows

```powershell
$ErrorActionPreference = 'Stop'

$hermesVersion = (& hermes --version | Out-String).Trim()
if ($hermesVersion -notmatch '0\.20\.') {
throw "Hermes 0.20.x is required; found: $hermesVersion"
}

$installDir = Join-Path ([IO.Path]::GetTempPath()) ("substrate-wiki-" + [guid]::NewGuid())
New-Item -ItemType Directory -Path $installDir | Out-Null
Set-Location $installDir

$base = 'https://github.com/Substrate-memory/Substrate-memory-plugins/releases/download/v2.0.5'
Invoke-WebRequest "$base/install_hermes_plugin.py" -OutFile 'install_hermes_plugin.py'
Invoke-WebRequest "$base/substrate_wiki.zip" -OutFile 'substrate_wiki.zip'
Invoke-WebRequest "$base/SHA256SUMS" -OutFile 'SHA256SUMS'

$installerSha = (Get-FileHash -Algorithm SHA256 'install_hermes_plugin.py').Hash.ToLowerInvariant()
$archiveSha = (Get-FileHash -Algorithm SHA256 'substrate_wiki.zip').Hash.ToLowerInvariant()
if ($installerSha -ne 'INSTALLER_SHA256_PENDING') {
throw 'Installer checksum mismatch'
}
if ($archiveSha -ne 'PLUGIN_SHA256_PENDING') {
throw 'Plugin archive checksum mismatch'
}
$published = Get-Content 'SHA256SUMS' -Raw
if (($published -notmatch [regex]::Escape($installerSha)) -or
($published -notmatch [regex]::Escape($archiveSha))) {
throw 'Published SHA256SUMS does not match the independently pinned checksums'
}

py -3 install_hermes_plugin.py `
--archive substrate_wiki.zip `
--sha256 PLUGIN_SHA256_PENDING `
--yes --json
```

Use `python` instead of `py -3` if that is the installed Python 3 launcher. Add `--headless` to the final command on a machine that cannot open a browser.

### Finish onboarding

The installer validates archive provenance, target-Hermes metadata, and packaged source digests; installs beneath `$HERMES_HOME/plugins/substrate_wiki`; activates `memory.provider: substrate_wiki`; and starts hosted device onboarding.

The installing agent must present `verification_uri_complete` exactly as the clickable authentication URL. Never present `verification_uri` alone: that bare URL requires manual code entry. The complete URL opens the passwordless email sign-in with the one-time Hermes code already attached.

After browser approval, answer the history prompt yourself:

- **Approve**: upload eligible direct conversations and explicit saved memories using durable checkpoints.
- **Decline**: do not upload past history; automatic future capture remains enabled.

If a non-interactive agent cannot present the consent prompt, have it ask you first and then run exactly one of:

```bash
hermes substrate_wiki onboard --mode device --wait --history approve --json
hermes substrate_wiki onboard --mode device --wait --history decline --json
```

Then verify the content-free state:

```bash
hermes substrate_wiki onboarding-status --json
```

Tenant credentials are stored in native credential custody, with an owner-private profile fallback; they never belong in ordinary configuration, logs, arguments, or diagnostics. See [configuration and operation](docs/operation.md) and the immutable [v2.0.5 release](https://github.com/Substrate-memory/Substrate-memory-plugins/releases/tag/v2.0.5).

## What it does

- Implements Hermes provider lifecycle hooks and tools.
- Prefetches bounded cited `memory_card` results for automatic recall.
- Captures only completed user/assistant dialogue turns.
- Redacts credential-shaped values before durable local spooling or network transfer.
- Replays Hermes history with deterministic event IDs and durable checkpoints.
- Keeps spool/checkpoint state beneath the active `$HERMES_HOME/substrate_wiki` profile.

It does **not** expose arbitrary filesystem writes, include Substrate server code, bundle credentials, or make private history public.
### Required behavior for the installing agent

Treat the sentence above as approval to install and configure the current Substrate plugin
for the active Hermes profile. Do not ask the user for repository paths, commands, commit
hashes, API keys, or TLS workarounds.

1. Confirm `hermes --version` is 0.21.x. Do not upgrade Hermes automatically.
2. Install the plugin subdirectory, not the repository root:

```sh
substrate_ref="$(git ls-remote https://github.com/Substrate-memory/Substrate-memory-plugins.git refs/heads/main | awk '{print $1}')"
printf '%s\n' "$substrate_ref" | grep -Eq '^[0-9a-f]{40}$'
hermes plugins install \
Substrate-memory/Substrate-memory-plugins/plugins/substrate \
--ref "$substrate_ref" --no-enable
```

3. Run `hermes plugins doctor substrate --ci`. Stop without changing the legacy provider
if installation or doctor fails.
4. Enable the plugin and let device onboarding run on first use. When a
`verification_uri_complete` link appears (from `memory_search` or the first turn),
present that exact clickable URL to the user. The user approves in their browser;
the agent must never approve it for them or ask for a pasted key.
5. The plugin stores the tenant-scoped key privately in the active profile's `.env` and
retries automatically. No manual credential migration is required.
6. Only after a memory call succeeds, disable `substrate_wiki`, clear an old
`memory.provider: substrate_wiki` selection with `hermes memory off`, and restart the
gateway. This prevents duplicate capture while preserving built-in local memory.
7. The optional `setup.py` can pre-connect during installation, but it is not required.
8. Verify the new `memory_search` tool in a new turn. `wiki_search` is legacy and is not a
valid verification of the current plugin.
9. Report success only after `memory_search` reaches the authenticated Substrate API.
Fail closed and report the exact safe error class otherwise.

TLS verification must remain enabled. The plugin supplements the host trust store with
unmodified public ISRG Root X1 and X2 certificates for current Let's Encrypt chains. It
never trusts a replacement leaf certificate or a private certificate.

## Current plugin

The `substrate` plugin is a thin, standard-library-only adapter. It provides:

- bounded next-turn memory context through `pre_llm_call`;
- nonblocking completed-turn capture through `post_llm_call`;
- `memory_search`, `memory_expand`, and `memory_evidence`;
- active-profile credential migration and passwordless device authorization;
- additive public-root TLS trust without disabling certificate or hostname checks.

The Substrate server remains the source of truth for storage, ranking, the associative
editor, and evidence.

## Legacy compatibility

`src/substrate_wiki` and its release tooling are retained for existing installations and
migration tests. It is not the current install target. Do not run
`scripts/install_hermes_plugin.py` for the prompt above. Legacy packaging state remains
unchanged: `v2.0.5` is not published yet, its historical URL would contain
`releases/download/v2.0.5`, and its unpublished checksum marker remains
`PLUGIN_SHA256_PENDING`. These strings document legacy release truth; they are not install
instructions. Immutable historical releases remain under `legacy-assets` and Git tags.

## Privacy boundary

Visible prompts and assistant output may be sent to the configured Substrate server after redaction. Tool calls, tool results, system messages, memory-write events, session metadata, and provider scope are not uploaded. Redaction is defense in depth, not proof arbitrary sensitive prose is absent.

Local failed deliveries remain in a bounded owner-private spool until delivered or explicitly removed. Status, progress, and receipts are content-free.

Read [SECURITY.md](SECURITY.md), [the threat model](docs/threat-model.md), [the source-ownership boundary](docs/source-of-truth.md), and the permanent [open/held commercial boundary](BOUNDARY.md) before deployment.

## Open and paid boundary

The open side is permissively licensed and includes the Hermes plugin/client, memory extraction and entity model, credential containment, privacy deletion, and policy compiler. This Hermes integration itself is hosted-only. [BOUNDARY.md](BOUNDARY.md) distinguishes the permanent commitment from the current `v2.0.5` implementation.
Visible prompts and assistant output may be sent to the configured Substrate server after
redaction. The current plugin does not upload tool results or system messages. It caches no
retrieved memory locally and returns empty context on any retrieval failure.

The paid hosted tier covers hosted brokerage, multi-user operation, cross-organizational graph services, audit/attestation, and insurance-backed decisions. Its meter is per authorized action, never seats. This repository does not contain or license those held services.
Read [SECURITY.md](SECURITY.md), [the threat model](docs/threat-model.md),
[the source-ownership boundary](docs/source-of-truth.md), and [BOUNDARY.md](BOUNDARY.md)
before deployment.

## Development

```bash
python3 -m venv .venv
. .venv/bin/activate
python -m pip install -e '.[dev]'
pytest -q
python -m compileall -q src scripts
python scripts/verify_public_plugin_candidate.py --root . --layout destination
uv sync --frozen --extra dev
uv run --frozen --extra dev ruff check .
uv run --frozen --extra dev python -m pytest -q
python3 scripts/verify_public_plugin_candidate.py --root . --layout destination
```

Builds are deterministic and require an immutable source commit for releases:

```bash
HERMES_PLUGIN_SOURCE_COMMIT="$(git rev-parse HEAD)" python scripts/build_plugin.py
python scripts/build_plugin.py --check
```

Version `1.4.1` remains an imported immutable release with its original Substrate-v2 provenance. Repository-native releases begin at `2.0.0` and use standalone provenance.

## Repository map

- `src/substrate_wiki/` — canonical plugin source.
- `scripts/` — deterministic builder, verified installer, publication scanner, import benchmark.
- `tests/` — provider, privacy, replay, packaging, and resource-boundary tests.
- `legacy-assets/` — immutable releases imported from Substrate-v2.
- `docs/` — architecture, compatibility, ownership, release, and threat-model contracts.
- `BOUNDARY.md` — permanent open-source/commercial boundary and current implementation gaps.
- `plugins/substrate/` — current Hermes 0.21.x retrieval plugin and setup flow.
- `src/substrate_wiki/` — legacy provider retained for migration compatibility.
- `scripts/` — deterministic legacy builder, installer, and publication scanner.
- `tests/` — provider, privacy, replay, packaging, retrieval, and setup tests.
- `legacy-assets/` — immutable historical releases.
- `docs/` — architecture, compatibility, ownership, and threat-model contracts.

## License

Expand Down
Loading
Loading