Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,10 +94,12 @@ All of the server's input and output artifacts live in one directory, the *io di
| `ledgers/ledger_<seq>.json` | persistent | the server (on success) | one record per closed ledger — `{sequence, txHash, closedAt, hash, headerXdr, metadataXdr}` — the ledger→transaction index behind `getTransactions` and `getLedgers`. Written in Python because the header artifacts are XDR, which K cannot construct. |
| `requests/request_<n>.json` | persistent | the server | an archive of each incoming JSON-RPC request, numbered by a monotonic counter, kept for debugging. |
| `wasms/<hash>.wasm` | persistent | the server | the raw bytes of each successfully uploaded wasm module, keyed by hex sha256. The K state stores modules parsed (`ModuleDecl`), so `getLedgerEntries` CONTRACT_CODE entries read the original bytes from here. |
| `events/events_<ledger>.json` | persistent | the server | one JSON array per ledger with the contract events emitted in it, already in the spec's Event shape. Built by the server from the staged records after a successful transaction; read back by the K `getEvents` rules. |
| `request.json` | transient | the server | the request envelope for the call in flight (`method`, `id`, `now`, and method-specific fields). The semantics remove it once they respond. |
| `response.json` | transient | the semantics | the JSON-RPC response (`{jsonrpc, id, result}`) for the most recent call. The server reads it back; it is absent when a transaction gets stuck. |
| `events_staged.jsonl` | transient | the K semantics | the contract events of the transaction in flight, one JSON record per `contract_event` call. The server converts them into `events/events_<ledger>.json` after a successful run and removes the file. |

Receipts, traces, ledger records, and request archives are split into one file per item — keyed by tx hash, ledger sequence, or a counter — so that no single file grows without bound as the chain advances. The server creates the `receipts/`, `traces/`, `ledgers/`, and `requests/` directories before the semantics run, because the K file-system hooks open files with POSIX `open()`, which does not create parent directories.
Receipts, traces, ledger records, events, and request archives are split into one file per item — keyed by tx hash, ledger sequence, or a counter — so that no single file grows without bound as the chain advances. The server creates the `receipts/`, `traces/`, `ledgers/`, `requests/`, `wasms/`, and `events/` directories before the semantics run, because the K file-system hooks open files with POSIX `open()`, which does not create parent directories.

The world state stays in KORE (rather than a JSON snapshot) because an uploaded wasm module is a `ModuleDecl` that the semantics cannot reconstruct from bytes — only `wasm2kast` (Python) can produce it. The receipts and the ledger counter, by contrast, are plain data and live in JSON files, which the semantics read and write directly via the file-system hooks.

Expand Down Expand Up @@ -178,6 +180,6 @@ sequenceDiagram
## What's not yet implemented

- `simulateTransaction` of upload/deploy host functions, auth recording, resource metering, `events`/`restorePreamble`/`stateChanges` (only invoke-contract dry runs with the return value are supported)
- `getEvents` and other read-only RPC methods (`getLedgerEntries` is implemented for the entry types the K state tracks: `ACCOUNT`, `CONTRACT_DATA`, `CONTRACT_CODE`)
- `system` events in `getEvents` (the semantics have no source of them; contract events are supported)
- `ExtendFootprintTTL` and `RestoreFootprint` operations
- `SCVec` / `SCMap` contract-argument types in the request encoder (`scval_to_json`)
5 changes: 4 additions & 1 deletion docs/node-semantics.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,8 @@ The semantics communicate with the Python process through files in the working d
| `receipts/receipt_<hash>.json` | K → Python | one stored receipt per transaction, keyed by tx hash |
| `traces/trace_<hash>.jsonl` | K → Python | one execution trace per transaction (per-instruction records), keyed by tx hash |
| `ledgers/ledger_<seq>.json` | Python → K | one record per closed ledger (`sequence`, `txHash`, `closedAt`, and the ledger-header XDR artifacts), written by the server and read back to serve `getTransactions`/`getLedgers` |
| `events_staged.jsonl` | K → Python | the contract events of the transaction in flight, one JSON record per `contract_event` call |
| `events/events_<ledger>.json` | Python → K | one finished JSON array of Event objects per ledger, served by `getEvents` |

---

Expand All @@ -41,7 +43,7 @@ insert-handleRequestFile → handleRequestFile
#dispatchMethod(method, request) ← routes on the "method" field
├─ getHealth / getNetwork / getLatestLedger / getVersionInfo / getFeeStats
│ / getTransaction / getTransactions / getLedgers / getLedgerEntries / traceTransaction → #respond(...)
│ / getTransaction / getTransactions / getLedgers / getLedgerEntries / getEvents / traceTransaction → #respond(...)
└─ sendTransaction → #runTx → run steps
→ #finalizeTx → record receipt + bump ledger → #respond(...)
Expand All @@ -68,6 +70,7 @@ If `request.json` is absent, `insert-handleRequestFile` does not fire and K halt
- `getTransaction` → reads the hash's `receipts/receipt_<hash>.json` file; returns the stored receipt merged with the ledger-range fields (`latestLedger`/`latestLedgerCloseTime`/`oldestLedger`/`oldestLedgerCloseTime`), or `{ "status": "NOT_FOUND", ... }` (with the same ledger-range fields) when the file is absent
- `getTransactions` / `getLedgers` → walk the per-ledger index files `ledgers/ledger_<seq>.json` from the envelope's `startSeq` up to the latest ledger, taking at most `limit` records (`#txInfos` / `#ledgerInfos`), and format the history page (`#txHistoryPage` / `#ledgerHistoryPage`). Parameter validation and cursor resolution happen in the server; the response `cursor` (`#pageCursor`) names the page's last record when the page is full — a TOID for transactions, the plain sequence for ledgers — and is empty otherwise. Serialization matches real stellar-rpc, including its quirks: per-transaction `createdAt` is a JSON number (unlike singular `getTransaction`), per-ledger `ledgerCloseTime` is a decimal string, and empty `resultXdr`/`resultMetaXdr` stubs are omitted (`#optXdrEntry`)
- `getLedgerEntries` → looks each *key descriptor* of the request up in the world-state cells (`<accounts>`, `<contracts>`, `<contractData>`, `<contractCodes>`) and responds with `{ "entries": [...], "latestLedger": <int> }`. The server decodes the base64 `LedgerKey` XDR into the descriptors beforehand and re-encodes the found entries as `LedgerEntryData` XDR afterwards (`ledger_entries.py`) — the entries in K's response are an intermediate JSON shape (per-kind payloads such as `balance`, `wasmHash`, or an ScVal value serialised by `#scVal2JSON`, the inverse of `#decodeArg`). Keys that match nothing are skipped via an `[owise]` rule — per the spec they are not an error
- `getEvents` → scans the `events/events_<ledger>.json` files of the requested window, applies the request's filters (type, contract ids, topic matchers with `*`/`**`), and paginates; returns `{ "latestLedger": ..., "events": [...], "cursor": ... }`. The events themselves were captured during `sendTransaction`: a rule in `node.md` shadows the upstream no-op `contract_event` host function, resolves the topics/data host objects, and stages them in `events_staged.jsonl`; the Python server then produces the finished per-ledger files (base64 SCVal XDR and strkey ids are XDR work K cannot do). A `startLedger` beyond the chain tip is answered with `#respondError` (JSON-RPC `-32600`)

`#respond(ID, RESULT)` is the shared terminal: it writes the JSON-RPC envelope to `response.json`, removes `request.json`, and sets the exit code to 0. `#respondError(ID, CODE, MSG)` is its error counterpart, writing `{jsonrpc, id, error: {code, message}}` instead; the unknown-method fallback uses it to answer `-32601 Method not found` (a safety net — the Python server filters unknown methods before they reach K).

Expand Down
6 changes: 4 additions & 2 deletions docs/notes.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ State lives in the io dir as `state.kore` (KORE world state) and `metadata.json`
## Tests (`src/tests/integration/`)

- `test_server.py` drives the running HTTP server end-to-end. It exercises the read-only methods, `sendTransaction` + `getTransaction`, the history methods (`getTransactions`/`getLedgers` response shapes, pagination, and parameter validation against the official spec and the Go protocol structs), ledger increments, the full lifecycle (create → upload wasm → deploy → invoke), the `traceTransaction` flows, and `getLedgerEntries` (account, contract code, contract instance, and persistent storage entries via `storage.wat`, plus its parameter validation). `test_call_tx_with_args` deploys `args.wat` and calls functions with `bool`, `u32`, `i32`, `u64`, `i64`, `u128`, `i128`, and `symbol` arguments, exercising the `scval_to_json` / `#decodeArg` pipeline.
- `test_get_events.py` covers `getEvents`: request validation, the empty-window shape, the full shape of an event emitted via `contract_event` (`data/wasm/events.wat`), filtering, and pagination.
- `test_integration.py` and `test_unit.py` hold small sanity checks.

Run with `make test` (requires `make kdist-build` first).
Expand All @@ -39,9 +40,10 @@ The tests do not yet cover `bytes` / `address` SCVal arguments or `SCVec` / `SCM
(komet-node does not track fees, entry changes, or events).
- `sendTransaction`'s `errorResultXdr` is always a generic `txMALFORMED` result (no per-cause codes such as `txBAD_SEQ` — sequence numbers, fees, and signatures are not modelled), and `diagnosticEventsXdr` is never populated (both fields are optional in the spec). `TRY_AGAIN_LATER` is never returned by design: it signals mempool backpressure, which cannot arise in a synchronous node without a mempool.
- `SCVec` / `SCMap` contract arguments are not yet encoded.
- The `xdrFormat` parameter is accepted on `getTransaction`, `sendTransaction`, `getTransactions`, `getLedgers`, and `getLedgerEntries`, but only the default `'base64'` is supported; `'json'` is rejected with `-32602`.
- The `xdrFormat` parameter is accepted on `getTransaction`, `sendTransaction`, `getTransactions`, `getLedgers`, `getLedgerEntries`, and `getEvents`, but only the default `'base64'` is supported; `'json'` is rejected with `-32602`.
- `simulateTransaction` only simulates invoke-contract host functions (not uploads/deploys); `auth` is always empty, `minResourceFee`/`transactionData` are synthetic constants, and `events`/`restorePreamble`/`stateChanges` and the `resourceConfig`/`authMode`/`xdrFormat` parameters are not supported. `ScMap` return values are not yet encoded.
- `getEvents` and TTL/footprint operations are not implemented.
- `getEvents` serves contract events only: `system` events are never emitted (the semantics have no source of them), and events whose topics/data have no staging representation (maps, errors, 256-bit ints) are dropped with a warning.
- TTL/footprint operations are not implemented.
- `getFeeStats` reports constant distributions (there is no fee market); only its `latestLedger` is live.
- `getTransactions` / `getLedgers` serve only ledgers with an index file under `ledgers/`; io-dirs created before the ledger index existed resume fine, but their earlier ledgers do not appear in the history.
- `getLedgerEntries` covers the entry types the K state tracks (`ACCOUNT`, `CONTRACT_DATA`, `CONTRACT_CODE`); other key types are reported as not found. `lastModifiedLedgerSeq` is approximated by the current ledger (per-entry modification ledgers are not tracked), and only the balance is real in `ACCOUNT` entries (sequence number, thresholds, etc. are synthesised constants).
Expand Down
29 changes: 27 additions & 2 deletions docs/server.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ class StellarRpcServer:
traces_dir: Path # io_dir / 'traces' — trace_<hash>.jsonl per transaction
ledgers_dir: Path # io_dir / 'ledgers' — ledger_<seq>.json per closed ledger
requests_dir: Path # io_dir / 'requests' — request_<n>.json archive
events_dir: Path # io_dir / 'events' — events_<ledger>.json per ledger
```

The server is a plain `http.server.HTTPServer` (not pyk's `JsonRpcServer`). A `BaseHTTPRequestHandler` reads each POST body and calls `_handle`, which parses the JSON-RPC frame and delegates to `handle_rpc`.
Expand Down Expand Up @@ -43,7 +44,7 @@ server = StellarRpcServer(io_dir=Path('out'))
server.handle_rpc('sendTransaction', {'transaction': xdr})
```

For `sendTransaction` it builds the request envelope with `encoder.build_tx_request` and runs it with `interpreter.run` (skipping the `<program>` wasm injection when the hash already has a receipt, so a duplicate is not re-executed — the semantics answer `DUPLICATE`); for `simulateTransaction` it builds the envelope with `encoder.build_simulate_request` and runs it without committing; for the read-only methods (`getHealth`, `getNetwork`, `getLatestLedger`, `getTransaction`, `getTransactions`, `getLedgers`, `getLedgerEntries`, `traceTransaction`) it builds a small envelope and runs it. For the history methods (`getTransactions`, `getLedgers`) the server also validates the pagination parameters — limit range, `startLedger` bounds, `cursor`/`startLedger` exclusivity — and resolves the cursor to the first ledger sequence to serve, because the JSON-RPC parameter-error path lives here. In every case the *content* of the response is produced by the semantics (`node.md`), not by Python — the exceptions are the failure fallback (below), the per-ledger XDR artifacts (`_record_closed_ledger`), and the XDR fields of the `simulateTransaction` response, which K cannot construct or serialize. Each call is logged to stderr.
For `sendTransaction` it builds the request envelope with `encoder.build_tx_request` and runs it with `interpreter.run` (skipping the `<program>` wasm injection when the hash already has a receipt, so a duplicate is not re-executed — the semantics answer `DUPLICATE`); for `simulateTransaction` it builds the envelope with `encoder.build_simulate_request` and runs it without committing; for the read-only methods (`getHealth`, `getNetwork`, `getLatestLedger`, `getTransaction`, `getTransactions`, `getLedgers`, `getLedgerEntries`, `getEvents`, `traceTransaction`) it builds a small envelope and runs it. For the history methods (`getTransactions`, `getLedgers`) and `getEvents` the server also validates the request parameters — pagination limits, ledger-window bounds, cursor format and exclusivity, filter/topic counts — because the JSON-RPC parameter-error path lives here. In every case the *content* of the response is produced by the semantics (`node.md`), not by Python — the exceptions are the failure fallback (below), the per-ledger XDR artifacts (`_record_closed_ledger`), the finished per-ledger event files (`_finalize_events`), and the XDR fields of the `simulateTransaction` response, which K cannot construct or serialize. Each call is logged to stderr.

---

Expand All @@ -66,7 +67,7 @@ At construction the server prepares the *io dir*, where `state.kore` lives at `i
- **`state.kore` absent** — `interpreter.empty_config()` produces the initial idle K configuration (a blank-slate state with no accounts, contracts, or storage) and writes it; `metadata.json` is seeded with `{"latest_ledger": 0}`.
- **`state.kore` present** — it is used as-is, and `metadata.json` is seeded only if missing. This lets you resume a previous session (ledger counter and stored receipts included) or start against a pre-built state.

In both cases the server creates the `receipts/`, `traces/`, `ledgers/`, and `requests/` directories if they do not already exist, because the K file-system hooks write into them but cannot create them.
In both cases the server creates the `receipts/`, `traces/`, `ledgers/`, `requests/`, `wasms/`, and `events/` directories if they do not already exist, because the K file-system hooks write into them but cannot create them.

Once the socket is bound, `serve` logs three lines to stderr: whether it is starting from a fresh state (an empty io-dir) or resuming an existing one (with the latest ledger), the io-dir path, and the listening address. Instruction tracing is always on, so every transaction the semantics run produces a trace. (Tracing only produces records for contract invocations.)

Expand Down Expand Up @@ -268,6 +269,30 @@ Per-ledger `ledgerCloseTime` is a *string* holding a decimal number (matching re

This is the one method whose response the server post-processes: the semantics look the keys up in the K state and answer with intermediate JSON entries, and `ledger_entries.py` re-encodes them as `LedgerEntryData` XDR (K cannot produce XDR). Because the K state keeps uploaded wasm parsed, the server stores the raw bytes under `wasms/<hash>.wasm` at upload time and reattaches them to `CONTRACT_CODE` entries. The semantics do not track per-entry modification ledgers, so `lastModifiedLedgerSeq` reports the current ledger; `ACCOUNT` entries carry the real balance but synthesised constants for the remaining required fields (sequence number 0, master weight 1, no signers).

### `getEvents`

`getEvents` returns the contract events emitted in a ledger window — `startLedger` (inclusive) to `endLedger` (exclusive) — optionally narrowed by up to five filters (`type`, `contractIds`, `topics` with `*`/`**` wildcards) and paginated with `pagination.cursor` / `pagination.limit` (default 100). A cursor replaces `startLedger`/`endLedger` and resumes strictly after the last returned event.

Events are captured during `sendTransaction`: the semantics intercept the `contract_event` host function and stage each event's topics and data, and after a successful run the server converts the staged records into one finished `events/events_<ledger>.json` per ledger (`_finalize_events` — the base64 SCVal XDR, strkey, and TOID encodings are XDR work K cannot do). The K `getEvents` rules scan, filter, and paginate those files. Parameter validation lives in `_get_events_envelope`; `xdrFormat: "json"` is not supported and is rejected with `-32602`.

**Response**:
```json
{
"latestLedger": 4,
"events": [
{
"type": "contract", "ledger": 4, "ledgerClosedAt": "2024-05-18T04:00:00Z",
"contractId": "C...", "id": "0000000017179873280-0000000000",
"inSuccessfulContractCall": true, "txHash": "<64-char hex>",
"topic": ["<base64 SCVal XDR>"], "value": "<base64 SCVal XDR>"
}
],
"cursor": "0000000021474836480-0000000000"
}
```

`system` events are never emitted (the semantics have no source of them), and events carrying values with no staging representation (maps, errors, 256-bit ints) are dropped with a warning rather than served with fabricated XDR.

---

## Failure fallback
Expand Down
Loading
Loading