Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
212 changes: 212 additions & 0 deletions docs/LIGHTWEIGHT-CONSTRAINTS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,212 @@
# Lightweight constraints an example author hits

The ladder's persistence layer is the [LASTRADA Lightweight](https://github.com/LASTRADA-Software/Lightweight)
ORM, pinned to one commit SHA by `examples/common/CMakeLists.txt` and, in
lockstep, `examples/bank/CMakeLists.txt`. Some of what that revision can and
cannot do is not obvious from its headers, and every rung that
rediscovered a limit wrote it down in **its own plan document** — so the next
rung rediscovered it again. This page is the single place those belong.

**Scope.** This is not morph's design (that lives in `docs/spec/`) and not a
plan (those live in `docs/superpowers/`). It is a record of *someone else's*
library's behaviour, kept here because example authors are the people who trip
over it.

**What every entry carries**, and what the plan documents did not:

- the constraint, stated as something you can act on;
- **the pinned revision it was verified at**, and how it was verified;
- the file and line in Lightweight that decides it;
- the workaround actually used in this tree;
- **what would retire it** — the condition under which the entry should be
deleted.

**An entry is only as good as its last verification.** When the pin moves,
re-run the probes; two of the three entries below were folklore that the pin
had already retired, and nobody noticed because nothing re-checked them.

Pinned revision at the time of writing:
`bbb972a78e1962b968a2c6ad93f7dade736eaa01`.

---

## 1. `Where()` cannot bind a binary value — **live**

**Constraint.** The fluent query builder's literal `Where()` overload cannot
take a `SqlBinary` or `SqlDynamicBinary<N>`. A comparison against a binary
column has to be expressed some other way.

**Verified at** `bbb972a78e1962b968a2c6ad93f7dade736eaa01`, by compiling it
(`g++ 16.2.1`, `-std=c++23 -fsyntax-only`, non-reflection build):

```cpp
Lightweight::SqlDynamicBinary<32> blob;
mapper.Query<Child>().Where(Lightweight::FieldNameOf<&Child::label>, "=", blob).All();
```

```
/usr/include/c++/16/bits/alloc_traits.h:716:28: error: no matching function for call to
'construct_at(Lightweight::SqlVariant*&, const Lightweight::SqlDynamicBinary<32>&)'
```

The identical call with `std::string{"x"}` in place of `blob` compiles, so the
refusal is about the value type and not about the call shape.

**Why.** `Where()`'s literal path stores the bound value in a
`std::vector<SqlVariant>` (`src/Lightweight/SqlQuery/Core.hpp:137`), and
`SqlVariant::InnerType` (`src/Lightweight/DataBinder/SqlVariant.hpp:49`) has
**zero** binary alternatives — the variant runs `SqlNullType`, `SqlGuid`,
`bool`, the integer and floating types, the string types, `SqlText`, `SqlDate`,
`SqlTime`, `SqlDateTime`, and stops. `SqlDataBinder<SqlDynamicBinary<N>>` exists
and works everywhere else; `Where()` simply does not go through it.

**Workaround used in this tree.** Store the value in a text column as
lower-case hex and compare against the hex string.
`examples/bank`'s offline queue does exactly that for
`OfflineQueueRecord::idempotencyKeyHex`, because enqueue-time dedup needs a
`WHERE` on that column. Note what it does *not* do: the adjacent `payload`
column stays a real `SqlDynamicBinary`, because nothing ever puts it in a
`Where()`. The constraint is on the *comparison*, not on binary storage, so hex
only where a `WHERE` reaches. Hex is NUL-free by construction, so the text
column carries an arbitrary byte string without truncating it or colliding two
keys that share a NUL prefix, and the conformance round-trip
(`tests/offline_queue_conformance.hpp`'s `checkNulPayloadRoundTrip`) passes
unedited.

**What retires this.** A Lightweight release that either adds a binary
alternative to `SqlVariant::InnerType` or routes `Where()`'s literal path
through `SqlDataBinder<>`. Tracked upstream as
[LASTRADA-Software/Lightweight#618](https://github.com/LASTRADA-Software/Lightweight/issues/618).
When it lands, the hex column can become a plain `SqlDynamicBinary<N>` again.

---

## 2. `Query<Record>()` has no `Update()` — **live, but not the constraint it was recorded as**

**Constraint.** The record-typed fluent builder
(`mapper.Query<Record>().Where(...)`) offers `All()`, `First()`, `Count()`,
`Delete()` and friends, but **no `Update()`**. To update, use
`DataMapper::Update(record)` on a fetched record, or drop to the untyped
builder on the connection.

**Verified at** the pin, by compiling it:

```cpp
mapper.Query<Parent>().Where(Lightweight::FieldNameOf<&Parent::id>, "=", 1).Update();
```

```
error: 'class Lightweight::SqlAllFieldsQueryBuilder<Parent, Lightweight::DataMapperOptions{true},
Lightweight::SqlQueryExecutionMode::Synchronous>' has no member named 'Update'
```

**This corrects what the plan archive says.** Three plan documents record this
as *"fluent `Query`/`Update` refuse `HasMany`-bearing records in the
non-reflection build"* (`docs/superpowers/plans/2026-08-16-kanban-backend.md`,
`docs/superpowers/specs/2026-08-16-kanban-rung4-design.md`,
`docs/superpowers/plans/2026-08-19-ledger-rung5.md`). At the pinned revision
that is **false in both halves**, and the control probes say so:

| probe | at the pin |
| --- | --- |
| `DataMapper::Update(parent)` where `Parent` has a `HasMany` member | **compiles** |
| `mapper.Query<Parent>().Where(...).All()` where `Parent` has a `HasMany` member | **compiles** |
| `mapper.Query<Parent>().Where(...).Update()` (`Parent` has `HasMany`) | fails |
| `mapper.Query<Child>().Where(...).Update()` (`Child` has **no** `HasMany`) | **fails identically** |

The last row is the one that matters: the refusal has nothing to do with
`HasMany`. `DataMapper::Update` skips relations explicitly — *"Relations
(HasMany, HasManyThrough, HasOneThrough, ...) have no column of their own"*,
`src/Lightweight/DataMapper/DataMapper.hpp:2047` and three sibling sites — so a
`HasMany` member costs an update nothing.

**Workaround.** `DataMapper::Update(record)`, which is what every ladder rung
already does.

**What retires this.** A Lightweight release that adds `Update()` to
`SqlAllFieldsQueryBuilder`. Re-run the probe when the pin moves.

---

## 3. `HasMany` resolves its foreign key by ordinal member index — **retired; do not believe this**

**This constraint is no longer true**, and it is recorded here only because
three plan documents still assert it and a fourth rung would otherwise design
around a limit that no longer exists.

**What the plans say.** That `HasMany` locates the child's foreign key by
matching **member position**, so reordering a record's members silently
repoints the relation.

**What the pinned revision does.** Matches by relationship **type**.
`src/Lightweight/DataMapper/Record.hpp:242-255` says so in its own words —
*"This is how `HasMany`, `HasManyThrough` and `HasOneThrough` locate their
foreign key column: by matching the relationship type, never by member
position"* — and `detail::InverseBelongsToResolver` (`Record.hpp:213-238`)
carries three `static_assert`s that turn every failure mode into a named
compile error: no `BelongsTo` at all, a `BelongsTo` whose column name does not
match, and an ambiguous pair of them.

**Verified at** the pin, with the child's `BelongsTo` declared **last**, which
positional resolution would get wrong:

```cpp
static_assert(Lightweight::InverseBelongsToIndexOf<Parent, Child> == 2);
static_assert(Lightweight::InverseBelongsToFieldNameOf<Parent, Child> == std::string_view{"parent"});
```

Both hold.

**What would re-open this.** A `HasMany` relation resolving to the wrong column
at a pin where the `static_assert`s above are absent. If that happens, this
entry becomes live again and the pin's `Record.hpp` is the first thing to read.

---

## 4. The migration DSL cannot express a partial index — **live**

**Constraint.** `SqlMigrationQueryBuilder`'s index API takes a name, a table
and a column list, and nothing else. There is no predicate, so
`CREATE UNIQUE INDEX … WHERE <expr>` — a partial index — has no spelling in the
DSL.

**Verified at** the pin, by reading the API surface. Every overload, in
`src/Lightweight/SqlQuery/Migrate.hpp:553-582`:

```cpp
CreateIndex(std::string indexName, std::string tableName, std::vector<std::string> columns, bool unique = false);
CreateUniqueIndex(std::string indexName, std::string tableName, std::vector<std::string> columns);
CreateIndex(std::string tableName, std::vector<std::string> columns, IndexType type = IndexType::NonUnique);
```

Also `SqlCreateTableQueryBuilder::Unique()` / `::UniqueIndex()`
(`Migrate.hpp:74`, `:80`) and `SqlAlterTableQueryBuilder::AddUniqueIndex(column)`
(`:207`) — none takes a predicate. `MigrationPlan.cpp` renders a trailing
`WHERE` only for UPDATE and DELETE plans (`:56-71`), never for an index.

**Why it bites.** Dedup on an optional key wants exactly a partial index:
`UNIQUE(key) WHERE key <> ''`. An *unconditional* unique index is not a
substitute — it makes the second empty-key row fail, which
`IOfflineQueue::enqueue` forbids.

**Workaround used in this tree.** `examples/bank`'s offline queue does the
check as a `SELECT` under its own mutex instead, and says so in
`LightweightOfflineQueue`'s class comment: as strong as `FileOfflineQueue`'s
linear scan (single process), deliberately weaker than
`SqliteOfflineQueue`'s real index (any writer). The other option, not taken
there, is `SqlMigrationQueryBuilder::Native(callback)` (`Migrate.hpp:594`),
which hands the plan a raw SQL string — an escape hatch that gives up the DSL's
cross-dialect rendering for that one statement.

**What retires this.** A `CreateIndex` overload taking a predicate expression,
or an equivalent in the plan renderer.

---

## Adding an entry

File the upstream issue first, so the entry has a retirement condition somebody
is tracking, then add a section here with the five bullets from the top of this
page. An entry with no verification revision is worth less than no entry: it
cannot be re-checked, and it will outlive the constraint it describes — which
is exactly how entries 2 and 3 above became folklore.
41 changes: 40 additions & 1 deletion docs/spec/offline/offline.md
Original file line number Diff line number Diff line change
Expand Up @@ -258,7 +258,7 @@ while offline; `SyncWorker` drains and replays them on reconnect.
| `enqueue` | `uint64_t enqueue(std::string payload, std::string idempotencyKey)` | Appends payload carrying the dedup key (stored on `QueueItem::idempotencyKey`). Virtual with a default that delegates to the one-arg `enqueue` then stamps the key via the protected `setIdempotencyKey`, so existing implementations keep working; the key is dropped by an implementation with no per-item storage that overrides neither. |
| `drain` | `std::vector<QueueItem> drain()` | Returns all pending items in enqueue order, without removing them. Safe to call multiple times — items survive between `drain()` and the corresponding `markDone()`. |
| `markDone` | `void markDone(uint64_t itemId)` | Removes the item identified by `itemId`. No-op if not found. |
| `setAttempts` | `void setAttempts(uint64_t itemId, uint32_t attempts)` | Persists an updated attempt count for an item. **Public** (unlike `setIdempotencyKey`) because `SyncWorker` calls it from outside the queue after every failed replay. Default no-op; `InMemoryOfflineQueue` overrides it to update the in-deque item. A queue that overrides it to store the count durably makes `SyncWorker`'s retry budget survive a process restart. |
| `setAttempts` | `void setAttempts(uint64_t itemId, Attempts attempts)` | Persists an updated attempt count for an item. **Public** (unlike `setIdempotencyKey`) because `SyncWorker` calls it from outside the queue after every failed replay. Default no-op; `InMemoryOfflineQueue` overrides it to update the in-deque item. A queue that overrides it to store the count durably makes `SyncWorker`'s retry budget survive a process restart. The count is an `Attempts`, not a bare `uint32_t` — see "`Attempts`: why the count has its own type" below. |
| `size` | `std::size_t size() const` | Number of pending items, without removing them. Default calls `drain().size()` — correct but O(n) and allocates a full snapshot to answer a size query; every shipped implementation overrides it with a direct count. |
| `maxDepth` | `std::optional<std::size_t> maxDepth() const` | The capacity `enqueue()` enforces, or `std::nullopt` if unbounded. Default: `std::nullopt` — preserves current behavior for any `IOfflineQueue` subclass written before this method existed. |
| `setIdempotencyKey` (protected) | `void setIdempotencyKey(uint64_t itemId, std::string key)` | Hook the default two-arg `enqueue` uses to stamp the key onto an already-enqueued item. Default no-op; `InMemoryOfflineQueue` records the key directly instead. **A conflicting non-empty key is skipped, never raised** — an implementation that deduplicates leaves the item unkeyed rather than failing, because the default `enqueue` has already inserted by the time it stamps, so the row exists either way and an exception could not undo it. That path therefore yields an extra *unkeyed* item, not a dedup hit; a caller wanting dedup uses the virtual two-arg `enqueue` (see below). |
Expand All @@ -267,6 +267,45 @@ while offline; `SyncWorker` drains and replays them on reconnect.
default can call it (and so can an application) without needing a non-`const`
reference to the queue.

### `Attempts`: why the count has its own type

`setAttempts(uint64_t itemId, uint32_t attempts)` was two adjacent, mutually
convertible unsigned integers, and `bugprone-easily-swappable-parameters` said
so at every implementation — which meant every implementor, in this tree and
out of it, hand-wrote a suppression for a hazard they did not choose. That is
the framework exporting its lint bill to its consumers, and the suppression
left the hazard in place.

The hazard is not theoretical, and it is **silent in both directions**:
`setAttempts()` on an unknown id is a documented no-op
(`tests/test_offline_queue.cpp`, `tests/test_file_offline_queue.cpp`), so a
transposed call writes an attempt count into an id nothing matches, returns
normally, and throws nothing. The real item's count never advances. The defect
surfaces much later, as a `SyncWorker` retry budget that never exhausts and a
poison payload that replays forever.

`morph::offline::Attempts` removes it rather than suppressing the warning
about it:

- **Implicit from a narrow integer**, so `setAttempts(id, 3)` and
`setAttempts(id, counter)` read exactly as before and no call site in the
tree changed.
- **Not constructible from a 64-bit integer**, which is what a `QueueItem::id`
is. `setAttempts(attempts, itemId)` therefore does not compile.
- A genuinely 64-bit count is still expressible, with the narrowing visible at
the call site: `setAttempts(id, static_cast<std::uint32_t>(count))`.

`QueueItem::attempts` stays a plain `uint32_t`: it is a struct field, reached
by name, with no adjacent same-typed field to transpose it with. The hazard was
in the *parameter list*, and that is where the type went.

Two `static_assert`s below `IOfflineQueue` hold the property in place, and they
are written as a pair on purpose: one asserts the transposed call is **not**
well-formed, the other that the ordinary call still is. Either alone would pass
against a degenerate definition — an alias for `uint32_t` fails the first, an
`explicit` constructor fails the second — so the pair is what makes the check
mean something.

**Enqueue order is the implementation's to keep; the id does not imply it.**
`drain()` requires enqueue order, and `QueueItem::id` does not supply it. All
three shipped implementations mint ids that increase with insertion — an
Expand Down
31 changes: 27 additions & 4 deletions docs/spec/security.md
Original file line number Diff line number Diff line change
Expand Up @@ -633,10 +633,33 @@ transport above is not a matter of degree:
configuration this project tests, the two spellings render an `errno` to
identical bytes. The property gained is that the guarantee now holds by
specification rather than by the implementation happening to be safe.
`TcpSocket::connect`'s `::gai_strerror` is deliberately untouched: it renders
`EAI_*` resolver codes, which are not `errno` values, so
`std::system_category()` cannot describe them and no drop-in substitution
exists.
`TcpSocket::connect`'s `::gai_strerror` is deliberately untouched, for two
separate reasons that morph#640 asked to be kept apart. The first is that no
substitution exists: it renders `EAI_*` resolver codes, which are not `errno`
values, so `std::system_category().message()` would describe them
confidently and wrongly. The second is that it does not have
`std::strerror`'s defect in the first place — **measured**, not assumed. On
glibc 2.44 it returns a pointer to a string literal in libc's own read-only
mapping, distinct per code and stable across calls:

```
EAI_AGAIN ptr=0x7fa71e9b62f2 "Temporary failure in name resolution"
EAI_FAIL ptr=0x7fa71e9b632e "Non-recoverable failure in name resolution"
dladdr -> /usr/lib/libc.so.6, anonymous (a literal in a data section)
7fa71e99f000-7fa71ea15000 r--p ... /usr/lib/libc.so.6
```

The `r--p` is the load-bearing part: a shared scratch buffer of the kind
`std::strerror` is permitted to return would have to be writable. Even an
unrecognised code yields a constant ("Unknown error") rather than a formatted
one, so there is no per-call buffer on any path. The musl implementation the
WASM build links is a `static const char msgs[]` table by inspection, with
the same property, and `TcpSocket` is POSIX-only, so Winsock's
documented-unsafe `gai_strerrorA` is never compiled. What is **not**
established: no other libc was checked, and POSIX itself is not read as
requiring this — so the entry is a measurement on the configurations morph
builds, not a portability guarantee. Re-check if a platform with a different
libc joins CI.

## Residual limitations & hardening checklist

Expand Down
8 changes: 8 additions & 0 deletions examples/IMPLEMENTATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,14 @@ could have provided is a defect in the stress test.** If the framework can't
provide it, that inability is a *finding* — record it per
[`FINDINGS.md`](FINDINGS.md), don't quietly code around it.

**Before you design around a Lightweight limitation, read
[`docs/LIGHTWEIGHT-CONSTRAINTS.md`](../docs/LIGHTWEIGHT-CONSTRAINTS.md)** — one
page, one entry per constraint, each with the pinned revision it was verified
at and what would retire it. It exists because three rungs in a row
rediscovered the same limits and recorded them in their own plan documents,
where the next rung could not find them and where two of them quietly went
stale. If you hit a new one, add it there rather than to your plan.

**The promotion rule (rule-of-three, from the round-7 review):** an
app-built answer to a framework gap (the polling helper with its timeout,
an op-id ledger, epoch tokens, a recursive validator, redaction-on-serve)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -170,7 +170,7 @@ class LightweightOfflineQueue final : public morph::offline::IOfflineQueue {
/// after the item was marked done does not fail.
/// @param itemId Id of the item whose attempt count changed.
/// @param attempts New cumulative attempt count to persist.
void setAttempts(std::uint64_t itemId, std::uint32_t attempts) override;
void setAttempts(std::uint64_t itemId, morph::offline::Attempts attempts) override;

private:
mutable std::mutex _mtx;
Expand Down
5 changes: 2 additions & 3 deletions examples/bank/src/offline/lightweight_offline_queue.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -172,14 +172,13 @@ std::size_t LightweightOfflineQueue::size() const {

std::optional<std::size_t> LightweightOfflineQueue::maxDepth() const { return _maxDepth; }

// NOLINTNEXTLINE(bugprone-easily-swappable-parameters) -- the signature is IOfflineQueue's, not ours
void LightweightOfflineQueue::setAttempts(std::uint64_t itemId, std::uint32_t attempts) {
void LightweightOfflineQueue::setAttempts(std::uint64_t itemId, morph::offline::Attempts attempts) {
const std::scoped_lock lock{_mtx};
auto record = _mapper.QuerySingle<OfflineQueueRecord>(itemId);
if (!record.has_value()) {
return;
}
record->attempts = attempts;
record->attempts = attempts.value();
_mapper.Update(*record);
}

Expand Down
Loading
Loading