Pre-implementation commit validation docs alignment

[owleyeview]

Align the commit-validation design and implementation documents before implementation begins. Resolve contradictions around the validation contract, finding provenance, StagedHolon state, CommitResponse rejection reporting, rule metadata and inventory, Capability 1 bootstrap sequencing, constraint coverage, and the prerequisites for transaction-wide relationship validation. Also reconcile kernel/evolution policies such as DS-SCHEMA-003 and DS-ENUM-003 with the runtime rule catalog.

This alignment is needed so implementation issues are grounded in one coherent architecture: semantic violations are serializable data rather than operational errors, acceptance is derived from a complete validation report, rule execution fails closed with stable provenance, and each capability has explicit dependencies and exit criteria. The resulting documents should agree with the canonical TDL and current map-holons architecture, reducing the risk of embedding contradictory contracts or obsolete rule classifications in code.

[owleyeview]

Questions for @evomimic

• Do validation rules really need both DefaultSeverity and MinimumBlockingBehavior? All rules currently carry the same values, so these fields contain zero information.

• Does default-value materialization belong only in the loader, where it is currently implemented? It seems like defaults should also be applied in other runtime holon-creation paths, and there may be a better shared location for this behavior.

• Type-descriptor and schema validation depend on inverse relationships being available. Because inverses can be guaranteed when both relationship endpoints are in the same Space, can we make it an explicit requirement that each Schema and its components be stewarded within a single Space?

• Is Commit intended to become the literal sole persistence gate? If so, should the direct production persistence, SmartLink, and delete externs be internalized or retired? Is LocalHolonSpace creation the only explicit bootstrap exception?

• Is the stale-read/concurrent relationship-mutation protection described in §12.4 of commit-validation-design-spec.md actually feasible in Holochain’s eventually consistent environment? Do you have an implementation strategy in mind, or should this be recorded as a known gap?

Additional observation: Validating schema-level rules means that whenever a type changes, its entire ComponentOf Schema needs to be validated. I think this makes sense, but it strengthens the argument for versioned Schema “modules” that are pushed out as a unit, so an update cannot expose a partially formed Schema that breaks validation.

[evomimic]

Questions for @evomimic

• Do validation rules really need both DefaultSeverity and MinimumBlockingBehavior? All rules currently carry the same values, so these fields contain zero information.

They are conceptually distinct—diagnostic presentation vs. persistence policy—but today they encode no variation. Commit rules are already defined as blocking, and configured constraints are unconditionally blocking. Keep severity on the emitted validation result, default it to error for now, and introduce rule/profile policy metadata only when a real non-blocking use case exists.

• Does default-value materialization belong only in the loader, where it is currently implemented? It seems like defaults should also be applied in other runtime holon-creation paths, and there may be a better shared location for this behavior.

Default population should be a Shared Objects construction concern:

• Nursery / TransientHolonManager attempts materialization when a holon is first created or cloned.
• It resolves the governing DescribedBy descriptor when available, obtains effective defaults, and writes explicit values into staged state.
• If the descriptor or DescribedBy relationship is not yet available, it returns a non-fatal deferred outcome, such as DefaultsDeferredNoDescriptor; it must not invent defaults or block bootstrap construction.
• Holon Loading resolves references first, then performs the required completion pass over deferred staged holons. That is the bootstrap backstop.
• Commit requires completed explicit state and validates it; it never supplies defaults itself.
  For non-loader construction paths, the same Shared Objects completion operation should run after descriptor resolution and before Commit. Otherwise “Loader as backstop” becomes a hole for programmatic creation.
  One important clone rule: cloning must not retroactively apply defaults to already persisted historical state. It may materialize only values omitted in the newly created staged clone, under its governing descriptor at clone creation time.

• Type-descriptor and schema validation depend on inverse relationships being available. Because inverses can be guaranteed when both relationship endpoints are in the same Space, can we make it an explicit requirement that each Schema and its components be stewarded within a single Space?

Uh "hard no" on that. Otherwise the entire schema would forever be confined to a single space. Section 12.5 handles physical inverse realization; it does not make an incomplete or unavailable semantic dependency pass validation.

• Is Commit intended to become the literal sole persistence gate? If so, should the direct production persistence, SmartLink, and delete externs be internalized or retired? Is LocalHolonSpace creation the only explicit bootstrap exception?

All public production create/update/relationship/delete paths should route through Commit. Direct persistence, SmartLink insertion, and SmartLink deletion operations remain internal storage capabilities that accept only a prepared Commit plan. LocalHolonSpace bootstrap should be the one narrowly documented exception; migration/recovery paths should still use Commit or have an explicitly reviewed exceptional contract.

The docs already promise that every persistence path reaches generalized Commit orchestration, but the implementation plan needs an explicit extern/API inventory and retirement/internalization task. Commit plan (line 170)

My understanding is that PR 623 closed the unintended public test-probe surface and added an audited production coordinator export manifest. Are you referring to a specific remaining production hdk_extern—for example create_path_to_holon_node, holon_storage_persist, or a SmartLink/delete extern—that can bypass Commit?
Please point to the function(s) and call path you believe remain exposed. If there is no concrete bypass beyond the legacy ingress already tracked in #622, we should avoid reopening the validation design’s persistence boundary.

• Is the stale-read/concurrent relationship-mutation protection described in §12.4 of commit-validation-design-spec.md actually feasible in Holochain’s eventually consistent environment? Do you have an implementation strategy in mind, or should this be recorded as a known gap?

Actually, this is just a specific example of a broader cross-cell conflict resolution issue. I've given thought to the solution, but don't want to put its resolution on the critical path for initial Validation track implementation.

You are right that §12.4, as written, names a stale-read protection requirement without yet selecting the relationship-authority model that would make it true across independently executing Holochain cells.

The important distinction is between Commit-local consistency and cross-cell conflict resolution.

For the current validation track we are only promising the former. Commit will validate a complete staged Nursery and the relationship operations for which its execution context has authoritative local write scope. This includes:

• descriptor and occurrence conformance;
• endpoint compatibility;
• constraint configuration validity;
• declared relationship planning;
• inverse derivation and paired persistence when both directional writes are local to the same Commit authority; and
• retry on local source-chain concurrency failure.

It will not claim that a DHT observation is a serializable, cross-cell snapshot of a shared relationship bucket.

That means strict aggregate rules whose truth depends on several independently writable cells—such as shared-bucket cardinality, duplicate exclusion, ordering, replacement conflicts, and remote inverse realization—require a separate Relationship Coordination capability. Until such a capability supplies an authoritative bucket scope and write authority, Commit must not accept a mutation by treating an incomplete cross-cell view as a passing validation result.

This keeps the basic validation path implementable without weakening its guarantees:

• local conformance is validated now;
• local relationship plans are prepared and persisted as one Commit-owned operation;
• cross-Space inverse work remains deferred;
• multi-cell aggregate correctness is explicitly deferred rather than implied.

We will revise the implementation plan to split relationship delivery accordingly:

1. Local relationship conformance and locally realizable inverse planning/persistence.
2. Deferred cross-cell relationship coordination, covering bucket ownership, mutation routing, ordering, concurrency preconditions, conflict outcomes, reconciliation, and Runtime Recognition/governance implications.

So I would record the present §12.4 wording as an unresolved design boundary, not as a claim that Holochain already provides generic distributed CAS or transaction isolation. The immediate track will define a precise local authority scope; the cross-cell conflict-resolution model will be designed separately.

Suggested re-working of §12.4

---

12.4 Commit-local write consistency

Commit validates and prepares only the relationship buckets for which its
execution context supplies authoritative local write scope. For a bucket whose
declared and required local inverse operations are written by the same
Holochain cell, Commit validates the prospective local collection and persists
the complete prepared plan in one normal-order zome-call transaction.

If the source-chain head moves before persistence, Commit discards the plan,
reloads the authoritative local state, revalidates, and retries or returns a
concurrency failure. Relaxed chain ordering is not used for an operation whose
validity depends on observed bucket state.

Commit does not treat a DHT observation as a serializable cross-cell bucket
snapshot. It does not use re-read-and-revalidate as a substitute for
cross-cell coordination.

A rule whose truth requires observations or writes controlled by more than one
cell requires an explicit Relationship Coordination capability. Until that
capability is supplied, Commit returns RelationshipCoordinationRequired rather
than accepting the mutation from an incomplete shared-bucket view.

---

Additional observation: Validating schema-level rules means that whenever a type changes, its entire ComponentOf Schema needs to be validated. I think this makes sense, but it strengthens the argument for versioned Schema “modules” that are pushed out as a unit, so an update cannot expose a partially formed Schema that breaks validation.

Yep!

[owleyeview]

• Type-descriptor and schema validation depend on inverse relationships being available. Because inverses can be guaranteed when both relationship endpoints are in the same Space, can we make it an explicit requirement that each Schema and its components be stewarded within a single Space?

Uh "hard no" on that. Otherwise the entire schema would forever be confined to a single space. Section 12.5 handles physical inverse realization; it does not make an incomplete or unavailable semantic dependency pass validation.

Noting here that subsequent conversation led to this revised understanding:

"
If the requirement is simply that a particular Schema and all of the TypeDescriptors that are its Components are stewarded in the same Space, then I agree. That gives the Schema a clear stewardship/locality boundary, and its Components inverse can be guaranteed locally.

...

A schema’s components will have many semantic relationships to descriptors outside that schema — Extends, UsesKeyRule, ValueType, SourceType/TargetType, dependencies, etc. If inverse availability implied co-location for all of those relationships, the requirement would propagate through the connected type graph and tend toward pulling the entire schema ecosystem into one Space.

So I think the distinction is:

Schema + its Components: yes, same stewarding Space.
Arbitrary relationships from those Components to descriptors in other Schemas/Spaces: no co-location requirement.

The intended containment hierarchy is strict:

Space → Schema → Module → Type

A Schema is wholly contained within a single Space. Its Modules are wholly contained within that Schema, and the Types belonging to a Module are wholly contained within that Module. None of those containment units are partitioned across Spaces.

Types can of course have relationships to Types outside their Module, Schema, or Space. Those relationships do not imply containment or co-location.
"