Skip to content

[research·architecture] One route-scoped market-context capability — shared observations, bounded research, no background news system #776

Description

@atomchung

Status

Research / architecture decision only — recorded 2026-08-02. Not an implementation request and not on the current M1 acceptance critical path.

Current execution authority remains #27 / #713: repair the bounded consider rule-effect and product-safe outcome projection, freeze a new exact SHA, then restart owner-live acceptance. Do not widen that repair with this work.

This issue exists because the repository now has good individual market-data and market-lookup pieces, but no single owner for their cross-route product semantics.

Milestone

None yet. Re-evaluate after the current M1 candidate is repaired and the ordinary / market-data recovery / bounded-unavailable owner journeys have completed.

User outcome

Every supported decision route receives the right market context for that user moment:

  • fresh public observations when the answer is genuinely about “now”;
  • one frozen observation frame when the user is answering, resuming, or amending the same review;
  • bounded current-event or decision-dimension research only when it can change the lead judgment or one necessary question;
  • no duplicate price sources, no observation-instant drift, no generic news dump, and no claim that a public event was the user's motive without confirmation.

The user should not need to know which internal module fetched a price, built a benchmark window, or verified an event. They should see one coherent answer with source, as_of, and an honest route-specific limitation.

Current repository truth / evidence

Verified against latest main@907627ce147a936fad43b5e58b8a6bfb5f90900d and the current execution index #27.

1. Deterministic observation acquisition is already independent

#605 / PR #611 introduced engine/market_data.py as the single acquisition boundary for:

  • instrument and benchmark closes/history;
  • split observations;
  • spot and historical FX;
  • source, as_of, coverage and stable gaps;
  • supplied-feed precedence and same-day cache reuse.

prepare and consider consume the same MarketDataBundle. The module explicitly owns no portfolio weights, returns, P&L, recommendation or state transition.

Latest PR #762 adds unresolved and stable throttle/request-failure gaps so a rate-limited symbol is no longer presented as delisted/unpriceable and no longer invalidates same-day cache reuse for the whole universe. This shows the acquisition seam is real and actively hardened, but also that provider behavior remains a reliability and latency concern.

2. Deterministic market backdrop exists as a projection

engine/market_context.py computes frozen SPY / QQQ window and YTD readings plus VIX level/delta. Supported prepare flows build that context from the shared price frame; weekly-market-read (#683) reads the settled plan and never refetches a provider.

#683 is implemented and CI-verified but owner-live UX-unaccepted. Its current slice is deliberately narrow: too_heavy × positive frozen VIX delta, then one read-only watch. It is not a live news recap or monitor.

3. Current-event / decision-dimension research is a separate host contract

#601 / PR #604 added references/market-lookup.md, currently prompt/docs only:

  • L0: standing position packet on every consider call — current price vs premise, recent move and 52-week range;
  • L1: triggered event lookup for vague/current why_now, a cited current claim, or a narrow contradiction;
  • L2: triggered lookup of a decision-central unchecked dimension such as valuation, operating evidence or an official macro release.

The contract correctly separates public_fact, agent_judgment, and user-confirmed why_now, uses one primary source plus at most one countercheck, and persists no research store.

4. The current split creates one unresolved cross-route gap

consider already asks market_data.py for current prices needed by deterministic portfolio arithmetic. The L0 contract separately instructs the host to fetch/transcribe current price and ready-made recent/range readings.

Therefore one answer can currently obtain “current market context” through two acquisition paths:

  1. engine MarketDataBundle for valuation/consequence;
  2. host L0 lookup for position-in-time judgment.

The repository does not currently prove that these use one source, one observation instant, one coverage result, or one retry/cache policy. The user may see a correctly computed consequence and a separately gathered “recent move / 52-week range” packet that was fetched later, failed differently, or disagreed in basis/provenance.

5. Freshness and failure semantics are correct locally but scattered

  • prepare: resolve once, then freeze the review frame; price gaps may degrade after bounded recovery.
  • resume, preview, finalize, add-cash: must reuse the frozen frame; refetching would change the review the user already answered against.
  • book-aware consider: current observations matter; after bounded recovery is exhausted, the route may refuse rather than answer on an unsafe basis.
  • no-book decision framing: host-side, fail-soft, no portfolio arithmetic, but L0/L1/L2 can still improve the decision framing.
  • weekly-market-read: retrospective window-frozen companion; omit when its connection gate is absent, never substitute a generic recap.
  • generic market/news questions: host informational behavior, not a FOMO Kernel decision-memory or review route.

These distinctions are spread across market_data.py, market-lookup.md, trade-consequence.md, decision-framing.md, weekly-review.md, weekly-market-read.md, SKILL.md and AGENTS.md. There is no one route policy declaring them.

Problem and consequence

1. “Market information” currently names three different truth classes

  1. reproducible observations — price, FX, volume/range/benchmark series;
  2. deterministic projections — recent move, range position, market backdrop;
  3. researched evidence — filing, guidance, macro event, operating or valuation claim.

Treating them as one module either puts non-deterministic research inside the deterministic engine or leaves deterministic numeric context in agent prose. Both violate existing boundaries.

2. L0 may duplicate acquisition and observation instants

A second host-side lookup costs latency and provider requests, has its own failure path, and can describe a different instant from the book consequence.

User consequence: “this becomes X% of the book” and “the stock is near Y point in its range” can both be individually sourced but not form one coherent decision frame.

3. Route freshness is implicit

“Always use the latest data” is wrong for resume, add-cash, preview/finalize and weekly retrospective reading. In those moments, freshness means preserving the exact frame the user already saw, not refetching a newer frame.

User consequence: the same review can change underneath their answers or explain a move that happened after the decision window.

4. A continuous market monitor would solve the wrong first problem

An always-on news/filing service would introduce scheduling, coverage, correction/replay, retention, notification, source licensing, latency and relevance problems before the existing route-scoped lookup has proved insufficient.

It would also move the product toward the research-terminal / investment-OS scope #475 explicitly excludes. “Know everything continuously” is not the product requirement; retrieve and freeze the smallest decision-relevant delta at the right moment is.

Decision

A. Keep three layers, with explicit ownership

Layer 1 — MarketDataBundle: acquisition facts

Keep engine/market_data.py as the single source-neutral acquisition boundary for reproducible observations. No recommendation, interpretation or user motive enters it.

Layer 2 — deterministic market-context projections

Build route-visible numeric context from the same resolved bundle whenever the engine already has the needed series. Candidate projections, names not yet final:

  • PositionMarketSnapshot: current close, premise/current relationship, recent return windows, 52-week high/low and range position;
  • MarketBackdrop: benchmark window readings and VIX context.

The engine owns window definitions, adjusted/raw basis, observation dates, missing coverage and rounding. The agent selects salience and explains relevance; it does not calculate a parallel L0 packet.

This is a candidate decision, not authorization to implement. First measure whether the current host L0 path actually duplicates calls or produces instant/basis divergence, and define the exact return/range semantics before changing the contract.

Layer 3 — bounded MarketEvidencePacket: researched facts

Keep L1/L2 event and decision-dimension research host-side and triggered. A future typed packet may be justified only if cross-host drift or provenance failure is observed; do not create a crawler, evidence database or generic research graph pre-emptively.

The packet, if introduced, would remain ephemeral to the decision unless a separate product owner names a later reader. It cannot become why_now without the user's own confirmation.

B. Declare one route-scoped market-context policy

Each route should explicitly state four things:

  1. observation mode: live | session_frozen | window_frozen | none;
  2. research mode: none | triggered_l1_l2;
  3. failure mode: degrade | refuse | omit;
  4. persistence: which facts are frozen in the existing route artifact and which remain conversation-only.

Initial policy to validate:

User moment Observation mode Research Failure behavior
first / weekly / snapshot prepare live once, then session-frozen normally none bounded recovery, then route-defined degradation
resume / preview / finalize / add-cash session-frozen only none fail closed if the frame is gone; never refetch
book-aware consider live decision snapshot triggered L1/L2 when material bounded recovery; refuse when truth-critical consequence is unavailable
no-book decision framing live public context, no book arithmetic triggered L1/L2 fail-soft and state the narrow unverified dimension
weekly-market-read window-frozen from the settled review none in current slice omit; never substitute generic recap
generic news / market recap not a FOMO Kernel product route host-defined no kernel state or decision-memory claim

C. Do not build an always-on market/news subsystem now

No background scheduler, feed ingestion, filing warehouse, KOL crawler, push alert, persistent “what happened” database or market-wide discovery loop is justified by current evidence.

Revisit only if repeated route-scoped owner-live use shows one of these concrete failures:

  • lookup latency dominates the decision moment despite cache/reuse;
  • the same relevant event must be re-retrieved across many real decisions;
  • delayed discovery causes a documented missed product outcome;
  • a named future reader needs a durable event record;
  • bounded host lookup cannot achieve acceptable source consistency across clients.

User before / after

Before

  • consider can price the book through the engine and separately fetch L0 context through the host;
  • each route locally knows whether to fetch or freeze, but no single contract makes the difference inspectable;
  • “market context” can mean a price bundle, SPY/QQQ/VIX projection, or researched event;
  • a user may experience repeated lookups, inconsistent as-of moments, a recovery refusal, or a weekly frozen read without understanding why those routes behave differently.

After the proposed smallest architecture

  • one acquisition bundle supplies all reproducible observations for a route;
  • deterministic projections own numeric L0/backdrop semantics;
  • the agent performs only the contextual event/dimension research that requires judgment;
  • every route declares live-vs-frozen behavior and degrade/refuse/omit semantics;
  • users get faster, coherent, source-dated answers without turning the product into a market terminal.

Best-practice assessment

This separation solves observed repository risks:

It is not justified as architecture cleanliness alone. The implementation should stop if measurement shows L0 does not duplicate acquisition, does not drift, or does not materially affect latency/answer quality.

A provider framework, second source, resident monitor, generic event ontology or persistent research store would be over-design at this stage.

Research scope

  1. Trace the actual supported call path for prepare, consider, no-book framing and weekly-market-read on current main.
  2. Measure provider/tool calls, observation timestamps and wall-clock for one synthetic case per route.
  3. Prove whether book-aware consider performs a second L0 retrieval after MarketDataBundle resolution and whether the visible facts can diverge in source/as-of/basis.
  4. Inventory every current numeric field required by L0 and classify it as:
    • already present in the bundle;
    • deterministically derivable from the bundle under a named window/basis;
    • provider-specific ready-made value that should remain host-transcribed;
    • unnecessary for the first useful answer.
  5. Produce the route policy table with one authoritative documentation location and thin references from route files.
  6. Run one owner-live book-aware consider and one no-book framing to decide whether the market context adds non-obvious value or merely more prose/latency.
  7. If and only if duplication/drift is observed, open one bounded implementation issue for a shared deterministic L0 projection.

Non-goals

Acceptance criteria

  • The supported route call graph is documented against one exact main@SHA.
  • Provider/tool call count, observation as_of, and wall-clock are recorded separately for prepare, book-aware consider, no-book framing and weekly-market-read.
  • The issue proves or rejects the L0 duplicate-acquisition / instant-drift hypothesis with a synthetic reproducible trace.
  • Every L0 numeric field has a named owner and basis; no field is left as “agent calculates it.”
  • One route policy table defines live/frozen mode, lookup trigger, failure behavior and persistence.
  • The policy preserves the current difference between review degradation, consider refusal, and weekly-read omission.
  • One owner-live verdict states whether current market context materially improved the decision, and whether its latency/extra prose earned its place.
  • The conclusion explicitly chooses one of: keep host L0; move bounded L0 projection over MarketDataBundle; or remove fields that do not earn their latency.
  • No implementation issue is opened unless the measured failure names the smallest user-visible correction.
  • Public evidence uses only fictional/generic instruments and contains no private trade, book, motive, source list or output.

Dependencies / relationships

Privacy boundary

Only synthetic, fictional or generic books, instruments, timestamps and public-source fixtures may be committed or posted. Real holdings, trades, reasons, lookup queries, source selections and delivered answers remain local.

Market-data cache keys reveal requested symbols and remain local under the protected state root. No new persistence is authorized here.

Rollout / compatibility / recovery

Research only, so no migration or compatibility change.

Any later deterministic L0 projection should be additive first, consume the existing MarketDataBundle, preserve supplied-feed mode, retain existing gap semantics, and leave canonical session/evaluation state readable. Rollback must be code/docs only; no new durable research state should exist to migrate.

Stop condition

Stop after the route trace, duplicate/drift measurement, policy table and two owner-live verdicts. Do not continue automatically into implementation, a second provider, event persistence, scheduling, notifications or a broader research system.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions