Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions docs/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -295,6 +295,17 @@ Server Event → Socket.IO → socket.js → React Component State Update
- Serves HTTPS with the shared Tailscale cert (`data/certs/`) when one is present, plain HTTP otherwise — matching the scheme the sidebar's `//<host>:5560` link inherits from the main app
- Fix history viewer, process status dashboard

### Local-model Hub metadata

`server/services/huggingFaceMetadata.js` owns the shared authenticated Hub reads,
request budgets, in-memory cache and publish-date enrichment used by the local
model catalog and MTPLX. Add reusable raw repo metadata reads here; disk cache
persistence remains in `huggingFaceRepoCache.js`. Model ranking, GGUF/MLX variant
selection, fit and installability belong in `huggingFaceCatalog.js`. Consumers
needing only metadata import its owner directly; the catalog retains its old
`fetchRepoPublishedDates` export for compatibility. `importScoping.test.js` guards
this boundary and the catalog workflow test checks shared request coalescing.

### Shell Service (`server/services/shell.js`)
- PTY-based web terminal via node-pty
- Session management with WebSocket I/O
Expand Down
2 changes: 1 addition & 1 deletion server/lib/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -538,7 +538,7 @@ pm` default, `NPM_CONFIG_PREFIX`, nvm/Volta) installed `codex` successfully and
| `arrayUtils.js` | `shuffle(arr, random?)` — Fisher-Yates shuffle (new array, never mutates). The canonical uniform shuffle — never `arr.sort(() => Math.random() - 0.5)`, which is biased. Shared by `meatspacePostCognitive.js` (Schulte table / mental rotation) and `meatspacePostMemory.js` (memory drill generators). `dedupeByKey(items, keyOf, pick?)` — one survivor per key, first-seen order. **Required before any multi-row `INSERT … ON CONFLICT (key) DO UPDATE`**: Postgres refuses the whole statement ("ON CONFLICT DO UPDATE command cannot affect row a second time") when its VALUES list names one conflict key twice, and the rows a batcher joins usually come from something that promises no uniqueness (a disk scan, a peer payload). `DO NOTHING` upserts are exempt. `pick(held, candidate)` defaults to last-seen-wins (what a sequential one-row upsert loop leaves); pass a comparator when the table's conflict rule isn't "latest write" — `memorySync.applyRemoteChanges` keeps the newest `updatedAt` so a peer's payload ordering can't flip a last-writer-wins outcome. Used by `services/mediaAssetIndex/db.js` and `services/memorySync.js`. |
| `assetRoutePrefixes.js` | Import-free leaf holding the URL prefixes the server owns: `ASSET_ROUTE_PREFIXES` (every `/data/**` static mount) and `SERVER_OWNED_PREFIXES` (what must never reach the SPA fallback, each with the exact `spaPaths` that ARE client routes). `scripts/dev-proxy-drift.test.js` checks the dev proxy's `^/data/` wildcard against the mounts, pins the route-registration order in `server/index.js` (a router added below the terminators is shadowed), and fails if a client route — from `NAV_COMMANDS` or `App.jsx`'s nested `<Route>` tree — is ever added under a server-owned prefix without being declared. |
| `asyncMutex.js` | Promise-based async mutex. |
| `concurrencyGate.js` | `createConcurrencyGate(limit)` → `run(fn)` — cap on simultaneous async work for ONE module-scoped budget, released FIFO. Sibling to `mapWithConcurrency.js`, which caps in-flight work *within one array map*; a gate is shared state, so several call sites fanning out at the same remote respect one budget instead of each respecting its own while the host sees the sum. `createMutex` (`asyncMutex.js`) is this with `limit` fixed at 1 — prefer it for mutual exclusion. Note the budget is per-MODULE, not per-host: two modules calling one host each get their own gate. Used by `huggingFaceCatalog.js` (4) and `ollamaRegistryCatalog.js` (16), whose cold catalog-enrichment bursts otherwise arrive at a free public API as a thundering herd — which the Hub answers with an HTTP/2 GOAWAY that surfaces as a bare `fetch failed`. |
| `concurrencyGate.js` | `createConcurrencyGate(limit)` → `run(fn)` — cap on simultaneous async work for ONE module-scoped budget, released FIFO. Sibling to `mapWithConcurrency.js`, which caps in-flight work *within one array map*; a gate is shared state, so several call sites fanning out at the same remote respect one budget instead of each respecting its own while the host sees the sum. `createMutex` (`asyncMutex.js`) is this with `limit` fixed at 1 — prefer it for mutual exclusion. Note the budget is per-MODULE, not per-host: two modules calling one host each get their own gate. Used by `huggingFaceMetadata.js` (4) and `ollamaRegistryCatalog.js` (16), whose cold catalog-enrichment bursts otherwise arrive at a free public API as a thundering herd — which the Hub answers with an HTTP/2 GOAWAY that surfaces as a bare `fetch failed`. |
| `dispatchLabels.js` | slashdo dispatch-hint contract: `model:light/medium/heavy` + `effort:low/medium/high/xhigh/max` vocabulary, prescribed forge colors, validation (`normalizeDispatchModel` / `normalizeDispatchEffort`), the read-side inverse (`dispatchHintFromLabels` — recovers `{model, effort}` from a raw label-name list, used by branch-reconcile's per-branch routing hint), GitHub/GitLab vs Jira label formatting, optional contributor labels (`good first issue` / `help wanted`, never implied by `model:light`, and released at claim time by `formatContributorLabelReleaseCommands` — one best-effort command per label, since a forge fails the whole edit when a named label is absent), the one shared volunteer-claim policy (`volunteerClaimLabels` / `formatVolunteerClaimCommands` — a human comment claiming an unassigned issue is resolved by BOTH issueWatcher.js's deterministic pass and the claim prompt's Phase 1 handoff, so both stamp `in-progress` and retire the invitations rather than writing opposite state), the open-ended planner-attribution axis (`planner:<model>`, `normalizePlannerId` / `resolvePlannerId` / `formatPlannerLabelGuidance` — records WHICH model wrote the plan, prefix-matched by `dispatchLabelSpec` so it lazily creates like the fixed labels; a filing agent takes the value from its prompt, never from self-identification), the workflow-state markers (`EPIC_LABEL`/`EPIC_DECOMPOSED_LABEL` and `IN_PROGRESS_LABEL` — state, not hints: shared with perpetualWork.js#isActionableIssue, issueReconcile.js's zombie scan, issueWatcher.js's volunteer assignment, and the claim prompts), lazy-create command text, the optional `--label` slots a rendered `issue create` example offers (`OPTIONAL_ISSUE_LABEL_FLAG_SLOTS` / `formatOptionalIssueLabelFlags` — one list so a new axis reaches every prompt template's copy-pasteable command, not just its prose), and shared dispatch plus issue-quality guidance (`ISSUE_QUALITY_GUIDANCE`, `DISPATCH_HINT_GUIDANCE`, `MANDATORY_DISPATCH_HINT_GUIDANCE`, `JIRA_DISPATCH_HINT_GUIDANCE`, `MANDATORY_JIRA_DISPATCH_HINT_GUIDANCE`, `REFERENCE_WATCH_LABEL_CONTRACT`, plus the consumer-side pair `DISPATCH_HINT_READING_GUIDANCE` / `DISPATCH_HINT_FANOUT_GUIDANCE` — what the labels mean to an agent that RECEIVES them (the preloaded `open-issues` task data input) and, for an orchestrator only, how to route each sub-agent by its own issue (the swarm block); the fan-out form is the reading form plus one line, never a second copy of the vocabulary). All forge filing callers require both dispatch axes and verify labels after creation; help wanted requires specific hardware validation or valuable input from real users; never invent `medium`; reject future-only/speculative work while keeping useful current refactors claimable. Consumed by work-tracker instructions, quota-burn audits, Layered Intelligence filing, and claim follow-up prompts. |
| `domainAutonomy.js` | Per-domain autonomy guardrails (pure). `AUTONOMY_DOMAINS`/`DOMAIN_IDS`/`DOMAIN_MODES` (`off`/`dry-run`/`execute`), `getDomainMode(config, id)`, and `normalizeDomainAutonomy(raw)` to coerce a hand-edited/partial map. Default per domain is `execute` (reproduces pre-#711 behavior, so no migration needed). Also `CREATIVE_DOMAIN`/`getCreativeAutonomyMode(config)` (#2183) — the Creative Director orchestrator domain, kept out of `DOMAIN_IDS` and defaulting to mirror the `cos` mode. |
| `domainBudgets.js` | Per-domain daily autonomy budgets (pure). `BUDGET_LIMIT_FIELDS` (`maxActionsPerDay`/`maxMinutesPerDay`), `getDomainBudget(config, id)`, `normalizeDomainBudgets(raw)`, `hasBudget(budget)`, and `evaluateBudget(budget, usage)` → `{ withinBudget, exceeded }`. A `null`/non-positive cap means unlimited (default per domain, so no migration needed). Token/$ caps are intentionally absent — CLI subscription providers expose no per-run metering. Usage ledger + gate wiring live in `services/domainUsage.js`. |
Expand Down
7 changes: 7 additions & 0 deletions server/lib/importScoping.test.js
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,10 @@ const reaches = (entry, target) => staticImportClosure(abs(entry)).files.has(abs
// Each row: the entry that was narrowed, the module it must no longer
// statically reach, and why the entry only ever needed a slice of it.
const NARROWED = [
['services/mtplxModelManager.js', 'services/huggingFaceCatalog.js',
'reads repository ages through shared metadata without catalog selection'],
['services/huggingFaceMetadata.js', 'services/pipeline/musicGen.js',
'owns Hub transport and caching independently of audio rendering'],
['lib/providerFamilies.js', 'lib/grok.js',
'shares browser-safe family identity without Grok filesystem helpers'],
['services/promptSections/instructions.js', 'services/taskScheduleRegistry.js',
Expand Down Expand Up @@ -83,6 +87,9 @@ describe('narrowed imports stay narrow (#6009)', () => {
// Positive controls. Without these the negatives above would also pass if
// `staticImportClosure` stopped resolving these files at all.
it('still sees the modules the narrowed entries were pointed AT', () => {
expect(reaches('services/mtplxModelManager.js', 'services/huggingFaceMetadata.js')).toBe(true);
expect(reaches('services/huggingFaceCatalog.js', 'services/huggingFaceMetadata.js')).toBe(true);
expect(reaches('services/huggingFaceMetadata.js', 'services/huggingFaceRepoCache.js')).toBe(true);
expect(reaches('services/promptSections/instructions.js', 'lib/scheduledTaskTypes.js')).toBe(true);
expect(reaches('services/agentAppWorkspace.js', 'lib/fileUtils.js')).toBe(true);
expect(reaches('lib/pipelineValidation.js', 'lib/editorial/checkInfra/taxonomy.js')).toBe(true);
Expand Down
Loading