diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index afe9786c36..e39d157fdb 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -295,6 +295,17 @@ Server Event → Socket.IO → socket.js → React Component State Update - Serves HTTPS with the shared Tailscale cert (`data/certs/`) when one is present, plain HTTP otherwise — matching the scheme the sidebar's `//:5560` link inherits from the main app - Fix history viewer, process status dashboard +### Local-model Hub metadata + +`server/services/huggingFaceMetadata.js` owns the shared authenticated Hub reads, +request budgets, in-memory cache and publish-date enrichment used by the local +model catalog and MTPLX. Add reusable raw repo metadata reads here; disk cache +persistence remains in `huggingFaceRepoCache.js`. Model ranking, GGUF/MLX variant +selection, fit and installability belong in `huggingFaceCatalog.js`. Consumers +needing only metadata import its owner directly; the catalog retains its old +`fetchRepoPublishedDates` export for compatibility. `importScoping.test.js` guards +this boundary and the catalog workflow test checks shared request coalescing. + ### Shell Service (`server/services/shell.js`) - PTY-based web terminal via node-pty - Session management with WebSocket I/O diff --git a/server/lib/README.md b/server/lib/README.md index 5628f71b77..5a98815c13 100644 --- a/server/lib/README.md +++ b/server/lib/README.md @@ -538,7 +538,7 @@ pm` default, `NPM_CONFIG_PREFIX`, nvm/Volta) installed `codex` successfully and | `arrayUtils.js` | `shuffle(arr, random?)` — Fisher-Yates shuffle (new array, never mutates). The canonical uniform shuffle — never `arr.sort(() => Math.random() - 0.5)`, which is biased. Shared by `meatspacePostCognitive.js` (Schulte table / mental rotation) and `meatspacePostMemory.js` (memory drill generators). `dedupeByKey(items, keyOf, pick?)` — one survivor per key, first-seen order. **Required before any multi-row `INSERT … ON CONFLICT (key) DO UPDATE`**: Postgres refuses the whole statement ("ON CONFLICT DO UPDATE command cannot affect row a second time") when its VALUES list names one conflict key twice, and the rows a batcher joins usually come from something that promises no uniqueness (a disk scan, a peer payload). `DO NOTHING` upserts are exempt. `pick(held, candidate)` defaults to last-seen-wins (what a sequential one-row upsert loop leaves); pass a comparator when the table's conflict rule isn't "latest write" — `memorySync.applyRemoteChanges` keeps the newest `updatedAt` so a peer's payload ordering can't flip a last-writer-wins outcome. Used by `services/mediaAssetIndex/db.js` and `services/memorySync.js`. | | `assetRoutePrefixes.js` | Import-free leaf holding the URL prefixes the server owns: `ASSET_ROUTE_PREFIXES` (every `/data/**` static mount) and `SERVER_OWNED_PREFIXES` (what must never reach the SPA fallback, each with the exact `spaPaths` that ARE client routes). `scripts/dev-proxy-drift.test.js` checks the dev proxy's `^/data/` wildcard against the mounts, pins the route-registration order in `server/index.js` (a router added below the terminators is shadowed), and fails if a client route — from `NAV_COMMANDS` or `App.jsx`'s nested `` tree — is ever added under a server-owned prefix without being declared. | | `asyncMutex.js` | Promise-based async mutex. | -| `concurrencyGate.js` | `createConcurrencyGate(limit)` → `run(fn)` — cap on simultaneous async work for ONE module-scoped budget, released FIFO. Sibling to `mapWithConcurrency.js`, which caps in-flight work *within one array map*; a gate is shared state, so several call sites fanning out at the same remote respect one budget instead of each respecting its own while the host sees the sum. `createMutex` (`asyncMutex.js`) is this with `limit` fixed at 1 — prefer it for mutual exclusion. Note the budget is per-MODULE, not per-host: two modules calling one host each get their own gate. Used by `huggingFaceCatalog.js` (4) and `ollamaRegistryCatalog.js` (16), whose cold catalog-enrichment bursts otherwise arrive at a free public API as a thundering herd — which the Hub answers with an HTTP/2 GOAWAY that surfaces as a bare `fetch failed`. | +| `concurrencyGate.js` | `createConcurrencyGate(limit)` → `run(fn)` — cap on simultaneous async work for ONE module-scoped budget, released FIFO. Sibling to `mapWithConcurrency.js`, which caps in-flight work *within one array map*; a gate is shared state, so several call sites fanning out at the same remote respect one budget instead of each respecting its own while the host sees the sum. `createMutex` (`asyncMutex.js`) is this with `limit` fixed at 1 — prefer it for mutual exclusion. Note the budget is per-MODULE, not per-host: two modules calling one host each get their own gate. Used by `huggingFaceMetadata.js` (4) and `ollamaRegistryCatalog.js` (16), whose cold catalog-enrichment bursts otherwise arrive at a free public API as a thundering herd — which the Hub answers with an HTTP/2 GOAWAY that surfaces as a bare `fetch failed`. | | `dispatchLabels.js` | slashdo dispatch-hint contract: `model:light/medium/heavy` + `effort:low/medium/high/xhigh/max` vocabulary, prescribed forge colors, validation (`normalizeDispatchModel` / `normalizeDispatchEffort`), the read-side inverse (`dispatchHintFromLabels` — recovers `{model, effort}` from a raw label-name list, used by branch-reconcile's per-branch routing hint), GitHub/GitLab vs Jira label formatting, optional contributor labels (`good first issue` / `help wanted`, never implied by `model:light`, and released at claim time by `formatContributorLabelReleaseCommands` — one best-effort command per label, since a forge fails the whole edit when a named label is absent), the one shared volunteer-claim policy (`volunteerClaimLabels` / `formatVolunteerClaimCommands` — a human comment claiming an unassigned issue is resolved by BOTH issueWatcher.js's deterministic pass and the claim prompt's Phase 1 handoff, so both stamp `in-progress` and retire the invitations rather than writing opposite state), the open-ended planner-attribution axis (`planner:`, `normalizePlannerId` / `resolvePlannerId` / `formatPlannerLabelGuidance` — records WHICH model wrote the plan, prefix-matched by `dispatchLabelSpec` so it lazily creates like the fixed labels; a filing agent takes the value from its prompt, never from self-identification), the workflow-state markers (`EPIC_LABEL`/`EPIC_DECOMPOSED_LABEL` and `IN_PROGRESS_LABEL` — state, not hints: shared with perpetualWork.js#isActionableIssue, issueReconcile.js's zombie scan, issueWatcher.js's volunteer assignment, and the claim prompts), lazy-create command text, the optional `--label` slots a rendered `issue create` example offers (`OPTIONAL_ISSUE_LABEL_FLAG_SLOTS` / `formatOptionalIssueLabelFlags` — one list so a new axis reaches every prompt template's copy-pasteable command, not just its prose), and shared dispatch plus issue-quality guidance (`ISSUE_QUALITY_GUIDANCE`, `DISPATCH_HINT_GUIDANCE`, `MANDATORY_DISPATCH_HINT_GUIDANCE`, `JIRA_DISPATCH_HINT_GUIDANCE`, `MANDATORY_JIRA_DISPATCH_HINT_GUIDANCE`, `REFERENCE_WATCH_LABEL_CONTRACT`, plus the consumer-side pair `DISPATCH_HINT_READING_GUIDANCE` / `DISPATCH_HINT_FANOUT_GUIDANCE` — what the labels mean to an agent that RECEIVES them (the preloaded `open-issues` task data input) and, for an orchestrator only, how to route each sub-agent by its own issue (the swarm block); the fan-out form is the reading form plus one line, never a second copy of the vocabulary). All forge filing callers require both dispatch axes and verify labels after creation; help wanted requires specific hardware validation or valuable input from real users; never invent `medium`; reject future-only/speculative work while keeping useful current refactors claimable. Consumed by work-tracker instructions, quota-burn audits, Layered Intelligence filing, and claim follow-up prompts. | | `domainAutonomy.js` | Per-domain autonomy guardrails (pure). `AUTONOMY_DOMAINS`/`DOMAIN_IDS`/`DOMAIN_MODES` (`off`/`dry-run`/`execute`), `getDomainMode(config, id)`, and `normalizeDomainAutonomy(raw)` to coerce a hand-edited/partial map. Default per domain is `execute` (reproduces pre-#711 behavior, so no migration needed). Also `CREATIVE_DOMAIN`/`getCreativeAutonomyMode(config)` (#2183) — the Creative Director orchestrator domain, kept out of `DOMAIN_IDS` and defaulting to mirror the `cos` mode. | | `domainBudgets.js` | Per-domain daily autonomy budgets (pure). `BUDGET_LIMIT_FIELDS` (`maxActionsPerDay`/`maxMinutesPerDay`), `getDomainBudget(config, id)`, `normalizeDomainBudgets(raw)`, `hasBudget(budget)`, and `evaluateBudget(budget, usage)` → `{ withinBudget, exceeded }`. A `null`/non-positive cap means unlimited (default per domain, so no migration needed). Token/$ caps are intentionally absent — CLI subscription providers expose no per-run metering. Usage ledger + gate wiring live in `services/domainUsage.js`. | diff --git a/server/lib/importScoping.test.js b/server/lib/importScoping.test.js index 4094521a46..148edbdd25 100644 --- a/server/lib/importScoping.test.js +++ b/server/lib/importScoping.test.js @@ -39,6 +39,10 @@ const reaches = (entry, target) => staticImportClosure(abs(entry)).files.has(abs // Each row: the entry that was narrowed, the module it must no longer // statically reach, and why the entry only ever needed a slice of it. const NARROWED = [ + ['services/mtplxModelManager.js', 'services/huggingFaceCatalog.js', + 'reads repository ages through shared metadata without catalog selection'], + ['services/huggingFaceMetadata.js', 'services/pipeline/musicGen.js', + 'owns Hub transport and caching independently of audio rendering'], ['lib/providerFamilies.js', 'lib/grok.js', 'shares browser-safe family identity without Grok filesystem helpers'], ['services/promptSections/instructions.js', 'services/taskScheduleRegistry.js', @@ -83,6 +87,9 @@ describe('narrowed imports stay narrow (#6009)', () => { // Positive controls. Without these the negatives above would also pass if // `staticImportClosure` stopped resolving these files at all. it('still sees the modules the narrowed entries were pointed AT', () => { + expect(reaches('services/mtplxModelManager.js', 'services/huggingFaceMetadata.js')).toBe(true); + expect(reaches('services/huggingFaceCatalog.js', 'services/huggingFaceMetadata.js')).toBe(true); + expect(reaches('services/huggingFaceMetadata.js', 'services/huggingFaceRepoCache.js')).toBe(true); expect(reaches('services/promptSections/instructions.js', 'lib/scheduledTaskTypes.js')).toBe(true); expect(reaches('services/agentAppWorkspace.js', 'lib/fileUtils.js')).toBe(true); expect(reaches('lib/pipelineValidation.js', 'lib/editorial/checkInfra/taxonomy.js')).toBe(true); diff --git a/server/services/huggingFaceCatalog.js b/server/services/huggingFaceCatalog.js index d087afd8b6..f5d17c6f94 100644 --- a/server/services/huggingFaceCatalog.js +++ b/server/services/huggingFaceCatalog.js @@ -1,38 +1,19 @@ +import { fetchModels, fetchRepoModel } from './huggingFaceMetadata.js' +// Compatibility for existing catalog consumers; new metadata callers use its owner. +export { fetchRepoPublishedDates } from './huggingFaceMetadata.js' import { ESTABLISHED_MODEL_PUBLISHERS, localModelSafety } from '../lib/localModelSafety.js' -import { getHfToken } from './hfToken.js'; import { formatBytes as formatBytesRaw } from '../lib/fileUtils.js' -import { fetchWithTimeout } from '../lib/fetchWithTimeout.js' -import { readResponseJson } from '../lib/readResponseJson.js' -import { createConcurrencyGate } from '../lib/concurrencyGate.js' -import { createSingleFlight } from '../lib/singleFlight.js' -import { describeFetchError, isReplayableConnectionError } from '../lib/fetchErrorChain.js' -import { readCachedRepoModel, writeCachedRepoModel } from './huggingFaceRepoCache.js' import { LOCAL_LLM_CATEGORIES, isBackend } from '../lib/localLlmCatalog.js' import { ENGINES } from './pipeline/musicGen.js' import { fetchOllamaRegistryVariants } from './ollamaRegistryCatalog.js' import { reconcileFit } from '../lib/localModelAssessment.js' -const HF_API_BASE = 'https://huggingface.co/api/models' -const HF_TIMEOUT_MS = 12_000 -// Pause before the single connection-blip retry (see hfFetch). -const HF_RETRY_DELAY_MS = 250 // Upper bound on how long the curated-catalog endpoint waits for HF variant // enrichment. The curated catalog must stay usable offline (it was a pure local // list before enrichment), so when HF is slow/down we return the catalog as-is // after this budget; in-flight probes keep running and warm the repo cache, so // the next load (or a recovered HF) enriches without delay. const CATALOG_ENRICH_TIMEOUT_MS = 5_000 -// Budget for the publish-date lookup behind a checkpoint search. Deliberately -// longer than CATALOG_ENRICH_TIMEOUT_MS: that bound exists because the curated -// catalog must stay usable with zero enrichment offline, whereas a search's ages -// ARE the enrichment, and on a cold cache these probes can sit behind the -// catalog's own fan-out in the shared gate. Still bounded — a hung Hub must not -// hold the search open indefinitely. -const PUBLISH_DATE_BUDGET_MS = 15_000 -// Hard cap on repos probed per publish-date lookup, independent of the caller's -// page size — abandoned probes keep draining through hfGate after the response. -const MAX_PUBLISH_DATE_PROBES = 24 - const CATEGORY_IDS = new Set(LOCAL_LLM_CATEGORIES.map((c) => c.id)) // Default browse phrases used when the search box is empty (and as the seed when // a category tag is clicked with no query). The Hub `search` param is AND-across @@ -708,167 +689,6 @@ function toResult(model, backend, requestedCategory, installedIds, installedAudi return result } -async function hfHeaders() { - const headers = { Accept: 'application/json' } - const token = await getHfToken() - if (token) headers.Authorization = `Bearer ${token}` - return headers -} - -// 4 at a time, shared by BOTH entry points: a cold catalog load fires ~36 -// `?blobs=true` probes and a keystroke fires up to 18 more, concurrently — a -// burst the Hub answers with an HTTP/2 GOAWAY. See concurrencyGate for why a -// shared gate rather than a per-map cap. -const hfGate = createConcurrencyGate(4) -// The interactive search's own LIST query gets a separate, tiny budget so it is -// never stuck behind the catalog's fan-out. `enrichCatalogWithVariants` bounds -// how long the *response* waits, not the probes themselves — abandoned probes -// keep draining through hfGate — so a user who lands on the curated tab and then -// switches to Hugging Face would otherwise queue behind up to ~32 waiters. On a -// degraded Hub each of those costs two timeouts, which is minutes of a search box -// that has not even issued its request yet. This query is 1–2 requests, not a -// fan-out, so a budget of 2 keeps the total offered to the Hub bounded (4 + 2) -// while making the path the user is actually waiting on independent. -const hfSearchGate = createConcurrencyGate(2) -// Coalesce concurrent probes of the SAME repo — a repo can appear in both the -// curated catalog and the live search, and neither caches until it resolves. -const repoModelFlight = createSingleFlight() - -// Statuses that are a real, durable "this repo has no data" answer and so are -// safe to cache. Everything else non-OK (including auth denials, rate limits, -// 5xx, and 408) may change and must be retried on a later lookup — mirrors -// `resolveRegistryBody` in ollamaRegistryCatalog.js, which has always drawn this -// line. -// Authentication denials are deliberately excluded: a user can add a token or -// gain access to a gated repo at any time, so caching a 401/403 would keep the -// catalog blank until the seven-day repo-cache TTL expires. A genuinely missing -// or gone repo remains a durable no-data answer. -const HF_PERMANENT_NOT_FOUND = new Set([404, 410]) - -// Single door to the Hub: bounded concurrency + the shared one-shot retry. -// -// Returns a discriminated outcome rather than a Response, because the body must -// be consumed INSIDE the gate slot. Releasing at response headers would bound -// only the header waits while every body streamed concurrently — i.e. exactly -// the many-simultaneous-streams-on-one-pooled-connection condition that earns -// the GOAWAY this gate exists to prevent. -// -// { outcome: 'ok', data } — 2xx with a parseable body -// { outcome: 'permanent', status, errorText } — a durable no-data answer; cacheable -// { outcome: 'transient', status, errorText } — a bad moment; MUST NOT be cached -// -// Throws only when both attempts failed at the connection level. -function hfFetch(url) { - return hfGate.run(async () => { - const res = await fetchWithTimeout( - url, - { headers: await hfHeaders() }, - HF_TIMEOUT_MS, - { retries: 1, retryDelayMs: HF_RETRY_DELAY_MS, shouldRetry: isReplayableConnectionError } - // Both attempts lost the connection. undici's own message is a bare `fetch - // failed`, which reaches the search box verbatim and reads like a bug in - // PortOS — name the actual condition so the user knows to just try again. - ).catch((err) => { - if (!isReplayableConnectionError(err)) throw err - throw new Error(`Hugging Face is not responding (connection dropped twice) — try again in a moment. [${describeFetchError(err)}]`) - }) - if (!res.ok) { - const errorText = await res.text().catch(() => '') - const outcome = HF_PERMANENT_NOT_FOUND.has(res.status) ? 'permanent' : 'transient' - return { outcome, status: res.status, errorText } - } - // A 200 whose body won't parse is a proxy/captive-portal error page, not an - // answer about the repo — transient, so it is never cached as "no data". - const data = await readResponseJson(res, { fallback: null }) - return data == null - ? { outcome: 'transient', status: res.status, errorText: 'unparseable response body' } - : { outcome: 'ok', data } - }) -} - -// `filter` is a Hugging Face library tag — 'gguf' for the GGUF query, 'mlx' for -// the Apple-MLX query, or null/'' to relax the format filter (audio category and -// the GGUF-signal fallback). Only one filter at a time; MLX runs as a separate -// query so its results don't pollute the GGUF list. -async function fetchModels(search, limit, filter) { - const params = new URLSearchParams({ - search, - sort: 'downloads', - direction: '-1', - limit: String(limit), - full: 'true' - }) - if (filter) params.set('filter', filter) - - const result = await hfSearchGate.run(() => hfFetch(`${HF_API_BASE}?${params.toString()}`)) - if (result.outcome !== 'ok') { - const detail = result.errorText ? ` — ${result.errorText.slice(0, 160)}` : '' - throw new Error(`Hugging Face search failed: ${result.status}${detail}`) - } - return Array.isArray(result.data) ? result.data : [] -} - -const repoModelCache = new Map() -const REPO_MODEL_CACHE_MAX = 500 -// A connection-level failure (network down / both retry attempts dropped) — -// distinct both from a durable no-data answer (401/403/404/410, cacheable) and -// from a transient HTTP status (429/5xx), which `hfFetch` reports as -// `outcome: 'transient'`. Neither transient form may be cached: doing so would -// disable enrichment for the repo until the TTL expires, a week later. -const TRANSIENT_FETCH = Symbol('transient-fetch') - -// Fetch (and cache) the per-model record WITH per-file sizes. The search -// endpoint returns siblings without sizes; only `?blobs=true` carries them. -// -// Three tiers, cheapest first: an in-process Map, then the disk cache -// (huggingFaceRepoCache.js), then the Hub. The disk tier is what stops the -// curated catalog — a KNOWN, fixed list of ~36 repos — from re-asking the Hub -// for all of them after every restart, self-update, or dev reload. Steady state -// on that path is zero network. -// -// `null` = fetched-but-unavailable, and it is cached at both tiers (per the -// absent-vs-empty sentinel rule) so a sizeless repo isn't re-probed every search -// — but ONLY when the Hub gave a durable answer (404/410). An auth denial, rate -// limit, 5xx, or dropped connection also returns null and is NOT cached, so a -// newly authorized or recovered Hub re-enriches on the next request instead of -// staying blank for the week the disk TTL would otherwise hold it. -async function fetchRepoModel(repoId) { - if (repoModelCache.has(repoId)) return repoModelCache.get(repoId) - return repoModelFlight.run(repoId, async () => { - const cached = await readCachedRepoModel(repoId) - // `hit` is separate from the value because a cached `model` of null is a - // real answer (gated/absent), not a miss. - if (cached.hit) { - rememberRepoModel(repoId, cached.model) - return cached.model - } - // repoId comes from the HF search response (untrusted upstream) — encode each - // path segment so a `?`/`#`/`..` in the id can't reshape the request path/query. - const safeRepoPath = String(repoId).split('/').map(encodeURIComponent).join('/') - const result = await hfFetch(`${HF_API_BASE}/${safeRepoPath}?blobs=true`) - .catch(() => TRANSIENT_FETCH) - // Transient — a rate limit, a 5xx, or a dropped connection. Return null so - // this load degrades gracefully, but do NOT cache it: persisting a transient - // as "no data" would bake a bad moment into the disk tier for the full TTL, - // and a restart would no longer clear it the way the old memory-only cache did. - if (result === TRANSIENT_FETCH || result.outcome === 'transient') return null - // 'permanent' (gone) IS a real answer — cache the null. Auth denials are - // transient because the user's credentials or repository access can change. - const model = result.outcome === 'ok' ? result.data : null - rememberRepoModel(repoId, model) - await writeCachedRepoModel(repoId, model) - return model - }) -} - -function rememberRepoModel(repoId, model) { - // Evict oldest entry when the cap is reached (insertion-order iteration). - if (repoModelCache.size >= REPO_MODEL_CACHE_MAX) { - repoModelCache.delete(repoModelCache.keys().next().value) - } - repoModelCache.set(repoId, model) -} - // Total resident size of an audio repo's weight files — audio generators ship // `.safetensors`/`.ckpt`/`.bin` weights rather than a single GGUF, so the quant // picker doesn't apply; sum the weight siblings instead. @@ -1332,44 +1152,3 @@ export function applyMeasuredFit(models, { backend, measured } = {}) { } return list } - -// Publish dates for a set of Hugging Face repos, as `{ repoId: createdAt|null }`. -// -// For lists whose rows come from somewhere OTHER than the Hub's own search — the -// MTPLX discover listing, which carries downloads and license but no dates — so -// the card can say how old a checkpoint is. Reuses fetchRepoModel's three tiers -// (memory → disk → Hub) and its gate, so a repeated search is free and a burst -// stays inside the same concurrency budget as everything else here. -// -// Never throws and never fails the caller's list: a repo the Hub has no answer -// for (gated, renamed, offline) resolves to `null`, which the UI renders as a -// missing age rather than an error. -export async function fetchRepoPublishedDates(repoIds = [], { timeoutMs = PUBLISH_DATE_BUDGET_MS } = {}) { - // Cap the fan-out independently of the caller's page size. The MTPLX search - // endpoint accepts limit=100, and every unresolved probe keeps draining through - // the shared hfGate after the response returns — starving curated-catalog and - // HF-search enrichment on a degraded Hub for as long as it takes. A page of - // ages beyond the first two dozen rows is not worth that. - const unique = [...new Set(repoIds.filter((id) => typeof id === 'string' && id.includes('/')))] - .slice(0, MAX_PUBLISH_DATE_PROBES) - // Seeded with nulls and filled in place, so the budget below can return early - // with a partial answer instead of an empty one. - const dates = Object.fromEntries(unique.map((repo) => [repo, null])) - const work = Promise.allSettled(unique.map(async (repo) => { - const model = await fetchRepoModel(repo) - dates[repo] = model?.createdAt || model?.created_at || null - })) - // Bound the wait the way enrichCatalogWithVariants does, so an unreachable Hub - // can never hang a search. Whatever resolved in time is already in `dates`; the - // rest stay null and the card simply omits that row's age. Note a TRANSIENT - // failure caches nothing (see fetchRepoModel), so those repos are re-probed on - // the next search rather than being remembered as dateless. - if (timeoutMs > 0) { - let timer - const budget = new Promise((resolve) => { timer = setTimeout(resolve, timeoutMs); timer.unref?.() }) - await Promise.race([work.finally(() => clearTimeout(timer)), budget]) - } else { - await work - } - return dates -} diff --git a/server/services/huggingFaceCatalog.test.js b/server/services/huggingFaceCatalog.test.js index acf006ab36..e3d35ffbb9 100644 --- a/server/services/huggingFaceCatalog.test.js +++ b/server/services/huggingFaceCatalog.test.js @@ -1,5 +1,6 @@ import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest' import { searchHuggingFaceModels, enrichCatalogWithVariants, applyMeasuredFit, fetchRepoPublishedDates } from './huggingFaceCatalog.js' +import { fetchRepoPublishedDates as fetchMetadataDates } from './huggingFaceMetadata.js' import { __resetOllamaRegistryCache } from './ollamaRegistryCatalog.js' // The disk cache resolves its file from the REAL PATHS.data, and fetchRepoModel @@ -1296,7 +1297,7 @@ describe('huggingFaceCatalog', () => { expect(catalog.every((e) => e.format === 'gguf')).toBe(true) }) - it('coalesces concurrent probes of the same repo into one request', async () => { + it('shares concurrent repo probes between catalog variants and metadata consumers', async () => { const repo = 'dedupe-pub/Shared-GGUF' let blobCalls = 0 fetch.mockImplementation(async (url) => { @@ -1304,7 +1305,7 @@ describe('huggingFaceCatalog', () => { if (u.includes('blobs=true')) { blobCalls += 1 await new Promise((resolve) => setTimeout(resolve, 5)) - return response({ id: repo, siblings: [{ rfilename: 'S-Q4_K_M.gguf', size: 4_000_000_000 }] }) + return response({ id: repo, createdAt: '2026-01-02T00:00:00.000Z', siblings: [{ rfilename: 'S-Q4_K_M.gguf', size: 4_000_000_000 }] }) } return response([]) }) @@ -1312,9 +1313,15 @@ describe('huggingFaceCatalog', () => { const catalogs = Array.from({ length: 3 }, () => ([ { id: repo, key: 'shared', name: 'Shared', category: 'chat', size: '2.0 GB' } ])) - await Promise.all(catalogs.map((c) => enrichCatalogWithVariants(c, { - backend: 'lmstudio', systemMemoryBytes: 128 * 1024 ** 3, installedIds: [], timeoutMs: 0 - }))) + const [dates] = await Promise.all([ + fetchMetadataDates([repo], { timeoutMs: 0 }), + ...catalogs.map((c) => enrichCatalogWithVariants(c, { + backend: 'lmstudio', systemMemoryBytes: 128 * 1024 ** 3, installedIds: [], timeoutMs: 0 + })) + ]) + + expect(dates).toEqual({ [repo]: '2026-01-02T00:00:00.000Z' }) + expect(await fetchRepoPublishedDates([repo])).toEqual(dates) expect(blobCalls).toBe(1) expect(catalogs.every((c) => c[0].format === 'gguf')).toBe(true) diff --git a/server/services/huggingFaceMetadata.js b/server/services/huggingFaceMetadata.js new file mode 100644 index 0000000000..5dc28a7830 --- /dev/null +++ b/server/services/huggingFaceMetadata.js @@ -0,0 +1,238 @@ +/** + * Shared Hugging Face metadata for local-model discovery. + * + * Owns authenticated list/repo reads, retry and concurrency budgets, and the + * memory → disk → Hub cache. Use fetchModels for raw search rows, + * fetchRepoModel for a size-bearing repo record, and fetchRepoPublishedDates + * for bounded best-effort age enrichment (including non-Hub listings). + * + * Catalog ranking, GGUF/MLX variants, fit and installability stay in + * huggingFaceCatalog.js; disk persistence stays in huggingFaceRepoCache.js. + * Keep consumers on this service so they share one cache and request budget + * without importing model selection or music rendering. + */ + +import { getHfToken } from './hfToken.js'; +import { fetchWithTimeout } from '../lib/fetchWithTimeout.js' +import { readResponseJson } from '../lib/readResponseJson.js' +import { createConcurrencyGate } from '../lib/concurrencyGate.js' +import { createSingleFlight } from '../lib/singleFlight.js' +import { describeFetchError, isReplayableConnectionError } from '../lib/fetchErrorChain.js' +import { readCachedRepoModel, writeCachedRepoModel } from './huggingFaceRepoCache.js' + +const HF_API_BASE = 'https://huggingface.co/api/models' +const HF_TIMEOUT_MS = 12_000 +// Pause before the single connection-blip retry (see hfFetch). +const HF_RETRY_DELAY_MS = 250 +// Budget for the publish-date lookup behind a checkpoint search. Deliberately +// longer than CATALOG_ENRICH_TIMEOUT_MS: that bound exists because the curated +// catalog must stay usable with zero enrichment offline, whereas a search's ages +// ARE the enrichment, and on a cold cache these probes can sit behind the +// catalog's own fan-out in the shared gate. Still bounded — a hung Hub must not +// hold the search open indefinitely. +const PUBLISH_DATE_BUDGET_MS = 15_000 +// Hard cap on repos probed per publish-date lookup, independent of the caller's +// page size — abandoned probes keep draining through hfGate after the response. +const MAX_PUBLISH_DATE_PROBES = 24 + +async function hfHeaders() { + const headers = { Accept: 'application/json' } + const token = await getHfToken() + if (token) headers.Authorization = `Bearer ${token}` + return headers +} + +// 4 at a time, shared by BOTH entry points: a cold catalog load fires ~36 +// `?blobs=true` probes and a keystroke fires up to 18 more, concurrently — a +// burst the Hub answers with an HTTP/2 GOAWAY. See concurrencyGate for why a +// shared gate rather than a per-map cap. +const hfGate = createConcurrencyGate(4) +// The interactive search's own LIST query gets a separate, tiny budget so it is +// never stuck behind the catalog's fan-out. `enrichCatalogWithVariants` bounds +// how long the *response* waits, not the probes themselves — abandoned probes +// keep draining through hfGate — so a user who lands on the curated tab and then +// switches to Hugging Face would otherwise queue behind up to ~32 waiters. On a +// degraded Hub each of those costs two timeouts, which is minutes of a search box +// that has not even issued its request yet. This query is 1–2 requests, not a +// fan-out, so a budget of 2 keeps the total offered to the Hub bounded (4 + 2) +// while making the path the user is actually waiting on independent. +const hfSearchGate = createConcurrencyGate(2) +// Coalesce concurrent probes of the SAME repo — a repo can appear in both the +// curated catalog and the live search, and neither caches until it resolves. +const repoModelFlight = createSingleFlight() + +// Statuses that are a real, durable "this repo has no data" answer and so are +// safe to cache. Everything else non-OK (including auth denials, rate limits, +// 5xx, and 408) may change and must be retried on a later lookup — mirrors +// `resolveRegistryBody` in ollamaRegistryCatalog.js, which has always drawn this +// line. +// Authentication denials are deliberately excluded: a user can add a token or +// gain access to a gated repo at any time, so caching a 401/403 would keep the +// catalog blank until the seven-day repo-cache TTL expires. A genuinely missing +// or gone repo remains a durable no-data answer. +const HF_PERMANENT_NOT_FOUND = new Set([404, 410]) + +// Single door to the Hub: bounded concurrency + the shared one-shot retry. +// +// Returns a discriminated outcome rather than a Response, because the body must +// be consumed INSIDE the gate slot. Releasing at response headers would bound +// only the header waits while every body streamed concurrently — i.e. exactly +// the many-simultaneous-streams-on-one-pooled-connection condition that earns +// the GOAWAY this gate exists to prevent. +// +// { outcome: 'ok', data } — 2xx with a parseable body +// { outcome: 'permanent', status, errorText } — a durable no-data answer; cacheable +// { outcome: 'transient', status, errorText } — a bad moment; MUST NOT be cached +// +// Throws only when both attempts failed at the connection level. +function hfFetch(url) { + return hfGate.run(async () => { + const res = await fetchWithTimeout( + url, + { headers: await hfHeaders() }, + HF_TIMEOUT_MS, + { retries: 1, retryDelayMs: HF_RETRY_DELAY_MS, shouldRetry: isReplayableConnectionError } + // Both attempts lost the connection. undici's own message is a bare `fetch + // failed`, which reaches the search box verbatim and reads like a bug in + // PortOS — name the actual condition so the user knows to just try again. + ).catch((err) => { + if (!isReplayableConnectionError(err)) throw err + throw new Error(`Hugging Face is not responding (connection dropped twice) — try again in a moment. [${describeFetchError(err)}]`) + }) + if (!res.ok) { + const errorText = await res.text().catch(() => '') + const outcome = HF_PERMANENT_NOT_FOUND.has(res.status) ? 'permanent' : 'transient' + return { outcome, status: res.status, errorText } + } + // A 200 whose body won't parse is a proxy/captive-portal error page, not an + // answer about the repo — transient, so it is never cached as "no data". + const data = await readResponseJson(res, { fallback: null }) + return data == null + ? { outcome: 'transient', status: res.status, errorText: 'unparseable response body' } + : { outcome: 'ok', data } + }) +} + +// `filter` is a Hugging Face library tag — 'gguf' for the GGUF query, 'mlx' for +// the Apple-MLX query, or null/'' to relax the format filter (audio category and +// the GGUF-signal fallback). Only one filter at a time; MLX runs as a separate +// query so its results don't pollute the GGUF list. +export async function fetchModels(search, limit, filter) { + const params = new URLSearchParams({ + search, + sort: 'downloads', + direction: '-1', + limit: String(limit), + full: 'true' + }) + if (filter) params.set('filter', filter) + + const result = await hfSearchGate.run(() => hfFetch(`${HF_API_BASE}?${params.toString()}`)) + if (result.outcome !== 'ok') { + const detail = result.errorText ? ` — ${result.errorText.slice(0, 160)}` : '' + throw new Error(`Hugging Face search failed: ${result.status}${detail}`) + } + return Array.isArray(result.data) ? result.data : [] +} + +const repoModelCache = new Map() +const REPO_MODEL_CACHE_MAX = 500 +// A connection-level failure (network down / both retry attempts dropped) — +// distinct both from a durable no-data answer (401/403/404/410, cacheable) and +// from a transient HTTP status (429/5xx), which `hfFetch` reports as +// `outcome: 'transient'`. Neither transient form may be cached: doing so would +// disable enrichment for the repo until the TTL expires, a week later. +const TRANSIENT_FETCH = Symbol('transient-fetch') + +// Fetch (and cache) the per-model record WITH per-file sizes. The search +// endpoint returns siblings without sizes; only `?blobs=true` carries them. +// +// Three tiers, cheapest first: an in-process Map, then the disk cache +// (huggingFaceRepoCache.js), then the Hub. The disk tier is what stops the +// curated catalog — a KNOWN, fixed list of ~36 repos — from re-asking the Hub +// for all of them after every restart, self-update, or dev reload. Steady state +// on that path is zero network. +// +// `null` = fetched-but-unavailable, and it is cached at both tiers (per the +// absent-vs-empty sentinel rule) so a sizeless repo isn't re-probed every search +// — but ONLY when the Hub gave a durable answer (404/410). An auth denial, rate +// limit, 5xx, or dropped connection also returns null and is NOT cached, so a +// newly authorized or recovered Hub re-enriches on the next request instead of +// staying blank for the week the disk TTL would otherwise hold it. +export async function fetchRepoModel(repoId) { + if (repoModelCache.has(repoId)) return repoModelCache.get(repoId) + return repoModelFlight.run(repoId, async () => { + const cached = await readCachedRepoModel(repoId) + // `hit` is separate from the value because a cached `model` of null is a + // real answer (gated/absent), not a miss. + if (cached.hit) { + rememberRepoModel(repoId, cached.model) + return cached.model + } + // repoId comes from the HF search response (untrusted upstream) — encode each + // path segment so a `?`/`#`/`..` in the id can't reshape the request path/query. + const safeRepoPath = String(repoId).split('/').map(encodeURIComponent).join('/') + const result = await hfFetch(`${HF_API_BASE}/${safeRepoPath}?blobs=true`) + .catch(() => TRANSIENT_FETCH) + // Transient — a rate limit, a 5xx, or a dropped connection. Return null so + // this load degrades gracefully, but do NOT cache it: persisting a transient + // as "no data" would bake a bad moment into the disk tier for the full TTL, + // and a restart would no longer clear it the way the old memory-only cache did. + if (result === TRANSIENT_FETCH || result.outcome === 'transient') return null + // 'permanent' (gone) IS a real answer — cache the null. Auth denials are + // transient because the user's credentials or repository access can change. + const model = result.outcome === 'ok' ? result.data : null + rememberRepoModel(repoId, model) + await writeCachedRepoModel(repoId, model) + return model + }) +} + +function rememberRepoModel(repoId, model) { + // Evict oldest entry when the cap is reached (insertion-order iteration). + if (repoModelCache.size >= REPO_MODEL_CACHE_MAX) { + repoModelCache.delete(repoModelCache.keys().next().value) + } + repoModelCache.set(repoId, model) +} + +// Publish dates for a set of Hugging Face repos, as `{ repoId: createdAt|null }`. +// +// For lists whose rows come from somewhere OTHER than the Hub's own search — the +// MTPLX discover listing, which carries downloads and license but no dates — so +// the card can say how old a checkpoint is. Reuses fetchRepoModel's three tiers +// (memory → disk → Hub) and its gate, so a repeated search is free and a burst +// stays inside the same concurrency budget as everything else here. +// +// Never throws and never fails the caller's list: a repo the Hub has no answer +// for (gated, renamed, offline) resolves to `null`, which the UI renders as a +// missing age rather than an error. +export async function fetchRepoPublishedDates(repoIds = [], { timeoutMs = PUBLISH_DATE_BUDGET_MS } = {}) { + // Cap the fan-out independently of the caller's page size. The MTPLX search + // endpoint accepts limit=100, and every unresolved probe keeps draining through + // the shared hfGate after the response returns — starving curated-catalog and + // HF-search enrichment on a degraded Hub for as long as it takes. A page of + // ages beyond the first two dozen rows is not worth that. + const unique = [...new Set(repoIds.filter((id) => typeof id === 'string' && id.includes('/')))] + .slice(0, MAX_PUBLISH_DATE_PROBES) + // Seeded with nulls and filled in place, so the budget below can return early + // with a partial answer instead of an empty one. + const dates = Object.fromEntries(unique.map((repo) => [repo, null])) + const work = Promise.allSettled(unique.map(async (repo) => { + const model = await fetchRepoModel(repo) + dates[repo] = model?.createdAt || model?.created_at || null + })) + // Bound the wait the way enrichCatalogWithVariants does, so an unreachable Hub + // can never hang a search. Whatever resolved in time is already in `dates`; the + // rest stay null and the card simply omits that row's age. Note a TRANSIENT + // failure caches nothing (see fetchRepoModel), so those repos are re-probed on + // the next search rather than being remembered as dateless. + if (timeoutMs > 0) { + let timer + const budget = new Promise((resolve) => { timer = setTimeout(resolve, timeoutMs); timer.unref?.() }) + await Promise.race([work.finally(() => clearTimeout(timer)), budget]) + } else { + await work + } + return dates +} diff --git a/server/services/mtplxModelManager.js b/server/services/mtplxModelManager.js index fddb0104f3..706517b6bd 100644 --- a/server/services/mtplxModelManager.js +++ b/server/services/mtplxModelManager.js @@ -28,7 +28,7 @@ import { safeJSONParse } from '../lib/fileUtils.js'; import { getHfCacheRoot } from '../lib/hfCache.js'; import { listMtplxCachedModels } from '../lib/mtplxModels.js'; import { findCommandOnPath } from '../lib/processEnv.js'; -import { fetchRepoPublishedDates } from './huggingFaceCatalog.js'; +import { fetchRepoPublishedDates } from './huggingFaceMetadata.js'; import { runStreamingCommand } from '../lib/streamingSpawn.js'; /** A Hugging Face search is one API call — short, and worth failing fast. */ diff --git a/server/services/mtplxModelManager.test.js b/server/services/mtplxModelManager.test.js index 94226aa6a4..d631a67419 100644 --- a/server/services/mtplxModelManager.test.js +++ b/server/services/mtplxModelManager.test.js @@ -10,7 +10,7 @@ import * as bufferedSpawnModule from '../lib/bufferedSpawn.js'; import * as mtplxModels from '../lib/mtplxModels.js'; import * as processEnv from '../lib/processEnv.js'; import * as streamingSpawn from '../lib/streamingSpawn.js'; -import * as hfCatalog from './huggingFaceCatalog.js'; +import * as hfMetadata from './huggingFaceMetadata.js'; import * as huggingfaceLora from '../lib/huggingfaceLora.js'; import * as hfToken from './hfToken.js'; @@ -24,7 +24,7 @@ describe('mtplxModelManager', () => { vi.spyOn(processEnv, 'findCommandOnPath').mockReturnValue(BINARY); vi.spyOn(mtplxModels, 'listMtplxCachedModels').mockResolvedValue({ models: [], error: null }); // Publish dates come from the Hub — no suite may reach it. - vi.spyOn(hfCatalog, 'fetchRepoPublishedDates').mockResolvedValue({}); + vi.spyOn(hfMetadata, 'fetchRepoPublishedDates').mockResolvedValue({}); vi.spyOn(hfToken, 'getHfToken').mockResolvedValue(null); vi.spyOn(huggingfaceLora, 'fetchHuggingfaceModel').mockResolvedValue({ usedStorage: 0 }); vi.spyOn(console, 'log').mockImplementation(() => {}); @@ -86,14 +86,14 @@ describe('mtplxModelManager', () => { { repo: 'Example/Qwen-MTP', downloads: 42 }, { repo: 'Example/Unlisted', downloads: 1 }, ]))); - hfCatalog.fetchRepoPublishedDates.mockResolvedValue({ 'Example/Qwen-MTP': '2026-01-02T00:00:00.000Z' }); + hfMetadata.fetchRepoPublishedDates.mockResolvedValue({ 'Example/Qwen-MTP': '2026-01-02T00:00:00.000Z' }); const { models } = await searchMtplxCatalog({}); expect(models[0].publishedAt).toBe('2026-01-02T00:00:00.000Z'); // A repo the Hub has no answer for still lists — the card just omits the age. expect(models[1].publishedAt).toBeNull(); - expect(hfCatalog.fetchRepoPublishedDates).toHaveBeenCalledWith(['Example/Qwen-MTP', 'Example/Unlisted']); + expect(hfMetadata.fetchRepoPublishedDates).toHaveBeenCalledWith(['Example/Qwen-MTP', 'Example/Unlisted']); }); it('refuses before spawning when MTPLX is not installed', async () => {