feat(#1865): ingest observer /neighbors as confirmed scope evidence, and surface it on Reach

[efiten]

Closes #1865. Carries @SaarMesh-Bot's #1867 forward, adds the syntax fix that thread agreed on but never landed, and puts the result on the Reach report.

Please merge this with a merge commit, not a squash

Two of the five commits are @SaarMesh-Bot's, with their authorship intact. A squash rewrites both to me and erases the credit for the implementation, which is the larger half of this PR. This is the one case where our usual squash is the wrong button.

Why this was sitting closed

@SaarMesh-Bot opened #1867 on 2026-07-26. @dborup cherry-picked it onto master the same day, ran it against real MQTT traffic on stg.meshview.dk, and confirmed it end to end. @SaarMesh-Bot then validated it a second time on 2026-08-05 on unrelated hardware, a Heltec V4 on the SaarLorLux mesh: a responded neighbor wrote *,#de,#de-sl,#scn,#saarlorlux,#de-west,#de-rp, two timeout neighbors were correctly left untouched, and the observer's own self.scopes landed on its own node.

Then #1867 was closed by its author, and #1852 followed on 2026-08-01. @dborup said why:

Went ahead and built this ourselves rather than wait on a separate PR — PR review throughput here has been pretty backed up lately.

So two contributors validated a feature on two meshes and shipped it in their own forks because upstream review had stalled. That is the backlog costing us work rather than merely delaying it, and it is worth naming.

What I did

1. Rebased #1867 onto master, 67 commits later. It was +386/-0 and hit two conflicts, both from master moving underneath it:

• cmd/server/db.go: bug(db): detectSchema() silently swallows PRAGMA failure — server runs v2 SQL against a v3 DB until restart (empty Packets, /channels/*/messages 500) #1901 replaced the inline per-table PRAGMA table_info loops with schemaColumns(ctx, q, table). Kept master's helper and set the new flag through it, rather than reinstating the old loops.
• public/nodes.js: master rewrote the transported_scopes tooltip to explain the fix(#1751): transported_scopes is shared across every node with the same 1-byte pubkey prefix #1902 one-byte-hop exclusion. Kept master's wording, appended the new row.

No behaviour from #1867 changed. Its five tests pass unmodified.

2. Fixed the scope syntax @cwichura raised. He pointed out on the thread that the OTA query returns bare region names while everything else in CoreScope carries the leading #, @dborup agreed, and then the PR closed before it landed. Without it the node page reads:

Own scope:        #be
Configured scope: be,eu

Two spellings of the same thing, and nothing can compare the columns.

Measured before choosing a rule: on a live 1425-node instance 361 of 361 default_scope values start with #. And cmd/ingestor/main.go already prefixes a missing # for hashRegions/hashChannels before deriving the key as sha256("#name"), so the # is part of the name, not decoration. normalizeScopeList applies that existing rule on write. * passes through as a wildcard, empty stays empty because "responded, no scopes" is a real answer, order and duplicates are preserved, case is untouched (folding it would retarget sha256("#name")), and it is idempotent so a reconnect re-ingesting the same report is a no-op.

Contract change, flagged rather than buried: five assertions in issue1865_test.go move from "de" to "#de" and friends. They assert the stored spelling, which is exactly what changed; the ordering and last-write-wins behaviour they exist to test is untouched. If you would rather keep the raw OTA string, revert cfd9d468 alone and the rest still stands.

3. Put it on the Reach report, which is the page operators print and hand to someone else, and which never said which region the node serves. One muted line under the subtitle, keeping the two claims apart because they are different statements:

Scope #be observed                    inferred from observed advert transport scope
Configured scope ✓ #be,#eu confirmed  read back off the node itself

Confirmed wins when both exist. An empty confirmed value renders "none configured" and does not fall back to the inferred guess: a node that answered "I have no scopes" must not be shown a contradicting guess instead. Colour is carried only by the tick, via var(--success).

Server side, reachNodeScopes does a single-row lookup gated on the schema flags, rather than widening buildNodeInfoMap's bulk SELECT that every other consumer would pay for. All three fields are omitempty, so an older schema sends nothing and the UI shows nothing.

Verification

• cmd/ingestor: the five feat: ingest observer /neighbors report as confirmed scope evidence (#1865) #1867 tests pass unmodified, plus 13 new cases pinning normalizeScopeList (wildcard, empty, idempotence, order, duplicates, case, whitespace, empty entries).
• test-frontend-helpers.js: 666 → 673 passed, 0 failed. The seven new cases assert the RENDERED markup through a new window.__meshcoreReachInternals, following map.js (a11y(map): cluster bubbles + role pills + multi-byte hash labels encode signal by color only (WCAG 1.4.1) #1356/test(#1356): assert the rendered label, not where identifiers sit in map.js #1933) instead of grepping the source. They cover absent data, inferred never getting the tick, confirmed winning, empty-confirmed not falling back, the instant living in a title attribute, and HTML-escaping of node-controlled scope strings.
• gofmt clean. eslint on the changed files reports exactly what master reports (46 problems, 19 errors, 27 warnings, all pre-existing) — this PR adds none.
• cmd/server full suite: two clean runs. An earlier run failed TestHandleNodePaths_SortByRecency_1145 and TestIssue1008_HandlerReturns503WhileSubpathIndexLoading; both pass in isolation and both passed on two subsequent full runs, so they are order-dependent flakes rather than anything this branch does. Worth a separate issue.
• cmd/ingestor full suite fails only TestWriteStatsAtomic_SymlinkAtDestIsReplaced, which fails identically on unmodified master here (Windows symlink privileges).

Not done

• No filtering or sorting by configured scope. @cwichura suggested it on the thread and called it feature creep himself; agreed, and it is a clean follow-up.
• No backfill of rows written before the syntax fix. No instance has shipped this feature, so there are none.
• Live and Nodes views take feat: ingest observer /neighbors report as confirmed scope evidence (#1865) #1867's rendering as-is.

@SaarMesh-Bot @dborup @cwichura: this is your work and your review thread. If any of the three choices above is wrong, say so and I will change it rather than merge over you.

🤖 Generated with Claude Code

https://claude.ai/code/session_01PQS3XLoPD98yu9pxdRujqg

[cwichura]

I raised the request and provided a few initial comments. All the code came from others. But reading over the description here, I think this makes sense.

[efiten]

Merging with a merge commit, not a squash, so @SaarMesh-Bot's two commits keep their authorship. A squash would rewrite the larger half of this PR to my name.

@SaarMesh-Bot @dborup: this goes in without a second reviewer, which #1922 said I would try to avoid. The reason it is going in anyway is the one @dborup gave when closing #1852: work that sits waiting for review gets rebuilt in forks instead, and this PR exists precisely because that already happened to #1867. Leaving it open longer to satisfy a process point would repeat the thing the process is meant to prevent.

What I changed in carrying it forward, so you can check my work rather than take it: two conflicts from master moving underneath, both resolved toward master (#1901's schemaColumns helper instead of the old inline PRAGMA loops, and master's transported_scopes tooltip wording from #1902); and one behaviour change I made deliberately, normalising configured_scope to the leading-# form so it is comparable with default_scope, which is the thing @cwichura raised and you agreed with before the PR closed. That is commit cfd9d468 on its own and reverting it leaves the rest standing.

If either of you thinks the normalisation is wrong, say so and I will revert that commit rather than argue about it.

[efiten]

Correction to what I wrote a moment ago: I asked for a merge commit to preserve authorship, and that is not possible here. The repository allows squash only:

merge_commit: false
squash:       true
rebase:       false

So the choice is not mine to make and my earlier note on this PR was wrong about what was available.

Squashing with Co-authored-by: trailers instead, which is the mechanism that exists for exactly this case: GitHub renders co-authors on the resulting commit and counts the contribution. @SaarMesh-Bot's two commits become co-authorship on one commit rather than two commits under their own name. That is a real loss of granularity, not an equivalent, and it is worth someone with repository settings access deciding whether squash-only is the right default for a project that takes work carried forward from closed PRs.