Repository navigation
Move the USGS earthquake source into Forge - #3
Merged
Merged
Conversation
Port Live Map's USGS provider and ingestion rules: one revisioned event object per USGS id, out-of-order corrections ignored, deleted features withdrawn from the public feed but kept in history, and source links limited to earthquake.usgs.gov. Live fetching is gated by a new source registry entry. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GcagdBzPRAWDhhc9Yipm4d
Zacharon
marked this pull request as ready for review
October 19, 2026 23:08
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Before: USGS earthquakes were collected only by Live Map. Forge's
/api/eventsfeed had no real source behind it.After:
osint-forge-usgsloads the USGS summary feed into the Forge ontology aseventobjects, and they show up in/api/events. It can read a saved feed with--file, or fetch the live feed with--live --confirm-live.How:
osint_forge/ingestion/usgs.pyports the rules from Live Map'ssrc/data/providers/usgs.jsandsrc/ingestion/usgs-pipeline.js:event:usgs-<id>.updatedtime is ignored.deletedfeature gets a withdrawn, non-public revision. It leaves the feed but stays in history.https://earthquake.usgs.gov.natural-disasterdomain match Live Map.Live collection goes through a new
usgsentry inconfig/source_registry.json. Its PII risk is low, it is live-capable, and it is off by default, so--livealone is refused.Tests run against the hand-built fixture
tests/fixtures/usgs/summary_feed.json. They cover creation, rejects, no-op reloads, corrections, out-of-order updates, withdrawal, the feed projection, invalid feeds and the CLI gate. The full suite gives OK (13 PostgreSQL tests skipped without a database). The ingest was also run against a local PostgreSQL 16. The live USGS fetch could not be tried from this environment because outbound access to earthquake.usgs.gov is blocked.Generated by Claude Code