Phase 2A: define opportunity discovery architecture - #5
Merged
Conversation
Record the architecture decisions Phase 2A implementation depends on, before any schema, adapter, or UI code is written. - ADR-016: TypeScript for opportunity ingestion and deterministic scoring, superseding ADR-003 for that scope only. Phase 1B resume parsing and Phase 6 market aggregation remain open, separately-decided questions. - ADR-017: local-first ingestion execution (an explicit, owner-run CLI, not a GitHub Actions schedule), because no hosted Supabase project exists yet and a cloud runner cannot reach the loopback-only local database. Records the later migration path (Supabase Edge Function, then Cloudflare Worker) and its trigger conditions, none of which are provisioned now. - ADR-018: manually entered opportunities are private, user-owned rows in a new `private_opportunities` table, structurally separate from the shared, service-role-written opportunity-identity domain -- not a nullable-owner column on `opportunities`. Retires the informal "manual source" sketch from earlier planning in favor of the smallest-blast-radius RLS shape. - Cross-reference notes added to ADR-002, ADR-006, and ADR-010 recording how each relates to the new decisions, without altering their own accepted or proposed decisions. - ARCHITECTURE.md, DATA_MODEL.md, RLS_POLICY_MATRIX.md, SECURITY_AND_PRIVACY.md, TESTING_STRATEGY.md, USER_WORKFLOWS.md, DEVELOPMENT_ROADMAP.md, DEPLOYMENT_STRATEGY.md, COST_MODEL.md, OBSERVABILITY.md, MVP_SCOPE.md, and DATA_FLOW.md updated so no diagram or prose still presents scheduled GitHub Actions Python ingestion as active or guaranteed; corrected fragilities (e.g. the Supabase pause-mitigation assumption) are called out honestly rather than left stale. - New docs/INGESTION_ARCHITECTURE.md: the adapter/orchestrator interface, identity/dedup and freshness/closure rules, sanitization, the first-adapter choice (Greenhouse), the SSRF-adjacent manual-import boundary, and the binding source-onboarding checklist. - Phase 2A PR sequence recorded in DEVELOPMENT_ROADMAP.md: 2A.0 (this PR, docs only) -> 2A.1 (foundation) -> 2A.2 (Greenhouse adapter + local ingestion, the minimum-useful milestone) -> 2A.3 (explainable ECE market relevance). No schema, dependency, adapter, HTTP request, or runtime code is introduced by this commit.
Rewrite the source-compliance registry as a dated (2026-08-03), evidence- labeled matrix, verified against official documentation and single, manual, low-volume requests during this research pass -- not assumed from technical reachability. - Greenhouse Job Board API: implement first -- unauthenticated, robots.txt- permitted, officially documented as intended for career-site/job-board reuse, complete-response semantics, strongest observed timestamp coverage. - Lever Postings API: candidate for later -- cleaner documented third-party- consumption basis than Greenhouse, but no relevant German/EU employer board found live and weaker timestamp fields. - Workable XML feed: rejected for the first adapter on measured cost (644 MB uncompressed / 132,490 records / no server-side filtering, measured this pass), not on compliance grounds -- it is officially documented as intended for exactly this kind of reuse. - SmartRecruiters Posting API: blocked pending clarification -- a documented public API answers unauthenticated requests in practice (Bosch 4,714 postings, Continental 890), but the API host's own robots.txt disallows every user agent except one named crawler. This is the single highest- coverage source found and is explicitly not implemented until written clarification, an appropriate credentialed relationship, or a robots.txt change resolves the conflict. - Personio XML feeds: candidate for later, pending a proper terms review. - Workday's undocumented internal /wday/cxs/ endpoint: do not use -- public accessibility of an internal SPA endpoint is not a documented API. - Bundesagentur fuer Arbeit Jobsuche: do not use -- no official API exists, and the BA's own terms of use (Sec 2a(3)) explicitly prohibit the read-out this would perform; recorded with the exact quoted clause so a future contributor finding the community API wrapper does not mistake technical availability for permission. Make it in Germany inherits the same restriction as a BA-sourced derivative. - EURES: licensed/partner-only (the public API endpoint returns 401; the documented APIs are input-only, for national employment services). EURAXESS and European Job Days: manual import only (no documented API/RSS). - Company career sites for the full target-employer watchlist (Airbus, Siemens, NXP, Texas Instruments, OHB, Rohde & Schwarz, Bosch, Infineon, Mercedes-Benz, Deutsche Bahn, Deutsche Telekom, BMW, Beckhoff, Phoenix Contact, Rheinmetall, Hensoldt, Fraunhofer, DLR, and Bremen/Hamburg university career pages): each investigated and recorded per-employer, every custom or SuccessFactors site routed to manual import only. - Honest coverage caveat recorded explicitly: the confirmed Greenhouse German boards skew Munich/Berlin deep-tech with very few student-level titles, so the Greenhouse adapter alone does not deliver Bremen/Hamburg student-role coverage -- manual import (Workflow 9) is what closes that gap, which is why both ship together rather than the adapter alone. - OPEN_QUESTIONS.md updated: the "which recruitment system does each target employer use" question is closed with the findings above; the ingestion runtime/execution-model/manual-ownership questions are closed by ADR-016/ 017/018 (see the prior commit); newly opened questions recorded explicitly (SmartRecruiters written clarification, which reviewed Greenhouse boards enter the first configuration, raw-payload/snapshot retention limits, the hosted execution platform choice after local validation, whether Workable's feed cost is ever justified). No adapter, dependency, HTTP source request beyond this pass's own read-only verification, or stored listing is introduced by this commit.
Markdown link-validation caught a stray placeholder link, [DATA_SOURCES_AND_COMPLIANCE.md](#), pointing at an empty fragment inside the file itself. Replaced with plain text.
Records a personal-first implementation policy (MVP_SCOPE.md): until the owner validates real-world usefulness, prioritize working product features over preparation for hypothetical future users, with an explicit list of essential safeguards that remain non-negotiable regardless of product stage. Combines the Phase 2A.1 (opportunity foundation) and Phase 2A.2 (Greenhouse adapter) milestones into a single Phase 2A.1 "Personal Opportunity MVP" — a feature is not useful merely because tables and empty screens exist. Renumbers the ECE-relevance milestone to Phase 2A.2 and updates every cross-reference across the docs set. Makes application tracking and interview-preparation notes part of that first milestone's exit condition instead of a later phase. Applications now pin the exact listing content they refer to for both shared opportunities (via the existing saved_opportunity_version_id) and private manually entered ones (via a new, immutable private_opportunity_snapshot_jsonb captured once at creation) so a later edit never rewrites what was actually applied to. Expands the application status enum and adds next-action/notes fields sized for a single-owner MVP rather than a full event-sourced history. Adds a reduced, explicit testing scope for Phase 2A.1-sized feature work (TESTING_STRATEGY.md §10): essential per-table RLS/API/frontend checks remain required, but exhaustive privilege-permutation matrices, admin-role testing, and abuse-prevention testing are deferred until they would actually apply. Source-compliance conclusions (Greenhouse first, Lever later, SmartRecruiters blocked, BA excluded, manual-only sources, no live network dependency in CI) are unchanged.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Documentation-and-architecture PR only. Records the decisions Phase 2A (Opportunity Discovery MVP)
implementation depends on, before any schema, adapter, or UI code is written. No database
migration, dependency, adapter, HTTP source request, ingestion run, stored listing, UI feature,
GitHub setting, or cloud resource is introduced by this PR.
Decisions changed / recorded
ADR-003 for that scope only. The realistic Phase
2A workload (fetch JSON, validate, normalize, sanitize, hash, upsert) doesn't need Python's
data/ML ecosystem, and the two realistic future hosted execution targets (Supabase Edge
Functions, Cloudflare Workers) both run TypeScript natively. Phase 1B resume parsing and Phase 6
market aggregation remain open, separately-decided questions — this ADR does not resolve them.
GitHub Actions schedule. Decisive reason: no hosted Supabase project exists yet, so a cloud
runner has nothing reachable to write to. Records the later migration path (Supabase Edge
Function → Cloudflare Worker → last-resort scheduled container) and its trigger conditions, none
provisioned now.
private_opportunitiestable — structurally separate from the shared, service-role-writtenopportunity-identity domain, not a nullable-owner column on
opportunities. This resolves theopen question the prior planning pass left unsettled, and retires the informal "manual source /
import_manual_opportunityRPC" sketch from that pass in favor of the smallest-blast-radius RLSshape (matches the existing
work_experiencetemplate exactly).new decisions, without altering their own Accepted/Proposed decisions.
ARCHITECTURE.md,DATA_MODEL.md,RLS_POLICY_MATRIX.md,SECURITY_AND_PRIVACY.md,TESTING_STRATEGY.md,USER_WORKFLOWS.md,DEVELOPMENT_ROADMAP.md,DEPLOYMENT_STRATEGY.md,COST_MODEL.md,OBSERVABILITY.md,MVP_SCOPE.md,DATA_FLOW.mdupdated so no diagram or prosestill presents scheduled GitHub Actions Python ingestion as active or guaranteed. One real
fragility is called out more honestly than before: the Supabase free-tier pause mitigation
previously assumed scheduled jobs would keep the project warm — under manual local execution
that mitigation doesn't apply, so this is currently a bigger risk, not a smaller one.
docs/INGESTION_ARCHITECTURE.md: adapter/orchestrator interface, identity/dedup andconservative freshness/closure rules, sanitization, the first-adapter choice (Greenhouse) and why,
the manual-import/SSRF boundary, and the binding source-onboarding checklist.
docs/DATA_SOURCES_AND_COMPLIANCE.md: dated (2026-08-03), evidence-labeled source matrix— Greenhouse (implement first), Lever/Personio (candidates for later), Workable (rejected on
measured 108 MB/refresh bandwidth, not compliance), SmartRecruiters (blocked pending
clarification — documented API vs. a
robots.txtdisallowing all agents but one), Bundesagenturfür Arbeit (do not use — its own terms of use explicitly prohibit this), Make it in Germany
(inherits that restriction), Workday's undocumented internal endpoint (do not use), EURES
(licensed/partner-only), EURAXESS (manual import only), and every researched target-employer
career site.
Correction: personal-first implementation priority
A follow-up commit (
docs: prioritize the personal opportunity MVP) records an explicitproduct-priority correction from the repository owner: CareerOS is currently personal-first,
single-owner, and local-first, and the immediate priority is a genuinely useful working
opportunity-discovery and application-tracking system — not extensive preparation for hypothetical
future users.
MVP_SCOPE.md): until the owner validates real-world usefulness,prioritize working product features over speculative multi-user/enterprise concerns (no admin
roles, teams, orgs, public registration, abuse prevention, or quotas at this stage). A practical
effort guideline is recorded (~70% product functionality, ~20% correctness/tests, ≤10% essential
security/auth work absent a concrete blocker). Explicitly not a relaxation of secret handling
or personal-data privacy — the safeguards that were already non-negotiable stay non-negotiable.
DEVELOPMENT_ROADMAP.md): the previous 2A.1(opportunity-foundation-only, empty screens) / 2A.2 (Greenhouse adapter) split is combined into a
single Phase 2A.1 — Personal Opportunity MVP, on the reasoning that a feature isn't useful
merely because tables and empty screens exist. The former 2A.3 (explainable ECE relevance) is
renumbered to 2A.2; every cross-reference across the docs set and ADR-016 is updated
accordingly.
phase (the old, now-redundant "Phase 4 — Save, track, act" roadmap phase is marked absorbed, not
deleted, so history stays traceable).
DATA_MODEL.md'sapplicationstable gains: an expandedstatus enum (
preparing → applied → awaiting_response → interview_scheduled → interview_complete → offer → accepted / rejected / withdrawn / closed),status_updated_at,optional
next_action/next_action_due_at,notes, andcontact_note— current-status-plus-timestamps, deliberately not a full event-sourced history.
applied-to content was already pinned via the existing
saved_opportunities .saved_opportunity_version_id→ immutableopportunity_versions. A private, manually enteredopportunity had no equivalent guarantee —
private_opportunitiesis ordinary mutable CRUD, notversioned — so
applicationsgainsprivate_opportunity_idplus aprivate_opportunity_snapshot_jsonbcaptured once at application-creation time; a later edit to the private entry (fixing a typo,
updating a copied deadline) never rewrites what the application and interview-prep notes refer
to. Exactly one of
saved_opportunity_id/private_opportunity_idis set per row. This is atargeted addition to
DATA_MODEL.md/USER_WORKFLOWS.md, not a broader schema redesign.TESTING_STRATEGY.md§10,appended so existing anchors referenced elsewhere in the docs are not disturbed): essential
per-table RLS/API/frontend checks remain required (owner CRUD, one representative cross-user
denial, browser-cannot-write-shared-records, version/snapshot pinning, real primary workflows).
Exhaustive privilege-permutation matrices, admin-role testing, public-registration abuse testing,
and multi-tenant isolation testing are explicitly deferred until they would actually apply.
Privileged functions and new external-content trust boundaries still get focused tests when
actually introduced — this narrows breadth for ordinary CRUD, not depth where real risk exists.
SmartRecruiters blocked, BA/Make-it-in-Germany excluded, manual-import-only sources, no live
network dependency in CI) are unchanged — no further source research was performed in this
correction.
Explicit absence of runtime implementation
No SQL migration, no
ingestion/code, no HTTP request beyond this research pass's own read-onlyverification, no manual-import RPC, no opportunity UI, no search, no ranking, no stored listing, no
cloud resource.
git diff --statagainstmaintouches only files underdocs/.Test plan
git status/git diff --checkclean; no whitespace errors(verified with a corrected slugifier after an initial false-positive; one real bug found and
fixed — a stray
(#)placeholder self-link)docs/adr/README.md) lists all 18 ADRs with consistent status/title/number;unaffected by the corrective commit (no ADR added or resolved by it)
mmdc(mermaid-cli 11.16.0); 4 rendercleanly. The 5th (
DATA_FLOW.md's main sequence diagram) fails to parse — confirmed pre-existingon
mainat5c3f20c, unrelated to this PR's edits (a semicolon inside a message label). Notfixed here (out of scope); flagged separately.
.github/workflows/ci.ymlparses as valid YAML; unchanged by this PR; all 5uses:actionsremain SHA-pinned
node_modules/dist/.envin the diffformat:check,lint,typecheck,buildall clean.test: 52/53 passing locally — the onefailure was a 5s test-timeout in
SignInPage.test.tsxthat reproduced against two differenttests across repeated local runs (a local-machine timing flake, not a deterministic failure tied
to any change in this PR, which touches no
app/source); GitHub Actions'frontendjob isauthoritative and is tracked below.
frontend/databasejobs) — both passed after the initial two commits;re-verified after the corrective commit