Skip to content

Phase 2A: define opportunity discovery architecture - #5

Merged
abdo2006-dev merged 4 commits into
mainfrom
phase-2a/opportunity-architecture
Aug 3, 2026
Merged

Phase 2A: define opportunity discovery architecture#5
abdo2006-dev merged 4 commits into
mainfrom
phase-2a/opportunity-architecture

Conversation

@abdo2006-dev

@abdo2006-dev abdo2006-dev commented Aug 3, 2026

Copy link
Copy Markdown
Owner

Summary

Documentation-and-architecture PR only. Records the decisions Phase 2A (Opportunity Discovery MVP)
implementation depends on, before any schema, adapter, or UI code is written. No database
migration, dependency, adapter, HTTP source request, ingestion run, stored listing, UI feature,
GitHub setting, or cloud resource is introduced by this PR.

Decisions changed / recorded

  • ADR-016 (new): TypeScript for opportunity ingestion and deterministic scoring, supersedes
    ADR-003 for that scope only. The realistic Phase
    2A workload (fetch JSON, validate, normalize, sanitize, hash, upsert) doesn't need Python's
    data/ML ecosystem, and the two realistic future hosted execution targets (Supabase Edge
    Functions, Cloudflare Workers) both run TypeScript natively. Phase 1B resume parsing and Phase 6
    market aggregation remain open, separately-decided questions — this ADR does not resolve them.
  • ADR-017 (new): local-first ingestion execution — an explicit, owner-run local CLI, not a
    GitHub Actions schedule. Decisive reason: no hosted Supabase project exists yet, so a cloud
    runner has nothing reachable to write to. Records the later migration path (Supabase Edge
    Function → Cloudflare Worker → last-resort scheduled container) and its trigger conditions, none
    provisioned now.
  • ADR-018 (new): manually entered opportunities are private, user-owned rows in a new
    private_opportunities table — structurally separate from the shared, service-role-written
    opportunity-identity domain, not a nullable-owner column on opportunities. This resolves the
    open question the prior planning pass left unsettled, and retires the informal "manual source /
    import_manual_opportunity RPC" sketch from that pass in favor of the smallest-blast-radius RLS
    shape (matches the existing work_experience template exactly).
  • Cross-reference notes added to ADR-002, ADR-006, ADR-010 recording how each relates to the
    new decisions, without altering their own Accepted/Proposed decisions.
  • ARCHITECTURE.md, DATA_MODEL.md, RLS_POLICY_MATRIX.md, SECURITY_AND_PRIVACY.md,
    TESTING_STRATEGY.md, USER_WORKFLOWS.md, DEVELOPMENT_ROADMAP.md, DEPLOYMENT_STRATEGY.md,
    COST_MODEL.md, OBSERVABILITY.md, MVP_SCOPE.md, DATA_FLOW.md updated so no diagram or prose
    still presents scheduled GitHub Actions Python ingestion as active or guaranteed. One real
    fragility is called out more honestly than before: the Supabase free-tier pause mitigation
    previously assumed scheduled jobs would keep the project warm — under manual local execution
    that mitigation doesn't apply, so this is currently a bigger risk, not a smaller one.
  • New docs/INGESTION_ARCHITECTURE.md: adapter/orchestrator interface, identity/dedup and
    conservative freshness/closure rules, sanitization, the first-adapter choice (Greenhouse) and why,
    the manual-import/SSRF boundary, and the binding source-onboarding checklist.
  • New docs/DATA_SOURCES_AND_COMPLIANCE.md: dated (2026-08-03), evidence-labeled source matrix
    — Greenhouse (implement first), Lever/Personio (candidates for later), Workable (rejected on
    measured 108 MB/refresh bandwidth, not compliance), SmartRecruiters (blocked pending
    clarification — documented API vs. a robots.txt disallowing all agents but one), Bundesagentur
    für Arbeit (do not use — its own terms of use explicitly prohibit this), Make it in Germany
    (inherits that restriction), Workday's undocumented internal endpoint (do not use), EURES
    (licensed/partner-only), EURAXESS (manual import only), and every researched target-employer
    career site.

Correction: personal-first implementation priority

A follow-up commit (docs: prioritize the personal opportunity MVP) records an explicit
product-priority correction from the repository owner: CareerOS is currently personal-first,
single-owner, and local-first, and the immediate priority is a genuinely useful working
opportunity-discovery and application-tracking system — not extensive preparation for hypothetical
future users.

  • New personal-first policy (MVP_SCOPE.md): until the owner validates real-world usefulness,
    prioritize working product features over speculative multi-user/enterprise concerns (no admin
    roles, teams, orgs, public registration, abuse prevention, or quotas at this stage). A practical
    effort guideline is recorded (~70% product functionality, ~20% correctness/tests, ≤10% essential
    security/auth work absent a concrete blocker). Explicitly not a relaxation of secret handling
    or personal-data privacy — the safeguards that were already non-negotiable stay non-negotiable.
  • Phase 2A PR sequence revised (DEVELOPMENT_ROADMAP.md): the previous 2A.1
    (opportunity-foundation-only, empty screens) / 2A.2 (Greenhouse adapter) split is combined into a
    single Phase 2A.1 — Personal Opportunity MVP, on the reasoning that a feature isn't useful
    merely because tables and empty screens exist. The former 2A.3 (explainable ECE relevance) is
    renumbered to 2A.2; every cross-reference across the docs set and ADR-016 is updated
    accordingly.
  • Application tracking pulled into the first milestone's exit condition, not a separate later
    phase (the old, now-redundant "Phase 4 — Save, track, act" roadmap phase is marked absorbed, not
    deleted, so history stays traceable). DATA_MODEL.md's applications table gains: an expanded
    status enum (preparing → applied → awaiting_response → interview_scheduled → interview_complete → offer → accepted / rejected / withdrawn / closed), status_updated_at,
    optional next_action/next_action_due_at, notes, and contact_note — current-status-plus-
    timestamps, deliberately not a full event-sourced history.
  • Pinned listing content for both shared and private opportunities: a shared opportunity's
    applied-to content was already pinned via the existing saved_opportunities .saved_opportunity_version_id → immutable opportunity_versions. A private, manually entered
    opportunity had no equivalent guarantee — private_opportunities is ordinary mutable CRUD, not
    versioned — so applications gains private_opportunity_id plus a private_opportunity_snapshot_jsonb
    captured once at application-creation time; a later edit to the private entry (fixing a typo,
    updating a copied deadline) never rewrites what the application and interview-prep notes refer
    to. Exactly one of saved_opportunity_id / private_opportunity_id is set per row. This is a
    targeted addition to DATA_MODEL.md/USER_WORKFLOWS.md, not a broader schema redesign.
  • Reduced, explicit testing scope for Phase 2A.1-sized feature work (TESTING_STRATEGY.md §10,
    appended so existing anchors referenced elsewhere in the docs are not disturbed): essential
    per-table RLS/API/frontend checks remain required (owner CRUD, one representative cross-user
    denial, browser-cannot-write-shared-records, version/snapshot pinning, real primary workflows).
    Exhaustive privilege-permutation matrices, admin-role testing, public-registration abuse testing,
    and multi-tenant isolation testing are explicitly deferred until they would actually apply.
    Privileged functions and new external-content trust boundaries still get focused tests when
    actually introduced — this narrows breadth for ordinary CRUD, not depth where real risk exists.
  • Source-compliance conclusions (Greenhouse first, Lever/Personio later, Workable rejected,
    SmartRecruiters blocked, BA/Make-it-in-Germany excluded, manual-import-only sources, no live
    network dependency in CI) are unchanged — no further source research was performed in this
    correction.

Explicit absence of runtime implementation

No SQL migration, no ingestion/ code, no HTTP request beyond this research pass's own read-only
verification, no manual-import RPC, no opportunity UI, no search, no ranking, no stored listing, no
cloud resource. git diff --stat against main touches only files under docs/.

Test plan

  • git status/git diff --check clean; no whitespace errors
  • Relative markdown links in every changed file resolve to existing files
  • Heading-anchor fragments in every changed file resolve against GitHub's actual slug algorithm
    (verified with a corrected slugifier after an initial false-positive; one real bug found and
    fixed — a stray (#) placeholder self-link)
  • ADR index (docs/adr/README.md) lists all 18 ADRs with consistent status/title/number;
    unaffected by the corrective commit (no ADR added or resolved by it)
  • All 5 mermaid diagrams in changed files rendered with mmdc (mermaid-cli 11.16.0); 4 render
    cleanly. The 5th (DATA_FLOW.md's main sequence diagram) fails to parse — confirmed pre-existing
    on main at 5c3f20c
    , unrelated to this PR's edits (a semicolon inside a message label). Not
    fixed here (out of scope); flagged separately.
  • .github/workflows/ci.yml parses as valid YAML; unchanged by this PR; all 5 uses: actions
    remain SHA-pinned
  • Secret scan and absolute local-path scan on the diff: clean
  • Tracked-file hygiene: no node_modules/dist/.env in the diff
  • Frontend regression (unaffected by this docs-only PR, run to confirm no incidental breakage):
    format:check, lint, typecheck, build all clean. test: 52/53 passing locally — the one
    failure was a 5s test-timeout in SignInPage.test.tsx that reproduced against two different
    tests across repeated local runs (a local-machine timing flake, not a deterministic failure tied
    to any change in this PR, which touches no app/ source); GitHub Actions' frontend job is
    authoritative and is tracked below.
  • CI (GitHub Actions frontend/database jobs) — both passed after the initial two commits;
    re-verified after the corrective commit

Record the architecture decisions Phase 2A implementation depends on, before
any schema, adapter, or UI code is written.

- ADR-016: TypeScript for opportunity ingestion and deterministic scoring,
  superseding ADR-003 for that scope only. Phase 1B resume parsing and Phase 6
  market aggregation remain open, separately-decided questions.
- ADR-017: local-first ingestion execution (an explicit, owner-run CLI, not a
  GitHub Actions schedule), because no hosted Supabase project exists yet and
  a cloud runner cannot reach the loopback-only local database. Records the
  later migration path (Supabase Edge Function, then Cloudflare Worker) and
  its trigger conditions, none of which are provisioned now.
- ADR-018: manually entered opportunities are private, user-owned rows in a
  new `private_opportunities` table, structurally separate from the shared,
  service-role-written opportunity-identity domain -- not a nullable-owner
  column on `opportunities`. Retires the informal "manual source" sketch from
  earlier planning in favor of the smallest-blast-radius RLS shape.
- Cross-reference notes added to ADR-002, ADR-006, and ADR-010 recording how
  each relates to the new decisions, without altering their own accepted or
  proposed decisions.
- ARCHITECTURE.md, DATA_MODEL.md, RLS_POLICY_MATRIX.md, SECURITY_AND_PRIVACY.md,
  TESTING_STRATEGY.md, USER_WORKFLOWS.md, DEVELOPMENT_ROADMAP.md,
  DEPLOYMENT_STRATEGY.md, COST_MODEL.md, OBSERVABILITY.md, MVP_SCOPE.md, and
  DATA_FLOW.md updated so no diagram or prose still presents scheduled GitHub
  Actions Python ingestion as active or guaranteed; corrected fragilities
  (e.g. the Supabase pause-mitigation assumption) are called out honestly
  rather than left stale.
- New docs/INGESTION_ARCHITECTURE.md: the adapter/orchestrator interface,
  identity/dedup and freshness/closure rules, sanitization, the first-adapter
  choice (Greenhouse), the SSRF-adjacent manual-import boundary, and the
  binding source-onboarding checklist.
- Phase 2A PR sequence recorded in DEVELOPMENT_ROADMAP.md: 2A.0 (this PR, docs
  only) -> 2A.1 (foundation) -> 2A.2 (Greenhouse adapter + local ingestion,
  the minimum-useful milestone) -> 2A.3 (explainable ECE market relevance).

No schema, dependency, adapter, HTTP request, or runtime code is introduced by
this commit.
Rewrite the source-compliance registry as a dated (2026-08-03), evidence-
labeled matrix, verified against official documentation and single, manual,
low-volume requests during this research pass -- not assumed from technical
reachability.

- Greenhouse Job Board API: implement first -- unauthenticated, robots.txt-
  permitted, officially documented as intended for career-site/job-board
  reuse, complete-response semantics, strongest observed timestamp coverage.
- Lever Postings API: candidate for later -- cleaner documented third-party-
  consumption basis than Greenhouse, but no relevant German/EU employer board
  found live and weaker timestamp fields.
- Workable XML feed: rejected for the first adapter on measured cost (644 MB
  uncompressed / 132,490 records / no server-side filtering, measured this
  pass), not on compliance grounds -- it is officially documented as intended
  for exactly this kind of reuse.
- SmartRecruiters Posting API: blocked pending clarification -- a documented
  public API answers unauthenticated requests in practice (Bosch 4,714
  postings, Continental 890), but the API host's own robots.txt disallows
  every user agent except one named crawler. This is the single highest-
  coverage source found and is explicitly not implemented until written
  clarification, an appropriate credentialed relationship, or a robots.txt
  change resolves the conflict.
- Personio XML feeds: candidate for later, pending a proper terms review.
- Workday's undocumented internal /wday/cxs/ endpoint: do not use -- public
  accessibility of an internal SPA endpoint is not a documented API.
- Bundesagentur fuer Arbeit Jobsuche: do not use -- no official API exists,
  and the BA's own terms of use (Sec 2a(3)) explicitly prohibit the read-out
  this would perform; recorded with the exact quoted clause so a future
  contributor finding the community API wrapper does not mistake technical
  availability for permission. Make it in Germany inherits the same
  restriction as a BA-sourced derivative.
- EURES: licensed/partner-only (the public API endpoint returns 401; the
  documented APIs are input-only, for national employment services).
  EURAXESS and European Job Days: manual import only (no documented API/RSS).
- Company career sites for the full target-employer watchlist (Airbus,
  Siemens, NXP, Texas Instruments, OHB, Rohde & Schwarz, Bosch, Infineon,
  Mercedes-Benz, Deutsche Bahn, Deutsche Telekom, BMW, Beckhoff, Phoenix
  Contact, Rheinmetall, Hensoldt, Fraunhofer, DLR, and Bremen/Hamburg
  university career pages): each investigated and recorded per-employer,
  every custom or SuccessFactors site routed to manual import only.
- Honest coverage caveat recorded explicitly: the confirmed Greenhouse German
  boards skew Munich/Berlin deep-tech with very few student-level titles, so
  the Greenhouse adapter alone does not deliver Bremen/Hamburg student-role
  coverage -- manual import (Workflow 9) is what closes that gap, which is
  why both ship together rather than the adapter alone.
- OPEN_QUESTIONS.md updated: the "which recruitment system does each target
  employer use" question is closed with the findings above; the ingestion
  runtime/execution-model/manual-ownership questions are closed by ADR-016/
  017/018 (see the prior commit); newly opened questions recorded explicitly
  (SmartRecruiters written clarification, which reviewed Greenhouse boards
  enter the first configuration, raw-payload/snapshot retention limits, the
  hosted execution platform choice after local validation, whether Workable's
  feed cost is ever justified).

No adapter, dependency, HTTP source request beyond this pass's own read-only
verification, or stored listing is introduced by this commit.
Markdown link-validation caught a stray placeholder link, [DATA_SOURCES_AND_COMPLIANCE.md](#),
pointing at an empty fragment inside the file itself. Replaced with plain text.
Records a personal-first implementation policy (MVP_SCOPE.md): until
the owner validates real-world usefulness, prioritize working product
features over preparation for hypothetical future users, with an
explicit list of essential safeguards that remain non-negotiable
regardless of product stage.

Combines the Phase 2A.1 (opportunity foundation) and Phase 2A.2
(Greenhouse adapter) milestones into a single Phase 2A.1 "Personal
Opportunity MVP" — a feature is not useful merely because tables and
empty screens exist. Renumbers the ECE-relevance milestone to
Phase 2A.2 and updates every cross-reference across the docs set.

Makes application tracking and interview-preparation notes part of
that first milestone's exit condition instead of a later phase.
Applications now pin the exact listing content they refer to for both
shared opportunities (via the existing saved_opportunity_version_id)
and private manually entered ones (via a new, immutable
private_opportunity_snapshot_jsonb captured once at creation) so a
later edit never rewrites what was actually applied to. Expands the
application status enum and adds next-action/notes fields sized for a
single-owner MVP rather than a full event-sourced history.

Adds a reduced, explicit testing scope for Phase 2A.1-sized feature
work (TESTING_STRATEGY.md §10): essential per-table RLS/API/frontend
checks remain required, but exhaustive privilege-permutation matrices,
admin-role testing, and abuse-prevention testing are deferred until
they would actually apply.

Source-compliance conclusions (Greenhouse first, Lever later,
SmartRecruiters blocked, BA excluded, manual-only sources, no live
network dependency in CI) are unchanged.
@abdo2006-dev
abdo2006-dev merged commit 658e4af into main Aug 3, 2026
2 checks passed
@abdo2006-dev
abdo2006-dev deleted the phase-2a/opportunity-architecture branch August 3, 2026 19:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant