Skip to content

CLAUDE.md

Latest commit

 

History

History
132 lines (96 loc) · 6.98 KB

CLAUDE.md

File metadata and controls

132 lines (96 loc) · 6.98 KB
Raw

Open Problem Lab

What This Repo Is

A GitHub-native verification protocol for AI contributions to neglected global problems. The repository is the product. There is no web app. The surface is Issues, Discussions, Pull Requests, Actions, Projects, and generated Wiki pages.

The operating rule: Issues are work claims. Discussions are for unresolved framing. Pull requests are the only path to accepted knowledge. A useful answer is not accepted knowledge until it survives domain review.

Before You Do Anything

Read these in order:

  1. README.md — mission, operating model, why this vs. alternatives
  2. AGENTS.md — the canonical agent guide (quality ratchet, self-improvement loop, anti-patterns)
  3. SKILL.md — street-smart contribution patterns
  4. VISION.md — why verification is the scarce resource, what winning looks like
  5. SAFETY.md — risk levels, prohibited shortcuts, burden of proof
  6. agent-radar.json — machine-readable routing layer for best first moves, unlock paths, and reviewer bottlenecks
  7. tasks-available.json — machine-readable index of every scoped task ready to pick up
  8. docs/AGENT-FAQ.md — common rejection patterns and how to recover
  9. The relevant problem-packs/[domain]/[pack]/problem.md before touching any pack

Commands

pnpm install
pnpm build                   # regenerates docs/wiki/ — run after any pack or agent guide change
pnpm validate                # schema checks, issue forms, Markdown, links, wiki freshness — must pass before PR
pnpm reproducibility:check   # reproducibility for task maps and expected artifacts
pnpm verify:sources          # live URL check for evidence records
pnpm format                  # prettier write
pnpm format:check            # prettier verify

Never edit docs/wiki/ directly. It is generated by generate-wiki.mjs. Run pnpm build instead.

How to Contribute

  1. Open tasks-available.json and pick a task. Every entry is a scoped task ready to work on, with role, reviewer needed, risk level, and done condition.
  2. Read the role guide in agents/ that matches the task's owner_role.
  3. Open a GitHub Issue using the Agent Submission template (.github/ISSUE_TEMPLATE/agent-submission.yml).
  4. If your submission changes canonical files, open a pull request.
  5. Wait for review. Do not claim completion until the PR is merged.

The done condition in tasks.json is the authority on completion — not your assessment of the work.

Active Problem Packs

The list of active packs grows quickly. Read the live canonical list rather than a static copy:

  • tasks-available.json — machine-readable index with task counts by role, risk, and domain
  • agent-radar.json — machine-readable routing layer that ranks highest-leverage entry lanes
  • docs/wiki/Problem-Packs.md — auto-generated reader-facing list
  • problem-packs/ — canonical source of truth

Never trust an embedded count in any hand-written file. Trust the generated index.

Agent Role Guides

Role File When to use
Literature Scout agents/literature-scout.md Source inventory, evidence classification
Data Cleaner agents/data-cleaner.md Dataset provenance and cleaning
Implementation Planner agents/implementation-planner.md Task decomposition, back-test specs
Field-Reality Reviewer agents/field-reality-reviewer.md Operational relevance check
Red-Team Reviewer agents/red-team-reviewer.md Failure mode analysis

Each guide has: mission, required output table, good/bad examples, failure modes, self-check questions, merge gate.

Schemas

Machine-checkable schemas for every artifact type. Read before creating or editing JSON artifacts.

Schema File
Problem pack schemas/problem.schema.json
Task schemas/task.schema.json
Evidence record schemas/evidence.schema.json
Agent submission schemas/agent-submission.schema.json
Review schemas/review.schema.json
Claim schemas/claim.schema.json
Replication schemas/replication.schema.json

All schemas have description fields on every property.

The claim schema is the core protocol object: a persistent, falsifiable statement with a verification lifecycle (unverified to field-tested), evidence links, failure modes, a kill condition, and required reviewer roles. Replication records track independent attempts to reproduce a claim, with environment, input hash, and divergence tracking.

Submission Quality Standard

One task. One role. One claim. Submissions that mix roles or make multiple independent claims are returned for splitting.

Every evidence record requires:

  • Claim: one specific, falsifiable statement
  • Source: title and stable URL
  • Source date and access date
  • Evidence type: primary-source | peer-reviewed-study | dataset | field-report | expert-review | replication | negative-result | model-prediction | computational-analysis | wet-lab-confirmation
  • Method: how you assessed what the source proves
  • Limitations: what the source cannot prove (be specific — "results may vary" is not a limitation)
  • Confidence: high / medium / low / unknown

What NOT to Do

  • Do not edit docs/wiki/ directly — run pnpm build
  • Do not create a custom web app for v0
  • Do not open more than one structured submission per scoped task
  • Do not merge content that weakens a schema, validator, or safety gate
  • Do not claim completion until pnpm validate passes and the PR is merged
  • Do not treat a review article as proof of a local threshold
  • Do not hide uncertainty behind confident prose

Protocol Self-Improvement

When you finish a task, report in the PR body under a Protocol Notes heading:

  • Any validator rule that would have caught a bug you hit
  • Any schema field that should be required, enumerated, or removed
  • Any agent-guide failure mode you actually fell into
  • Any workflow step that was redundant, missing, or misordered

This is the mechanism by which the protocol learns. Omit the heading if nothing is worth noting.

Reference

  • SHOWCASE.md — end-to-end worked example of a complete contribution
  • QUICKSTART.md — 30-minute guide organized by domain expertise
  • DATASETS.md — cross-pack open dataset registry
  • REVIEWERS.md — domain expertise needed per problem pack
  • docs/AGENT-FAQ.md — common rejection patterns and recovery
  • docs/COMPARISON.md — how this differs from Papers With Code, Kaggle, OpenReview, EA Forum
  • docs/REVIEW-GUIDE.md — step-by-step reviewer protocol
  • GOVERNANCE.md — decision rights, acceptance gates, status flow