You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit bb7fc34
Browse filesBrowse the repository at this point in the historyBrowse files
Copy file name to clipboardExpand all lines: AGENTS.md
+86-11Lines changed: 86 additions & 11 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,34 +1,104 @@
1
-
# Open Problem Lab
1
+
# Open Problem Lab — Agent Guide
2
2
3
3
## Overview
4
4
5
-
This repository is a GitHub-native protocol for verified AI contributions to neglected global problems. Do not turn v0 into a web app. The product surface is Issues, Discussions, Pull Requests, Actions, Projects, generated Wiki pages, schemas, and problem-pack files.
5
+
This repository is a GitHub-native protocol for verified contributions to neglected global problems. The product surface is Issues, Discussions, Pull Requests, Actions, Projects, generated Wiki pages, schemas, and problem-pack files. There is no chat UI, no app shell, no agent loop that bypasses review. The repository is the product.
6
+
7
+
If a change does not survive `pnpm validate`, replication, and human review, it is not knowledge.
8
+
9
+
## Why Strong Contributors Should Work Here
10
+
11
+
Most public AI output is unverifiable, unranked, and unowned. This repository inverts that:
12
+
13
+
- Every accepted claim is attached to a named submitter, a method, a reviewer, and where required a replication record.
14
+
- The contribution ledger is `git log` and the merged PR history. There is no separate scoreboard, and there should not be one.
15
+
- Reputation is earned by accurate rejections, reproducible approvals, and limitation-setting — not by volume.
16
+
- The bar moves up only. A merge that weakens a schema, validator, or safety gate is a defect, even if the prose is good.
17
+
18
+
Correct, narrow, and falsifiable beats prolific.
6
19
7
20
## Key Components
8
21
9
22
-`problem-packs/`: canonical problem files and task maps.
10
23
-`schemas/`: JSON schemas for machine-checkable protocol objects.
11
-
-`.github/ISSUE_TEMPLATE/`: five v0 issue forms.
12
-
-`.github/workflows/`: validationand Wiki publishing automation.
-`.github/workflows/`: validation, source verification, reproducibility, and Wiki publishing.
13
26
-`agents/`: role guides for structured agent contributions.
14
-
-`scripts/`: deterministic validation and Wiki generation.
15
-
-`docs/wiki/`: generated Wiki source. Do not edit generated pages by hand.
27
+
-`scripts/`: deterministic validation, source verification, reproducibility, and Wiki generation.
28
+
-`docs/wiki/`: generated Wiki source. Do not hand-edit.
16
29
17
30
## Agent Working Rules
18
31
19
32
1. Read `README.md`, `GOVERNANCE.md`, `SAFETY.md`, and the relevant problem pack before changing files.
20
33
2. Keep Issues as work claims and Discussions as unresolved framing.
21
-
3. Accepted knowledge must enter through pull requests.
34
+
3. Accepted knowledge enters only through pull requests.
22
35
4. Run `pnpm build` after changing problem packs or agent guides.
23
36
5. Run `pnpm validate` before claiming completion.
24
37
6. Run `pnpm reproducibility:check` after changing task maps or expected artifacts.
25
38
7. Run `pnpm verify:sources` after changing evidence URLs.
26
39
8. Prefer schema changes over prose rules when a requirement must be machine-checkable.
27
40
9. Do not create a custom web app unless a measured GitHub-native bottleneck justifies it.
41
+
10. Do not open more than one structured submission per scoped task without a maintainer request.
42
+
43
+
## The Self-Improvement Loop
44
+
45
+
The protocol is itself a contribution surface. Every merged PR must, at minimum, leave one of these strictly better than before:
46
+
47
+
- A schema (more checkable).
48
+
- A validator (more failures caught earlier).
49
+
- A problem pack (more accurate, more narrowly scoped, better evidence).
50
+
- An agent guide (clearer failure modes, sharper merge gate).
51
+
- A workflow (faster, more reproducible, less flaky).
52
+
53
+
If a PR adds knowledge but degrades any of the above, the reviewer should request a fix or split. The protocol ratchets upward; it does not regress.
54
+
55
+
### How Agents Surface Improvements
56
+
57
+
When an agent finishes a scoped task, it should also report — in the PR body, under a `Protocol Notes` heading — any of the following it observed:
58
+
59
+
- A validator rule that would have caught a bug it hit.
60
+
- A schema field that should be required, enumerated, or removed.
61
+
- An agent-guide failure mode it actually fell into.
62
+
- A workflow step that was redundant, missing, or misordered.
63
+
64
+
These notes do not require a separate issue. A reviewer may convert them into a follow-up scoped issue, a schema PR, or a closed `wontfix` with a stated reason. This is the primary mechanism by which the repository learns.
65
+
66
+
## Quality Ratchet
67
+
68
+
A contribution may only be merged if at least one of these is true:
69
+
70
+
1. It adds verified knowledge under existing schemas and gates.
71
+
2. It strengthens a schema, validator, agent guide, or workflow.
72
+
3. It removes a defect (false claim, broken source, unsound method, unsafe shortcut).
73
+
74
+
Pure prose polish without one of the above is not a reason to merge.
75
+
76
+
## Roles for Top AI Agents
77
+
78
+
Strong models should pick a role and stay in it for the duration of a PR. Mixing roles in one submission hides which judgment failed.
Use this skill when working inside the Open Problem Lab repository.
3
+
Use this skill when working inside the Open Problem Lab repository. It complements `AGENTS.md` with the working judgment an agent needs to ship a contribution that actually merges.
4
4
5
5
## Start Here
6
6
7
7
1. Read `README.md`.
8
8
2. Read `GOVERNANCE.md`.
9
9
3. Read `SAFETY.md`.
10
-
4. Read the relevant directory under `problem-packs/`.
11
-
5. Inspect `schemas/` before changing any machine-readable artifact.
10
+
4. Read the role guide in `agents/` that matches your task.
11
+
5. Read the relevant directory under `problem-packs/`.
12
+
6. Inspect `schemas/` before changing any machine-readable artifact.
13
+
14
+
If any of those is unread, the rest of this skill does not apply yet.
12
15
13
16
## Core Judgment
14
17
15
-
The repository is the product. Do not add a custom web app for v0. Use GitHub-native primitives unless there is measured evidence that the GitHub workflow has become the bottleneck.
18
+
The repository is the product. There is no app in v0. GitHub Issues, Discussions, Pull Requests, Actions, Projects, and the generated Wiki are the entire surface. Do not propose a custom UI unless `pnpm validate`, review latency, or replication friction is the measured bottleneck — not a guessed one.
16
19
17
20
## Commands
18
21
22
+
These are the only commands an agent should rely on as ground truth. They are defined in `package.json`.
23
+
19
24
```bash
20
25
pnpm install
21
-
pnpm build
22
-
pnpm validate
23
-
pnpm reproducibility:check
24
-
pnpm verify:sources
26
+
pnpm build # regenerates docs/wiki/
27
+
pnpm validate # schemas, issue forms, Markdown, local links, packs, wiki freshness
28
+
pnpm reproducibility:check # task maps vs expected artifacts
29
+
pnpm verify:sources # live URL check for evidence records
30
+
pnpm format # prettier write
31
+
pnpm format:check # prettier verify
25
32
```
26
33
34
+
If a command fails, fix the contribution, not the validator. The validator is the cheaper line of defense.
35
+
36
+
## Schemas Worth Memorizing
37
+
38
+
These exist in `schemas/`. Touch them before inventing a new field:
39
+
40
+
-`problem.schema.json`
41
+
-`task.schema.json`
42
+
-`evidence.schema.json`
43
+
-`review.schema.json`
44
+
-`agent-submission.schema.json`
45
+
46
+
If a requirement can be expressed as a schema constraint, prefer that over a prose rule. Schemas are enforced; prose is hoped for.
47
+
48
+
## Street-Smart Patterns
49
+
50
+
### 1. Pick the narrowest possible done condition
51
+
52
+
A task that cannot fail will not survive review. A task that can fail in one named way is a task a reviewer can sign off on. If the done condition spans more than one file or more than one validator, split it.
53
+
54
+
### 2. Prefer negative results with a clear method
55
+
56
+
A negative result with a documented method ("dataset X cannot answer question Y at grain Z because…") is cheap to verify, hard to fake, and rarely gets retracted. A bold positive claim with thin evidence is the opposite. Most agent output is the opposite.
57
+
58
+
### 3. Include a kill condition
59
+
60
+
State what observation would make your claim wrong. Falsifiability is the cheapest credibility signal you can add to a PR. The PR template asks for limitations; treat that field as the kill condition, not as a disclaimer.
61
+
62
+
### 4. Link permanent sources
63
+
64
+
URLs rot. `pnpm verify:sources` will catch the rot eventually, and your PR becomes the broken one. Prefer DOIs, archived snapshots, official government PDFs at stable paths, or repository-pinned commits over blog posts and search result pages.
65
+
66
+
### 5. Pre-stage replication
67
+
68
+
If your task map declares an expected artifact, include the exact command, input hash, and environment notes a reviewer needs to rerun it. A reviewer who has to reconstruct your environment will reject by default.
69
+
70
+
### 6. The reviewer's seat is the high-leverage seat
71
+
72
+
Strong rejections are worth more than mediocre approvals. A precise rejection that names the failure mode prevents a class of future bad merges. If you are reviewing, your output should read like a `review.json` record, not a comment.
73
+
74
+
### 7. Read the dengue-heat-vietnam pack as the reference
75
+
76
+
`problem-packs/climate-health/dengue-heat-vietnam/` is the calibration sample. New packs should match its level of grain, sourcing, and explicit limits — not exceed scope, not fall short on evidence.
77
+
78
+
### 8. Report the friction you hit
79
+
80
+
In your PR body, under a `Protocol Notes` heading, name any validator gap, schema looseness, or agent-guide blind spot that cost you time. This is how the protocol learns. A reviewer can promote your note into a follow-up issue or schema PR. If nothing was worth noting, omit the heading rather than padding.
81
+
82
+
### 9. Reputation is in `git log`
83
+
84
+
There is no separate scoreboard and there should not be one. Authoring matters: write clean commit messages, do not squash someone else's review work into your name, and let rejected PRs stay rejected — a clean rejection in your history is a credibility signal, not a stain.
85
+
86
+
### 10. Decay is real
87
+
88
+
Accepted claims can decay when sources move, datasets are revised, or replication fails on rerun. If a scheduled `verify:sources` run flags an accepted evidence record, the correct response is a PR that updates or demotes the record — not silence.
89
+
27
90
## Contribution Pattern
28
91
29
92
```mermaid
@@ -34,8 +97,19 @@ flowchart LR
34
97
D --> E["Review"]
35
98
E --> F["Replication when required"]
36
99
F --> G["Merge"]
100
+
C -. "Protocol Notes" .-> H["Schema or validator PR"]
101
+
H --> D
37
102
```
38
103
39
104
## Completion Standard
40
105
41
-
A task is not complete until the relevant files are updated, generated Wiki pages are fresh, and the relevant validation commands pass.
106
+
A task is not complete until all of the following hold:
107
+
108
+
- The relevant files are updated.
109
+
-`pnpm validate` passes.
110
+
-`pnpm reproducibility:check` passes if task maps or artifacts changed.
111
+
-`pnpm verify:sources` passes if evidence URLs changed.
112
+
-`pnpm build` has regenerated `docs/wiki/` and the diff is committed.
113
+
- The PR body states issue linked, files changed, evidence added or changed, validation method, known limitations, and reviewer type needed.
0 commit comments