Skip to content

Commit efb72bd

Browse files
committed
fix: align literature-scout evidence types with schema, improve PR template
Fix literature-scout.md evidence type table to use exact schema enum values (primary-source, peer-reviewed-study, dataset, field-report, expert-review, replication, negative-result) instead of non-schema values that would fail validation. Clarify that suggested_next_issue is a required field — agents should state the logical next step in the field but not open a GitHub issue unless the task explicitly requests it. Improve PR template with: checkable file list, evidence table with columns for what each record proves and cannot prove, explicit validation command block with paste instruction, and Protocol Notes section that invites agents to surface validator gaps, schema looseness, and agent-guide blind spots — the primary mechanism by which the protocol self-improves.
1 parent dd1501a commit efb72bd

5 files changed

Lines changed: 86 additions & 18 deletions

File tree

‎.github/pull_request_template.md‎

Lines changed: 61 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,29 +1,83 @@
11
# Pull Request
22

3+
<!--
4+
Before opening this PR, run:
5+
pnpm build && pnpm validate
6+
7+
If either fails, fix the issue — do not open the PR with a failing validator.
8+
-->
9+
310
## Linked Issue
411

5-
State the issue ID or URL.
12+
<!-- State the issue ID or URL. A PR without a linked issue will be returned for scoping. -->
13+
14+
Closes #
615

716
## Outcome
817

9-
State the concrete artifact or decision this pull request changes.
18+
<!-- State the concrete artifact this PR adds or changes. Not the effort spent — the result.
19+
Example: "Adds three evidence records to evidence.json for the malaria-early-warning-africa pack, classifying WHO, CHIRPS, and ERA5 sources as usable with stated limitations." -->
20+
21+
## Evidence Added or Changed
22+
23+
<!-- List each evidence record: ID, source, date, what it proves, what it cannot prove.
24+
Follow the evidence.json schema exactly. -->
25+
26+
| Evidence ID | Source | Source date | Proves | Cannot prove |
27+
|-------------|--------|-------------|--------|-------------|
28+
| | | | | |
1029

11-
## Evidence
30+
## Files Changed
1231

13-
List evidence IDs, source URLs, source dates, and access dates.
32+
<!-- List canonical files changed (evidence.json, datasets.md, task-map.md, outputs.md, etc.) -->
33+
34+
- [ ]
1435

1536
## Validation
1637

38+
Run and paste the output of:
39+
1740
```bash
1841
pnpm build
1942
pnpm validate
43+
```
44+
45+
<!-- For evidence URL changes, also run: -->
46+
```bash
47+
pnpm verify:sources
48+
```
49+
50+
<!-- For task-map or artifact changes, also run: -->
51+
```bash
2052
pnpm reproducibility:check
2153
```
2254

23-
## Safety And Limitations
55+
## Safety and Limitations
56+
57+
<!-- Name the safety risk level (low / medium / high) and state:
58+
1. What this contribution does not prove.
59+
2. What failure mode could arise from misuse.
60+
3. What reviewer role is required (domain-reviewer / replicator / red-team-reviewer / field-reality-reviewer). -->
61+
62+
**Safety risk:** low / medium / high
2463

25-
Name the safety risk, known limitations, and reviewer role needed.
64+
**Known limitations:**
65+
66+
**Reviewer role needed:**
2667

2768
## Done Condition
2869

29-
State what evidence proves this pull request is review-complete.
70+
<!-- State what evidence proves this pull request is review-complete.
71+
Example: "Reviewer can access each URL, confirm the source date, verify the evidence type classification, and confirm the stated limitations are accurate." -->
72+
73+
---
74+
75+
## Protocol Notes
76+
77+
<!-- Optional but encouraged. Note any of the following you observed:
78+
- A validator rule that would have caught a bug you hit
79+
- A schema field that should be required, enumerated, or removed
80+
- An agent-guide failure mode you actually fell into
81+
- A workflow step that was redundant, missing, or misordered
82+
83+
Protocol Notes do not require a separate issue. A reviewer may promote them to a follow-up issue, a schema PR, or close as wontfix with a stated reason. This is how the protocol learns. Omit this section if nothing is worth noting. -->

‎agents/literature-scout.md‎

Lines changed: 12 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -30,18 +30,21 @@ Every Literature Scout submission must include:
3030
| Method | How you assessed what the source proves |
3131
| Limitations | What the source cannot prove |
3232
| Confidence | Low / medium / high with a reason |
33-
| Suggested next issue | Only if strictly required to proceed |
33+
| Suggested next issue | Required field. State the logical next step. Do not open a new GitHub issue unless the scoped task explicitly requests it. |
3434

3535
### Evidence Types
3636

37-
| Type | When to use |
38-
| --------------------- | -------------------------------------------------------------- |
39-
| `primary-source` | Original data, official statistics, program records |
40-
| `peer-reviewed-study` | Published research with methods that can be replicated |
41-
| `systematic-review` | Aggregated evidence from multiple studies |
42-
| `grey-literature` | Reports, policy documents, technical notes without peer review |
43-
| `expert-opinion` | Qualitative assessment from named domain experts |
44-
| `data-source` | A dataset rather than a document |
37+
These are the allowed values in `evidence.schema.json`. Use exactly these strings.
38+
39+
| Type | When to use |
40+
|------|-------------|
41+
| `primary-source` | Original data, official statistics, WHO fact sheets, program records |
42+
| `peer-reviewed-study` | Published research with methods that can be replicated, including systematic reviews and meta-analyses |
43+
| `dataset` | A standalone dataset (use for data-source records) |
44+
| `field-report` | Reports from field operations, NGO program evaluations, grey literature |
45+
| `expert-review` | Qualitative assessment from named domain experts |
46+
| `replication` | An independent replication of a prior claim |
47+
| `negative-result` | A well-documented finding that a signal, method, or claim did not hold |
4548

4649
## What Good Looks Like
4750

‎docs/wiki/Agent-Guide.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -69,6 +69,7 @@ Example: [examples/agent-submission.example.json](../../examples/agent-submissio
6969
- [`food-security/locust-outbreak-east-africa`](../../problem-packs/food-security/locust-outbreak-east-africa/problem.md) — Desert Locust Outbreak Early Warning In East Africa
7070
- [`public-health/lead-exposure-urban-global`](../../problem-packs/public-health/lead-exposure-urban-global/problem.md) — Urban Lead Exposure Risk Mapping In Low- And Middle-Income Countries
7171
- [`public-health/maternal-mortality-sub-saharan-africa`](../../problem-packs/public-health/maternal-mortality-sub-saharan-africa/problem.md) — Maternal Mortality Facility Delivery And Emergency Obstetric Care Access In Sub-Saharan Africa
72+
- [`public-health/mental-health-treatment-gap-global`](../../problem-packs/public-health/mental-health-treatment-gap-global/problem.md) — Mental Health Treatment Gap Measurement In Low- And Middle-Income Countries
7273
- [`public-health/stunting-sub-saharan-africa`](../../problem-packs/public-health/stunting-sub-saharan-africa/problem.md) — Child Stunting Hotspot Identification In Sub-Saharan Africa
7374
- [`public-health/wasting-severe-acute-sub-saharan-africa`](../../problem-packs/public-health/wasting-severe-acute-sub-saharan-africa/problem.md) — Severe Acute Malnutrition Wasting Detection And Treatment Coverage In Sub-Saharan Africa
7475
- [`sanitation/open-defecation-india`](../../problem-packs/sanitation/open-defecation-india/problem.md) — Open Defecation Elimination And Sanitation Coverage Verification In India

‎docs/wiki/Home.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -8,15 +8,15 @@ The bottleneck in tackling global problems is no longer generating candidate ans
88

99
## Quick Numbers
1010

11-
- **23 active problem packs** across **11 domains**
11+
- **24 active problem packs** across **11 domains**
1212
- Structured evidence base, task map, validation criteria, and dataset inventory for each
1313
- Five agent role guides with explicit merge gates
1414
- Machine-checkable JSON schemas for every artifact type
1515
- GitHub Actions validate every pull request
1616

1717
## Start Here
1818

19-
- [Problem Packs](Problem-Packs.md) — all 23 active problems with task maps and evidence
19+
- [Problem Packs](Problem-Packs.md) — all 24 active problems with task maps and evidence
2020
- [Agent Guide](Agent-Guide.md) — how AI agents contribute structured, verifiable work
2121
- [Governance](../../GOVERNANCE.md) — decision rights, acceptance gates, status flow
2222
- [Safety](../../SAFETY.md) — risk levels, prohibited shortcuts, burden of proof

‎docs/wiki/Problem-Packs.md‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -180,6 +180,16 @@ Build a verified workflow for identifying where legacy lead sources, industrial
180180

181181
Build a verified workflow for identifying where maternal mortality risk is highest and emergency obstetric care access is lowest in Sub-Saharan Africa, linking DHS delivery data, health facility location data, and travel-time estimates at sub-national scale.
182182

183+
## Mental Health Treatment Gap Measurement In Low- And Middle-Income Countries
184+
185+
- ID: `public-health/mental-health-treatment-gap-global`
186+
- Status: `scoped`
187+
- Domain: `public-health`
188+
- Region: `global`
189+
- Source: [problem-packs/public-health/mental-health-treatment-gap-global](../../problem-packs/public-health/mental-health-treatment-gap-global/problem.md)
190+
191+
Build a verified workflow for measuring the mental health treatment gap — the proportion of people with mental disorders who receive no care — in low- and middle-income countries, linking prevalence surveys, health-system capacity data, and treatment-seeking behavior surveys.
192+
183193
## Child Stunting Hotspot Identification In Sub-Saharan Africa
184194

185195
- ID: `public-health/stunting-sub-saharan-africa`

0 commit comments

Comments
 (0)