Two halves. Below, the work I built and the work I landed in other people's projects. Every pull request in the second half is a commit that shipped upstream: a defect I reproduced first, the smallest fix I could defend, and the tests that pin the behaviour. Both halves are rendered from the GitHub API on a schedule, so neither can drift from what is actually public.
Focus · AI agent runtimes and their memory subsystems · multi-agent orchestration · MCP and tool plumbing · multimodal agents · Python
   
These 14 labs were each built from scratch and run on a laptop CPU. Every one ships a verifier, multi-seed measurements, and a README whose results table is rendered from a committed JSON by a script the CI byte-compares against, so no number in any of them is hand-typed. The descriptions below are pulled live from each repository, so this table cannot rot either.
| Repository | What it demonstrates |
|---|---|
| align-lab CPU-only post-training study: SFT vs DPO vs ORPO vs SimPO on a verifiable digit-addition task. Multi-seed; shows DPO/SimPO win-rate climbing to ~0.99 while real generation accuracy collapses (likelihood displacement). |
Four preference-optimisation objectives on one task and one policy: the pair-wise win-rate is a flattering metric that can rise while generation gets worse. |
| grpo-repro A tiny, fully-verifiable GRPO / REINFORCE / DPO comparison lab on a Reverse-Polish-Notation puzzle env. Same policy, same reward, CPU-second ablation with real measured numbers. |
GRPO, REINFORCE and DPO as three arms on identical weights and an identical verifiable reward. With a partial credit given to any legal answer, all three collapse to a degenerate output; the reward shape, not the algorithm, was the bug. |
| reward-hacking-lab CPU-only study of reward-model over-optimisation (Goodharting) in RLHF-style rejection-sampling self-improvement: optimise a verifiable reward and accuracy climbs; optimise a learned reward and the proxy climbs while true accuracy collapses. |
Same loop, two rewards: optimise an exact verifier and true accuracy climbs, optimise a learned proxy and the proxy climbs while accuracy falls. Goodhart on demand, measured. |
| starlab CPU-reproducible STaR self-improvement: a ~100k-param transformer bootstraps column-addition reasoning from its own verifier-checked chains, with a matched-compute control. |
STaR bootstrap: the model samples its own chains, an exact arithmetic verifier filters them, survivors become next round's training set - reported against a matched-compute control so the gain cannot be mistaken for simply training longer. |
| data-select-lab A CPU-only LoRA/PEFT + LESS data-selection study: influence-scored LoRA gradient features pick the few fine-tuning examples that teach a frozen model a held-out capability, with forgetting reported honestly. |
LESS-style LoRA gradient features pick the few dozen examples that teach a frozen model a held-out skill; the same table reports how much of the skill it already had that costs. |
| Repository | What it demonstrates |
|---|---|
| skill-lab CPU-only, dependency-free study of a Voyager-style self-evolving skill library: a budgeted BFS planner distils reusable first-order skills from its own verified plans and gets measurably cheaper the more it is used. |
Voyager-style skill library distilled from the agent's own verified plans: the budget spent on the next task falls as the library grows, which is the whole claim. |
| autprompt-lab CPU-only study of budgeted automatic prompt optimisation (OPRO / Promptbreeder-style): random vs greedy vs population-evolution search against a frozen tiny transformer on a verifier-scored task, at an equal query budget. |
OPRO / Promptbreeder-style prompt search under an equal query budget - random, greedy and population evolution scored against one frozen policy, so the comparison is about search, not compute. |
| evolve-lab A CPU-only, dependency-free self-optimizing agent: evolution rediscovers and blends single-machine dispatching rules purely from an exact tardiness verifier. |
A (mu, lambda) evolution strategy over dispatch rules rediscovers WSPT from an exact tardiness verifier. Dependency-free, and honest that the margin it keeps over WSPT sits inside the seed noise. |
| Repository | What it demonstrates |
|---|---|
| lagent Local-first ReAct tool-use agent with a measured scaffold ablation: identical behaviour-cloned weights, four harnesses, ground-truth verifier. CPU-only tiny transformer. |
ReAct scaffold ablation on identical behaviour-cloned weights: strip the observation scratchpad and the solve rate collapses. The harness loop is load-bearing machinery, not decoration around the model. |
| agent-harness-eval Agent Harness Eval: instrument, score, and statistically compare agent rollouts. Paired bootstrap + McNemar, behavior metrics, zero-dependency core. |
Paired bootstrap and McNemar for comparing agent rollouts, because a three-seed difference is not a result and most harness comparisons never check. |
| hier-memo-agents Hierarchical planner-executor agents with shared blackboard memory, topic-level reuse, and verifiable citation provenance. Zero deps, deterministic, CI-tested. |
Planner and executor agents over a shared blackboard with citation-linked reuse: memory matched by topic rather than by question, so reuse is transfer and not a cache hit. |
| Repository | What it demonstrates |
|---|---|
| llama-anatomy From-scratch ~100k-parameter LLaMA decoder (RMSNorm, RoPE, SwiGLU, GQA), plus an equal-parameter ablation of each choice, a train-short/test-long probe of RoPE against learned absolute positions, and a parameter-free sweep of RoPE's rotation base. CPU-only, multi-seed; every README number renders from a committed artifact and CI byte-compares it. |
A LLaMA decoder written from the papers at 10^5 parameters, with every component swapped for an equal-parameter control so a difference is never just size. The rotation-base sweep is the one axis that adds no parameters at all, and it separates a floor its seeds agree on from a ceiling they do not. |
| consistency-lab Adaptive early-stopping self-consistency: a sequential stopping rule that halts CoT sampling once the vote is statistically decided, measured on the accuracy-vs-compute frontier vs fixed-N self-consistency. CPU-only, verifiable. |
A sequential stopping rule for self-consistency: sample chains until the vote is statistically decided, then spend the chains you saved on the questions that are actually contested. |
| vlm-distill-bench Reproducible CPU-only distillation & quantization benchmark for compact vision-language models on procedural mini-CLEVR. Real numbers, committed seeds, no downloads. |
Distillation plus quantisation of a compact vision-language model on procedural mini-CLEVR, including the negative result that temperature-scaled KD can lose to plain cross-entropy at this scale. |
Projects with 1,000+ stars that have merged my pull requests upstream.
| Project | Stars | Merged | Pull requests |
|---|---|---|---|
| bytedance/deer-flow | 83.0K | 8 | #5555 · #5593 · #5609 · #5586 · #5588 · #5801 · #5821 · #5607 |
| agentscope-ai/agentscope | 32.3K | 2 | #2754 · #2808 |
| deepset-ai/haystack | 26.6K | 2 | #12810 · #12905 |
| mrexodia/ida-pro-mcp | 12.3K | 4 | #531 · #529 · #533 · #539 |
Merged per month
2026-09 ######################################## 16
All merged pull requests
| Merged | Project | Pull request |
|---|---|---|
| 2026-09-24 | bytedance/deer-flow | #5607 fix(memory): reject a Honcho base_url that can never resolve |
| 2026-09-24 | deepset-ai/haystack | #12905 fix(core): compare Ellipsis callable parameters by return type |
| 2026-09-24 | bytedance/deer-flow | #5821 fix(community): fall back to the default SearXNG max_results on an unparseable value |
| 2026-09-24 | bytedance/deer-flow | #5801 fix(gateway): treat a blank GATEWAY_HOST/GATEWAY_PORT as unset |
| 2026-09-24 | bytedance/deer-flow | #5588 fix(skills): load SKILL.md saved as UTF-8 with a BOM |
| 2026-09-24 | bytedance/deer-flow | #5586 fix(memory): keep a zero confidence from scoring as the default |
| 2026-09-24 | agentscope-ai/agentscope | #2808 fix(formatter): forward the images of collapsed messages in xAI multi-agent history |
| 2026-09-23 | deepset-ai/haystack | #12810 fix(auth): deserialize listed secrets when recursive is enabled |
| 2026-09-22 | mrexodia/ida-pro-mcp | #539 fix(rpc): treat a blank IDA_MCP_URL as unset for download URLs |
| 2026-09-22 | bytedance/deer-flow | #5609 fix(subagents): report an explicit zero batch limit instead of defaulting it |
| 2026-09-22 | bytedance/deer-flow | #5593 fix(skills): render an explicit empty allowed-tools as no tools, not as all |
| 2026-09-22 | agentscope-ai/agentscope | #2754 fix(agent): keep tool-result metadata and timestamps when truncating |
| 2026-09-21 | mrexodia/ida-pro-mcp | #533 fix(idalib): fall back to defaults for blank IDA_MCP_* values |
| 2026-09-21 | mrexodia/ida-pro-mcp | #529 fix(installer): keep IPv6 brackets in generated transport URLs |
| 2026-09-21 | mrexodia/ida-pro-mcp | #531 fix(installer): let --uninstall run when IDA Free is installed |
| 2026-09-20 | bytedance/deer-flow | #5555 fix(memory): report malformed backend_config values by key name |
In review right now (49 open pull requests)
| Project | Open | Pull requests |
|---|---|---|
| agno-agi/agno | 7 | #10281 · #10287 · #10297 · #10301 · #10320 · #10323 · #10325 |
| langflow-ai/langflow | 5 | #15223 · #15225 · #15227 · #15229 · #15231 |
| TencentCloud/Octop | 5 | #910 · #912 · #1146 · #1147 · #1150 |
| agentscope-ai/agentscope | 4 | #2741 · #2799 · #2805 · #2834 |
| browser-use/browser-use | 4 | #5838 · #5845 · #5847 · #5849 |
| crewAIInc/crewAI | 4 | #7610 · #7612 · #7615 · #7617 |
| mrexodia/ida-pro-mcp | 4 | #537 · #541 · #542 · #543 |
| strands-agents/harness-sdk | 3 | #4553 · #4554 · #4591 |
| bytedance/deer-flow | 2 | #5850 · #5852 |
| deepset-ai/haystack | 2 | #12813 · #12907 |
| HKUDS/LightRAG | 2 | #4097 · #4098 |
| livekit/agents | 2 | #7468 · #7470 |
| PrefectHQ/prefect | 2 | #23190 · #23191 |
| QwenLM/Qwen-MM-Plugins | 2 | #69 · #71 |
| andrewyng/openworker | 1 | #690 |
Plus 1 merged pull request(s) in 1 smaller repository, below the 1,000 star bar of this table.
Rendered by scripts/build_readme.py from the GitHub API — the lab table reads each repository's live description, the PR tables read the search API — and refreshed by .github/workflows/refresh.yml · record last changed 2026-09-25 11:47 UTC.