Supporting code and data for the systematic identification of documented rainfall-induced pluvial muddy flood (PMF) events in the loess and loamy agricultural regions of north-western Europe.
The MUDCAP Interreg North-West Europe project investigates the occurrence, characteristics, and impacts of pluvial muddy floods — rainfall-driven surface runoff events that mobilise fine sediment from agricultural loess and loamy soils and deliver it to roads, buildings, and watercourses downstream.
This repository documents the automated screening pipeline used to identify scientific publications reporting real, observed PMF events or closely related rainfall-induced flood events in the study area.
Study area: Belgium, Luxembourg, The Netherlands, northern France (Normandy, Hauts-de-France, Picardy, Île-de-France, Grand Est, Champagne-Ardenne, Lorraine, Alsace), and western Germany (North Rhine-Westphalia, Rhineland-Palatinate, Saarland).
Publication period covered: 1970–2025.
Data source: Scopus (Elsevier).
┌────────────────────────────────────────────────────────────────────────┐
│ Phase 1 — Query Generation │
│ config.toml → generate_mudcap_queries.py → Scopus query string │
└───────────────────────────────┬────────────────────────────────────────┘
│
┌───────────────────────────────▼────────────────────────────────────────┐
│ Phase 2 — Data Ingestion │
│ main.py → data_collector/ → SQLite DB (1 615 publications) │
└───────────────────────────────┬────────────────────────────────────────┘
│
┌───────────────────────────────▼────────────────────────────────────────┐
│ Phase 3 — Autonomous Screening │
│ screen.py → extraction/screening_agent.py (6 Mistral agents) │
│ Each publication screened independently by all 6 agents │
└───────────────────────────────┬────────────────────────────────────────┘
│
┌───────────────────────────────▼────────────────────────────────────────┐
│ Results │
│ results/relevant_publications.csv — 304 relevant publications │
│ results/pmf_relevant_publications.csv — 46 PMF-strict subset │
└────────────────────────────────────────────────────────────────────────┘
The Scopus query is built from three term blocks defined in config.toml
and combined with Boolean AND:
| Block | Role | Example terms |
|---|---|---|
| Driver | Rainfall triggering mechanism | rainfall, flash flood*, cloudburst*, convective storm* |
| Process | Flood or sediment transport event | flood*, muddy flood*, overland flow, debris flow*, ephemeral gully erosion |
| Geography | Target region anchor | Belgium, Hesbaye, Meuse, Pays de Caux, North Rhine-Westphalia |
The full reproducible query is generated by:
python generate_mudcap_queries.pyTerms within each block are joined with OR; blocks are joined with AND.
Wildcard * handles plural and derived forms.
Each publication was assessed on three independent boolean dimensions
using a structured AI screening protocol (see extraction/screening_agent.py
for the full system prompt).
| Dimension | Variable | What qualifies |
|---|---|---|
| Flood event relevance | flood_event_relevance |
Paper reports a real, observed rainfall-induced hazard event — with field data, measurements, damage records, or post-event surveys |
| Geographic relevance | geographic_relevance |
Study area falls within the target region (see below) |
| Muddy flood relevance | muddy_flood_relevance |
Paper documents observed PMF-type processes: muddy overland flow, sediment-laden runoff, ephemeral gully erosion in loess/loamy catchments |
included = (flood_event_relevance OR muddy_flood_relevance) AND geographic_relevance
A publication is included if it passes either event gate (flood or muddy flood) and the geographic gate. This dual-gate design ensures that papers framed as erosion studies (rather than floods) are not missed, and vice versa.
The screening prompt explicitly excludes:
- Climate projections, future scenarios, or model-based flood risk assessments
- Purely numerical or hydrological modelling studies without documented real events
- Laboratory or field experiments simulating rainfall or runoff
- Statistical or probabilistic flood frequency analyses without event documentation
- Risk assessments, vulnerability analyses, and exposure mapping
- Review papers and meta-analyses that synthesise existing literature without new event data
- Papers where events are hypothetical, simulated, or future-projected
Administrative units: Belgium (all regions), Luxembourg, The Netherlands, northern France (Normandy, Hauts-de-France, Picardy, Île-de-France, Grand Est, Champagne-Ardenne, Lorraine, Alsace), western Germany (North Rhine-Westphalia, Rhineland-Palatinate, Saarland).
Natural regions and river basins also recognised:
- Rivers: Meuse, Scheldt, Ourthe, Vesdre, Geul, Dyle, Sambre, Demer, Gete, Lesse, Somme, Oise, Scarpe, Authie, Canche, Moselle, Rur, Erft, Ahr, Lippe
- Belgian natural regions: Hesbaye, Herve, Condroz, Entre-Sambre-et-Meuse, Thiérache, Brabant Wallon
- French loess landscapes: Pays de Caux, Pays de Bray, Artois, Cambrésis, Santerre, Beauvaisis, Boulonnais, Vermandois
- German loess Börde: Jülicher Börde, Zülpicher Börde, Kölner Bucht, Soester Börde
- Dutch regions: Zuid-Limburg, Heuvelland
Each dimension also carries a confidence rating (low / medium / high):
- high — the abstract is explicit and unambiguous.
- medium — the abstract strongly implies it, but key details are absent.
- low — the abstract is insufficient; the paper is flagged for human review.
Hard rule: A dimension cannot be true with confidence = "low".
Uncertainty results in false + low confidence, flagging the paper for review.
Six Mistral language models were used as independent screening agents. Each publication was submitted to all six agents; their decisions were aggregated by counting the number of agents that voted for inclusion.
| Agent name | Mistral model |
|---|---|
large-2512 |
mistral-large-2512 |
medium-2508 |
mistral-medium-2508 |
small-2603 |
mistral-small-2603 |
small-2506 |
mistral-small-2506 |
mini-8b-2512 |
ministral-8b-2512 |
mini-14b-2512 |
ministral-14b-2512 |
All agents used temperature = 0.0 and were prompted to return structured
JSON (response_format: json_object) to enforce parseable output.
The agent_votes_overall column in the results CSV records how many agents
independently classified a publication as relevant. Publications are ranked
in descending vote order: a publication voted relevant by 5 out of 6 agents
has stronger multi-model consensus than one voted by a single agent.
| Metric | Value |
|---|---|
| Publications retrieved from Scopus | 1 615 |
| Duplicate publications removed | 2 |
| Unique publications independently screened by all 6 agents | 1 613 |
| Relevant publications (≥ 1 agent vote) | 304 |
| — of which: consensus (≥ 3 agent votes) | 172 |
| — of which: strong consensus (5 agent votes) | 84 |
PMF-strict subset (muddy_flood_relevance — see below) |
46 |
| Average pairwise inter-agent agreement | 91.9 % |
Relevant publications per agent:
| Agent | Model | Relevant found |
|---|---|---|
large-2512 |
mistral-large-2512 | 194 |
medium-2508 |
mistral-medium-2508 | 178 |
small-2603 |
mistral-small-2603 | 176 |
small-2506 |
mistral-small-2506 | 230 |
mini-8b-2512 |
ministral-8b-2512 | 142 |
mini-14b-2512 |
ministral-14b-2512 | 0 |
Note: ministral-14b-2512 applied highly conservative screening and flagged no
publications as relevant. Its decisions still contribute to the vote tally but
effectively act as an abstention. This outlier behaviour reflects the sensitivity
of smaller instruction-tuned models to strict exclusion criteria.
All 304 publications where at least one agent voted for inclusion.
Sorted by agent_votes_overall descending (strongest consensus first).
| Column | Description |
|---|---|
rank |
Rank by consensus (1 = highest agent agreement) |
agent_votes_overall |
Number of agents (out of 6) that voted relevant |
title |
Publication title |
authors |
Semicolon-separated author list |
year |
Publication year |
journal |
Journal or conference name |
doi |
Digital Object Identifier |
doi_url |
Resolved DOI URL |
abstract |
Full abstract |
A stricter subset of 46 publications where muddy_flood_relevance = true
was voted by a majority of agents — i.e., papers that specifically document
pluvial muddy flood processes (sediment-laden runoff, muddy overland flow,
ephemeral gully erosion events) rather than generic flood events.
These are the core references for the MUDCAP event inventory.
.
├── README.md # This file
├── config.toml # Search terms, agent config, DB settings
├── pyproject.toml # Python dependencies
├── uv.lock # Locked dependency versions
├── .env.example # API key template
│
├── generate_mudcap_queries.py # Phase 1 entry point: generate Scopus query
├── main.py # Phase 1+2 orchestrator: query + Scopus ingestion
├── screen.py # Phase 3 CLI: run all agents against the DB
├── db_models.py # SQLModel ORM definitions (Publication, Screening, …)
├── database_manager.py # DB session management and CRUD operations
│
├── query_generator/ # Search query builder
│ ├── models.py # SearchBlock, QueryProject data classes
│ ├── generator.py # QueryGenerator: dispatches to formatters
│ └── formatters.py # ScopusFormatter: builds Scopus syntax
│
├── data_collector/ # Scopus ingestion layer
│ ├── base.py # BaseProvider + PublicationMetadata Pydantic model
│ ├── scopus.py # ScopusProvider: search + retrieval API with backoff
│ └── collector_manager.py # CollectorManager: orchestrates providers
│
├── extraction/ # AI screening layer
│ ├── models.py # Pydantic models: ScreeningDecision, BotAgentConfig
│ └── screening_agent.py # ScreeningAgent: Mistral client + system prompt
│
└── results/
├── relevant_publications.csv # 304 relevant publications (all event types)
└── pmf_relevant_publications.csv # 46 PMF-strict publications
- Python ≥ 3.13
- uv package manager
- Scopus API key with institutional access (apply at dev.elsevier.com)
- Institutional network access — Scopus API requests must originate from an IP address registered with Elsevier. If working off-campus, connect to your institution's VPN before running Phase 1+2.
- Mistral API key (obtain at console.mistral.ai)
# Install dependencies
uv sync
# Configure API keys
cp .env.example .env
# Edit .env and add your keyspython main.pyThis generates the Scopus query from config.toml, queries the Scopus API,
and stores all results in mudcap_review.db. Expect ~1 600 records for the
current query and year range.
Note:
reset_db = falseinconfig.tomlby default. Set totrueonly if you want to wipe and reinitialize the database.
# Dry run first (no writes to DB)
python screen.py --dry-run
# Full screening run
python screen.pyRuns all agents configured in [[agents]] in config.toml. Each agent
screens all unique (non-duplicate) publications that have no existing
decision for that agent name.
To screen a limited batch:
python screen.py --limit 100