Train split: 372 of 9,428 gold queries cannot run against the shipped train databases (gold-as-prediction scores 0.9605; retail_world schema mismatch)

[christian-gheorghe]

Submitting train.json's own gold SQL as the prediction through the official evaluator (evaluation_ex.py, --meta_time_out 30.0, --num_cpus 8, databases from train_databases.zip, 69 dbs) scores 9,056 / 9,428 = 0.9605, not 1.0.

Executing each gold statement on its own (no timeout, no scoring) shows the two causes:

cause	count	where
gold references tables the shipped database does not contain	~350	retail_world ~330 (gold uses Order Details, Territories, EmployeeTerritories; the shipped file has OrderDetails and no territory tables), works_cycles 10, app_store 6, shooting 4
gold exceeds 30 s	13–15 (membership jitters at the boundary between runs)	e.g. 2142, 2143, 6720, 6759, 6771, 6775, 6776, 6797, 6836, 6891, 8190, 8820, 8997, 9002

So the train answer key and the train databases were not shipped from the same state, and roughly 4% of train examples have SQL that cannot run. Anything that learns from train (few-shot retrieval, fine-tuning, selector calibration) inherits them silently.

Reproduce: build {str(i): f"{SQL}\t----- bird -----\t{db_id}"} from train.json in file order, score it with evaluation_ex.py against the extracted train databases (see the separate issue about the difficulty JSONL — train has no difficulty field), then execute the gold statements alone to separate errors from timeouts.

Suggested fix: re-export retail_world (and the three smaller dbs) so the schema matches the gold, or correct the gold; publish the list of train ids whose gold cannot execute.