What happened
While reviewing the paraphrase study (#60), a reviewer read the eight Spanish passages line by line against src/SignsOfAI.Core/Rules/Packs/rules.es.json and reported that every Spanish rule the study saw move fired on prose no machine wrote and no careful writer would avoid:
| Rule |
Where it fired |
lex.ademas |
además as an ordinary connector |
lex.asimismo |
asimismo, standard formal register |
lex.utilizar |
utilizar, the neutral verb |
lex.profundo |
profunda influencia |
lex.panorama |
panorama artístico |
These are pre-2022 Wikipedia articles, so each hit is a false positive by construction — the same argument Docs/CALIBRATION.md makes about the English rules it ranks.
Why this is not already covered by the thresholds
tools/SignsOfAI.Calibration -- thresholds derives a humanRatePer1000 per rule from the corpus, and a rule used at a human rate is shown without scoring. That machinery works. The problem is upstream of it: the Spanish corpus is 25 texts, all encyclopedia entries, which is thin enough that a rate derived from it may be measuring one genre rather than the language. Thresholds.MinimumTexts is 8, so a rule appearing in 8 Spanish texts gets a threshold from a very small base.
The English side has 65 texts across two genres and two writer populations. Spanish has one of each. This is the gap Docs/Calibration/README.md already names as what it most wants contributed, and the study made it concrete.
What to do
Two things, in order:
- Grow the Spanish corpus, which is the real fix and the one the README asks for. SciELO and Redalyc are the obvious sources for academic Spanish and neither was reachable when the corpus was first assembled. A Spanish arm with academic prose in it would let the thresholds do their job.
- Then re-derive and re-examine these five rules. If they survive a larger corpus they stay; if they turn out to measure formal Spanish rather than machine Spanish, they are reweighted or retired — the same treatment
Docs/CALIBRATION.md prescribes for a rule "measuring the genre rather than the machine".
Do not hand-edit the five rules out of the pack. The whole point of the calibration machinery is that rules leave on evidence rather than on somebody's judgement, and a Spanish pack curated by taste is the thing this project criticises other tools for.
Note on provenance
The reviewer read the passages as a native reader and also judged the Spanish rewrites themselves sound — meaning, register, figures and citation markers preserved in all eight pairs. So this finding is about the pack, not about the study material.
What happened
While reviewing the paraphrase study (#60), a reviewer read the eight Spanish passages line by line against
src/SignsOfAI.Core/Rules/Packs/rules.es.jsonand reported that every Spanish rule the study saw move fired on prose no machine wrote and no careful writer would avoid:lex.ademaslex.asimismolex.utilizarlex.profundolex.panoramaThese are pre-2022 Wikipedia articles, so each hit is a false positive by construction — the same argument
Docs/CALIBRATION.mdmakes about the English rules it ranks.Why this is not already covered by the thresholds
tools/SignsOfAI.Calibration -- thresholdsderives ahumanRatePer1000per rule from the corpus, and a rule used at a human rate is shown without scoring. That machinery works. The problem is upstream of it: the Spanish corpus is 25 texts, all encyclopedia entries, which is thin enough that a rate derived from it may be measuring one genre rather than the language.Thresholds.MinimumTextsis 8, so a rule appearing in 8 Spanish texts gets a threshold from a very small base.The English side has 65 texts across two genres and two writer populations. Spanish has one of each. This is the gap
Docs/Calibration/README.mdalready names as what it most wants contributed, and the study made it concrete.What to do
Two things, in order:
Docs/CALIBRATION.mdprescribes for a rule "measuring the genre rather than the machine".Do not hand-edit the five rules out of the pack. The whole point of the calibration machinery is that rules leave on evidence rather than on somebody's judgement, and a Spanish pack curated by taste is the thing this project criticises other tools for.
Note on provenance
The reviewer read the passages as a native reader and also judged the Spanish rewrites themselves sound — meaning, register, figures and citation markers preserved in all eight pairs. So this finding is about the pack, not about the study material.