Skip to content

A clause is not a concept - #45

Merged
jepegit merged 1 commit into
mainfrom
concept-fragments
Sep 9, 2026
Merged

jepegit merged 1 commit into
mainfrom
concept-fragments

Conversation

@jepegit

@jepegit jepegit commented Sep 9, 2026

Copy link
Copy Markdown
Owner

38 of the corpus's 482 concepts were clause fragments, and the programme asked questions about them:

"What did they report for figure shows?"

One paper had figure shows, shows the XRD pattern and scan shows the xrd among its 27 concepts. Another had see Figure and see Table. Also becomes amorphous, containing species, consistent with the presence, peaks characteristic, capacity associated, against sodium metal counter, discharged to mv, above mv.

Both guards were written in the past tense

PHRASE_BOUNDARY_VERBS is 35 entries and every one is a past participle — measured, observed, shown, reported… So a phrase opening or closing on a finite verb went straight through.

CLAUSE_LIKE, whose own docstring says "a finite verb makes a phrase a clause rather than a term", had the auxiliaries and nothing else. A verb in the middle was therefore invisible to both: the boundary check cannot reach it, and CLAUSE_LIKE did not know the word. Hence scan shows the xrd.

The lead family is deliberately absent from CLAUSE_LIKE. In a battery corpus "lead" is the metal, and a rule that loses lead acid battery to catch "leads to capacity fade" is a bad trade. Tested.

Corpus

before after
clause fragments 38 8
concepts 482 442
SEG-03 27 23
errors 2 2

A segment reported as introducing five new terms, when two of them were the same noun phrase cut in different places, was the working-memory rule being broken by arithmetic rather than by content.

The eight that remain are reduced relatives — electrodes cycled with LiPF6, electrode cycled in EC. They are at least comprehensible as terms, and the judgement separating them from cycled electrode surface is not one a word list can make.

What this does not fix, and why I stopped

43 pairs where one concept is a sub-phrase of another: metal counter beside metal counter electrodes beside sodium metal counter electrodes.

_drop_redundant already decides these on the ratio of their mention counts (drop the shorter at ≥0.8, the longer at <0.35). Measuring every surviving pair in the corpus says the ratio cannot separate them:

0.75  'thin film' (12) inside 'thin film electrodes' (9)      <- both real terms
0.75  'specific surface' (8) inside 'specific surface area' (6) <- both real
0.75  'metal counter' (8) inside 'metal counter electrodes' (6) <- a truncation

What distinguishes them is that a truncation ends on a modifier rather than a head noun — metal counter, a-Si thin, Si hollow, TOF-SIMS depth — which is a judgement about English that a regex does not have. I tried two window-level discounts before measuring this; one was too weak (it compared against a single containing phrase instead of their union) and the positional one cut Irisarri from 32 concepts to 5, taking irreversible capacity and graphitization degree with it. Reverted both. Left alone rather than guessed at — this is the shape of thing docs/PLAN-ai.md assigns to the critic role.

🤖 Generated with Claude Code

Thirty-eight of the stress corpus's 482 concepts were clause fragments, and the
programme asked questions about them: "What did they report for figure shows?"
One paper had "figure shows", "shows the XRD pattern" and "scan shows the xrd"
among its twenty-seven concepts, another had "see Figure" and "see Table".

Both guards were written in the past tense. PHRASE_BOUNDARY_VERBS is thirty-five
entries and every one of them is a past participle, so a phrase opening or closing
on a finite verb went straight through. CLAUSE_LIKE -- the rule whose own docstring
says "a finite verb makes a phrase a clause rather than a term" -- had the
auxiliaries and nothing else, so a verb in the *middle* was invisible to both: the
boundary check cannot see it and CLAUSE_LIKE did not know the word.

The lead family is deliberately absent from CLAUSE_LIKE. In a battery corpus "lead"
is the metal, and a rule that loses "lead acid battery" to catch "leads to capacity
fade" is a bad trade.

Clause fragments across the corpus fall from 38 to 8, concepts from 482 to 442, and
SEG-03 from 27 to 23 -- a segment reported as introducing five new terms when two
of them were the same noun phrase cut in different places was the working-memory
rule being broken by arithmetic rather than by content.

The eight that remain are reduced relatives: "electrodes cycled with LiPF6",
"electrode cycled in EC". They are at least comprehensible as terms, and the
judgement that separates them from "cycled electrode surface" is not one a word
list can make.

What this does not fix is the overlap: forty-three pairs where one concept is a
sub-phrase of another, "metal counter" beside "metal counter electrodes" beside
"sodium metal counter electrodes". _drop_redundant already decides those on the
ratio of their mention counts, and measuring the corpus says the ratio cannot: the
truncation "metal counter" sits at 0.75, between "thin film" and "specific
surface", which are both real terms. What separates them is that a truncation ends
on a modifier rather than a head noun, which is a judgement about English that a
regex does not have. Left alone rather than guessed at.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@jepegit
jepegit merged commit 64ad259 into main Sep 9, 2026
8 checks passed
@jepegit
jepegit deleted the concept-fragments branch September 9, 2026 07:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant