Skip to content

About

Grammar error detector for ESL learners: a fine-tuned DistilBERT classifier combined with rule-based checks to flag errors, identify the offending word, and explain each mistake in plain English. Includes a Flask web app and CLI.

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

ESL Error Detector

Flags and explains grammar errors in learner English. A fine-tuned DistilBERT classifier decides whether a sentence probably contains an error; about 25 rules over a spaCy dependency parse then find which error it is, which word is responsible, and why, in plain English. Many of the rules target errors I saw constantly while teaching English in Spain.

demo

How it works

  1. Classifier. DistilBERT fine-tuned on W&I+LOCNESS (about 38k learner sentences with corrections) predicts error / no error. The threshold is 0.25, well below the F1-optimal one, so nearly every flawed sentence reaches the rules. A false alarm costs nothing: a sentence only shows an error if a rule fires on it.
  2. Article checks run on every sentence regardless of the classifier: a/an, missing, unnecessary and wrong-choice articles.
  3. Rules identify the error type and the exact token from the parse.
  4. Output maps each error to its tokens, highlighted in the web app with an explanation.

What it catches

  • Agreement and form: subject–verb agreement (including compound and elided subjects), pronoun case, irregular forms, missing plural -s after numbers, double comparatives
  • Tense and aspect: tense errors, stative verbs in the continuous, since / for / ago
  • Verb patterns: infinitive after a preposition, gerund after stop, enjoy, avoid…
  • Structure: frequency-adverb position, misplaced modifiers, faulty parallelism, doubled subjects ("The teacher, she explained"), I/me order in lists, comma splices, double negatives
  • Word choice: prepositions, interested / interesting, adjective vs adverb, there / their / they're, your / you're, its / it's, spelling

Results

Held-out 20% split, 5 epochs on a Colab GPU:

Accuracy F1 Precision Recall
76.5% 0.770 0.776 0.763

Limits

The rules depend on the parser, and parsers are least reliable on short, heavily ungrammatical sentences, which is exactly what learners write. Out of reach for now:

  • errors that need context: a wrong tense with no time marker, or peoples meaning individuals
  • coreference: myself for me needs to know what the pronoun refers to
  • argument structure: "I was arguing the cashier" parses as valid English
  • dialect: on line vs in line is standard in some varieties

Next: suggested corrections, and a token-level sequence-labelling model in place of the classifier and rules, which would reach some of the errors above.

Running it

pip install -r requirements.txt
python -m spacy download en_core_web_sm
python app.py        # web app at http://localhost:5000
python demo.py       # terminal version

The trained checkpoint isn't in the repo (size). Train with python main.py or, faster, with notebook/train_colab.ipynb on a GPU.

Licence: MIT (LICENSE).

About

Grammar error detector for ESL learners: a fine-tuned DistilBERT classifier combined with rule-based checks to flag errors, identify the offending word, and explain each mistake in plain English. Includes a Flask web app and CLI.

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages