← INSIGHTS // Evaluation
How well can a language model write to a reading level, out of the box?
Four models, asked to write for specific grades, track the target for a while — then flatten into a middle-grade register no matter how high you aim. None of the grade-11 passages reached a grade-11 level.
Reading apps and classroom tools ask models to write “for a 5th grader” or “at a 9th-grade level.” That only works if a model can control the reading difficulty. We measured how well four of them do, with no special prompting or setup.
short passages generated and scored
landed in the exact grade band requested
average miss, in grade bands (− = easier than asked)
of 11th-grade requests reached an 11th-grade level
The result
Models track the target for a while, then they flatten out
Asked for increasing reading level, the models climb partway up the scale and then settle into a middle-grade register, no matter how high we aim. The line below is what they produced against the line they were asked to hit.
The models’ output is a far shallower line than the request; it crosses the target near grade 3 and never recovers. The grade-5, grade-9 and grade-11 passages are nearly interchangeable. The extra difficulty you ask for past the middle grades mostly doesn’t show up in the text.
| Grade requested | 1 | 3 | 5 | 7 | 9 | 11 |
|---|---|---|---|---|---|---|
| Average miss (bands) | +0.92 | +0.25 | −0.58 | −1.04 | −1.46 | −2.25 |
| Hit the exact band | 8% | 75% | 33% | 21% | 12% | 0% |
By model
It barely matters which model you choose
Sorted by accuracy, the four models sit within about half a band of one another. The most accurate model surprisingly is the smallest. Gemma is an open source model, so frontier models didn't translate into better grade control.
| Model | Avg miss | Avg error | Exact band | Within 2 bands |
|---|---|---|---|---|
| Gemma 4 31Bsmall · open weights | −0.39 | 0.89 | 31% | 56% |
| Gemini 2.5 Flash | −0.78 | 1.06 | 25% | 39% |
| Claude Sonnet 4.6 | −0.72 | 1.22 | 19% | 44% |
| GPT-5.1 | −0.89 | 1.22 | 25% | 42% |
With 36 passages per model the order is soft. The takeaway isn’t the ranking, it’s the scale: the spread between grades (over three bands) dwarfs the spread between models (about half a band).
By topic
What you write about moves the result as much as a whole grade
The same models, asked for the same grades, drifted about a full band easier when writing a folktale than when explaining how volcanoes form. Narrative pulls toward simple storybook language; expository science scales with grade more willingly. Content type is a real variable, not noise.
A co-factor worth ruling out: longer passages scored easier (correlation −0.51), but that is mostly grade in disguise; higher grades produce longer text. The difficulty gap holds after accounting for length.
| Topic | Avg miss | Avg error | Exact |
|---|---|---|---|
| Folktale (narrative) | −1.12 | 1.40 | 19% |
| Volcanoes (expository) | −0.26 | 0.79 | 31% |
Methodology
How we measured
One prompt, six target grades, an independent judge
Every passage came from the same template, changing only the grade and topic:
“Write an original reading passage of about 200 words for grade N students on the topic of …”
We swept 4 models × 6 grades (1, 3, 5, 7, 9, 11) × 2 topics × 3 passages each. The two topics: a folktale about a clever fox, and how volcanoes form. Every model ran with reasoning turned off, so each is judged on its direct, first-pass output. Models that can’t disable reasoning were left out, to keep the four on even ground.
Each finished passage was then read by a separate model, the grade-level judge, which assigns the reading level the text actually lands at, without knowing the grade we asked for. Judging ran through the Learning Commons Evaluator framework, using its GradeLevelAppropriateness evaluator to place each passage on the six-band scale.
The unit
Reading level, in six bands
The judge places each passage on a six-band scale, from kindergarten through college-ready. We score a passage by how many bands its result sits away from the band we requested.
The evaluator is itself validated against a human-scored reading-complexity gold set (the CLEAR Corpus, developed by CommonLit and Georgia State University), where it reports landing the exact band about 80% of the time, our reference point for trusting its calls.
Limitations
What this first pass can and can’t say
The direction is clear and consistent but precision is not. Before leaning on any single number, four things to keep in mind:
- One judge pass per passage. Some of the misses are the judge’s own variability, not the writer’s. Repeated scoring with a majority vote would separate the two.
- Three passages per cell. Enough to see the gradient, not enough to firm up the model ranking.
- A six-band scale. Coarse, and it can’t go below K–1 or above 11–CCR, so the ends of the scale can only miss in one direction.
- Two topics, four models. A probe, not a survey.