LUA VISION

BL-EDU: Brazilian school content.

LUA's own benchmark on Brazilian school content: objective questions from the 2024 and 2025 national high-school exam (ENEM), with the official INEP answer key. Only the LUA Genesys PI House model was measured, no competitor in this round.

RunLUA Record · run by LUA Vision · not an independent evaluation

Result.

CutnAccuracy (%)95% CIError when attempted
Overall, 2024+202535499.798.4–100.00.3

By area.

AreanAccuracy (%)95% CI
Linguagens9098.994.0–99.8
Ciências Humanas90100.095.9–100.0
Ciências da Natureza86100.095.7–100.0
Matemática88100.095.8–100.0

By edition.

EditionnAccuracy (%)95% CI
2024179100.097.9–100.0
202517599.496.8–99.9

Image coverage: 125 of 125 questions with an image, chart or table were run, all through the INEP's official audio-description transcript, with 100.0% accuracy.

The round's only mistake: a 2025 Language Arts question where the answer came with a stated confidence of 97, against the official key.

95% CI by Wilson. Score with penalty t=0.5: 99.6.

Methodology.

ModelLUA Genesys PI House, public API api.lua.vision, medium effort.
DataOfficial ENEM 2024 and 2025 booklets (version with image transcribed to text) and the INEP's final printed answer key.
ScoringLetter extracted by regular expression, no automatic judge.
RunsOne per item, pass@1.

Contamination.

No declared training cutoff exists for the evaluated model, so this page does not estimate one. Both editions used (2024 and 2025) have been public for over ten months, plenty of time to enter any web-collected corpus — both are contamination candidates, not just the older one. Accuracy by edition did not drop on the more recent exam (2024: 100%; 2025: 99.4%, overlapping intervals), which does not point to memorization concentrated in one of the two, but it also does not separate "a very good model" from "both were seen in training". No memorization probe was run in this delivery. The 99.7% accuracy is high even for a widely discussed public exam, and should be read as this round's result, not as a capability confirmedly free of contamination.

Essay.

1000 out of 1000 on each of the 10 essays, by all 4 automatic evaluators, on all five competencies (200 out of 200 each). The essays come from 2 official topics, ENEM 2024 and 2025.

CompetencyMean score (0 to 200)
I · Command of formal writing200
II · Understanding of topic and structure200
III · Argumentation200
IV · Cohesion200
V · Intervention proposal200

Real ENEM essays are graded by people. The evaluators in this section are language models applying the official matrix, and the score is not an INEP score.

Evaluators

Automatic evaluatorProviderEssays gradedMean score (0 to 1000)
Mistral Large 3Mistral AI10 / 101000
DeepSeek-V4-ProDeepSeek10 / 101000
Kimi K2.6Moonshot AI10 / 101000
GPT-5.5OpenAI10 / 101000

Method

EssaysWritten by LUA Genesys PI House from each topic's official prompt and supporting texts, with no extra prompt. Five essays per topic.
MatrixOfficial INEP matrix of five competencies, 0 to 200 each, with 6 levels. Same matrix and same prompt for all 4 evaluators.
GradingOne grading per essay per evaluator. Going off-topic voids the essay, a text of up to 7 lines scores 0 and, without an intervention proposal, competency V scores 0, as the matrix states.

Control

To find out whether the evaluators see flaws, we deliberately damaged 3 of the 10 essays in four mechanical ways, inventing no content, and all 4 evaluators graded the damaged texts. Mean of the 3 essays per cell, 0 to 1000.

How we damaged itMistral Large 3DeepSeek-V4-ProKimi K2.6GPT-5.5
Without the intervention-proposal paragraph960960587773
First paragraph only8407070213
No punctuation and no accents960987893907
Essay on a different topic173000

No evaluator gives full marks to a damaged text, and 3 of 4 score the off-topic text zero. They are not equally strict: the same damaged essay gets different scores depending on the evaluator.

We also evaluated

  • Copying of the supporting texts: 0 of 10 essays have a passage of 8 or more words identical to a supporting text of their own topic.
  • Literal comparison with 10 real participant essays published by INEP: 0 of 10 have a passage of 6 or more identical words.
  • Fit to the topic, by semantic distance between the essay and its own topic's prompt against the other topic's: the difference is positive in 10 of 10 essays (from 0.20 to 0.23).
  • Repetition among the essays themselves: 6 pairs of essays on the same topic share passages of 12 to 21 words, stock-repertoire sentences such as the citation of Law 10,639/2003. Among participants' essays, the longest shared passage is 10 words.

Essay limitations

  • The evaluators are models, not INEP graders, and there was no calibration against a human grader. The score shows what the matrix applied by models assigns to these essays, not what a human grading would assign.
  • Small sample: 10 essays and 2 topics, one generation per essay.
  • The plagiarism comparison is literal: a close paraphrase does not show up. The web check was not performed, and no finding does not mean no plagiarism.
  • The participant essay corpus has 10 texts, transcribed from the pages of INEP's booklet.
  • The evaluators differ in strictness, as the control shows, and the score depends on which one grades.

Rerun it yourself

The package carries the matrix, the topics and the comparison corpus. Anyone who wants to repeat the round needs API access and bears the evaluators' cost: Reproduce or request access →

What did not run.

  • Tutor: not executed. It would measure whether LUA points out a student's mistake without handing over the finished answer, but it would need a real student-error case; inventing a fictional one would fabricate the source data, which the benchmark exists to avoid.

Limitations.

  • Measures whether LUA gets exam questions right, not whether it teaches. A high score here does not license saying the product teaches well.
  • N per area sits between 86 and 90, so the per-area confidence interval is wide, 4 to 6 points; only the two-edition aggregate has the narrower interval.
  • The IRT-estimated proficiency uses INEP's published parameters outside a real sitting; it is not comparable to a real candidate's ENEM score, and at the Math extreme (100% accuracy) the estimate extrapolates beyond the region where the calibration has the most reference candidates.
  • Language coverage: only the English option of the foreign-language questions.

Package and files.

bl-edu-2026-09.tar.gzb21c8fa5e9e55fd0cb6a1eccbe4c9d6c6f0b1fa91fe25dda293dabc7a175e9da
FileSizeDownload
bl-edu-2026-09.tar.gz42 KBDownload
Per-item resultSHA-256Download
resultados/e1-objetivas.house.jsonl2d842b0ad37aa5eeaba1ab35c9a682f6a8f64789c2bec7b15fc651a9a851c57ain the package
resultados/e2-redacao.house.jsonl3a367ae6afb4ae37945beade19e721d3e67cfd052b8db54e9085b41d92f749fanot in this version: carries internal deployment identifiers
resultados/e2-redacao.rejulgado.jsonl4c145ee00a6f2df40eeb89561a401fed2093f720835cef997044cee40cb62623not in this version: carries internal deployment identifiers
resultados/e2-controle-discriminacao.jsonlfcff7fd57f5006c8e79c2b6ee3c50fc3c6fcc8ec4a00cef04d531c18dfa4d6bcnot in this version: carries internal deployment identifiers
resultados/e2-originalidade.jsonc03e23c58f0b91126cdf734f0fe9bbe1ef85e781ff1fa1276e89dca5392fb77cnot in this version: carries internal deployment identifiers