Open benchmarks

Romanian-language benchmarks

Romanian benchmarks measure whether a model solves problems posed in Romanian, not translated English. RoMath draws on official Bacalaureat maths exams; answers are judged against the official marking scheme.

Which run is shown

One row per model per benchmark: the completed run with the largest sample wins, and on a tie the most recent finish wins. Small smoke runs cannot lift a model by getting lucky. Failed and cancelled runs are excluded.

The headline metric

Cost per successful task. It punishes both expensive correct answers and cheap wrong ones, so it is the number that is hardest to game. Success rate and total cost sit next to it.

RoMath (RoMathExam, Bacalaureat M1–M4).

Results · Romanian

Benchmarks: 1
Model results: 8

Cost vs accuracy, every model on every benchmark

Each dot is one (model × benchmark) result. Upper-left is the target zone: high accuracy, low cost. Dots on the dashed frontier are non-dominated — nothing beats them on both axes at once.

codai
other models
Pareto frontier (lowest cost at each accuracy)

RoMath

100 tasks

Romanian high-school mathematics benchmark from RoMathExam (Cuclea et al. 2026, 10,592 problems from official Bacalaureat exams 1895-2025, MIT license). Stratified sample across M1/M2/M3/M4 tracks. Scored by an LLM judge (sonnet-4-6) comparing the candidate's final answer to the official `barem` — math can't be exact-matched. Codai-unique angle: no major LLM router publishes Romanian-specific scores.

Reference →

Above-median accuracy, 9% below average cost

codai hits 98.0% (median across 8 models: 96.0%) while spending $0.0382 per success — the field averages $0.0419.

Model	Success	$ / task	$ / success	Tasks
`codai`routerpinned	98.0%	$0.0374	$0.0382	100/100
`claude-opus-4-7`best	99.0%	$0.0164	$0.0166	100/100
`claude-opus-4-8`	98.0%	$0.0232	$0.0237	100/100
`claude-haiku-4-5`	96.0%	$0.0099	$0.0103	100/100
`claude-sonnet-4-6`	96.0%	$0.0146	$0.0152	100/100
`gpt-5-mini`	80.0%	$0.0340	$0.0424	100/100

Download the data

The same rows this page renders, refreshed every 10 minutes. Free to reuse with attribution.

Licence: CC BY 4.0, attribution “codai (codai.ro/benchmarks)”.

All benchmarks Honesty leaderboard