Compare
Which model should do this job
Twelve tasks with exactly one right answer each. Pick the models you want, run them, and get a score, a timing and a list of what each one got wrong. The judging happens in your browser — nothing about your run is sent anywhere unless you save it.
Sign in to run a test
Running a set costs API calls, so it needs an account. Free to create, and the results are yours to keep.
Sign in or create an account How billing works
Already signed in somewhere else? This page reads the same session as the console. Open the console once, then come back.
The task set
Twelve tasks, one point each. No partial credit — a task is right or wrong, and the checker says which. They are chosen so that a weak model cannot pass by guessing, and so that each one isolates a different failure mode.
| Group | What it isolates | Tasks |
|---|---|---|
| Numeric reasoning | Carrying, percentages, ratios, enumeration — the four places models cut corners | 4 |
| Code repair | Off-by-one, wrong return type, pending promise, bad seed value | 4 |
| Structured output | Emitting exactly the requested shape, with no prose around it | 2 |
| Constrained writing | Satisfying a content rule and a length rule at the same time | 2 |
Every task is scored by a strict comparison, not a model judging another model. The checker code ships with the task set, so you can read exactly what counts as correct.
Why the score is worth trusting
A model judging a model is an opinion. This is not that. Each task declares a checker: a number to match, a regular expression the corrected code must satisfy, a JSON shape to equal, a word count to stay under. Nothing is graded on style.
The trade is that the set is small. Twelve tasks will not tell you which model is better at a 40-file refactor. They will tell you which model follows instructions and which one quietly drops a constraint, which is usually the thing that ruins an afternoon.
What gets saved
Nothing, until you press save. A run happens entirely in your browser: your key calls the API directly, the checker runs locally, and the result is rendered on the page.
Saving a run records the date, the models you tested, the task set version and the score, so a result can be pointed at later. Past runs appear under Records.
Comparisons
How FreeModel compares to the other gateways — including where they are the better choice, and what a migration actually involves.