Records

Every run you saved, with the date on it

最后更新：2026-10-28

A score without a date is a rumour. Models get updated, providers swap the weights behind a name, and a result from March says nothing about July. So each saved run keeps the date, the model ids, the task set version and the score together.

Your runs

A single score vs a dated record: the second one can be compared, the first cannot.

What a record contains

Field	Why it is kept
Date and time	Model behaviour changes; a score is only meaningful against a date
Model ids	The exact ids tested, including any `auto/` route; a route can resolve differently later
Task set version	So a score from v1 is never compared against a v2 run
Score per model	The headline number
Per-task failures	More useful than the total: it says which kind of task broke
Total time	Makes cost visible: the same score at half the latency is a different answer

Where these are stored

Runs saved from Compare are stored in this browser only. They are not uploaded, and they are not visible to anyone else — which also means they do not follow you to another machine.

A shared record — every run, by everyone, on one page — needs a server-side store that does not exist yet. Until it does, this page reads your local runs and says so, rather than showing an empty list and implying nobody has ever run a test.

How a record gets here

Records are not written on this page. They are written on Compare, when you decide a run is worth keeping, and this page reads them back.

Step 1: run a test. Sign in on Compare, paste an sk- key, choose the model ids and the task groups, and press Run the test.
Step 2: press save. Save this run is the only thing that writes anything. A run you do not save leaves no trace at all.
Step 3: come back here. The saved run appears above with its date, the model ids, the task set version and the score for each model.
Step 4: pin the id you tested. If a record covers a modality route such as auto/image rather than a named model, the note field is the only thing that will tell you later what it resolved to. Name the model in your own notes if the result matters.

Why the id and the date travel together

A score is one half of a fact. The other half is what answered and when, because both move. A named model can be updated behind the same id, and a route resolves per request — six of the 199 listed ids are modality routes, 3% of the list, and the model one of them picks on Monday is not guaranteed to be the model it picks on Friday.

The task set is the third variable, which is why its version is stored. Twelve tasks with exact checkers do not change quietly, but a change to the set would still make two scores incomparable, so v1 runs are never mixed with another version in the same comparison.

One more limit worth knowing before you read anything into a score: the set is text only. Of the 199 ids, 23% are not chat models and cannot answer these tasks at all; the 153 chat entries are the whole pool this set can exercise. A high score here says a model follows instructions exactly. It does not say it is the right model for your agent loop.

How to verify a record

A record is only useful if you can reproduce the run from it. Pick one record, copy its id and date, then re-run the same prompt with the same model. If the output differs, the record still tells you which model served the original run; that is the first thing to check before blaming the prompt.

Check the model field against what you requested.
Check the date against the price page in force that day.
Check the input size; a truncated input explains most output differences.

Records FAQ

Where are my saved runs stored?

In this browser, in local storage. They are not uploaded and they are not visible to anyone else, which also means they do not follow you to another machine or another browser. Clearing site data deletes them.

Can I see other people's runs?

No, because there is nowhere to see them yet. A shared record (every run, by everyone, on one page) needs a server-side store that does not exist. This page reads your local runs and says so, rather than showing an empty list and implying nobody ever tested anything.

Why does a record keep failures instead of only the score?

Because a total of eight out of twelve tells you less than which four broke. A model that misses the numeric tasks and a model that emits prose around its JSON both score below twelve, and they are not the same problem. The per-task list is the part you act on.

Why is the total time recorded?

The same score at half the latency is a different answer. If two models both solve ten tasks and one does it in half the time, that is a result the score alone hides — and time is recorded per run, not estimated afterwards, so it reflects the models that actually answered.

How old is too old for a record to mean anything?

There is no fixed expiry, which is exactly why the date is stored next to every score. A model can be updated behind an unchanged id, and free capacity moves between providers. Treat anything older than the last model update you know about as history rather than as a current measurement, and re-run it.

Records and privacy

A record contains the run id, date, model, and a summary of the request; it does not need to contain the full document. Store the minimum needed to reproduce the decision, and set a retention period. If a record includes customer text, treat it with the same rules as the source document.

Run a testTwelve tasks, any models, judged in your browser ModelsThe ids you can put in the test RoutingLet a route pick instead of testing by hand

Sources

Every third-party figure quoted on this site was read from the vendor documentation linked below, on the date stated. None of these links are affiliate links and none of these vendors paid to be listed. If a number here disagrees with the page it came from, the page is right — tell us and it gets corrected.

Key numbers

6 个字段
1 个日期
3 个版本维度
6 fields kept per record: date, model ids, task-set version, score, per-task failures, total time.
1 date per run — a score without a date is not comparable.
Task-set version separates runs: a v1 score is never compared with a v2 run.

How to use it, step by step

Step 1. Run the model set against a fixed task set.
Step 2. Save the run: the date, the exact model ids, the task-set version and the per-task results go in together.
Step 3. Compare only runs with the same task-set version, and read per-task failures before the headline score.

One worked example

A record for a 3,000-token run stores the run id, date, model, token counts, and a short request summary — not the full document. That is enough to re-run and compare. Keeping full inputs turns the record store into a second copy of your data, with the same privacy obligations and a larger bill.

What a record does not prove

A record proves a run happened with a given model and date. It does not prove the output was correct, and it does not prove the input was complete. Treat it as a receipt, not a review.

Record checklist

Run id and date are present and immutable.
Model field matches what actually served the request.
Token counts are stored per request.
Retention period is defined and enforced.

A record that cannot answer "which model, when, how many tokens" is a log line, not a record.