Notebook: data and tokenizer
This notebook is for results about what text the model trains on and how that text is cut into tokens. It has no rows yet. The three records dated 19 October 2026 are preparation: an approved plan, a checker and a tokenizer trainer. None of them ran the model or produced a number against a bar.
Results
Sources: archive/catalog.jsonl. Verdict words are defined on the format page.
| Date | What ran | Number | Bar | Verdict | Artifact |
|---|---|---|---|---|---|
| No result rows yet. |
Why the table is empty
Three records on this line fall on or after 12 October 2026. None is a result:
- 2026-10-19, d41-offline-donor (process): the owner approved a one-time exception. A large open model (gpt-oss-120b) may generate a training corpus offline and is never used again by the student. Running it needs a separate owner release.
- 2026-10-19, distill-d1-consumer-verifier (tooling): a host-only checker that validates the planned donor corpus against its registration sheet before admission. Its 32 self-tests pass; no GPU run and no generation took place.
- 2026-10-19, rust-bpe-trainer (tooling): a deterministic host-only Rust command to train a byte-level BPE tokenizer for a planned 16,384-token vocabulary. It does not change the active tokenizer and produces no model result.
What a row here will need
A row needs the data and tokenizer identity (hashes), a bar fixed beforehand and a comparison on the same evaluator. A tokenizer that changes the vocabulary changes what every panel row means, so it is compared on the fixed development panel and reported with its cost.
Go deeper
- Data and tokenizer and the tokenizer mechanism.
- Format and verdict words.
archive/CATALOG.md, filtered by linedata-tokenizer.