Notebook: discrete attention for the language base (M1)

This is the running table of results for the line that carries the project today. It asks whether a model that uses only integers and small geometric moves can learn to read earlier text well enough to predict the next word. The rows below run from 13 October to 19 October 2026. None of them is a trained language result: the best trained discrete model gets the right next word first in 463 of 14,287 development rows, while a simple count table (T2) gets 6,300.

flowchart LR
  A["14 Oct<br/>A2 pilot: rank counts up,<br/>loss total up"] --> B["15 Oct<br/>Wider shortlist at test time<br/>fails: 2,400 vs 4,894"]
  B --> C["18 Oct<br/>Row-parallel scoring:<br/>about 56x faster"]
  C --> D["18 to 19 Oct<br/>Four K16 runs:<br/>no keep"]
  D --> E["19 Oct<br/>Address cells at zero training:<br/>1,351 vs 127 of 3,847"]
  E --> F["19 Oct<br/>A8 dose stopped at<br/>batch 4,096 for diagnosis"]

The turning points in the rows below. Each box is one or more rows; the numbers are in the table.

The longer story of this line is on its line page. The mechanisms involved are described in address cells, admission, rank margin and relation read.

Terms used in the table

• DEV panel. The open development panel of 14,287 rows. Each row is a context and the true next token. P_succ (the copy-successor subset) is 3,847 of those rows; it is the subset where the answer follows an earlier occurrence of the same token.
• Admission. The read can only choose among a shortlist of candidate cells (K of them, usually K=8). A row is admitted when the true next token's cell is on the shortlist. A token outside it can never be chosen, so admission is a ceiling on top-1.
• Top-1 / top-8. The true token ranked first / within the first eight.
• Batches. Training steps. A full registered dose is 16,404 batches.
• FIT. An independent review of a run's pre-registration sheet before spending compute. A FIT releases nothing by itself.
• Control (C-R1 to C-R6, V2 and the like). A check built to fail if the instrument is wrong. A passing control is not a model result.
• Status words (positive, negative, diagnostic stop, blocked) are defined in the format page.

Results

Sources: each row links its record. Counts and one-line descriptions come from archive/catalog.jsonl; numbers and bars were checked against the record itself. Records dated before 18 October 2026 are in archive/.

What the rows say, in plain terms

• The integer path is built and its finite controls pass. Those are construction results, and they say nothing about language.
• Training moves the rank counts up but has not yet beaten the context-free count table. The best trained result in the rows is 463 of 14,287, against 6,300 for T2 and 1,226 for the unigram table (CHARTER.md, section 4.8).
• The main obstacle found so far is admission: a token outside the shortlist can never be chosen. Widening the shortlist at test time did not fix it (row of 15 October). Placing cells by address structure fixed it at the start (1,351 against 127 of 3,847), but training then pushed target cells out of the shortlist (the A8 row).
• Six of the 30 rows are blocked by tooling or build faults and are not results about the model. The row-parallel kernel (about 56x faster proposals with identical losses) is the largest measured speed change.
• The next step recorded in the A8 row is a choice among three diagnoses by the PM: an acceptance rule that refuses moves which eject a target cell, a per-checkpoint cell census, or a finer checkpoint cadence. Each new training run needs a reviewed sheet and a typed owner release.

Not in this table

• Plans, reviews, tooling and process records of the same period are not results. They are listed in archive/CATALOG.md and in docs/labs/ under this line.
• The GPU rental blockers for the baseline-reuse timing runs (17 October) are filed under the compute-ops line, not here.
• The float "starling" reads of 12 October are filed under m1-float-rung0, which has no notebook yet.

Go deeper

• Line page: discrete attention for the language base
• Format and verdict words
• Prime and zeta starting codebook notebook, whose table was the starting point for the 19 October address-cell rows
• archive/CATALOG.md, filtered by line m1-discrete-attention