You Are the Optimizer

The model is deliberately dumb: 9 logits — one bet per cell. It can't even see the board: each turn it softmaxes the logits of the free cells and samples. Even THIS can be trained.

Games: model plays X, always first. Opponent O moves randomly. Reward: win 1.0 · draw 0.5 · loss 0.0.

Each round = one real training step: ① rollout: watch 8 games. ② advantage: compute the group mean, mark each game +/0/− vs that bar. ③ step: weights unlock — push cells from «+» games up ▲, from «−» games down ▼ (yes, ALL their cells, even lucky ones — that's broadcast advantage).

Advantage is simplified: reward − mean. Real GRPO also divides by std.

Noise is honest: 8 games is tiny; rewards jump. Real GRPO averages 128 such groups per step. Trust the trend, not points.

Eval round: plays 8 games and records the score without any training — use it to check whether a spike was skill or luck.

you	loss.backward() + optimizer.step() — turning outcomes into weight moves
8 games	one GRPO group (rollouts_per_example = 8); a real step runs 128 groups
reward − mean	advantage: each game scored against its own group's bar
your ▲▼ of ±0.3	learning rate × gradient sign
“check vs GRPO”	the true policy gradient Σ adv·∇logπ over the whole group
reward curve	the training chart — jagged in real runs too; final ≠ best