We checked the textbook answer

Offline reinforcement learning has a standard answer to "was this decision a mistake": learn Q(s,a) from recorded play, then compare what was done with the best alternative. Fitted Q Iteration and IQL are the usual methods. Early on we set this route aside for a simpler one. On 26 Oct 2026 we tested it properly.

Our credit score is plain. It takes the change in win chance over the next 40 seconds and compares the decision with what strong players do in the same situation. No learned future, no bootstrapping.

The setup. 1,616 ranked runs at the top bracket: 800 to train, 400 to tune, 416 held-out test runs that no model saw. One shared vocabulary of 21 actions and one shared reward, the 40-second change in win chance. Three Q-learning variants: FQI at two horizons, and IQL. Two folds split by run, so every bootstrapped value comes from a model that never saw that run.

The result, on the held-out test runs. We measure how well each score tracks what a decision actually led to (correlation, higher is better).

Ours: 0.0198. FQI at the longer horizon: 0.0157. IQL: 0.0170. Ours is 26% ahead of FQI and the gap is outside the confidence interval. Against IQL the gap is not significant, so we do not claim it.

More iterations did not help. The model after one step and after four were the same. The likely reason is that the state 40 seconds later is set by the situation more than by the one action, so bootstrapping from the model's own estimates adds noise rather than signal.

What this changes: nothing in the product. It confirms the direction. We will improve the score by reading the situation better, not by switching to heavier reinforcement learning. Full numbers are in R10 in research.