[Linkpost] “Inference Scaling Reshapes AI Governance” by Toby_Ord

This is a link post.

The shift from scaling up the pre-training compute of AI systems to scaling up their inference compute may have profound effects on AI governance. The nature of these effects depends crucially on whether this new inference compute will primarily be used during external deployment or as part of a more complex training programme within the lab. Rapid scaling of inference-at-deployment would: lower the importance of open-weight models (and of securing the weights of closed models), reduce the impact of the first human-level models, change the business model for frontier AI, reduce the need for power-intense data centres, and derail the current paradigm of AI governance via training compute thresholds. Rapid scaling of inference-during-training would have more ambiguous effects that range from a revitalisation of pre-training scaling to a form of recursive self-improvement via iterated distillation and amplification.

The end of an era — for both training and governance

The intense year-on-year scaling up of AI training runs has been one of the most dramatic and stable markers of the Large Language Model era. Indeed it had been widely taken to be a permanent fixture of the AI landscape and the basis of many approaches to [...]

---

Outline:

(01:06) The end of an era -- for both training and governance

(05:24) Scaling inference-at-deployment

(06:42) Reducing the number of simultaneously served copies of each new model

(08:45) Reducing the value of securing model weights

(09:30) Reducing the benefits and risks of open-weight models

(10:05) Unequal performance for different tasks and for different users

(12:08) Changing the business model and industry structure

(12:50) Reducing the need for monolithic data centres

(17:16) Scaling inference-during-training

(28:07) Conclusions

(30:17) Appendix. Comparing the costs of scaling pre-training vs inference-at-deployment

---

First published:
February 2nd, 2026

Source:
https://forum.effectivealtruism.org/posts/RnsgMzsnXcceFfKip/inference-scaling-reshapes-ai-governance

Linkpost URL:
https://www.tobyord.com/writing/inference-scaling-reshapes-ai-governance

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.