The shutdown regime

The authors' outline of what the world would need to do to stop escalation to superintelligence.

Eliezer Yudkowsky and Nate Soares's framework, from If Anyone Builds It, Everyone Dies, pp. 139 to 195.

Consolidate compute
Gather all computing power that could train or run more powerful AIs into places monitored by multiple treaty powers.
Set the threshold low
Limit unmonitored hardware to about eight of the most advanced GPUs from 2024.
Ban efficiency research
Make it illegal to publish research into more efficient and more powerful AI techniques.
Enforce everywhere
Warn, then sabotage or strike, any datacentre that defies the treaty, while offering the holdout equal terms to join.
Keep the coalition broad
Do not bundle the ban with any other cause, so that nearly everyone can agree.

From chapter: PART III: FACING THE CHALLENGE, pp. 139–195

More from this book