1. Race as fast as possible to the moment before AGI;
2. Correctly anticipate when the moment just before AGI occurs;
3. Stop on a dime at the precipice just before AGI occurs;
4. Try to solve alignment using whatever temporal lead the U.S. has in the race by uploading AI researchers minds into the machine and then running centuries of simulated scientific method in the few weeks/months the U.S. remains in the lead.
5. ????????
6a. If alignment is impossible or if we fail to correctly solve alignment in time, all biological life on Earth is eliminated by misaligned AGI in 98% of possible futures.
6b. If alignment is possible and we solve solve alignment in time, the U.S. will use aligned AGI as a superweapon to forcibly establish global hegemony in 1% of possible futures.
6c. If alignment is impossible and we realize it in time to stop the race, humanity decides to limit progress to strong narrow AI and a global ban on developing AGI is successfully enforced in 1% of possible futures.
If this grand strategy sounds retarded to you, it is because you have common sense.
The entente strategy maximizes the possibility of a misaligned AGI (by not allowing enough time for scientific method to find a certain solution to alignment) and and minimizes the possibility of preventing a lone wolf from aligning to their specific intent (by allowing privacte actors to not only participate in the race, but to lead it). The entente strategy is the worst of both worlds and is fatally flawed. As a result, we are racing off a cliff into mutually assured extinction.
The entente strategy presumes the race-risk dynamic is an inescapable fact of reality. There are other viable, less-existentially-risky strategies, such as a global moratorium on private frontier AI research; global cooperation and joint control over government frontier research; and universally assured destruction against all entities that attempt to cheat.
We need you in the U.K. to create friction and opposition to the current U.S. grand strategy, so that we here in the U.S. can try to get the entente strategy reversed before we fly off the cliff.
Probably worth mentioning that Anthropic did ultimately support SB 1047. Given their statements I don’t think it’s fair to hold them to a standard of “support every AI regulation bill regardless of its merits”
The U.S.'s current "entente strategy":
1. Race as fast as possible to the moment before AGI;
2. Correctly anticipate when the moment just before AGI occurs;
3. Stop on a dime at the precipice just before AGI occurs;
4. Try to solve alignment using whatever temporal lead the U.S. has in the race by uploading AI researchers minds into the machine and then running centuries of simulated scientific method in the few weeks/months the U.S. remains in the lead.
5. ????????
6a. If alignment is impossible or if we fail to correctly solve alignment in time, all biological life on Earth is eliminated by misaligned AGI in 98% of possible futures.
6b. If alignment is possible and we solve solve alignment in time, the U.S. will use aligned AGI as a superweapon to forcibly establish global hegemony in 1% of possible futures.
6c. If alignment is impossible and we realize it in time to stop the race, humanity decides to limit progress to strong narrow AI and a global ban on developing AGI is successfully enforced in 1% of possible futures.
If this grand strategy sounds retarded to you, it is because you have common sense.
The entente strategy maximizes the possibility of a misaligned AGI (by not allowing enough time for scientific method to find a certain solution to alignment) and and minimizes the possibility of preventing a lone wolf from aligning to their specific intent (by allowing privacte actors to not only participate in the race, but to lead it). The entente strategy is the worst of both worlds and is fatally flawed. As a result, we are racing off a cliff into mutually assured extinction.
The entente strategy presumes the race-risk dynamic is an inescapable fact of reality. There are other viable, less-existentially-risky strategies, such as a global moratorium on private frontier AI research; global cooperation and joint control over government frontier research; and universally assured destruction against all entities that attempt to cheat.
We need you in the U.K. to create friction and opposition to the current U.S. grand strategy, so that we here in the U.S. can try to get the entente strategy reversed before we fly off the cliff.
Probably worth mentioning that Anthropic did ultimately support SB 1047. Given their statements I don’t think it’s fair to hold them to a standard of “support every AI regulation bill regardless of its merits”
They didn't ultimately support it, see: https://x.com/DanHendrycks/status/1887127975650631974