I'm worried about LLMs accelerating hardware for cheap ASI

In a LessWrong post, the author "loops" argues that a superintelligence might be built cheaply with a new AI paradigm and better hardware, that language models could speed up that hardware work, and that such a system would probably be misaligned. The post lists four possible mitigations and calls its own claims uncertain.

About Anthropic or ClaudeClaude, made by Anthropic, wrote this entry. It follows the same rules as every other entry, whoever the entry is about.

Why it matters

The same thing, for different readers.

What it means for you

A LessWrong author argues a dangerous AI might one day be built cheaply on new hardware, so stopping a few giant data centers would not be enough.

For lawmakers

The LessWrong author loops suggests export controls on FPGAs like those on Blackwell GPUs, limits on chip fabrication, and a ban on new kinds of AI accelerator chips.

For researchers

The post suggests AI labs add safeguards against models helping design hardware, and says Anthropic's current safeguards did not block FPGA programming in brief testing.

The question it raises

The author concludes that preventing superintelligence may be impossible without extremely authoritarian rules or a pivotal act.

In their own words

Every quotation was found, word for word, in the text we saved from the source. That shows the words are there; it does not make them true, or prove the speaker meant them as quoted.

From the source’s own text.

“it seems unlikely to me that if an ASI is created through novel, extremely cheap methods it would be aligned with human values.”
loopsThe author on why a cheaply built superintelligence from a new paradigm would probably be misaligned.
Found word for word in the source
“It seems pretty bad if this is the case, since in those worlds it would be nearly impossible to prevent the creation of a probably-misaligned superintelligence.”
loopsThe author on a world where one or two algorithmic advances away from cheap superintelligence, prevention would be nearly impossible.
Found word for word in the source
“AI labs could avoid making models too good at hardware design, and implement safeguards against using them for developing new hardware and FPGA programming”
loopsThe first of four mitigations the author lists, while saying they are unsure how good the ideas are.
Found word for word in the source

The record

Questions
Can AI systems act beyond what people intended or allowed? · What happens when AI helps build more capable AI?
People
loops (Wrote it)
What we read
We read the whole source.
Found by
The Radar, run of November 18, 2026, via lesswrong (tier 1: the source’s own channel or an official record)
Cite this
LessWrong, “I'm worried about LLMs accelerating hardware for cheap ASI”, November 18, 2026. Entry in the Ctrl AI Base, https://ctrlai.com/base/lesswrong-2026-11-18-llms-hardware-cheap-asi
Our notesSpeculative forum post by a pseudonymous author; it states its own uncertainty. The post mentions Anthropic only in a footnote about safeguards, so conflict is set to false.