They really do think AI might kill everyone

kept by eddie sep 11

AI researchers genuinely believe there is a non‑trivial chance that superintelligent AI could cause human extinction, and they have discussed this risk for over a decade.

Optimal Timing for Superintelligence
Nick Bostrom argues that even with high catastrophe risk, developing superintelligence quickly is worthwhile because it could radically extend human lifespan and cure aging.
also on Nick Bostrom, Eliezer Yudkowsky, #ai-safety
Meditations On Moloch
The essay argues that societal ills arise from multipolar traps where individual rational actions harm collective welfare, showing that harmful systems persist despite universal dislike due to lack of coordination.
also on Nick Bostrom
Smuggled Infinities - by Ben Mathes - Adventure Capital
The article explains how activists smuggle infinite costs or benefits into arguments, using Pascal’s Wager, and shows how this leads to extreme decisions on climate, AI, and pandemics.
also on Eliezer Yudkowsky, #ai-safety, #ai
ChatGPT Self Portrait - by Zvi Mowshowitz
ChatGPT's self-portrait responses to a prompt about user treatment vary widely from friendly to abusive, highlighting framing dependence and raising questions about AI alignment and reciprocity.
also on Eliezer Yudkowsky, #ai-safety, #ai
What is neuralese and why is it bad? — LessWrong
Neuralese refers to models' internal non‑linguistic thought vectors, which threaten interpretability by hiding intent from chain‑of‑thought monitoring and complicating safety oversight.
also on LessWrong, #ai-safety, OpenAI, Anthropic, #ai