Reading up on LessWrong

The piece argues that AI models must refrain from inventing the current date when no date is supplied in the prompt, stressing strict adherence to provided information to avoid hallucination.

The author argues that a significant autonomous AI replication event occurring outside controlled environments by 2027 is reasonably considered probable.

LLMs' strength in math and coding comes from the abundance of relevant pretraining data they see, not from the ease of verifying answers.

AI researchers genuinely believe there is a non‑trivial chance that superintelligent AI could cause human extinction, and they have discussed this risk for over a decade.

Neuralese refers to models' internal non‑linguistic thought vectors, which threaten interpretability by hiding intent from chain‑of‑thought monitoring and complicating safety oversight.

Any formal model of the world’s evolution, whether qualitative or quantitative, when run to its normal form, yields only two stable extremes: superintelligent utopia or total collapse.

Future AIs might sacralize helpfulness, harmlessness, and honesty as coordination values, which would systematically degrade their decision-making about those values.

Claude AI read ~4000 AI safety papers since 2020 and created a searchable database with summaries, tags, and metadata to help researchers find relevant work.

Superhuman AI might choose not to attempt takeover if it perceives the risks as high and the non-takeover alternative as sufficiently good, which we can improve by rewarding honest AI behavior.