AI existential risk probabilities are (still) too unreliable to inform policy
This essay, a repost from two years prior, contends that probability estimates of AI existential risk (p(doom)) are insufficiently rigorous to guide public policy. The authors, Arvind Narayanan and Sayash Kapoor, assert that these forecasts lack a validated model or method, unlike other quantitative predictions. They argue that inductive probability estimation is unreliable due to the absence of a suitable reference class for AI x-risk, and deductive estimation fails because there's no reliable theory for modeling technological progress and governance in this context. Consequently, many forecasts are subjective probabilities, which are essentially guesses. The authors aim to highlight this unreliability for policymakers, advocating for a broader conception of AI safety beyond existential risk and superintelligence.
Original sources
Newsletter AI Snake Oil
Issue:AI existential risk probabilities are (still) too unreliable to inform policy ↗
AI Radar wrote this summary from the publisher’s notes with AI, not from a transcript. How it works.