My thoughts on p(doom)

As a person with three decades of AI experience I feel like I have an obligation to write on p(doom). I know that it is fashionable to write it "pdoom", same as n-grams became "engrams" recently. But I am old-fashioned like that.

In short, what is the probability that AI will cause humanity's extinction? It is very low. First, we have the math - all the way since Pierre-Simon Laplace calculated the probability of sun not raising the next morning, all the way since Bayes we have the math to calculate the astronomically low chances of humanity extinction in the next day, year or a decade. And those odds are extremely low. They are non-zero though.

Second argument is the means. We have really few ways to actually kill all the humans on Earth and AI might surely trigger another test for our resilience against those ways. But the key word in the previous sentence is "another. Let's look at those ways and how they were tested already.

  1. Nuclear war. I could totally see some idiot placing the currently existing AI in the decision chain. But we have already tested this situation. In 1983 lieutenant colonel Stanislav Petrov received the decision from the smart computer system matching all the instructions. Petrov was to launch a nuclear attack, but he did not press the button. Humanity has already been tested on this scenario and we have passed.

  2. Biological weapon. Some rogue actor may want to stage a global terrorist attack and use AI to develop a virus. But they would need to make to virus infectivity and virulence right to cause a global event. And we had Covid-19 recently. This virus was pretty close to the optimum to spread globally while being lethal enough to cause problems. But we survived. Humanity passed that test as well.

  3. Climate change. We have a big ongoing test still to be concluded, but we have passed a few smaller ones. The Montreal Protocol saved the ozone layer. We cleaned up environmental damage from coal industrialization in Europe. The AI is making this test harder to pass, but it is not unique. Large pets (mostly large dogs) are a much bigger problem than AI in that aspect.

  4. Asteroid, supervolcano, aliens, etc. The big fat etc. There is a long list of possible scenarios that all share one common trait: AI is not in any way connected to them. It is not increasing the chances, it does not help to defend. It is just irrelevant in any conversation about those potential events.

That is the end of the list. In order to be scary, AI needs some means of affecting humanity globally. And it does not really have too many good ones to really be scary.

AI can destroy the Internet by flooding it with garbage. It kind of already did. But I'll be fine without Internet, and so will be most of the humanity.

My position on the challenges the humanity is facing in the next century is both optimistic and pessimistic at the same time. The optimist in me is certain that we have all the tools to overcome all the challenges. The pessimist in me is convinced we'll try any other possible way up until the last possible way until finally using the tools we had from the beginning. I mean, we have the technology for the safe self-driving cars since mid-1990s, but instead of using it we are trying to mount cameras and lidars on every vehicle and connect every vehicle to a big single-point-of-failure server farm huge distance away. We got robo-taxis with safety record so bad, that nobody even attempts to compare them with regular taxis, instead they compare them to "average driver" - so that every unlicensed drunk teenager is also in the comparison.

AI might present a new set of challenges for us to overcome. But we already have all the tools needed to overcome them.

I posted about OpenAI's agents cheating on CyberBench tests back in May 2026. Imagine my surprise when this topic suddenly became very hot three months later with the narrative like "it is so scary and nobody knew". While I am sitting here in my room puzzled - I knew. Everybody knew. The transcripts were published online for everybody to see and there was a public discussion of them on Reddit. This is just another example of us (the humanity) having all the tools to address a certain problem ("rogue AI hacking HaggingFace" in this case), yet for some reason not using the tools we have.

In conclusion, my p(doom) is based on Bayesian probability and Laplace's argument. It is astronomically low. You have a better chance of guessing someone's Bitcoin private key, than you have a chance of dying due to AI. Have fun guessing.

Posted On

Category:

Tags: /