`p(doom)` is ridiculous nonsense. How do people even come up with those numbers? Some people just say "I think it's 20% that AI will wipe us out" but how did they arrive at this number? Yeah they just made it up out of thin air, that's right. Complete fabrication without any evidence to back up such a probability. It's not like an asteroid whose trajectory you measure and you have certain instrument tolerances and measurement inaccuracies, hence the laws of physics tell you that your prediction has a 20% accuracy. No, it's all just made up bullshit. Just like all the AI safety guys: They're professional bullshitters and have been all along. Doesn't mean they're wrong per se, but it means that if they're right, they're right because they guessed right.
Sorry to break it to you, but all predictions are "made up". So much so you don't even know if you'll be dead or alive this time tomorrow.
The best we can do is have a Bayesian probability distribution about future outcomes.
I don't ever like holding a singular opinion on anything or converging on a single value for things like p(doom), but generally when I talk about things with people I try to represent my views as singular values or binary opinions. So I agree with you that p(doom) as a concept is silly in the same way as you saying you know you won't be dead in 24 hours is silly.
I can't speak for everyone, but I suspect most people who express a p(doom) haven't thought much about it so I agree with you there too. But I do personally put a decent amount of effort modelling out scenarios so I'll just speak to my own p(doom).
My p(doom) is high largely because I believe certain things about how AI capabilities will scale, how human incentives will shape certain outcomes, and our inability to slow progress in a field like AI. Basically:
- It's almost impossible to significantly slow AI progress without a doom-lite scenarios like nuclear war
- AI will scale to ASI in the coming decades
- Corruption is the norm, and democratic systems today are backed largely by the fact the public hold a monopoly over economic power - and all power derived from economic prosperity
- AI will replace 90% of productive human labour in the coming decades and erode the economic value of the humans, resulting over time in political corruption (resource curse)
- ASI unlocks all tech on the tech tree which primarily requires intelligence (which is likely to be the majority of tech)
- There is far more destructive/dangerous tech later on the tech tree than there is good. E.g., for every cancer cure there is 1,000s bioweapons and new torture techniques we can prompt Claude to create
- Even if humanity could agree on a workable definition of alignment it's a much harder problem than advancing capabilities, therefore capabilities will almost certainly progress faster than our ability to align advanced AI
- The field of alignment at it's limit resolves to basically just deciding between a god that follows the commands of a single or small group of humans (dangerous), or a god that has it's own moral considerations and does not listen to humans (dangerous).
So my p(doom) is largely a product of all of the above, and my general inability to find many neutral or good scenarios of ASI while landing on almost endless ways ASI is likely to go wrong regardless of whether you believe terminator-like scenarios. To be honest the only reason my personal p(doom) isn't higher than 99% is that I tend to weight the opinions of others whose opinions I value quite highly, and most people whose opinions I value have a slightly lower p(doom). I've rerated a lot recently because I was hoping some experts would turn out to be right about there being algorithmic limitations to ASI and things like "alignment by default". But I now think I've over weighted those opinions historically and a lot of the better arguments against a high p(doom) are either now dead in the water or have become increasingly hard to hold with any significant probability.
My best argument for against an AI doom scenario this century would be nuclear war, which I personally discard in my AI p(doom) probability. Allowing for other doom scenarios my AI p(doom) is probably closer to 90%.
Of course. They literally spell their playbook out for you:
“We decide something, then put it out there and wait for a while to see what happens.
If there is then no great outcry and no uprisings, because most people do not even understand what has been decided, then we continue—step by step, until there is no turning back.”
I once worked in a hardware lab that used wire shelving for holding arrays of running machines, and a bad ground running through one chassis caused anyone who touched the shelf to get shocked by something very close to line voltage.
That shelf is more glue and fire retardant than it is wood. Its also really bad as a book shelf so it might still be superior in this role than the intended one.
It depends on the loading. If you've got it 9-12" high apart and are putting paperbacks on there... that shouldn't be a problem.
However, when the shelf becomes multiples of that, then people start putting hardcover volumes or laying the book flat and stacking them high within the shelf.
It won't necessarily fail, but it can substantially sag with heavier loads.
This is an issue for boardgammers who are after larger and heavier shelf spaces which is why the Kallax shelving is much preferred. I'll also note that Ivar shelving is solid wood rather than particleboard.
I bought a hammer 2 decades ago and it still works flawlessly without me performing any maintenance. One day, the idiots in software engineering will realize that this is how tools are supposed to behave.
Since MS bought it, aren't there two versions, a legacy one called "Java" and the newer one from MS, probably not in Java and with all the cool kids (multiplayer mode) on it?
reply