Eliezer Yudkowsky is on Facebook. To connect with Eliezer, join Facebook today.
Join
or
Log In

Eliezer Yudkowsky

"Readers made insomnious by 'newspaper talk' of terrific atomic war weapons... may now get sleep."

I don't think it would be inappropriate to just copy and paste this tweet URL every time an AI researcher says almost exactly this about superintelligence.

In reality the comparison is unfair to the 1940s SciAm writer, who at least offers the reader a technical discussion. Sure, the conclusion falls flat in the face of the apparently unimaginable possibility of two more years of technological advancement, intelligent optimization around the supposed obstacle, and additional research that hadn't yet reached the mainstream press. Sure, all the scientists or the reporter ever knew of the meaning of the word 'impossible' was something they didn't know how to do right that minute, a state not labeled reachable in their personal search tree. Sure, that can change in an arbitrarily short time period, literally two years, for all that it seems unimaginable to those living in the moment who just can't see how to do it with current technology. Sure, they should have been able to see that coming from the previous historical case of the Wright Brothers, and heavier-than-air flight being declared 'fifty years off' two years before it was done. But for all of that, this discussion is still on a qualitative level above modern discussion of superintelligence in the press; people offer *technical reasons* for saying what they say, rather than just a dismissive sniff. Standards of discourse were higher back then, I guess.

Still, I don't think it's invidious comparison to just copy and paste the URL to this tweet, every time an AI researcher helplessly dances through this exact repetition of mistaken cognition.

EDIT: Leif K-Brooks says that the researchers interviewed were part of Leo Szilard's conspiracy! 'At the end of October Halban, Joliot and Kowarski deposited a sealed envelope with the Academy of Science. This was opened in 1949 and the paper inside “Sur la possibilit´e de produire dans un milieu uranif`ere des r´eactions nucl´eaires en chaine illimit´ee”'. This puts a rather different face on things and invalidates the newspaper article as a comparison for modern AGI dismissals.

March 14 at 4:41pmPublic

David Manley

"The value misalignment poohs out as the intelligence increases." I love "poohs out" apparently used here as a technical expression.
Edited3Mar 14

Lauren Chris Horne

> Standards of discourse were higher back then, I guess.

I predict that you do not think [edit for clarification: before updating on this message] that they would have discussed your superintelligence fears any differently then vs now. I'd make that bet at $0.66:$0.34, and will take it with no more than one person.
Edited4Mar 14

Yevhenii Diomidov

WTF? Why is there a paywall on a 77-year-old article?
3Mar 14

Leif K-Brooks

The tweeter rules out this being disinformation. I Googled around, and I believe they are incorrect to do that. See here: https://arxiv.org/pdf/physics/0206076.pdf The team SciAm quoted as disproving the possibility knew it was possible before this article was written, but they sealed that finding.

>Szilard started to persuade all physicists working on fission to cease publishing; he sent a cable to Joliot on April 6th [1939] requesting a delay in further publications “in view of possible misuse in Europe”

>At the end of October Halban, Joliot and Kowarski deposited a sealed envelope with the Academy of Science. This was opened in 1949 and the paper inside “Sur la possibilit´e de produire dans un milieu uranif`ere des r´eactions nucl´eaires en chaine illimit´ee”
Edited30Mar 14

Jai Dhyani

Full text, taken from http://documentslide.com/documents/50-and-100-years-ago-57a0690e8f61c.html:

> SOANDIOO YEARSAGO MAY, 1940: "Just about a year ago, two German physicists who had been gunning at the metal uranium with neu­tron bullets suddenly found that they had caused the biggest explosion in atomic history. The new phenomenon was called 'nuclear fission.' Besides the two main fragments, a few spare neu­trons were thrown off from the original nucleus. Early last summer a chilly sensation began tingling up and down the spines of the experimenters. Wasn't there a dangerous possibility that the uranium would at last become explo­sive? Now, a year after the original dis­ covery, word comes from Paris that we don't have to worry-at least, probably not. Scientists there have found that, in­stead of building up to a grand climax, the uranium-fission chain reaction runs down and stops like an unwound dock."
Mar 14

Cameron Cowan

I read Superintelligence by Nick Bostrom and while the tech stuff was not helpful to me, the lay person, it was helpful conceptually on things. I just don't thing AI is a close as we think it is. I think AI is very hard and even though times have changed quantum computing and new forms of processing not thought of will be needed to make a true AI.
Mar 14

Ben Henley

So the conclusion I draw from this post and the subsequent edit is that there's a secret AI arms race, superintelligent AI is 2 years away, and the experts who say it's decades away are all part of a US disinformation campaign.

I've always suspected that since 9/11 most of the NSA's budget has been diverted to emulating Dick Cheney's mind with ever increasing fidelity and at at increasing multiples of real time.
6Mar 14