The AI Timelines Scam

[epistemic status: that’s just my opinion, man. I have highly suggestive evidence, not deductive proof, for a belief I sincerely hold]

“If you see fraud and do not say fraud, you are a fraud.”Nasim Taleb

I was talking with a colleague the other day about an AI organization that claims:

  1. AGI is probably coming in the next 20 years.
  2. Many of the reasons we have for believing this are secret.
  3. They’re secret because if we told people about those reasons, they’d learn things that would let them make an AGI even sooner than they would otherwise.

His response was (paraphrasing): “Wow, that’s a really good lie! A lie that can’t be disproven.”

I found this response refreshing, because he immediately jumped to the most likely conclusion.

Near predictions generate more funding

Generally, entrepreneurs who are optimistic about their project get more funding than ones who aren’t. AI is no exception. For a recent example, see the Human Brain Project. The founder, Henry Makram, predicted in 2009 that the project would succeed in simulating a human brain by 2019, and the project was already widely considered a failure by 2013. (See his TED talk, at 14:22)

The Human Brain project got 1.3 billion Euros of funding from the EU.

It’s not hard to see why this is. To justify receiving large amounts of money, the leader must make a claim that the project is actually worth that much. And, AI projects are more impactful if it is, in fact, possible to develop AI soon. So, there is an economic pressure towards inflating estimates of the chance AI will be developed soon.

Fear of an AI gap

The missile gap was a lie by the US Air Force to justify building more nukes, by falsely claiming that the Soviet Union had more nukes than the US.

Similarly, there’s historical precedent for an AI gap lie used to justify more AI development. Fifth Generation Computer Systems was an ambitious 1982 project by the Japanese government (funded for $400 million in 1992, or $730 million in 2019 dollars) to create artificial intelligence through massively parallel logic programming.

The project is widely considered to have failed.  From a 1992 New York Times article:

A bold 10-year effort by Japan to seize the lead in computer technology is fizzling to a close, having failed to meet many of its ambitious goals or to produce technology that Japan’s computer industry wanted.

That attitude is a sharp contrast to the project’s inception, when it spread fear in the United States that the Japanese were going to leapfrog the American computer industry. In response, a group of American companies formed the Microelectronics and Computer Technology Corporation, a consortium in Austin, Tex., to cooperate on research. And the Defense Department, in part to meet the Japanese challenge, began a huge long-term program to develop intelligent systems, including tanks that could navigate on their own.

The Fifth Generation effort did not yield the breakthroughs to make machines truly intelligent, something that probably could never have realistically been expected anyway. Yet the project did succeed in developing prototype computers that can perform some reasoning functions at high speeds, in part by employing up to 1,000 processors in parallel. The project also developed basic software to control and program such computers. Experts here said that some of these achievements were technically impressive.

In his opening speech at the conference here, Kazuhiro Fuchi, the director of the Fifth Generation project, made an impassioned defense of his program.

“Ten years ago we faced criticism of being too reckless,” in setting too many ambitious goals, he said, adding, “Now we see criticism from inside and outside the country because we have failed to achieve such grand goals.”

Outsiders, he said, initially exaggerated the aims of the project, with the result that the program now seems to have fallen short of its goals.

Some American computer scientists say privately that some of their colleagues did perhaps overstate the scope and threat of the Fifth Generation project. Why? In order to coax more support from the United States Government for computer science research.

(emphasis mine)

This bears similarity to some conversations on AI risk I’ve been party to in the past few years. The fear is that Others (DeepMind, China, whoever) will develop AGI soon, so We have to develop AGI first in order to make sure it’s safe, because Others won’t make sure it’s safe and We will. Also, We have to discuss AGI strategy in private (and avoid public discussion), so Others don’t get the wrong ideas. (Generally, these claims have little empirical/rational backing to them; they’re based on scary stories, not historically validated threat models)

The claim that others will develop weapons and kill us with them by default implies a moral claim to resources, and a moral claim to be justified in making weapons in response. Such claims, if exaggerated, justify claiming more resources and making more weapons. And they weaken a community’s actual ability to track and respond to real threats (as in The Boy Who Cried Wolf).

How does the AI field treat its critics?

Hubert Dreyfus, probably the most famous historical AI critic, published “Alchemy and Artificial Intelligence” in 1965, which argued that the techniques popular at the time were insufficient for AGI. Subsequently, he was shunned by other AI researchers:

The paper “caused an uproar”, according to Pamela McCorduck.  The AI community’s response was derisive and personal.  Seymour Papert dismissed one third of the paper as “gossip” and claimed that every quotation was deliberately taken out of context.  Herbert A. Simon accused Dreyfus of playing “politics” so that he could attach the prestigious RAND name to his ideas. Simon said, “what I resent about this was the RAND name attached to that garbage.”

Dreyfus, who taught at MIT, remembers that his colleagues working in AI “dared not be seen having lunch with me.”  Joseph Weizenbaum, the author of ELIZA, felt his colleagues’ treatment of Dreyfus was unprofessional and childish.  Although he was an outspoken critic of Dreyfus’ positions, he recalls “I became the only member of the AI community to be seen eating lunch with Dreyfus. And I deliberately made it plain that theirs was not the way to treat a human being.”

This makes sense as anti-whistleblower activity: ostracizing, discrediting, or punishing people who break the conspiracy to the public. Does this still happen in the AI field today?

Gary Marcus is a more recent AI researcher and critic. In 2012, he wrote:

Deep learning is important work, with immediate practical applications.

Realistically, deep learning is only part of the larger challenge of building intelligent machines. Such techniques lack ways of representing causal relationships (such as between diseases and their symptoms), and are likely to face challenges in acquiring abstract ideas like “sibling” or “identical to.” They have no obvious ways of performing logical inferences, and they are also still a long way from integrating abstract knowledge, such as information about what objects are, what they are for, and how they are typically used. The most powerful A.I. systems … use techniques like deep learning as just one element in a very complicated ensemble of techniques, ranging from the statistical technique of Bayesian inference to deductive reasoning.

In 2018, he tweeted an article in which Yoshua Bengio (a deep learning pioneer) seemed to agree with these previous opinions. This tweet received a number of mostly-critical replies. Here’s one, by AI professor Zachary Lipton:

There’s a couple problems with this whole line of attack. 1) Saying it louder ≠ saying it first. You can’t claim credit for differentiating between reasoning and pattern recognition. 2) Saying X doesn’t solve Y is pretty easy. But where are your concrete solutions for Y?

The first criticism is essentially a claim that everybody knows that deep learning can’t do reasoning. But, this is essentially admitting that Marcus is correct, while still criticizing him for saying it [ED NOTE: the phrasing of this sentence is off (Lipton publicly agrees with Marcus on this point), and there is more context, see Lipton’s reply].

The second is a claim that Marcus shouldn’t criticize if he doesn’t have a solution in hand. This policy deterministically results in the short AI timelines narrative being maintained: to criticize the current narrative, you must present your own solution, which constitutes another narrative for why AI might come soon.

Deep learning pioneer Yann LeCun’s response is similar:

Yoshua (and I, and others) have been saying this for a long time.
The difference with you is that we are actually trying to do something about it, not criticize people who don’t.

Again, the criticism is not that Marcus is wrong in saying deep learning can’t do certain forms of reasoning, the criticism is that he isn’t presenting an alternative solution. (Of course, the claim could be correct even if Marcus doesn’t have an alternative!)

Apparently, it’s considered bad practice in AI to criticize a proposal for making AGI without presenting on alternative solution. Clearly, such a policy causes large distortions!

Here’s another response, by Steven Hansen (a research scientist at DeepMind):

Ideally, you’d be saying this through NeurIPS submissions rather than New Yorker articles. A lot of the push-back you’re getting right now is due to the perception that you haven’t been using the appropriate channels to influence the field.

That is: to criticize the field, you should go through the field, not through the press. This is standard guild behavior. In the words of Adam Smith: “People of the same trade seldom meet together, even for merriment and diversion, but the conversation ends in a conspiracy against the public, or in some contrivance to raise prices.”

(Also see Marcus’s medium article on the Twitter thread, and on the limitations of deep learning)

[ED NOTE: I’m not saying these critics on Twitter are publicly promoting short AI timelines narratives (in fact, some are promoting the opposite), I’m saying that the norms by which they criticize Marcus result in short AI timelines narratives being maintained.]

Why model sociopolitical dynamics?

This post has focused on sociopolotical phenomena involved in the short AI timelines phenomenon. For this, I anticipate criticism along the lines of “why not just model the technical arguments, rather than the credibility of the people involved?” To which I pre-emptively reply:

  • No one can model the technical arguments in isolation. Basic facts, such as the accuracy of technical papers on AI, or the filtering processes determining what you read and what you don’t, depend on sociopolitical phenomena. This is far more true for people who don’t themselves have AI expertise.
  • “When AGI will be developed” isn’t just a technical question. It depends on what people actually choose to do (and what groups of people actually succeed in accomplishing), not just what can be done in theory. And so basic questions like “how good is the epistemology of the AI field about AI timelines?” matter directly.
  • The sociopolitical phenomena are actively making technical discussion harder. I’ve had a well-reputed person in the AI risk space discourage me from writing publicly about the technical arguments, on the basis that getting people to think through them might accelerate AI timelines (yes, really).

Which is not to say that modeling such technical arguments is not important for forecasting AGI. I certainly could have written a post evaluating such arguments, and I decided to write this post instead, in part because I don’t have much to say on this issue that Gary Marcus hasn’t already said. (Of course, I’d have written a substantially different post, or none at all, if I believed the technical arguments that AGI is likely to come soon had merit to them)

What I’m not saying

I’m not saying:

  1. That deep learning isn’t a major AI advance.
  2. That deep learning won’t substantially change the world in the next 20 years (through narrow AI).
  3. That I’m certain that AGI isn’t coming in the next 20 years.
  4. That AGI isn’t existentially important on long timescales.
  5. That it isn’t possible that some AI researchers have asymmetric information indicating that AGI is coming in the next 20 years. (Unlikely, but possible)
  6. That people who have technical expertise shouldn’t be evaluating technical arguments on their merits.
  7. That most of what’s going on is people consciously lying. (Rather, covert deception hidden from conscious attention (e.g. motivated reasoning) is pervasive; see The Elephant in the Brain)
  8. That many people aren’t sincerely confused on the issue.

I’m saying that there are systematic sociopolitical phenomena that cause distortions in AI estimates, especially towards shorter timelines. I’m saying that people are being duped into believing a lie. And at the point where 73% of tech executives say they believe AGI will be developed in the next 10 years, it’s a major one.

This has happened before. And, in all likelihood, this will happen again.

3 thoughts on “The AI Timelines Scam

  1. Hi Jessica. Nice post and I agree with many of your points. Certainly, I believe—as you do—that a number of bad actors are wielding the specter of AGI sloppily and irresponsibly, either to consciously defraud people or on account of buying into something that speaks more to the messianic than to science. Perhaps ironically, one frequent debate that I have had with Gary in the past is that while he is vocally critical of exuberance over deep learning, he is himself partial to speaking rosily of nearish-term AGI, and of claiming progress towards it. On the other hand, I am considerably more skeptical.

    While I enjoyed the post and think we agree on many points, if you don’t mind I would like to respectfully note that I’ve been quoted here slightly out of context and would like to supply that missing context. To be sure, I think your post is written well and with honest intentions, and I know how easy it is to miss some context in Twitter threads.

    Regarding my friend Gary Marcus. I like Gary and we communicate fairly regularly, but we don’t always agree on the science or on the discourse.

    In this particular case, he was specifically insisting to be “the first” to make a set of arguments (which contradicted my understanding of history). When I say “Saying it louder ≠ saying it first.”, I am truly just pushing back on this specific point—the assertion that he said it first. Among others, Judea Pearl has argued the limits of curve fitting far more rigorously and far earlier in history.

    There is nothing dishonest or contradictory about agreeing with a broader point and simultaneously disagreeing with a man’s claim to have originated it. I took exception to the hyperbole, not the message.

    After the quote, your post notes: “but, this is essentially admitting that Marcus is correct, while still criticizing him for saying it” — what is there to “admit”? I myself have made similar critical arguments both in technical papers, position papers, blog posts, and the popular press. Characterizing “agreement” as “admitting” something, makes the false insinuation that somehow I have been on the wrong side of the debate.

    For the record, I am acknowledged in Gary’s paper on limitations of deep learning (which you reference here) for giving him a large amount of constructive feedback and have myself, perhaps in defiance of Adam Smith’s aphorism, been vocally critical within the technical forums, recently publishing “Troubling Trends in Machine Learning Scholarship” at the ICML debates (https://arxiv.org/abs/1807.03341, subsequently released by CACM), which I suspect is in the spirit of the sort of critical writing that you appear to encourage.

    Further, I’d like to address the second part of the discussion. When I say “Saying X doesn’t solve Y is pretty easy. But where are your concrete solutions for Y?” My point here is that Gary doesn’t just push back on the false claims made about current technology that doesn’t do Y. He also makes public attacks on the people working on X. Essentially their crime is that they haven’t developed technical solutions to Y. If failing to solve that particular moonshot is a crime, then Gary too is just as guilty. These attacks strike me as inappropriate and misplaced (compared to the more reasonable pushback on misinformation in the public sphere.

    Like

  2. (sorry if this is a repeat, it’s hard to tell if it’s actually posting & being held, or if wordpress just dumped my response. Since I’m curious for your thoughts, trying again just in case!)

    Hey Jessica, confused about one point in your article – but agree with most of it, particularly the point re: people with bad arguments like “you need to have a counter proposal to criticize”, and the points about incentives.

    I think the points re “many reasons why we believe this aren’t public” and “writing publicly about technical arguments/ correcting epistemic mistakes might be bad” make sense under a view that seems likely to be held by the AI safety people.

    E.g. suppose you believe there are two hard chunks of work, completing one of which (A) has a significant chance of destroying the world, unless the accessory work (B) was already completed. Most people are interested in A for money-seeking reasons, and don’t really pay attention to arguments for danger.

    Given that view, helping people notice their mistakes in trying to complete A seems potentially pretty bad for the world, at least at first glance. Similarly, if you say “A is close” and expect that to attract more attention to A but not B (A for the money-seeking reasons & because most people don’t pay attention to danger), that could possibly also be pretty bad for the world.

    But I may be misinterpreting your point – if your point is “this is not persuasive from the outside/ hard to distinguish from a lie”, makes sense.

    I just also think it makes sense for them to say what they said, given a view that they likely hold (but could be persuaded otherwise, haven’t thought about it much). Would love to hear your thoughts!

    Like

    1. I think there are possible beliefs one could have that would imply that certain things are bad to talk about publicly. My main point is that, from the outside, it’s hard to distinguish from a lie, and that furthermore such lies are common (there are pressures towards inflating the importance of one’s secrets).

      It isn’t impossible that you could simultaneously believe X and that the public shouldn’t believe X (i.e. you want the public to keep being misinformed). But this implies:

      There isn’t a conflict of interest if someone publicly argues for not-X. You wanted the public to be misinformed about X anyway.
      The public has little reason to think you’re acting in the public interest (after all, you’re trying to keep them misinformed)

      (Note, this doesn’t imply that people should be actively trying to market their ideas to a broad audience; by default, if you post an idea on some niche website, it doesn’t get public traction. By “public” I mean “the specific intellectual public that would be interested in listening to your ideas if you were saying them”.)

      Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s