AI Isn’t Omnipotent. It’s Janky.

• Read original article here

Updated at 10:45 a.m. ET on April 3, 2023

In the past few months, artificial intelligence has managed to pass the bar exam, create award-winning art, and diagnose sick patients better than most physicians. Soon it might eliminate millions of jobs. Eventually it might usher in a post-work utopia or civilizational apocalypse.

At least those are the arguments being made by its boosters and detractors in Silicon Valley. But Amba Kak, the executive director of the AI Now Institute, a New York–based group studying artificial intelligence’s effects on society, says Americans should view the technology with neither a sense of mystery nor a feeling of awed resignation. The former Federal Trade Commission adviser thinks regulators need to analyze AI’s consumer and business applications with a shrewd, empowered skepticism.

Kak and I discussed how to understand AI, the risks it poses, whether the technology is overhyped, and how to regulate it. Our conversation has been condensed and edited for clarity.

Annie Lowrey: Let’s start off with the most basic question: What is AI?

Amba Kak: AI is a buzzword. The FTC has described the term artificial intelligence as a marketing term. They put out a blog post saying that the term has no discernible, definite meaning! That said, what we are talking about are algorithms that take large amounts of data. They process that data. They generate outputs. Those outputs could be predictions, about what word is going to come next or what direction a car needs to turn. They could be scores, like credit-scoring algorithms. They could be algorithms that rank content in a way, like in your news feed.

Lowrey: That sounds like technology that we already had. What’s different about AI in the past year or two?

Kak: You mean “generative AI.” Colloquially understood, these systems generate text, images, and voice outputs. Like many other kinds of AI, generative AI relies on large and often complex models trained on massive data sets—huge amounts of text scraped from sites like Reddit or Wikipedia, or images downloaded from Flickr. There are image generators, where you put in a text prompt and the output is an image. There are also text generators, where you put in a text prompt and you get back text.

Read: What have humans just unleashed?

Lowrey: Do these systems “think”? Are they more “human” or more “intelligent” than past systems working with huge amounts of data?

Kak: The short answer is no. They don’t think. They’re not intelligent. They are “haphazardly stitching together sequences of linguistic forms” they observe in the training data, as the AI researchers Emily M. Bender, Timnit Gebru, Angelina McMillan-Major, and Margaret Mitchell put it. There are vested interests that want us to see these systems as being intelligent and a stepping stone to the singularity and “artificial general intelligence.”

Lowrey: What does singularity mean in this context?

Kak: It has no clear meaning. It’s this idea that machines will be so intelligent that they will be a threat to the human race. ChatGPT is the beginning. The end is we’re all going to die.

These narratives are purposefully distracting from the fact that these systems aren’t like that. What they are doing is fairly banal, right? They’re taking a ton of data from the web. They’re learning patterns, spitting out outputs, replicating the learning data. They’re better than what we had before. They’re much more effective at mimicking the kind of interaction you might have with a human, or the kind of result you might get from a human.

Lowrey: When you look at the AI systems out there, what do you see as the most immediate, concrete risk for your average American?

Kak: One broad, big bucket of concerns is the generation of inaccurate outputs. Bad advice. Misinformation, inaccurate information. This is especially bad because people think these systems are “intelligent.” They’re throwing medical symptoms into ChatGPT and getting inaccurate diagnoses. As with other applications of algorithms—credit scoring, housing, criminal justice—some groups feel the pinch worse than others. The people who might be most at risk are people who can’t afford proper medical care, for instance.

A second big bucket of concerns has to do with security and privacy. These systems are very susceptible to being gamed and hacked. Will people be prompted to disclose personal information in a dangerous way? Will outputs be manipulated by bad actors? If people are using these as search engines, are they getting spammed? In fact, is ChatGPT the most effective spam generator we’ve ever seen? Will the training data be manipulated? What about phishing at scale?

One third big bucket is competition. Microsoft and Google are well poised to corner this market. Do we want them to have control over an even bigger swath of the digital economy? If we believe—or are being made to believe—that these large language models are the inevitable future, are we accepting that a few companies have a first-mover advantage and might dominate the market? The chair of the FTC, Lina Khan, has already said the government is going to scrutinize this space for anticompetitive behavior. We’re already seeing companies engage in potentially anticompetitive behavior.

Lowrey: One issue seems to be that these models are being created with vast troves of public data—even if that’s not data people intended to be used for this purpose. And the creators of the models are a small elite—a few thousand people, maybe. That seems like an ideal way to amplify existing inequalities.

Read: Why are we letting the AI crisis just happen?

Kak: OpenAI is the company that makes ChatGPT. In an earlier version, some of the training data was sourced from Reddit, user-generated content known for being abusive and biased against gender minorities and members of racial and ethnic minority groups. It would be no surprise that the AI system reflects that reality.

Of course the risk is that it perpetuates dominant viewpoints. Of course the risk is that it reinforces power asymmetries and inequalities that already exist. Of course these models are going to reflect the data that they’re trained on, and the worldviews that are embedded in that data. More than that, Microsoft and Google are now going to have a much wider swath of data to work from, as they get these inputs from the public.

Lowrey: How much is regulating AI like regulating social media? Many of the concerns seem the same: the viral spread of misinformation and disinformation, the use and misuse of truly enormous quantities of personal information, and so on.

Kak: It took a few tech-driven crisis cycles to bring people to the consensus that we need to hold social-media companies accountable. With Cambridge Analytica, countries that had moved one step in 10 years on privacy laws all of a sudden moved 10 steps in one year. There was finally momentum across political ideologies. With AI, we’re not there. We need to galvanize the political will. We do not need to wait for a crisis.

In terms of whether regulating AI is like regulating other forms of media or tech: I get tired of saying this, but this is about data protection, data privacy, and competition policy. If we have good data-privacy laws and we implement them well, if we protect consumers, if we force these companies to compete and do not allow them to consolidate their advantages early—these are key components. We’re already seeing European regulators step in using existing data-privacy laws to regulate AI.

Lowrey: But we don’t do a lot of tech regulation, right? Not compared with, say, the regulation of energy utilities, financial firms, providers of health care.

Kak: Big banks are actually a useful way of thinking about how we should be regulating these firms. The actions of large financial firms can have diffuse, unpredictable effects on the broader financial system, and thus the economy. We cannot predict the particular harm that they will cause, but we know they can. So we put the onus on these companies to demonstrate that they are safe enough, and we have a lot of rules that apply to them. That’s what we need to have for our tech companies, because their products have diffuse, unpredictable effects on our information environment, creative industries, labor market, and democracy.

Lowrey: Are we starting from scratch?

Kak: Absolutely not. We are not starting with a blank slate. We already have enforcement tools. This is not the Wild West.

Generative AI is being used for spam, fraud, plagiarism, deepfakes, that kind of stuff. The FTC is already empowered to tackle these issues. It can force companies to substantiate their claims, including the claim that they’ve mitigated risks to users. Then there are the sectoral regulators. Take the Consumer Financial Protection Bureau. It could protect consumers from being harmed by chatbots in the financial sector.

Kak: There are bills that have been languishing on the Hill regarding algorithmic accountability, algorithmic transparency, and data privacy. This is the moment to strengthen them and pass them. Everybody’s talking about futuristic risks, the singularity, existential risk. They’re distracting from the fact that the thing that really scares these companies is regulation. Regulation today.

This would address questions like: What training data are you using? Where does it come from? How are you mitigating against discrimination? How are you ensuring that certain types of data aren’t being exploited, or used without consent? What security vulnerabilities do you have and how are you protecting against them? It’s a checklist, almost. It sounds boring. But you get these companies to put their answers on paper, and that empowers the regulators to hold them accountable and initiate enforcement when things go wrong.

In some legislative proposals, these rules won’t apply to private companies. They’re regarding the government use of algorithms. But it gives us a framework we can strengthen and amend for use on private businesses. And I would say we should go much further on the transparency and documentation elements. Until these companies do due diligence, they should not be on the market. These tools should not be public. They shouldn’t be able to sell them.

Lowrey: Does Washington really have its head around this?

Kak: It’s always tempting to put the blame on lawmakers and regulators. They’re slow to understand this technology! They’re overwhelmed! It’s missing the point and it’s not true. It works in the interest of industry. OpenAI and Anthropic and all these companies are telling lawmakers and the public that nobody’s as worried about this as they are. We’re capable of fixing it. But these are magic, unknowable systems. Nobody but us understands them. Maybe we don’t even understand them.

There are promising signs that regulators aren’t listening. Regulators at the FTC and elsewhere are saying, We’re going to ask questions. You’re going to answer. We’re going to set the terms of the debate, not you. That’s the crucial move. We need to place the burden on companies to assure regulators and lawmakers and the public. Lawmakers don’t need to understand these systems perfectly. They just need to ask the companies to prove to us that they’re not unleashing them on the public when they think they might do harm.

Lowrey: Let’s talk about the hypothetical long-range risk. A recent public letter called for a six-month halt on AI development. Elon Musk and hundreds of other tech leaders signed it. It asked, and I quote: “Should we develop nonhuman minds that might eventually outnumber, outsmart, obsolete and replace us? Should we risk loss of control of our civilization?” Are these concerns you share? What do you make of those questions?

Kak: Yeah, no. This is a perfect example of a narrative meant to frighten people into complacency and inaction. It shifts the conversation away from the harm these AI systems are creating in the present. The issue is not that they’re omnipotent. It is that they’re janky now. They’re being gamed. They’re being misused. They’re inaccurate. They’re spreading disinformation.

Lowrey: If you were a member of Congress and you had Sam Altman, the head of OpenAI, testifying before you, what would you ask him?

Kak: Apart from the laundry list of gaps in knowledge on training data, I would ask for details about the relationship between OpenAI and Microsoft, information about what deals they have under way—who’s actually buying this system and how are they using it? Why did the company feel confident enough that it had mitigated enough risk to go forward with commercial release? I would want him to show us documentation, receipts of internal company processes.

Let’s really put him on the spot: Is OpenAI following the laws that exist? My guess is he’d answer that he doesn’t know. That’s exactly the problem. We’re seeing these systems being rolled out with minimal internal or external scrutiny. This is key, because we’re hearing a lot of noise from these executives about their commitments to safety and so on. But surprise! Conspicuously little support for actual, enforceable regulation.

Let’s not stop at Sam Altman, just because he’s all over the media right now. Let’s call Satya Nadella of Microsoft, Sundar Pichai of Google, and other Big Tech executives too. These companies are competing aggressively in this market and control the infrastructure that the whole ecosystem depends on. They’re also significantly more tight-lipped about their policy positions.

Lowrey: I guess a lot of this will become more concrete when folks are using AI technologies to make money. Companies are going to be using this stuff to sell cars soon.

Kak: This is an expensive business, whether it’s the computing costs or the cost of human labor to train these AI systems to be more sophisticated or less toxic or abusive. And this is at a time when financial headwinds are affecting the tech industry. What happens when these companies are squeezed for profit? Regulation becomes more important than ever, to prevent the bottom line from dictating irresponsible choices.

Lowrey: Let’s say we don’t regulate these companies very well. What does the situation look like 20 years from now?

Kak: I can definitely speculate about the unreliable and unpredictable information environment we’d find ourselves in: misinformation, fraud, cybersecurity vulnerability, and hate speech.

Here’s what I know for sure. If we don’t use this moment to reassert public control over the trajectory of the AI industry, in 20 years we’ll be on the back foot, responding to the fallout. We didn’t just wake up one morning with targeted advertising as the business model of the internet, or suddenly find that tech infrastructure was controlled by a handful of companies. It happened because regulators didn’t move when they needed to. And the companies told us they would not “be evil.”

With AI, we’re talking about the same companies. Rather than take their word that they’ve got it covered, rather than getting swept up in their grand claims, let’s use this moment to set guardrails. Put the burden on the companies to prove that they’re going to do no harm. Prevent them from concentrating power in their hands.

This article previously misstated Lina Khan’s affiliation and the name of Anthropic.