Why AI is Ew
Someone on reddit responded to an “Ew AI” comment with a request for an explanation of the “visceral reaction”. A lot has been written about this so I was a little skeptical it was a good faith question, but I’ve been meaning to write some of this down in one place anyway, so I answered in good faith. This is a mildly cleaned up and expanded version of that response.
I have been to many presentations on genAI by its users and proponents, and have read lots by both its proponents and opponents, but I most certainly do not claim to be an expert. All of this has been expounded upon by people who’ve put in a lot more time and rigorous effort, and I would encourage you to seek them out (uh-oh, I’m going to need to make a bibliography at some point, aren’t I?). I’ve simply been asked about my position enough times that I wanted to have it written down in one place.
The below is intended to be the rationale behind the visceral reaction, but, to be clear, I think a straight “Ew, AI” is entirely warranted and, in most contexts, sufficient.
Oh, one final note: lots of genAI proponents will play a game where they find a use case where some of these don’t apply. Sure, yes, fine. I don’t care. If your tool manages to avoid IP theft but still boils the oceans, I don’t care. If you avoid boiling the oceans but still centralize power, I don’t care. Come back when you’ve found a way to not make the world worse, not when you’ve found a way to make it worse in fewer ways.
tl;dr:
Current “genAI” systems have numerous ethical issues which make using them responsibly impossible in most cases. Those ought to be enough to rule out their use. They additionally have many technical problems which make their use irresponsible or at least unwise in most cases.
Expanded Edition
Ethics
First (and really for me it stops here): these are an ethical nightmare.
Any time you hear someone say “ethical issues aside”, throw up all the red flags you have. I’m not saying it’s never worth doing that as an academic exercise, but proceed with extreme caution. At a minimum, make sure you come back to those ethical issues before settling the issue.
Environmentally Catastrophic
Training these models is environmentally catastrophic. It requires exorbitant energy and water, at a time when we ought to be working hard to reduce consumption of both of those. The proponents of these systems will tell you that each use uses relatively little electricity, and that’s true, but requires you to completely ignore how the thing is created. If, somehow, we managed to decide collectively “OK, these are good enough, we don’t need to train any new ones, and we will all commit to not use any newer ones”, you could maybe, maybe justify using them as sunk cost. But of course that isn’t the reality we live in. The people who created the current generation of models are already working on the next generation, and the next one after that. And using more water and burning more rainforest to do it. Use of the current generation creates demand for the next generation, providing the financial incentive and, given how the investments for these things work, the funding needed to do it.
IP Theft (and its inconsistent enforcement)
All these models are built on massive intellectual property theft. Maybe you care about that, maybe you don’t. I’ve always been something of a copyright minimalist (not quite an abolitionist), and could, in some ways, mostly get over this one, in some contexts, if it wasn’t so out of line with the massive damage we’ve caused to people’s lives for things like downloading a Metallica MP3. If Aaron Swartz was still walking around, maybe I could get over it. The theft only goes one way, though, only towards the already powerful, and that’s unacceptable.
Questionable Legal Status of the Output
Relatedly, if you do care about copyright and related issues at all, the output of these is also really problematic, and still sort of an unresolved legal question. I would not consider this a settled matter of law, but in the US, at least, the dominant position held by the highest courts these cases have gotten to is that the output of these programs is simply not subject to copyright. Maybe that’s fine with you, I am certainly not going to argue that people need to care more about strong copyrights, but the hypocrisy of places like GNU or Debian and others that have always claimed to care about license status as a method for “protecting freedom” embracing these tools is really disturbing. The second most popular legal theory (again, I am only really aware of this part of the issue in the United States) is that the output qualifies as a derivative work of the training data, which, given how the training data was collected, means they are almost all functionally illegal, since the license terms of the many components of the training data are incompatible.
Corporate Social Impact
Most or all of the companies behind the commercial models have really terrible social impact, up to and including supporting or outright endorsing fascist actors. Use of these models, even if not paying them explicitly, increases their power to support these terrible policies. And, of course, there’s regularly actual money involved. I include here use of AI tech in weapons targeting systems, of course, but it also applies to things like who these corporations are giving money to. They are actively supporting fascist elements in the United States and Europe, at least, because those are the elements most inclined to go along with the con.
Not unrelatedly, the majority of these tools in use today are run by landlords looking for renters. They turn what have largely been independent, accessible, often free (or nearly so) activities into things you need pay them for the tools you depend on. They centralize the benefits of this new model, but in terms of power and money.
Social Impact of the Tools Themselves
The tools themselves also have big problems with social impact. The racial and gender-based biases in these systems are well-documented, and yet they keep comping up. Defenders of these tools will point out that they’re “simply” reflecting the biases in the training data, which reflects the biases inherent in our society. Sure, that’s true. But that is not a defense of these tools, it’s yet another reason not to use them.
Labor Issues
These tools, and the people pushing them, see human labor as an inefficiency to be removed, or a bottleneck on production. That’s (literally; obviously) dehumanizing on the entire process. I want human judgement involved in most of these processes. I want a human being to have to say “is this a good idea? Is it worth my effort?”. I don’t want things to be slower or harder for no reason, but having human judgement in the process has reasons. I want a human being—actually, many human beings—to have to make ethical judgements. In practical terms, I want workers to be able to have a voice when a organization is doing something bad. I want labor—human labor— to have more power, not less.
Additionally, so much of “AI” systems really amount to something of a combination mechanical turk and a shell game, moving the cost of labor from one place to another. In the programming world, for example, automated systems for generating changes and “fixes” increase the burden of review (since competently reviewing code you didn’t write is harder than writing the code) while also shifting that work onto a different set of people (typically not the ones who’ve bought into the AI toolset, and typically ones who’re already overworked and undercompensated).
Technical
OK, let’s move on from a purely ethical. This is where it gets easier for the genAI proponents to play the “well this use case doesn’t have X set of problems” game. Again, I consider the ethical issues above sufficient to write off these tools. But it is worth understanding what we’re giving up by doing so (spoiler: a lot less than proponents would have you believe).
De-skilling Workers (and Learners)
There is lots of evidence that use of these tools diminishes the skills of the people using them. Much of this evidence comes not from fringe opponents of the technology, but from studies conducted by Anthropic themselves, who have found it in multiple cases (they are simply unconcerned with that happening, since having less skilled workers benefits them). And this effect is on people who have the skills to start with; I have not seen explicit studies on new learners, but I am convinced the impact will only be worse there.
More ≠ Better
There is also a fundamental values misalignment: “more” is not always better. “This tool lets me create code that I don’t fully understand way more quickly” sounds like a nightmare scenario. In the software engineering world, it’s been repeatedly documented for decades that bugs are more or less constant with lines of code. genAI has not demonstrated any ability to beat that trend (and there’s some evidence it’s below the curve), absent a lot of extra tooling.
Code Generation, not Architecture or Refinement
I am not ignorant to the capabilities of these tools. I readily acknowledge them. I’ve been in multiple talks, presentations, and demonstrations by people explaining what they’ve created with these tools. But I also know what their limits are—largely from listening to the same people. Overall, these tools create code that is functional, but architecturally unwieldy, at best, when doing anything complex. This means that revising the code produced will be much more difficult than it otherwise has to be. The assumption is that such revisions will be done by relying on the tool, either by getting it to revise the code (an exercise that has largely had poor results to date) or having it simply recreate the whole thing from revised prompts. It is a implicit form of insidious lock in.
Over-promising
There is also a big mismatch between what the tools are actually capable of and what they are perceived to be capable of. The marketing from the vendors is inconsistent on this point, so it’s unclear how much blame they deserve, but the end result is that people assume these things are more capable and safer than they are. All of the horror stories of production data being destroyed are the strongest evidence for this, but it shows up in less dramatic places over and over.
Likely Limits or Diminishing Returns Over Time
This is all about the state of the tools today. Long term, there a whole bunch of other concerns. These tools, by design, do not create anything genuinely novel; they function by remixing things in their training data. That has a few very negative implications. First, it’s a real upper limit on creativity, compared to what humans do naturally. Second, the quality of output declines sharply when the tools are fed their own output, a phenomenon called “model collapse”. Without a dramatic change to how the input data for these systems is gathered, the quality of that training data will only go down from here. For a little while, throwing more power at the problem, generating larger and larger models, will offset this problem, but lots of theoretical research suggests there’s an upper limit there. And it isn’t far away. When that happens, the tools will become less useful for the problems at hand, and the lock-in and dependency will become an ever-increasing problem.