Eve Fairbanks, a professional writer and editor, in the Atlantic:
The problem is that the efficiency and frictionlessness that make AI appealing to writers are the same qualities that make it feel untrustworthy to readers. And readers are right not to trust it. No matter how much we may tell ourselves that AI is just a tool like spell-check, it isn’t. When we use AI to flesh out ideas, we lose the most important part of the writing process: thinking.
[…]
We end up with canned perfection—writing that can’t really be argued with, because it has no underlying deliberative reasoning process, no train of thought. As I wrote on X recently, AI writing is almost impossible to edit, because even when it sounds plausible, a closer look will show that every element is equally off: The tone is bland; individual word choices are baffling; the structure lacks sense; key pieces of the argument are missing; facts are false. Working on AI text, as an editor, is like trying to operate on a body whose skin, muscles, veins, bones, and organs are all compromised. There’s nothing to leave intact, nowhere to begin.
This is the best diagnosis I’ve read of what’s wrong with AI writing. It doesn’t mean anything. It’s not the end result of planning, composing and revising the author’s thoughts. It’s just probabilistic text.