My friend John Gruber is incensed by the idea that AI watermarking schemes adjust the output of a given LLM so that the output can later be assessed as likely coming from it in particular:

They say “imperceptible” and “doesn’t change the meaning, quality, or readability”. Their words. Not almost imperceptible. Not slightly changes the meaning, quality, or readability. That made sense to me, because that’s absolutely what I want — nay, demand — from any tools I use personally.

I can understand why Gruber, a nearly-lifetime professional writer, would balk at the idea that the words in a given stretch of prose might be altered to satisfy such an unpoetic goal as watermarking. But to be offended by “the perversion of writing” you need actual writing to pervert. Even if you grant that whatever pops out of an LLM is “writing”, a watermarking process would have to take place after the writing was complete for any treachery to occur.

I don’t claim to be an expert about AI or the watermarking schemes that are being used by Anthropic and Google (SynthID), but as I understand it these techniques work by biasing the internal selection of tokens within the LLM, so that the output of the model is attributable to it. The “writing” in this case is the result of a proprietary, model-based selection of tokens generated from the prompt.

While generating output, LLMs routinely come up with a variety of candidate tokens at each step, and select one based on their respective probabilities. They don’t always pick the “most probable” option, or the results would always be the same. In this context, “most probable” just means “most in-line with the model’s weights and the current context.” The most probable token at any given point might turn out to be worse by human interpretation than a less probable one. To complain about increasing or decreasing the bias in token selection suggests that there is “one true token.” As far as the LLM is concerned, all of the candidates are acceptable. Gruber continues:

It’s unacceptable for a tool to sacrifice an iota of clarity, coherence, meaning, quality, etc. for the purpose of embedding hidden clues within the text to suggest its provenance. That’s what I would and will demand.

Because any of the candidates are acceptable, biasing against one and towards another is as likely to improve the writing as it is to diminish it. The result of one watermarked Claude generation might be clearer, more coherent, even more artistic, than the result would have been without the watermark. Or, it might be worse. That’s the nature of LLM output, and it’s why it’s so easy to criticize it. With LLMs, you get what you get. Sometimes it’s good, sometimes it’s bad, And it’s always biased by some method of choosing from candidates.

An LLM is a device that takes a prompt and generates a result. You might compare it to any algorithm whose output is deemed to be either satisfactory or unsatisfactory. Consider a random number generator: it takes a seed value as input, which prevents it from generating the same random number every time, and produces a number that has been proven to have a certain amount of “randomness” by the algorithm’s designers. There are ways of “adulterating” the output of a random number generator so that it is no longer random. For example, if the random number generator internally replaced all odd numbers with “1” and all even numbers with “2”, the resulting number would no longer exhibit the required amount of randomness. But there are also ways of tweaking the algorithm that would not diminish its randomness. For example if a random number generator routinely reversed the order of the digits in a generated number, the randomness would remain the same.

For something to be adulterated, it has to be pure to begin with. The internal “work in progress” of an LLM is anything but pure. Everything about a model, particularly a proprietary one, is opaque to the user, such that the only way to assess its correctness is by judging the output. Anthropic and Google claim they can watermark output without diminishing the quality, and I don’t see a particular reason to doubt that. They have developed methods to “reverse the order of the random number” in a way that can be traced back to them. Like Gruber, I care about the integrity of the written word, but to my mind written words come from, or are at least edited by, humans. AI output isn’t writing, and the internal, work-in-progress draft by one most certainly is not.