We are building Maxwell, an AI developmental and line editor, for a world that is both excited about AI and increasingly wary of it. Every week I write down what we have learned trying to navigate that, our failures included, because a writer deciding whether to trust us with a manuscript deserves to see how the work is actually done. One principle sits under all of it. Maxwell is an editor, not a writer. It will tell you what a sentence has to accomplish. It will not write the sentence for you. Not One Word.
Before any change to Maxwell reaches you, we run the updated code on a real book. Not a sample and not a file we made up for the occasion, but an actual chapter of an actual novel, going through the same path your manuscript goes through. Then we open the record Maxwell saved and read it to check that what got stored is what we meant to store.
We look at real saved work before we write anything new. If a new piece of code is going to read something Maxwell filed away earlier, we pull up a real saved record first and check what is there.
And every screen you touch gets checked at five screen sizes, from a small phone to a desktop monitor, every time we change anything. If a page slides sideways, if a column of text squashes down to nothing, or if a button is too small to tap with a thumb, the change is refused. It cannot ship.
One rule sits under all three practices we’re discussing here. Do the thing most likely to prove you wrong, and do it first. For our Maxwell updates, the lesson was to first check that the change worked on a real browser using real writing, not the pile of internal checks that all agree with each other. Maxwell does the same thing for your manuscript. Before it shows you an editorial note, it argues against that note. Is that repetition deliberate? Is that passive construction doing real work? A note that survives the argument comes to you as a correction. A note that only half survives is neither thrown away nor stated as fact: Maxwell marks it as less certain and puts it to you as a question. So you can tell which notes it is sure of and which ones it is asking you about.
The checks never looked at a real screen.
On the fifteenth of July, every click in our genre picker closed the picker. You would tell Maxwell you had written a police procedural, then click on crime fiction to find it. But the panel would shut. There was no way through it. Our genre picker, and everything else, passed 167 of our automated code checks. Nonetheless, I reported that picker as broken three separate times, and our internal checks still came back clean.
The reason is dull, but it is the whole point. Our internal checks do not run in a real browser. They run in a stripped-down imitation of one, which is fast and cheap and has no idea how wide anything is or what happens when a real click lands on a real screen. It could not have caught it. Nobody outside our team ever ran into it, because we were still two weeks from opening to writers. That is luck, not process.
Our solution. We run Maxwell tests in a real browser at five widths—five screen sizes from phone to desktop—that our customers actually use. Our software now enforces this real-world check on every single change.
The test data was mine. So was the mistake.
We did it again. Eleven days later, we shipped a feature that never once worked. Settle an editorial note about a character—say, “Donald appears with no setup”—and Maxwell was supposed to identify all other editorial notes related to that one. If the author’s edit to the first instance of the note also resolved downstream notes, it was supposed to offer the opportunity to clear them. It did that on the deeper-look notes examined after the main editorial pass. On the main editorial notes, the critical ones where it would have mattered most, it did not find the related notes about “Donald.” 756 checks passed with a green light.
Maxwell files every note with a small label saying where the note came from. I believed that label always showed the note’s source. On the main editorial notes, it showed something else: the heading the note was filed under rather than the source label. So the feature looked at every one of those notes and didn’t recognize any of the other main editorial notes that related to “Donald.” It passed over all of them. It didn’t trip over any of the checks, because as far as the code knew, nothing was wrong.
The practice notes I built by hand to test this had the note’s source labeled the way I believed it would appear in the other main editorial notes, instead of the way it actually appeared in real notes. So my test data was wrong in exactly the same way the code was wrong. The two agreed, and agreement is all a passing test is. Something you build yourself out of your own assumptions cannot tell you the assumption is wrong. It can only tell you that you were consistent. Opening a real saved note would have settled it in about a minute, and it was the last thing I did, not the first. That’s why it is the first thing I do now—check the code against something real.
Nothing was broken. Nothing happened either.
The third failure is why we run every change on a real book, all the way through.
A button at the top of the editorial notes says: “Re-assess the score.” Press it, and Maxwell is meant to take another look and rescore the chapter. What it actually did was type the question into the message box at the very bottom of the pane and wait for you to press “Send.” That box is a long way down: past every note, past every follow-up, past the whole manuscript. You would never see it. From where you were sitting, you pressed a button, and, as far as you could tell, nothing happened.
This shipped on the twenty-seventh of July, and we fixed it four days later. When we went back through the saved sessions, that request to re-assess the score had been sent zero times. Not zero out of a sample. Zero across every analysis anyone had ever run.
No test failed, and none could have. Nothing was broken. The button did exactly what it had been written to do. Tests confirm that the code does what you meant it to do. But tests have nothing whatever to say about whether what you meant to do was of any use to the person sitting with the mouse trying to edit a manuscript.
Of the three things we do now, the five-width check is the one I trust most, and not because layout matters more than the rest. The other two depend on us remembering to run the tests against real-world conditions, and anything that depends on memory can fall through the cracks. The software enforces layout. The five-width check was built against three specific faults that had already reached writers, so if any of them come back, the change is refused. It asks nothing of anybody’s memory or good intentions.
Our test coverage was never the question. Sequencing was. Do the thing most likely to prove you wrong, and do it first.
Next week in Not One Word: why a poem and a novel cannot be judged by the same standard, and the month a tester uploaded a textbook we had no box for.