The quote that was not on the page

I don’t run a law practice. I build the AI operating systems that run other people’s — AI-enablement for legal, full stop. Over two days this month I built a research agent around a legal-research subscription I already pay for, to feed the 50-state brief: find the new appellate decisions in every jurisdiction, check whether each authority is still good law, read the opinions, and gather the statutes, court rules and bar guidance a trusts-and-estates solo actually needs. The agent worked. What it taught me is about the one step most people skip. Notes twenty-two through twenty-four.

22. A quote the AI gives you is a claim about a page

On the second day I ran eight research assistants in parallel. Their job was narrow: for each of the fifty states and the District of Columbia, read the official source — the legislature’s code, the supreme court’s rules of professional conduct, the bar’s guidance on generative AI, the admission and continuing-education rules — and bring back the exact words, with the link. They came back with 356 quoted facts. Then a short script did the dull thing: it fetched every cited page again, normalized the text, and looked for each quote. Twenty-eight quotes were not on the page. One “quotation” about an admission rule had been assembled from the gist of the rule, in quotation marks, with a straight face. Nine pages could not be fetched at all, and sixty-one facts arrived with no quote to check, so none of those entered the record either. Only the 256 quotes found word-for-word on the official page made it through.

Here is what I want a practitioner to take from that: the assistants were not careless, and the failure rate was not high. Eight percent is low. It is also twenty-eight fabrications in an afternoon, and every one of them read exactly like the 256 that were real. Nothing about the prose told them apart. The only thing that did was re-reading the page. So the rule is mechanical and it is not optional: a quotation from an AI is a claim that a page says something, and the page is one fetch away. If you cannot re-fetch it, the fact does not go anywhere a client might see it. The step is boring. It cost one script. It is the whole product.

23. A summary of an opinion is a lead, not a reading

The first day was the courts. The research tool surfaced thirty-nine candidate decisions, and I drafted thirty-one entries for the brief from what I read in the tool: the header, the disposition, the opening paragraphs, the court’s own “we hold” sentences, the close. Careful work, I thought. Then every one of the thirty-one was checked against the court’s own opinion text, downloaded from the court. Seven were wrong in ways a lawyer would have relied on. A New York decree I had written as “affirmed” had been reversed on undue influence. An Ohio undue-influence case I had described as a will was a trust amendment. An Oregon holding about a limitations statute was misstated. A Utah caption and disposition were both wrong. A North Carolina remand was broader in my draft than in the court’s. Every error traced to the same cause: I had read a summary — the tool’s overview, a fetch tool’s digest, a headnote — and taken it for the opinion.

Two of the seven were not even errors in the end. In Alabama and Missouri the overview seemed to contradict my draft, and reading the opinion dissolved the conflict: the overview had summarized a different part of the same decision. That is the shape of the problem. A summary is a lead; it tells you where to look. It gets three things wrong more than anything else — the disposition, the caption, and the holding — and those are the three things you will quote to a client. The practice I kept: the entry is not written until the court’s own text has been read, and if the court does not publish the text online, the official docket plus the full text in the subscription is the floor, never the summary. Reading in the subscription is fine. Citing it is not; the page cites the court or the official reporter, every time.

24. Gate the words, not just the links

The entries that survived the quote check were headed for a public page, one per jurisdiction, each stating in my own words what the official source says, with the link and the date it was read. When I read the drafts as a reader would, the prose was full of the research itself: “matches the citator,” “same section as item twenty-one,” “returned a 403 to my fetch,” and the names of the research vendors whose text the pages must never carry. Every link on those entries was good. The link check passed all of them. A word filter — a short list of process words and vendor names the gate refuses — cut the publishable set from 215 candidate entries to 152. The other sixty-three go back for a plain-English rewrite before they can ship.

And one more thing, because this series is only worth reading if the failures are mine. On the first day my assistant deleted a court opinion from a shared folder, believing it was its own download. It was mine; I had saved it from the court’s site an hour earlier. A system notice had listed the file as “changed this session,” and the assistant read that as ownership. I had never told it what it may delete. The fix was a rule, not a scolding: nothing in a shared folder is yours to remove; check the size, the time and the source before any deletion, and ask. If you give an assistant a folder, decide in advance what it may take out of it.

These notes aren’t a practice diary — they’re a record of what I keep finding while building AI systems for legal work, written for the solo and small-firm attorneys those systems are for. This one you can act on tonight: take the last three quotations an AI gave you, open the sources, and find them on the page.

Field Notes are written for licensed attorneys and are not legal advice. Mike Moss is a Utah-admitted attorney doing AI-enablement work — not operating a law practice.