OpenAI shipped a model built only for lawyers this week. It searches almost every published American court opinion.

Then it published the score. On 200 questions from an independent legal research benchmark, at the highest reasoning setting, it passed the overall correctness check on 54.0 percent. Ordinary web search managed 38.7.

I want to be careful about how that lands. It is not a gotcha. Going from 38.7 to 54.0 is a forty percent relative jump and that is what real domain engineering buys. It also means forty six percent did not pass, and both of those are true at the same time.

The thing I keep coming back to is not the number. It is that a number exists at all. Law can now plan around a measured figure. We cannot. Nobody is running a benchmark for what happens when a student points a general model at your assignment, and nobody is going to.

Which is why I think the syllabus move is to stop naming tools. My own policy named ChatGPT for two terms. That policy did not cover Codex, it did not cover a firm's own interface on the same model, and it will not cover whatever ships in March. Same system, different name on the screen, policy silent.

So today's session is about writing a verification clause instead: which claims have to be checkable, what the check actually is, what the student hands in that proves it happened, and the honest gap, which is how somebody says I could not confirm this one without it costing them everything.

I will go first. In my own course the claim class I would most hate to see unverified is a cited figure that a student pulled from a summary rather than the original table, because it always looks right and it is always slightly wrong.

The session: https://www.harvestkernel.com/the-verification-clause/
The post behind it: https://www.harvestkernel.com/astra-for-law-fifty-four-percent-verification-clause/

Your turn, one sentence. Which claim class in your next assignment would you least want to see go unchecked?