Skip to main content

OpenAI shipped a model built only for lawyers this week. It searches almost every published American court opinion.

Then it published the score. On 200 questions from an independent legal research benchmark, at the highest reasoning setting, it passed the overall correctness check on 54.0 percent. Ordinary web search managed 38.7.

I want to be careful about how that lands. It is not a gotcha. Going from 38.7 to 54.0 is a forty percent relative jump and that is what real domain engineering buys. It also means forty six percent did not pass, and both of those are true at the same time.

The thing I keep coming back to is not the number. It is that a number exists at all. Law can now plan around a measured figure. We cannot. Nobody is running a benchmark for what happens when a student points a general model at your assignment, and nobody is going to.

Which is why I think the syllabus move is to stop naming tools. My own policy named ChatGPT for two terms. That policy did not cover Codex, it did not cover a firm's own interface on the same model, and it will not cover whatever ships in March. Same system, different name on the screen, policy silent.

So today's session is about writing a verification clause instead: which claims have to be checkable, what the check actually is, what the student hands in that proves it happened, and the honest gap, which is how somebody says I could not confirm this one without it costing them everything.

I will go first. In my own course the claim class I would most hate to see unverified is a cited figure that a student pulled from a summary rather than the original table, because it always looks right and it is always slightly wrong.

The session: https://www.harvestkernel.com/the-verification-clause/
The post behind it: https://www.harvestkernel.com/astra-for-law-fifty-four-percent-verification-clause/

Your turn, one sentence. Which claim class in your next assignment would you least want to see go unchecked?

Which of the three did you actually use last time? Reply with one, two or three. No judgement, I want to know which one is the default.

Today's teaching session runs the same essay prompt through AI three ways, live on screen.

One. "Write me a 1,500 word essay on this prompt." Eleven seconds, and a window with a timestamp you cannot account for.

Two. "Make this better." It comes back tighter, smoother, and the one sentence that was genuinely yours is gone. It kept the shape of an argument and quietly removed the argument.

Three. "Do not write anything for me. Ask me the four questions a hostile reader would ask, then give me the strongest case against my claim."

I will answer my own question first so the thread opens with an example rather than a prompt. I used two for years, on emails and proposals, and I did not notice what it was costing me until I read an old paragraph and could not remember why I had framed it that way. That is the tell. If you cannot defend it in an office hour, you did not write it.

The free working session that goes with today's video is here, and it is the third move done by hand:
https://www.harvestkernel.com/the-blank-page-breaker/

One question today, and you do not need to have read anything to answer it.

A lawyer was fined five thousand dollars this month because he signed a document he had not checked. The court's finding was not about the tool he used. It was about the step that was missing between making the thing and putting his name on it.

Most of our assignments have that same missing step. There is a produce step and a grade step and a submit button in between that nobody has ever defined.

So, honestly:

What is the one thing in your students' work that you assume is true and never actually check?

One sentence is a complete answer. Mine is citations. I read the argument closely and I almost never open the source.

Teaching Day. Faculty. The First Grading Weekend.

Before this weekend is over you will write "develop this further" for the fortieth time. The question is whether one student uses it.

Today's session is about why the comments that cost you the most are the ones students do the least with, and the three line structure that changes it. What happened. The fix. What it is for.

There is a reason this matters more than it used to. A working paper from Berkeley's Center for Studies in Higher Education found the share of A grades in AI exposed courses rose by 13 percentage points after ChatGPT came out, about 30 percent relative to the 2022 baseline. When the grade stops separating students, your comment is the only part of the transcript still saying something true about a specific person's work.

The free working session is here. Twenty minutes, five real comments, done by hand: https://www.harvestkernel.com/three-line-rewrite/

What is the one comment you write so often you could type it with your eyes closed?

One line is a complete answer.

Somebody wrote the free AI curriculum showing up on your campus. Almost nobody on the committee reads the byline.

Today's piece works through what that looked like the first time, when it was coding rather than AI, and what a department can actually do about it now. The short version is that higher education has a lever a school district does not: a vendor cannot adopt anything on your campus without a faculty member agreeing to teach it.

So here is the question, and it takes one sentence to answer.

What is the last piece of teaching material you were handed that you did not choose, and who made it?

Publisher courseware, a vendor AI module, a required PD platform, a textbook bundle with a portal attached. Any of it counts. One line is plenty.

Post: https://www.harvestkernel.com/free-ai-curriculum-byline/

I studied. I still failed.

If that sentence has been in your head this week, here is the mechanism, and it is not a character flaw.

Rereading a page produces a feeling of ease. Your brain reads that ease as evidence that you know the material. It is only evidence that you have seen it before. Recognizing and retrieving are two different abilities, and college quietly starts testing the second one.

Five researchers rated ten common study techniques for Psychological Science in the Public Interest in 2013. Highlighting and rereading both landed in the low utility group. Practice testing and distributed practice were the two rated high utility. Their own sentence: "Most students report rereading and highlighting, yet these techniques do not consistently boost students' performance, so other techniques should be used in their place."

Today's free working session runs the high utility version by hand: https://www.harvestkernel.com/the-closed-book-ten/

Now the question, and it is answerable in one sentence by anyone, whether or not you watched anything:

Which concept can you not say out loud right now?

Post the concept. Not the course, not the excuse, just the thing you cannot currently say out loud. Naming it is the whole first step.

Rereading Is Not Studying. Close the book and find out.

Two groups of mathematicians spent this week fighting in public over a claimed proof of a ninety year old problem.

Here is what neither of them did. Neither asked the other to prove a human did the typing. Both had already published a file a machine can check, so that question was closed before the argument opened.

What they argued about instead was the trail. Dates. Who was told what, and when. Whether a model had access to somebody's drafts.

The mathematician in the middle of it wrote that the real significance is what this means for the way we train students, assign credit, and referee. His words, not mine.

Today's free working session takes that idea and puts it into one assignment you already give: https://www.harvestkernel.com/the-checkable-step/

Open your next assignment. What is the one thing in it you are taking on trust?

Pull up the rubric for your first assignment of the term. Find the word strong.

That word is a meeting you have already scheduled.

You wrote it in July, in a template with the boxes already drawn, before you had met anybody. It made sense then. In October it costs you forty minutes at your desk, explaining what strong means to a student who decided it means something else. And they are not wrong to ask. You did not tell them. You told them a feeling.

The test takes three seconds a line. Read the criterion, then ask whether a student could point at the thing. Point at it, in their own draft, with a finger.

Nobody can point at strong critical thinking. Anybody can point at names a counterargument and says why it fails.

Today's working session is The Soft Word Sweep. One rubric, three lines, twenty minutes, done by hand: https://www.harvestkernel.com/the-soft-word-sweep/

Open the rubric for your first assignment. What is the softest word in it?

Almost every AI policy in almost every syllabus is written as a place.

It names a product. It names a category of site. It tells a student where not to go. All of that rests on one assumption: that using AI is a trip out of the document and back into it.

On September 3, Google put a conversational assistant inside Docs, Gmail and Keep. There is no trip now. The student talks to the file the assignment is written in.

The fix is not a ban and it is not a policy overhaul. It is one sentence, rewritten to name a behavior instead of an address. "Work you submit has to be work you can explain" is enforceable in a hallway, survives every product launch, and does not require anyone to keep a list of software.

One question, and you can answer it with a single assignment name:

Which one assignment on your schedule this term assumes the AI is somewhere else?

Thursday morning your students could not reach the tool your assignment told them to use. The due date did not move.

Grok went first, at about 9:30 Eastern. ChatGPT followed around 11, and the outage took logins, file uploads, voice mode and image generation with it. Claude went down at roughly the same time. Anthropic called it an infrastructure issue and had it back by about 12:15.

Here is the part worth arguing about. The reflex is to read that as a reason to keep AI out of coursework. I think that is the wrong read, and it is the same reflex that would remove library databases from a syllabus because the library site went down during finals.

The smaller and more useful lesson is that naming one tool in one assignment is a procurement decision. You picked a vendor, committed your students to it, and gave yourself no way to see whether it will be there on the morning it is needed.

The fix is one sentence in the assignment. We are calling it the outage clause, and there is a free working session that walks you through writing yours.

Run it: https://www.harvestkernel.com/the-outage-clause/

Which one assignment on your schedule this term stops working if the tool is down?