New on the blog.

OpenAI Economic Research and Bocconi University ran a randomized experiment with more than 1,000 first year undergraduates. Four groups: ChatGPT access, training in causal reasoning, both, or neither. Same real business case. Trained human graders on one five point rubric, plus automated text analysis on the same submissions.

The ChatGPT group scored almost a full point higher.

The causal reasoning group produced a wider range of more distinct ideas and scored no higher at all.

Two instruments pointed at the same stack of student work, and only one of them saw the thinking.

The post walks through what the rubric caught, what it missed, why the Berkeley grade inflation paper is the same finding from the other direction, and the one row to write before your next stack lands.

Read it: https://www.harvestkernel.com/rubric-rewards-polish-openai-bocconi-study/

The free working session that goes with it: https://www.harvestkernel.com/the-missing-row-rubric-audit/