An AI does its job by following a sheet of paper that says how to do it.
If it keeps messing up in the same place, it is usually not stupid. It is that the paper wasn't clear.
So the thing to fix is the paper, not to scold the AI.
2
People have always edited that paper
BEFORE
One mistake, one scolding, one fix by hand
You have to remember where it went wrong last time, and the time before. When you get busy, you forget. By the fiftieth time, you may not even realize it's the same flaw. Each time looks like a one-off, and you say a quick word and move on.
NOW
Bring in a second AI whose only job is looking back
It does no real work. Its job is going through the old records: spread out every recent mistake, spot the one that keeps coming back, then hand in a note saying "change line such-and-such of the paper like this." Whether to change it is up to you.
One does the work up front, another watches it from behind, and the watcher only edits the manual.
3
How does the one looking back work?
① Dig up the recordsLay out on the table every slip-up and every correction from the past while.
② Pick out repeatsOne or two times is just chance. By the third time, admit it is a real flaw.
③ Check the paperNot written down? Add a line. Written down and still failing? Even more valuable: it means the way it was written doesn't work.
④ Hand it in, wait for a nodIt only writes suggestions: which line, changed to what, and why. Without a human nod, not a single word gets touched.
4
Step three is the most useful
"It's already written, yet it still fails." Then writing it a second time won't help.
Ask instead: why didn't it work? Here is a real example.
Paper one"Always write on the envelope which letter you are replying to"
Paper two"When replying to the newest letter, no need to write it"
Both papers sit right in front of it, and they say different things.
Following either one makes sense, so it slips. Copying paper one again, bolding it, or
writing it three times does nothing. What helps is saying it outright: when the two clash, obey paper one,
and here is why.
Once it knows why, it won't improvise the next time it meets a case it has never seen.
5
Why must a human nod?
It may misreadMistakes chance for a pattern and writes a worse rule
It gets carried awayInvents work, hands in piles of useless advice weekly
Bad edits can be undoneEvery change is logged, so you can roll back anytime
You stay the bossIt proposes, you approve. Say no and nothing changes
One more hard rule: if you rejected it once, it may not suggest it again. Otherwise you will see it twice, stop reading, and the whole setup becomes useless.
6
What it can't do
It doesn't get smarterThis setup changes the manual, not the AI itself. It is still the same brain,
just with clearer instructions.
It also can't know what you never said. A flaw you never corrected won't turn up in the records,
because it isn't in the ledger.
So it cures one kind of problem: "told it many times and it still doesn't change."
Other problems you still have to watch yourself.
So what did it actually change?
From "the same mistake fifty times, each treated as the first"
to "by the third time, someone looks back and asks: maybe our own rule is what's wrong?"
