What is
Strata?

The full narration of the video.

Li Xianzhi asked in the comments: what is Strata? First, two numbers. Alibaba's Qwen team has a smart new AI that lives in data centers, needing hundreds of gigabytes of graphics memory. A gaming PC's graphics card has just twelve.

Strata is a free little program: install it, and a gaming PC can run that big AI. It went online on September 24th. In under three weeks, over twenty-one thousand people saved it. If the card can't hold it, how does it run? Let's follow the AI as it writes one word.

First, what does it look like? It isn't one big brain; it's a crowd of little experts. Forty-eight floors, five hundred and twelve on each: over twenty-four thousand in all. To write one word, it walks from the first floor to the last. But on each floor, only ten experts get called; the rest stay idle.

So the team is huge, but one word uses less than a fiftieth of it. So where do the idle experts go? Think of your kitchen. Salt and oil sit by the stove, in easy reach; the rarely used stuff goes in the pantry.

The graphics card is the stove: fast but small. The RAM is the pantry: big, but further away. Strata keeps the few thousand most-called experts on the card, and all twenty-four thousand in RAM. And it learns as you go: it spots the busiest experts and moves them to the stove.

What if a called expert isn't on the stove? The PC has another helper: the processor. It does the work right there in the pantry. Both work at once; neither waits for the other. There's a second trick: guess, then check.

A little apprentice guesses the next three words, and the big AI checks them in one go, keeping the right ones, rewriting the rest. The big AI decides every word, so the answer is the same, just sixty to eighty percent faster. The result? The author tested a twelve-gigabyte card with sixty-four gigabytes of RAM: dozens of words a second, faster than you can read.

And nothing you say ever leaves your PC. There are costs: it's a compressed version, a bit less smart than the original; it needs at least thirty-two gigabytes of RAM; and on first start, the PC can freeze for one to three minutes. The author measured the speed; your PC may differ. So when the card can't hold it all, Strata doesn't force it: the common stuff close at hand, the rest in storage, the processor pitching in, and an apprentice guessing ahead.

Keeping what you use close at hand: where else at home do you do that? Li Xianzhi, you go first.