VRAM Swap: The SSD Trick That Puts a Giant AI on Your Desk, and Who Will Benefit

Can the best open AI models run on the computer you already own? What would it take for you to run the best models on site and what technologies make it more affordable?

DeepSeek V4.1 Flash weighs about 500 GB to download, while the biggest gaming graphics card, the RTX 5090, has 32 GB of VRAM. For years that gap ended the conversation: what did not fit in memory did not run at any useful speed. Today a version of it runs on that card, and on laptops with less.

VRAM is no longer a hard constraint, and who benefits from AI over the next few years depends on what this technique can and cannot do.

How a giant model can fit on a small card

Picture a huge consulting firm with 384 specialists on every floor, where each word of the answer needs only six of them per floor. DeepSeek’s model is built like that, a design called a mixture of experts: while it writes each token, only about 2% of its 763 billion parameters are in use.

Keeping the whole firm in the small office, the card’s VRAM, is hugely expensive when nearly everyone sits there doing nothing. VRAM swap leaves them in a cheap building across town, your SSD, and calls in only those you need. The regulars stay in the office, so most calls never leave the room.

This is a real change because space across town is far cheaper. One recent estimate puts an SSD at $0.34 per GB against $62.47 for VRAM, about 180 times less.

It also runs today: laptops with 24 GB of VRAM have been reported at 12 to 16 tokens per second on this model, with free software. Model size has stopped being capped by the VRAM you can afford.

What VRAM swap does not do

It only works on models built as a mixture of experts. The whole idea depends on most specialists sitting idle, so a traditional dense model, where every part works on every word, gains nothing from it: everyone would be on the road at once. If the best models of tomorrow are not built this way, VRAM swap cannot help them.

In practice that limit is smaller than it sounds, because open models have gone this way almost entirely. Among open models with a current score on the Artificial Analysis Intelligence Index, the eleven best are all mixtures of experts, led by DeepSeek V4.1 Flash and GLM-5.3.

The closed leaders keep their weights private, so you can’t run them yourself, and they stay far ahead: the best open model scores 32 against 59 for the top closed one. They also run their own data centres, where they can adopt VRAM swap wherever their designs allow, so I expect them to be the first to benefit. We should see it in the price per token going down.

It does not make the model fast, at least not yet. Every specialist who isn’t in the office has to travel down the same narrow road, and on a top-end card the SSD moves data about 130 times more slowly than VRAM. Reports on this model run from about 7 tokens per second in short tests on an RTX 5090 to 13 to 16 on a 24 GB laptop, depending on the engine and how hard the model is compressed, so a 2,000-token answer takes two to five minutes.

So the bottleneck has moved from how much VRAM you can afford to the bandwidth between the SSD and VRAM, how many gigabytes per second that road can carry, and the new technology is going there. A fast SSD today moves about 14 GB/s against about 1.8 TB/s for the VRAM of a top card. PCIe 6.0 drives already in production double the SSD figure to 28 GB/s, although they reach servers long before home machines. Spreading a model across two drives already helps, and software such as Edge0 shortens the queue by predicting which specialists to call next.

The big jump is High Bandwidth Flash (HBF), flash built right next to the chip, which SK hynix and Sandisk are standardising at up to 3 TB/s. On paper that turns the country road into a motorway, faster than the VRAM of an RTX 5090.

Big inputs are where reports disagree. The laptop test fell to about 7 tokens per second after a 28,000-token document, because the document pushed the regulars out of the office. Another build claims 24 tokens per second on a far longer prompt. Until those two are reconciled, I see this as a tool for questions and background jobs first, and a side-by-side coding partner only maybe.

It also does not mean you are running the model DeepSeek released. One test file is a text-only conversion and “not a lossless copy”, and the fastest community builds are quantized much harder, down to about 2 bits per weight. That is like a heavily compressed photo: it may look fine, but neither of those reports measures how much the answers suffer. Speed is now well reported; equal quality is not.

Why running it at home does not undercut the cloud

If anyone can run these models at home, the obvious conclusion is that the price of AI collapses and the closed companies are in trouble. Yet the way big companies serve LLMs is similar to how a restaurant with millions of tables serves its food, and at home you are cooking for one.

The model’s weights are the ingredients, and the memory and drives that hold them are the pantry. The graphics card is the stove, and each token is a plate leaving the kitchen. A restaurant stocks one pantry for all its tables, and a data centre keeps one copy of the model for thousands of users, so the pantry adds almost nothing to any single bill. VRAM swap gives you a cheaper pantry, which is great news at home, but the restaurant gets the cheaper pantry too, and it was already splitting that cost across millions of plates.

The gap widens with waste. A Penn State study estimated that the average US household wastes about a third of the food it acquires, while restaurant kitchens are usually put at 4 to 10% of what they buy. The two numbers aren’t measured the same way, but the reason behind the gap carries over: a busy kitchen cooks to order and its stock keeps moving.

In AI, what goes off is machine time. A home graphics card sits idle most of the day, you have already paid for those hours, and each dish it does cook waits on the road from the SSD. A data centre keeps its stoves lit with the next order always waiting and keeps the whole model in fast memory, so it loses very little to idle time or lag.

DeepSeek charges $0.60 to $1.20 per million tokens of output, plus a fee for what you send in, while a new $9,999 machine running flat out, day and night, at its best claimed speed works out at about $6 over a three-year life. To break even inside those three years it would need about 140 tokens per second, and the best claim today is 24. At 24, on the cheapest power, it catches up after about 19 years.

Real homes do worse, because idle hours still have to be paid for. Use that machine a tenth of the time, at the speed in published tests, and each million tokens costs about $150. At home, what falls is the ticket to enter, not the price per token.

If you already own the machine, the sums change. A gaming laptop drawing an assumed 200 watts at 15 tokens per second spends about $0.90 of electricity per million tokens, close to DeepSeek’s own price, for a model squeezed harder than theirs. So buying hardware for this still loses to the cloud. A computer you already paid for is different: its price is spent either way, so only the power bill counts, and the answer depends on where you live. It beats DeepSeek’s top price wherever power costs under about $0.32 per kWh, which covers most homes in the world, and its bottom price under about $0.16, roughly where Asia’s and the Americas’ averages sit. In most of Europe it is roughly a draw, and it stays private.

That does not mean nothing changes. I see four real shifts, and only the first one helps ordinary buyers directly:

• Private AI gets a new floor. Any firm that cannot send its data outside can now run a very large model for the price of a good workstation, slowly but privately. In eight hours one machine writes about 200,000 tokens with no bill per word.
• Model size breaks free from VRAM. Labs can ship models far larger than any home card and still expect people to run them. DeepSeek’s model carries 196 billion parameters of pure lookup memory, and a 2.8-trillion model is already listed for this hardware.
• The big companies get a stronger version. High Bandwidth Flash goes to data centres first, and Google is already in the group. That is where I expect the price per token to fall first.
• SSDs are getting more expensive. Flash is in short supply, with one common SSD’s contract price up 303% in a year. Data centres cause most of that, and filling home drives with AI models only adds to it.

Open models were already winning on price without any of this. In late September they carried 72% of the tokens among OpenRouter’s top models but, by one investor’s count, only about a quarter of the spend. My reading is that people pay more when a wrong answer is expensive, and a cheaper place to store a model doesn’t change that.

What I expect to happen, and when

These are my own estimates, and the percentages show how sure I am. A speed only counts if someone outside the maker measures it on this class of model, with a prompt of at least 4,000 tokens and no loss of quality:

• Outsiders confirm at least 5 tokens per second on an RTX 5090, by mid-2027 (estimated probability 85%). The code is public, and free engines already report more.
• LM Studio or Ollama offers VRAM swap as a normal option, by end of 2027 (estimated probability 70%). It is too useful to stay a specialist project.
• A home build under $5,000 reaches 15 tokens per second on this model with no measured loss of quality, by end of 2027 (estimated probability 50%). Compressed builds already beat that speed, so the open question is quality. A second drive is also reported to add 37.5%.
• An RTX 5090 holds 20 tokens per second on a 30,000-token conversation, by end of 2027 (estimated probability 25%). Short test prompts are the easy case, although free engines are lifting the odds.
• A trillion-parameter model reaches 20 tokens per second on a machine under $3,000, by end of 2028 (estimated probability 35%). It probably needs techniques like Edge0, plus faster drives at sane prices.
• A big cloud provider runs fast flash like HBF in production, by end of 2028 (estimated probability 50%). The standard exists, but no product does yet.
• New home hardware beats cloud prices for someone using it two hours a day, by end of 2028 (estimated probability under 10%). Sharing beats owning while your machine sits idle most of the day.
• A leading closed lab names home VRAM swap as a reason for lost revenue, by end of 2028 (estimated probability 10 to 15%). OpenAI alone reports about $50 billion in annualised revenue, and a slow home machine competes for very little of it.

I would change my mind fast if an open model matched the best closed one on hard work, because then the closed labs’ high prices would have far less to stand on.

Who gains and who gets hurt

I ranked these by how much I expect this one technique to change each group’s profit by the end of 2028, from most helped to most hurt. Nobody on the list gets knocked out. The AI that helped write this comes from Anthropic, a closed lab, so weigh that entry with this in mind.

• Flash makers (SK hynix, Sandisk, Samsung, Kioxia, Micron): biggest winners. Storage turns from an afterthought into part of the AI engine, in homes and in data centres alike.
• Drive-controller makers such as Phison: clear winners. The speed of that road now decides the speed of the model, and buyers pay for speed.
• Open-model labs (DeepSeek, Moonshot, Zhipu, Alibaba): solid winners. Giant models that people can run on the gaming machines they already own earn reach and trust, although not much direct income.
• Leading closed labs (OpenAI, Anthropic, Google): first to benefit. They lose some privacy-minded customers, but they get the data-centre version first, and a lower cost per token lets them cut prices and defend their lead. Google, already in the High Bandwidth Flash group, may gain the most.
• Nvidia: small winner. Ordinary cards become useful for giant models, although a few buyers will skip the card with more memory.
• Apple: roughly even. Large-memory Macs lose part of their appeal as the easy way to run big models at home, while fast internal drives suit this technique.
• Mid-priced closed models: real pressure. When “good enough” can be private and paid for once, the middle of the market is the hardest place to defend.
• Private-AI box sellers and memory-heavy mini PCs: biggest losers. They charge a premium for capacity, and capacity just got cheaper. Those that adopt VRAM swap themselves can escape, and fast memory still wins on speed.

Send each job where it belongs

The practical answer to all of this is to stop choosing between home and cloud and send each job where it belongs. A night of summarising private documents can crawl along on your own machine, while the question you need answered in ten seconds goes to the fastest model you can reach.

Doing that by hand gets old quickly, so the sorting belongs in the agent. TinkerClaw, my fork of OpenClaw, already talks to any local server that speaks the OpenAI format, which is how these engines present themselves, and its Thalamus plugin already picks the model for each call. Sending slow overnight jobs to a VRAM swap machine is on its way in. If you want to see how an agent can make that choice, the Thalamus paper lays out the design, and our AI analysis map shows what each model costs against how smart it is.

A slow machine also changes how you prompt. The laptop test above lost half its speed after a 28,000-token document, so every token you don’t send is time back. Budget prompting collects the techniques for sending less without losing quality. It was written to cut bills, and at home the same techniques cut the wait.

Cheap local models also feed a bigger trend. Building your own tools gets easier every year, and that raises an awkward question for anyone who sells software, which I looked at in When code gets cheap, what is a software company selling?