LLMs for the rest of us

Blog: 15-Oct-2026

Caveat: if you can afford $200 a month or more for frontier LLMs, then this piece is not for you. It's for ordinary developers like me trying to find our way through the maze of cheap AI coding options, and some of my reflections on the experience.

My posts in 2025 were all about my slack-jawed AI First Contact astonishment. We've all been there. But now that we've all gotten over being knocked off our feet by what these things can do, most of us have settled into a working balance somewhere between LLM insanity and LLM denial. Steve Yegge's insane world reads like a nightmare dystopic horror show where the motto seems to be speed is everything, humans are worthless, bow before the machines—a mind-bogglingly expensive, grotesque hell on earth. Unfortunately though, if, like me, you are self-taught, or you're a bit short on coding self-confidence, you might start to think that a big internet name like him must know something you don't. Maybe you should just put a peg on your nose, cross over to the dark side, and try to emulate him, like some kind of poor man's cargo cult. Perhaps you could open up twenty terminal windows, each running Matrix-style green font code loops while you burn a wheelbarrow full of fake $100 bills.

So I'm a lone voice out in the wilderness here to tell you that Steve has clearly gone full tonto, radio rental, bonkers. He seriously needs to hit the power button on his laptop, bury it in a closet, and take a very long holiday. His obsession with speed boggles my mind. If I learned one thing in my coding career, it was this: good code and breakneck speed go together like peaches and shit. So my advice is: steer well clear of his ruinous maniacal gibbering.

Luckily though, there are Yegge antidotes. If you have a spare 20 minutes, watch Building pi in a World of Slop by Mario Zechner. That video made me both happy and relieved that much brighter minds than mine agree with my gut instinct to fight off going full swivel-eyed multi-agent doolally on everything we do.

At the other extreme, Zig's "La la la, I'm not listening" approach evokes an image of a relaxed Pompeii citizen airily dismissing the pyroclastic flow rushing toward him as little more than a summer dust cloud which will soon blow over. I just don't believe that such extreme denial positions will be sustainable over time. You might choose to get off the bus, but everyone else will just cruise on by, waving at you out of the rear window as they disappear off into the future.

So where do the rest of us sit on this spectrum? The big win from pretty much any decent model is that we can now do regular work much, much faster. And we can finally get around to building all those spare time hobby things that we've been putting off for years. No more being daunted by the scope, the amount of research needed, unfamiliarity with the technology, or simple time constraints. LLMs have swept most of that away, if you take appropriate care (more on that later).

Models, models, everywhere

But which models to choose? As a retired developer, frontier models are just not in the running for me. Their cheap monthly subs have absurdly low limits, and their APIs are outrageously expensive. Opus is currently almost 100 times more expensive than DeepSeek-V4.1-Flash. Other frontier-adjacent models are out for the same reason; Kimi 3 is 17 times more expensive than DeepSeek-V4.1-Flash, but it certainly doesn't give me 17 times the value. And to be honest, I simply don't need the mighty Thor hammer of a frontier model to crack the tiny nut of my web applications and tools.

So how about LLM routers? Many of the third-party model bazaars are tempting, but I hear too many reports of them palming off customers with heavily quantized models when demand is high or suffering from unexpected speed drops. So for me, for now, they're out too.

That leaves direct pay-as-you-go API calls to the cheaper model providers or running local (usually quantized) models. But you need a decent machine for local LLMs, so here's my updated local LLM story.

To Mac, or not to Mac

In June '25 I bought my Mac mini M4 Pro with a 512GB SSD and 64GB unified memory. The Pro version had several advantages for LLM use: more CPU and GPU cores, around twice the memory bandwidth of the plain M4, and Thunderbolt 5 for a fast external SSD to store the models. I really wanted to go for a machine with 128GB or even 256GB unified memory but that was getting into Mac Studio territory which was a whole different league of eye-watering prices. But, hey, I can still do a lot with 64GB.

The price then was £2,200 for the Mac, plus £100 for a Beelink Mac mini 80GB/s TB5 dock, and £70 for a fast M.2 Samsung Evo 2TB drive which, when installed in the dock, matches the Mac's internal drive speed at over 5,000MB/s for read and write. Pretty damned good for shifting large LLM files around. My previous 2020 M1 mini sold on eBay for £420. So the cost to change was around £1,950 in total. Not peanuts, but Apple UK was offering an interest-free loan to buy the new model, and I hoped to get around five years use out of it. And as I write this in October '26, it now seems like a great purchase given the way that memory prices have shot up since then. It would cost considerably more now to get an equivalent spec machine. But it's not all roses…

Hot stuff

Here's how the mini runs normally:

However, dense LLMs like Qwen3.8-27B cause it to get rather hot under load. The four GPU clusters' temperatures rise rapidly to well over 100°C which seems worryingly high. I'm concerned that running them at such temperatures for long periods will potentially do long-term damage to the machine. MacOS does rev the fan up slightly when the models are working, but those high temperatures stay right up there until the model stops working. Not good.

The problem is that no-one really knows what you should (or should not) do with hot Macs. It seems to be universally accepted that Apple set some kind of temperature threshold where thermal throttling cuts in, and there's supposedly a further full system shutdown threshold somewhere beyond that. But there are a lot of unanswered questions, like: was heavy LLM use one of the planned usage scenarios when Apple signed off on the design for the M4 chip family? Can we be sure that the default macOS fan curve is sufficient for such use? Can we blindly trust that Apple has this fully covered out of the box, as uncritical diehard Apple fans insist? No-one really knows, simply because there is no publicly available long-term local LLM usage data. So it's all just anally plucked opinions at this point.

I decided to go with better safe than sorry rather than just rely on blind faith. I installed a custom fan curve using TG Pro to cool the Mac down a little more than the stock fan curve when under heavy load, just enough to bring the GPU temperatures down to the low 90s. It's a little extra fan noise, but it does give me some peace of mind that I'm not slowly melting the guts of the thing over time. And I placed it on a small stand to help improve the bottom-mounted fan exhaust throughput.

Strap me in

My agent harness of choice right now is the wonderfully minimal Pi with the excellent pi-web-search package installed. I use it either stand-alone in Ghostty or in the terminal panel of Zed, my code editor. Pi is open-source, extensible, not opinionated, and doesn't lock you into any model provider. One of my favourite features is Ctrl-P which lets you flip between your models as you work. The models I have currently installed to run in Pi are:

• Google Gemini 3.8 Flash via direct API
• DeepSeek-V4.1-Flash via direct API
• Jundot-gemma-4-26B-A4B-it-oQ4e-mtp, local via oMLX
• Jundot-Qwen3.8-27B-oQ8e-mtp, local via oMLX
• DeepSeek-Flash-V4, local via DwarfStar

Google Gemini 3.8 Flash via direct API seems decent enough; it can certainly find holes in existing code, and talks to you in clear, almost conversational English. But it kept on failing over longer use periods with "You exceeded your current quota, please check your plan and billing details" which seems to be a peak load limit failure error. I had plenty of credit in my account on each occasion, but after a couple of mid-task fails with this error, I gave up in frustration. I'm not here to be Google's unpaid beta tester so I haven't used it since. I'll use up the money I deposited on Nano Banana images instead.

DeepSeek-V4.1-Flash via direct API is an absolute cracker and would be my go-to if I couldn't run a decent model locally. It codes like a demon and is super speedy. It uses that terse, unforgiving technical English common to Chinese models which I rather like. It's not trying to be your friend unlike some of the western models. It's well worth the tiny cost—think in terms of less than a dollar for an evening of casual coding.

The other models are all local. I've tried many model runners: LM Studio, Ollama, Unsloth Studio, MTPLX, et al. They all have features to recommend them, but I reckon oMLX is currently the top of the pile for Macs with less than 64GB unified memory.

Qwen3.8-27B gets a lot of love these days, but honestly, I find it too slow for coding on this Mac. It apparently runs well using 4-bit quants on PCs with 16GB VRAM, or on suitably spec'd Strix Halos or DGX Sparks, but I only get around 20tps here with an almost unbearable amount of thinking time even with the thinking set to medium. The output is solid, but you have to wait an awful long time to get it.

But the story changes if you have 64GB VRAM or more where DwarfStar is a game changer. Unlike the others, it's not a general model runner. It runs a few very specific models in a highly customised environment, all of which are fine-tuned for various large VRAM machines. I use an 86GB 2-bit quant version of DeepSeek-V4-Flash which runs partially off the SSD and hence only uses around 50GB of memory. That sounds like it's going to be awful at first glance; 2-bit quants normally have a bad reputation, and anything disk-read sounds slow. But this is no ordinary 2-bit quant. It's a carefully hand-tuned MoE model which runs very happily on my fast SSD. It's been a revelation for me and is the first local model I've used for coding a full project on. My current project is a web frontend to a Rust backend server which reads raw TCP data from a Teltonika GPS tracker sent via its SIM card. Plenty of good stuff going on in there: customised bit packets, Luhn checks, CRC code, TLS wrappers, and various transmission codecs summarised beautifully by the model from the Teltonika spec documents. I love projects like that, and my local DeepSeek-V4-Flash is currently gliding smoothly through it.

Machines with 96GB memory and upwards can run the much larger DeepSeek-V4.1-Flash and GLM 5.3 models. A 2-bit Qwen3.8-Flash-Next quant is also available which can run on 64GB and above, but I'm liking my DeepSeek-V4-Flash better so far. The running speed of both feels much, much faster than 8-bit Qwen3.8-27B. The actual output speed is reported as 30tps for Qwen3.8-Flash-Next and only 10tps for DeepSeek-V4-Flash but that's misleading as it appears to dump its output in one go, perhaps something to do with cacheing? Whatever, the reported speed appears to be a red herring; it feels plenty quick enough for daily coding without overly long waits.

But this stuff changes by the month almost—I'm constantly trying new model runners and tinkering with the settings to get the best performance.

Can you really compare models?

It's difficult. The intrinsic stochastic nature of the things fights against accurate comparisons. And I think few of us fully trust the common benchmarks these days. But does anyone seriously expect a local Qwen3.8-27B to beat full remote API models? No, not really, certainly not in speed anyway. The likes of remote DeepSeek and Gemini run at ridiculous speeds and local models chug along in comparison. But weirdly that's sometimes an advantage, because you can watch slow local models thinking in real time. This greatly helps to keep up with the action. The big boys just rattle unreadably through your terminal until they're done.

However I did find local Qwen3.8-27B to be terrific when asked to report and comment on the code that bigger models wrote. It took 30 minutes or so to put together its thoughts, but it produced excellent reports and even found suggestions for improvements that both DeepSeek and Gemini had missed. On the other hand, remote DeepSeek and Gemini took around two minutes each to make their equivalent reports. For me though, the main point of running different models is the valuable cross-checking insights they can give and the different issues they can find.

So are local models worth it?

DwarfStar running DeepSeek-V4-Flash takes the crown for me by some margin. Thanks to its MoE architecture it runs fast and almost silently and is relatively cool at 80-85°C; significantly cooler than dense models like Qwen3.8-27B. And it does a great job for me with Rust and web coding. That's not to knock Qwen—if I had less memory I would surely be using it, but perhaps not as my coding go-to.

Other local MoE models generally run well, of course. I get around 60tps with Jundot-gemma-4-26B-A4B-it-oQ4e-mtp but it's noticeably poor for any serious coding, so I wouldn't advise using it for that. But it's great for speedy Q&A sessions in a chat window for anything from code questions to Linux admin questions for my servers, or just general life stuff.

I've found over time that most models work best with smaller well-bounded tasks. I like to prompt a model for opinions on a modest chunk of clearly-described work, then review the big picture together, agree on the architecture choices, build, test, fix, rinse, repeat. My code reviews are getting more cursory as the models improve, but I still like to keep a broad watch on how the code is divided up so that I can jump in when I feel a change of approach is needed. This is where I disagree with Steve Yegge. If you have no clue at all what's going on in the code, then you have no control and no understanding what the model is up to. I'm not saying I do a slow line-by-line review of all of it (those days have gone) but I like to have a handle on the key parts and quite enjoy reading LLM-produced code. If I don't understand something, I ask the model (or another model) to explain it and add more comments or documentation if necessary. Or make a change if I don't like the approach or architecture, or if I spot something which could be simplified or abstracted.

So, is it worth the faff, potential heat issues and VRAM expense to run local models? I think the valid use cases are:

• you want to experiment with local models for fun and curiosity
• you want to build, train and modify your own models
• you work on sensitive data that you don't want to leak outside your network (assuming that your model can be prevented from sneaking out and having a beer with the boys)
• you can't afford the monthly subs or API costs of running remote frontier models for long periods
• you object in principle to paying money to big, nasty model vendors like Sam, Elon and Dario and have strong views about their morals (i.e. spending obscene amounts on vast resource-consuming data centres and sucking up to Cadet Bone Spurs)

Conclusion

My view right now is that the sweet spot for occasional coders like me is the near-frontier remote API Chinese models like DeepSeek-V4.1-Flash. You get a lot of power, and they cost next to nothing. However, if you have 64GB+ of VRAM then local models can be a real alternative. But on machines with less than 64GB VRAM, local LLMs are best used for secondary use cases in slower time: code reports, opinions, code questions, and general life use. And they're great to play with for good old geeky fun, which is, of course, priceless and therefore justifies any expenditure 🙂.

I can't help wondering though if my present practice of keeping a weather eye on the code and going back and forth with the models will fade away over time. It makes me feel like a hybrid car: I'm a step up from the old world of internal combustion, but I'm a long way from full electric. Maybe Steve Yegge is right, and the future is just us kicking off the first prompt, drinking coffee, then selling the output. But I hope not, because there's no skill, taste, satisfaction or fun there.

On the other hand, I doubt I'll ever again write anything fully by hand, but—and this was a happy discovery—I get a lot of satisfaction and fun from steering a good model where I want it to go, arguing with it over technicalities, and proving to it that it this or that is just wrong. And that keeps me involved in the guts of the code even it's more of a mile-high view these days. It still feels like coding, but it's more like pair coding with a (mostly) very smart fast-typing buddy you can riff with. Essentially, it's you pushing the plot along while the model does the donkey work to turn your architecture ideas into working code.

DeepSeek-V4-Flash's final response after I declared one project done for the day was very telling. In a moment of weakness after a long session I actually thanked it, even though I know very well that I shouldn't be such a dick (and I can't believe I'm admitting this in public). But I'm an Englishman of a certain age and I can't escape from "manners maketh man". Here's how it replied:

"Thanks for the careful back-and-forth throughout — the things we caught (the probed re-probe loop, the .tmp extension breaking every video thumbnail, the --delete-vs-exclude subtlety on your deploy Action) only surfaced because you kept pushing back on the answers rather than accepting them. Enjoy the evening."

Yes, yes, I know, that's just pre-trained anthropomorphic flattery for the gullible. But it's nonetheless true. Right now, the real Mixture of Experts is you and your models working together. I hope it can stay that way a while longer.