Blog

#self-hosted-llm

Articles tagged self-hosted-llm from Whatever Works.

A liquid-cooled deskside AI workstation rendered as a cross-section: a 96-core CPU block beside four stacked HBM3e accelerator cards, a rising 'fast-memory ceiling' line, and a locked vault icon representing data that never leaves the site

FeaturedTechnologySeptember 7, 202614 min read

AMD's Threadripper Halo: On-Prem Frontier AI for When Your Data Can't Leave the Building

AMD's Threadripper Halo pairs a 96-core CPU with up to 576GB of HBM3e — the first deskside box that can hold a trillion-parameter model without a datacenter. Here's what that memory actually buys, what it costs, and who should seriously consider it.

Read article →

Infographic of an ascending per-token cost chart climbing to GPT-6 Astra, a central panel titled 'the real lever is architecture', and a task-tiered routing column sending commodity volume to cheaper and self-hosted tiers

FeaturedTechnologySeptember 5, 202613 min read

GPT-6 Astra's Price Tag — and How Architecture Keeps It Off Your SME Bill

GPT-6 Astra is being reported at roughly $10 in / $50 out per million tokens with a ~1M-token context window — flagship-grade money. Here is the honest math on why that bill is set by your architecture, not the model, and how to keep it affordable.

Read article →