AI Development 13 min read July 20, 2026

The Half-Closet Server: A Field Guide to Property-Local AI

Spec, build, and govern a self-hosted AI node — hardware tiers, model serving, a records vault with RAG, a bounded agent loop, and the power math to keep it running. No cloud required.

Patrick Roden

Founder, DevForge Academy

Rows of server racks in a data center — the concentrated compute a household node quietly declines to rent

There's a civic argument for owning your own compute, and I've made it at length elsewhere — see Fight Big Tech and Big Govt — Bring the Power Home for the why. This guide is the how. Purely as engineering, a property-local AI node earns its shelf space on four counts: your documents and prompts never leave the building, inference has no per-token meter, latency is a LAN hop, and nothing about the system changes unless you change it. What follows is the full build — hardware, software, governance, and power — sized to fit half a closet and a realistic budget.

Write the spec before the purchase order

The most expensive mistake in home-lab AI is buying hardware before writing acceptance criteria. So do this the disciplined way: one page, seven requirements, testable before a dollar moves.

R1 — Inference. Runs current open-weight models at conversational speed for daily work, with headroom for larger models at batch speed.
R2 — Locality. All documents, prompts, and outputs stored and processed on-property by default. Cloud is opt-in per task, never a fallback.
R3 — No inbound. Zero open inbound ports. Remote access only through an outbound overlay tunnel.
R4 — Records. Every household or business document scanned, OCR'd, indexed, and answerable by the model.
R5 — Governance. Any agentic action is proposed, reviewed, and logged before it executes. No unbounded loops.
R6 — Power. Rides through outages gracefully; sustained draw under 1 kW; a path to running on stored energy.
R7 — Footprint. Everything fits half a closet and stays quiet enough to live near.

Every choice below traces to one of those. When a shiny component doesn't serve a requirement, it doesn't get bought.

The hardware layer

Compute — pick your tier, and match it to model architecture, not just size. The single most misunderstood fact in local inference right now: memory capacity decides what fits, but memory bandwidth decides how fast dense models generate. The new 128 GB unified-memory boxes hold a dense 70B easily and then generate at single-digit tokens per second, because ~250–300 GB/s of bandwidth is the ceiling. The same boxes fly on mixture-of-experts models, which only activate a fraction of their weights per token. So: buy for the model class you'll actually run, and distrust any hero benchmark that doesn't state framework, quantization, and dense-vs-MoE.

Two numbers, two different jobs: memory capacity decides what model fits, memory bandwidth decides how fast a dense model generates — a 128GB box holds a dense 70B but generates at single-digit tokens per second, while flying on mixture-of-experts models

The three tiers, at today's street prices — the half-closet tier is where most builders should land:

Three hardware tiers: Starter, a used 24GB GPU for 700 to 1000 dollars, fast at 7B to 32B dense; Half-closet, a 128GB unified-memory box for 2000 to 4700 dollars, the sweet spot for MoE and mid-size models; Deep end, dual 3090 rigs or a 512GB Mac Studio from 1600 to over 10000 dollars for fast dense 70B or the biggest MoE quants

The half-closet tier is where most builders should land: appliance form factor, 100–250 W under load, and enough memory that next year's models still fit.

Storage — the records vault. A two-bay NAS with mirrored drives, snapshots on, is the memory of the whole system. Follow 3-2-1: the mirror on-site, one encrypted copy off-site. This box outlives every model you'll ever run; don't cheap out on the drives.

Network — the property line. A real firewall (a small box running OPNsense/pfSense, or a serious prosumer router) with the AI node and NAS on their own VLAN. Default-deny inbound satisfies R3; remote access rides an outbound WireGuard/Tailscale overlay, so nothing on the internet can knock on your door. Add DNS filtering while you're in there.

A UPS. Not optional. It bridges flickers, rides out the seconds a generator or battery needs to pick up, and lets everything shut down clean in a real outage. Size it to your node's draw plus the network gear.

The software layer

Serving. Start simple: llama.cpp or Ollama serving GGUF quantizations, fronted by Open WebUI, exposing an OpenAI-compatible endpoint so every tool you already use points at localhost instead of a cloud. Graduate to vLLM when you need real concurrency. Quantization rule of thumb: Q4–Q5 is the daily-driver zone; leave several GB of headroom for KV cache, because long contexts eat memory fast.

Models. Track the current open-weight families (Llama, Qwen, Mistral, DeepSeek, gpt-oss and their successors) rather than marrying a checkpoint — the leaderboard churns monthly, your architecture shouldn't. Verify checksums on download, pin the version you're running, and record it. Model provenance is a supply chain; treat it like one.

The records vault, animated. Run paperless-ngx against the NAS: every statement, contract, court filing, and warranty gets scanned, OCR'd, tagged, and full-text indexed. Then put retrieval on top — an embedding model plus a local vector store (pgvector or LanceDB) — so the LLM answers questions about your own documents with citations back to the page. This is the single highest-value workload a household node runs, and it's a straight application of the patterns in our RAG Pipelines track. A filing cabinet that can be interrogated in a sentence changes how you deal with every institution that assumed you couldn't.

The governance layer: a bounded loop, not a loose intern

Here is where most home-lab guides stop and most real damage starts. An ungoverned agent with tool access is a very fast intern with no procedure: it will happily loop forever, act on truncated output, and "complete" work it never actually did. The fix isn't more intelligence. It's structure — the same spec-first discipline the whole build follows. Four roles, one record:

The governed agent loop: a spec becomes a plan, then bounded generation attempts capped at three before quarantine, then a review gate checking produced output against the plan, then execution that is reversible by default with a human sign-off gate for anything irreversible like deploy, spend, or send, and every decision lands in an append-only log

Five rules, all learned the hard way somewhere:

Bound every loop. Any retry or regenerate path gets a hard attempt ceiling, and hitting it quarantines the work for a human instead of spinning until a watchdog kills it.
Guard truncation. Long generations get cut off mid-file more often than you'd think. Detect it (unclosed structures, missing terminators) and re-request; never pass partial output downstream as if it were whole.
Verify plan vs. produced. If the plan says ten artifacts, count ten actual artifacts — not ten mentions of artifacts. Agents reference work they didn't do with total confidence.
Gate the irreversible. Generating a branch is reversible; deploying, spending, and sending are not. The reversible side can run autonomously; the irreversible side waits for a named human at a gate.
Log append-only. Every decision — proposed, reviewed, approved, executed — lands in a record nothing can quietly rewrite. When something goes sideways at 11pm, the log is the difference between diagnosis and archaeology.

Slow is smooth, and smooth is fast: the bounded loop feels slower per attempt and is dramatically faster per delivered result, because you stop paying the rework tax on confident garbage. This is the same engineering posture our Design & Specify and Testing tracks teach — the node is just its smallest complete habitat.

The layer nobody ships: the operator console

Everything above assumes someone comfortable in a terminal. That's fine for the builder; it's a dead end for the household. The loop only becomes livable when its roles become screens: the plan readable before anything runs, the review gate a button anyone in the house can press, the quarantine an inbox instead of a log grep, the decision record a ledger you can scroll at 11pm. If property-local AI is going to belong to families and small operators — not just to people who enjoy SSH — the console is the product.

That is the layer I'm building SQUEIL to be: a sovereign Property OS for exactly this stack — the governed loop this guide just taught, with its gates, sign-offs, and append-only record behind a GUI, proven first on my own service business before anyone else is asked to run theirs on it. It is not generally available yet; it's getting close. Until it ships, build the loop by hand — every habit transfers one-for-one, and the console will assume you have them. The spec-first discipline you practice now is the operating skill you'll bring to it later.

The power layer

Do the arithmetic before the shopping. A half-closet-tier node averages maybe 150 W with bursts to 250 W — call it 3–4 kWh/day, roughly a refrigerator. The NAS and network add another 1 kWh. Against a typical home's ~30 kWh/day, the AI node is a rounding error; the dryer is the extravagance.

Resilience comes in three stages, and you can stop at any of them. Stage one is the UPS you already bought: minutes of bridge, clean shutdowns. Stage two is stored energy: a wall-mount battery bank (10–40 kWh) behind a transfer-capable inverter keeps the node, network, fridge, and lights alive overnight. Stage three is generation: rooftop solar sized against your local sun-hours (daily kWh ÷ average sun-hours ≈ array kW, then oversize for winter) recharges the wall and demotes the grid to backup. Two boundaries to respect: panels, batteries, and transfer equipment are licensed-electrician and permit territory — the DIY line stops at the wall outlet — and budget honestly, since the U.S. federal residential credit for solar and batteries ended after 2025, leaving state and utility programs as the incentives that remain.

Threat model and operations

Keep it boring and written down. Egress: default-allow is fine for a household, but log DNS and know what your node talks to; an inference box has no business calling anywhere during a chat. Updates: pin versions, update on a schedule you choose, read the changelog — R2 means nothing changes unless you change it. Secrets: no credentials in anything a browser can fetch; environment files stay out of version control. Backups: a backup you haven't test-restored is a rumor — restore one file a month. And monitor the basics (disk health, temps, UPS state) so hardware tells you it's dying before it does.

Build order

Crawl, walk, run — each stage useful on its own:

NAS + paperless-ngx. The records vault pays for itself before any AI arrives.
Inference box + Open WebUI. A private assistant on your LAN, one weekend in.
RAG over the vault. Your documents become answerable. This is the "wow" milestone.
The governed loop. Give the model hands only after you've built the gates and the log.
Power stages. UPS on day one; battery and solar when budget and permits allow.

A starter build lands under $1,500. The half-closet build — the one this guide is really about — runs $3,500–6,500 for compute, storage, and network, and it is the most leverage per dollar a technical household can currently buy: a private model, an interrogable archive, a governed agent, and a machine that keeps working when the internet doesn't.

The why behind all of this — what it means for a household to compute and generate its own needs — lives in the companion essay, Bring the Power Home. The how now lives on your shelf.