Daily build · Private AI / Infrastructure

Server Closet

AI in a box for firms that won't use the cloud: model size and users in, hardware and cost vs cloud out.

Everything stays in your browser. No inputs or estimates are uploaded.

Start with a firm

Law firm loads by default. Pick a scenario, then tune any input.

All prices are rough illustrative estimates; performance figures are directional, not quotes. Validate workload, networking, thermals, software licensing, and vendor pricing before purchase.

Sizing inputs

Model & memory

Model class Quantization Context length Concurrent users

Usage

Total staff users Requests / user / day Input tokens / request Output tokens / request Uptime need Air-gapped requiredFlags cloud as prohibited; the hypothetical comparison remains visible.

Cost assumptions

Cloud comparisonDrives breakeven, TCO, chart, and verdict. Dedicated cloud $ / GPU-hr (80GB)24GB and 48GB classes scale proportionally from $0.70 and $1.20. Electricity $ / kWh Amortization years Admin hours / month Admin rate $ / hour Open-model input $ / 1M Open-model output $ / 1M

Assumptions & formulas

These are directional planning estimates. Real throughput depends on model architecture, serving software, prompt mix, batching, speculative decoding, GPU interconnect, and latency targets.

Weights GB = parameters (billions) × bytes per parameter. FP16 = 2, 8-bit = 1, and 4-bit = 0.56 including overhead.
KV bytes/token = 2 × layers × KV heads × head dimension × 2 bytes. 8-bit and 4-bit model modes assume 8-bit KV, halving this result.
KV total GB = KV GB/token × context × concurrent users × 0.6 average fill.
Total VRAM = (weights + KV) × 1.10 + 2 GB runtime. Only 85% of listed memory is treated as usable.
Peak target = concurrent users × 25 output tok/s. Approximate per-node capacity is bandwidth GB/s ÷ weights GB × min(concurrency, batch cap) × 0.6; extra nodes are added when needed.
Capex = hardware × nodes + 15% integration + $5,000.
Monthly opex = kW × hours × electricity × 1.4 PUE + 10% annual maintenance ÷ 12 + admin hours × rate. On-prem power hours are 220 for business hours and 730 for 24x7.
Dedicated cloud monthly = GPU rate × GPUs in the selected configuration × billed hours × 1.15 for storage, egress, and private networking. Billed hours are 260 for business hours and 730 for 24x7/redundant. Base rates are $0.70 for 24GB, $1.20 for 48GB (including each Mac Studio-class equivalent), and $2.80 for each 80GB datacenter GPU; editing the 80GB rate scales all three proportionally.
Token API monthly = staff × requests/day × workdays × (input tokens × input price + output tokens × output price) ÷ 1M. Workdays are 22 for business hours and 30 for 24x7. Frontier API uses $3 input / $15 output per 1M tokens.
Workstation and Mac Studio-class configurations are limited to four base nodes before redundancy; larger requirements advance to a server tier.
Breakeven is the first month, up to month 60, where cumulative on-prem cost is no greater than cumulative cloud cost.

Model	Layers	KV heads	Head dim	FP16 KV / token
7B	32	8	128	0.000131 GB
13B	40	8	128	0.000164 GB
32B	64	8	128	0.000262 GB
70B	80	8	128	0.000328 GB
120B-class	96	8	128	0.000393 GB