Private AI Sizing: Users, Agents, Rooms, Cost per Token

The author outlines a practical method for sizing private AI infrastructure by starting from the workload rather than the GPU count. The approach accounts for active requests in flight using Little's law, distinguishes between chat assistants and agentic loops, and factors in memory, facility constraints, cost per token, and regulatory compliance. Open article →