WiCi releases WiCi Infer for SSD streaming 800B LLMs
WiCi has released WiCi Infer, a free desktop application designed to run 800B-parameter AI models locally on personal computers by streaming weights from the SSD. Built on the open-source InferSSD engine, the app stores large model files on a local drive and loads only the active mixture-of-experts weights into memory as tokens are generated. According to the company, the software enables consumer hardware with 16GB to 48GB of memory to run massive models locally while keeping prompts private. Open article →