OmniJev/Flide: Fast inference infrastructure for decision models

github.com5 min read

Flide is a fast inference infrastructure designed for Jev-style decision models and chat models. Utilizing exact state forking, the engine delivers significant speed improvements over vLLM and SGLang while maintaining high throughput and decision accuracy. Open article →

More on this topic