Files

T

Xiaoze Fan 3af9d90ee5 feat: initial open-source release

Co-authored-by: Shuo Yang <andy_yang@berkeley.edu>

2026-08-11 22:53:25 +00:00

1.6 KiB

Raw Blame History

FreeToken

A local, MoE-offload inference runtime with an OpenAI- and Anthropic-compatible HTTP API — Run DeepSeek-V4-Flash on your 5090 with 20+ TPS.

Quick start

See docs/install.md for requirements and installation.

ft serve --model ~/models/Qwen3.6-35B-A3B   # API server on http://127.0.0.1:1919
ft launch claude                            # point an agent at it (codex / opencode / openclaw)
ft shell                                    # or chat in the terminal

Documentation

Install — requirements and setup
Supported models — model × quantization
CLI reference — ft commands and environment variables

Acknowledgment

FreeToken was deeply inspired by mini-sglang, and learned the design and reused code from the following projects: SGLang, vLLM, FlashInfer, flash-linear-attention, LightLLM and llama.cpp.

License

Apache License 2.0.