e0a3bbc046
Merge pull request #112 from FlashML-org/feat/split-residency
...
feat(moe): per-layer host-bank residency
2026-08-23 17:39:31 -07:00
831d38a66b
fix(moe): report the residency banks actually settle at
2026-08-23 23:06:56 +00:00
eebb3f53f3
feat(moe): estimate expert-bank bytes from the model config
2026-08-23 23:06:56 +00:00
c41833b717
feat(moe): per-layer host-bank residency -- split lock-CPU/pin-GPU layers, auto selection under capped pin quota
2026-08-23 21:02:59 +00:00
184a4f114d
build(kernel-cache): add sm_80 (A100/A800) to default arches ( #75 )
...
Signed-off-by: jasonfan <36730318+jason-fxz@users.noreply.github.com >
2026-08-22 23:38:42 -07:00
f0abe587a1
docs: add CONTRIBUTING.md
2026-08-22 22:22:15 -07:00
0ab982f109
chore(README): Add Community WeChat link to README
...
Added a link to the Community WeChat in the README.
2026-08-20 11:43:47 -07:00
bc6b269cbc
chore(assets): Add FreeToken wechat group invite
2026-08-20 11:41:16 -07:00
9db1a39455
chore(release): 0.1.2
Release wheels / build (push) Has been cancelled
Release wheels / publish-testpypi (push) Has been cancelled
Release wheels / publish-pypi (push) Has been cancelled
Release wheels / github-release (push) Has been cancelled
2026-08-19 05:57:18 +00:00
3f02e3c5a8
docs(README): use absolute URLs so PyPI renders assets and links
2026-08-18 20:49:53 -07:00
c0a4a60b2c
Merge branch 'dev/refine-readme': README overhaul + quick start guide
2026-08-18 20:38:49 -07:00
9ecbc9f33f
docs(README): add a Desktop screenshot to Getting Started
...
The Desktop paragraph promised a GUI for running models and tuning the engine
without showing either. The shot is the console with DeepSeek-V4-Flash FP4
resident on a 32 GiB card, so the cache sliders carry real numbers: 512K KV
tokens for 3.40 GiB against 1024 of 11008 expert slots for 12.8 GiB.
Framed as a floating window (rounded, drop shadow, transparent margin) so it
sits on both the light and dark README themes.
2026-08-18 20:36:08 -07:00
12ed695386
feat(stats): record per-request TTFT and report its mean
...
The request ring only carried whole-request duration, so /v1/stats exposed no
first-token latency — the number that says how responsive a local model feels,
independent of decode speed. Streaming generations now stamp the first delta
they emit; non-streaming rows carry None and are skipped by the mean, so a
/health probe can never drag it toward zero.
Surfaced as requests.ttft_mean_ms in /v1/stats and in `ft ctl stats`.
2026-08-18 19:08:52 -07:00
586a6c16ce
perf(fp8): run per-tensor fp8 as W8A8 via scaled_mm ( #7 )
2026-08-18 18:04:39 -07:00
f4373f8a45
docs(README): add install commands to Getting Started
2026-08-18 16:09:32 -07:00
5528af06a4
docs(README): add Citation
2026-08-18 15:50:32 -07:00
fca27580a1
docs: refine README.md, add quick start guide
...
Signed-off-by: jason-fxz <jason341132@qq.com >
2026-08-18 15:24:47 -07:00
5812fbd084
chore(README.md): add links.
2026-08-18 13:25:02 -07:00
6223cda1b0
Support muse glimmer ( #4 )
...
* feat(muse_glimmer): serve Muse-Glimmer-30B
2026-08-18 12:26:05 -07:00
b86a904f86
build(release): pypi-ready wheels, metadata, and publish workflow ( #6 )
2026-08-17 23:03:44 -07:00
76edc16282
build(release): pypi-ready wheels, metadata, and publish workflow
2026-08-18 05:00:47 +00:00
711325d17e
fix(server): stop API server when backend dies
...
Signed-off-by: jason-fxz <jason341132@qq.com >
2026-08-16 14:34:17 -07:00
2635246ba2
ci: move nightly wheels to 11:00 UTC
2026-08-16 12:11:17 -07:00
d7cd6ed88b
[Feature] unify reasoning effort on one shared scale ( #5 )
...
* feat(server): accept every reasoning-effort dialect
* refactor(server): derive thinking gears from the checkpoint, not a registry
2026-08-15 23:40:52 -07:00
a2572083e9
feat(launch): support launch deepseek harness ( #3 )
...
* feat(launch): support launch deepseek harness
* fix(launch): node version gate, cloud key clearing, and dsh docs
2026-08-14 23:50:44 -07:00
4f510f1ba6
feat(server): enable reasoning_effort on /v1/chat/completions ( #2 )
...
* feat(server): enable reasoning_effort on /v1/chat/completions
* fix(server): glm thinking-off mislabeling and reasoning_effort mapping edge cases
2026-08-14 00:02:32 -07:00
c28e93a8c0
chore(assets): update icons & logos
2026-08-13 18:09:01 -07:00
fdfcacd491
chore: Enhance README with logo
2026-08-13 16:28:39 -07:00
9dc3ff4ca5
Support minimax m3 ( #1 )
...
* feat(minimax_m3): serve MiniMax-M3-NVFP4 single-GPU (block-sparse GQA + MXFP8 dense + swigluoai NVFP4 experts)
2026-08-13 16:22:41 -07:00
17af154bb5
feat(assets): add dark/light logos
2026-08-13 16:10:28 -07:00
3e5214d4a3
build(deps): pin apache-tvm-ffi, flashlib, triton to exact versions
2026-08-13 14:51:26 -07:00
97eb4e25b5
Merge branch 'main' of github.com:FlashML-org/FreeToken
2026-08-13 13:27:51 -07:00
a8669e4950
chore(assets): add freetoken icon & logo
2026-08-13 13:26:53 -07:00
ea5348b4e3
feat(kvcache): checkpoint decode state at the tool-call opener token
2026-08-13 18:32:02 +00:00
ee54f638fd
fix(gemma4): read head_dim / kv heads per attention type from per_layer_config
2026-08-13 03:57:59 +00:00
3af9d90ee5
feat: initial open-source release
...
Co-authored-by: Shuo Yang <andy_yang@berkeley.edu >
2026-08-11 22:53:25 +00:00
Shuo Yang