Files
jason-fxz 12ed695386 feat(stats): record per-request TTFT and report its mean
The request ring only carried whole-request duration, so /v1/stats exposed no
first-token latency — the number that says how responsive a local model feels,
independent of decode speed. Streaming generations now stamp the first delta
they emit; non-streaming rows carry None and are skipped by the mean, so a
/health probe can never drag it toward zero.

Surfaced as requests.ttft_mean_ms in /v1/stats and in `ft ctl stats`.
2026-08-18 19:08:52 -07:00
..
2026-08-11 22:53:25 +00:00