KVBoost

KVBoost is a new technique that enables chunk-level KV cache reuse for HuggingFace models, achieving 5-48x faster Time-To-First-Token (TTFT) performance. The project demonstrates significant improvements in inference speed by reusing key-value cache data across multiple requests.