VLLM

  1. vLLM: High-Throughput LLM Inference with PagedAttention

    Serving LLMs in production is fundamentally a memory management problem. The KV cache — the set of attention key-value pairs stored during generation...

    AI

  2. IndexTTS-vLLM: Accelerated Open-Source Text-to-Speech with vLLM Inference

    Text-to-speech technology has advanced dramatically in the past three years. Zero-shot voice cloning, where a system can synthesize speech in a novel...

    Multimedia