Model Serving

  1. vLLM: Inferencia de LLMs de Alto Rendimiento con PagedAttention

    Serving LLMs in production is fundamentally a memory management problem. The KV cache — the set of attention key-value pairs stored during generation...

    IA

  2. NVIDIA Triton: Servidor de Inferencia de Modelos IA Multi-Framework

    Training machine learning models has become accessible to a broad audience of developers and organizations. Serving those models in production —...

    IA