TensorRT-LLM: NVIDIA's Open-Source Library for Optimized LLM Inference
Deploying large language models in production requires more than just loading weights onto a GPU. To achieve acceptable throughput and latency, …
Tags
Optimization
Prompt Poet: Character.AI's Open-Source Prompt Engineering Framework
Prompt engineering has evolved from a niche skill into a critical discipline in AI application development. The difference between a good prompt …
Pezzo: Open-Source LLM Operations Platform
Managing LLM-powered applications in production has become one of the most challenging operational problems in AI engineering. Teams that deploy …