GPTQModel

GPTQModel: Production-Ready LLM Quantization Toolkit for GPU and CPU

Large language models are powerful, but their size makes them expensive to deploy. A 70-billion-parameter model in 16-bit precision requires …