Preference Optimization

DPO: Direct Preference Optimization for LLM Alignment Without RL

For most of the history of large language model alignment, the dominant paradigm has been Reinforcement Learning from Human Feedback (RLHF) …