GRPO on SoloSoft

OpenManus-RL: Reinforcement Learning Tuning for LLM Agents

Fri, 01 May 2026 00:00:00 +0000

OpenManus-RL is an open-source research project at the intersection of reinforcement learning and LLM agent systems, developed collaboratively by Ulab-UIUC (University of Illinois Urbana-Champaign) and MetaGPT. The project provides a comprehensive framework for reinforcement learning tuning of LLM-based agents, with implementations of GRPO (Group Relative Policy Optimization), supervised fine-tuning (SFT), and advanced rollout strategies designed specifically for agentic tasks.

As LLM agents become increasingly capable of complex multi-step reasoning and tool use, the need for targeted reinforcement learning optimization has grown dramatically. OpenManus-RL addresses this by providing a modular, reproducible pipeline for training agents on agent-specific tasks, with built-in support for diverse environments including software engineering (SWE-Bench), web navigation (WebArena), and general tool use.

Verifiers: Modular RL Environment Library for Training LLM Agents

Fri, 01 May 2026 00:00:00 +0000

Verifiers is a modular Python library developed by PrimeIntellect-ai that provides a comprehensive framework for creating reinforcement learning environments tailored to training LLM agents. Designed for researchers and practitioners working on RL-based LLM alignment and agent optimization, Verifiers offers a clean, composable API with components for parsing model outputs, evaluating responses against rubrics, computing rewards, and running GRPO-based training loops.

The library addresses a growing need in the AI research community: as RL-based methods like GRPO, PPO, and rejection sampling become standard for LLM fine-tuning, researchers need standardized, reusable environment components rather than building training infrastructure from scratch for each experiment. Verifiers provides exactly this – a modular toolkit where environments are assembled from interchangeable building blocks.

VeRL: ByteDance's Reinforcement Learning Framework for LLMs

Fri, 01 May 2026 00:00:00 +0000

The most exciting frontier in large language model research in 2025-2026 has not been about making models bigger. It has been about making them smarter through reinforcement learning. DeepSeek-R1 demonstrated that RL training – specifically GRPO (Group Relative Policy Optimization) – can dramatically improve a model’s reasoning capabilities, enabling chain-of-thought reasoning, self-correction, and structured problem solving that rivals much larger models. ByteDance, one of the world’s largest technology companies and the creator of TikTok and Douyin, has been applying these same techniques at scale to train its own models. VeRL is the framework behind that effort.

VeRL (Voltron Reinforcement Learning) is ByteDance’s open-source reinforcement learning framework designed specifically for LLM training. It implements state-of-the-art RL algorithms including PPO (Proximal Policy Optimization) and GRPO, integrates tightly with vLLM for efficient inference during training, and supports distributed training across hundreds of GPUs. VeRL is the production framework that powers ByteDance’s internal LLM development, including the Doubao (豆包) AI assistant.