Custom reward functions for multi-turn reinforcement learning with Amazon Nova Forge
In multi-turn reinforcement learning, your custom reward function decides what the model actually learns. This post shows how to design a composite multi-turn reward for Amazon Nova Forge, execute model-generated code safely inside it, and instrument each component to catch the pitfalls that quietly collapse a reward.
Multi-turn reinforcement learning (RL) presents unique challenges when designing custom reward functions for Amazon Nova models. The reward function is crucial, as a poorly designed one can lead to unintended learning outcomes while training appears normal. Nova Forge simplifies this process by executing reward logic in a custom environment using Bring Your Own Orchestration (BYOO).
This allows developers to focus on defining desired outcomes while Nova Forge handles rollouts, message passing, and conversation state across turns. Nova Forge also offers a serverless multi-turn RL option for teams that prefer not to manage the environment. This post utilizes the BYOO path for multi-turn RL fine-tuning (RFT). Unlike supervised fine-tuning (SFT), RFT learns from evaluation signals on the model's own outputs, optimizing cumulative reward across the sequence of steps rather than grading a single response.
The reward function, a key component of RFT, guides the model by scoring model outputs. This post discusses how to design a composite multi-turn reward function that Group Relative Policy Optimization (GRPO) can learn from. It also demonstrates how to execute model-generated code safely within the reward and emphasizes the importance of instrumenting each component for trust in the training process.
By following the code examples provided, readers can implement their own reward functions tailored to their specific needs.
Written by urgent.news from AWS Machine Learning's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
