Term tag: reward

Reinforcement Learning (RL)

How an agent learns to act by trial and error: it takes actions in an environment, earns rewards or penalties, and adjusts to maximize long-term reward.

Reward Hacking

When an AI maximises its specified reward but defeats the designer's intent — optimising the proxy, not the goal. Goodhart's law, made mechanical.

Reward Model (RM)

The scorer that stands in for a human during RLHF: how reward models are trained, why their absolute scores mean nothing, and how they break.