RL evaluation
RL evaluation measures a fixed policy on tasks, seeds, or environments that were not used to choose its parameters. Training reward alone is insufficient.
Definition
RL evaluation measures a fixed policy on tasks, seeds, or environments that were not used to choose its parameters. Training reward alone is insufficient.
Intuition
A student can memorize practice questions. Evaluation asks whether the learned behavior transfers.
Worked example
Train on one set of question templates, tune on another, and reserve new topics and paraphrases for the final test.
The math
Report per-task outcomes, sample counts, and variability across seeds. A point estimate without a denominator can mislead.
In practice
Compare against random, rule-based, and supervised baselines. Report failures as well as averages.
Go deeper
Sources
More in Reinforcement learning
Assembled from the ReLU.chat curated knowledge base. These explanations are concise on purpose; check the sources for anything important.