Termination and truncation
Termination means the task reached an end state. Truncation means an external limit stopped observation, even if the underlying task could continue.
Definition
Termination means the task reached an end state. Truncation means an external limit stopped observation, even if the underlying task could continue.
Intuition
Running out of test time is different from reaching the goal. Confusing them changes the value target.
Worked example
A robot reaches its destination: termination. A simulator stops after 200 steps while the robot is moving: truncation.
The math
For a genuine terminal state, the next-state value is zero. Time-limit truncation may still require bootstrapping when the task continues.
In practice
Store both flags in a replay buffer and test the target for each case. Use the environment semantics to decide whether the horizon is part of the task.
Go deeper
Sources
More in Reinforcement learning
Assembled from the ReLU.chat curated knowledge base. These explanations are concise on purpose; check the sources for anything important.