An episode can stop because the task ended or because an external limit interrupted it. Those events look similar in a training loop but can require different value targets.
Termination means the modeled process reached an end state. Truncation means observation stopped for a reason outside that process, such as a simulator’s maximum step count. The correct interpretation comes from the environment definition, not the name of the variable in your code.
The numerical consequence
Suppose a transition gives reward 1, the discount factor is 0.9, and the estimated value of the next observation is 5. For a continuing transition, the one-step target is 1 + 0.9 × 5 = 5.5.
For a genuinely terminal transition, the target is just 1. Future return after termination is zero by the task definition. Using 5.5 would invent future opportunities that do not exist.
For an external time-limit truncation of a continuing task, retaining the bootstrap estimate can be appropriate. Treating it as terminal gives target 1 and can systematically undervalue states near the arbitrary cutoff.
A finite horizon is different
Sometimes the horizon is part of the actual task. A game may end after ten moves, or a controller may have a fixed number of actions to reach a target. Remaining time then belongs in the state if it affects optimal decisions.
A genuine end at the task horizon can be termination. An external training limit imposed on an otherwise continuing task is truncation. Do not decide solely by checking whether a timer was involved.
Store enough information
A replay record should preserve reward, the next observation, and separate terminated and truncated flags. If an environment automatically resets, ensure that the next observation in the record is the final observation of the transition, not the first observation of the next episode.
The update also needs a policy for bootstrapping each boundary type. Write down that policy next to the environment definition so future refactors do not silently change the learning objective.
A compact test
Construct one continuing transition, one terminal transition, and one externally truncated transition with the same reward and next-state estimate. Verify the expected targets explicitly. Then test the auto-reset path, where a final observation can be easy to lose.
This check is cheap and often more informative than another long training run. A model cannot reliably compensate for a target that erases the wrong future rewards.
See Gymnasium’s time-limit explanation for the API distinction, or ask about termination and truncation.