Good question. My guess is that it depends how stressful the stimulus was, all things considered.
Heart rate is a measurement of total stress, not intensity. Two people of equal physiology and fitness, but different life stress, would likely see heart rate behave differently, both during and after workouts.
Within training, if heart rate falls quickly after a workout, then I think it’s safe to assume it wasn’t that stressful (which is usually a good thing). If not, then the opposite.