A couple of thoughts. I like the way you record things to be consistent season to season and year to year so you can compare apples to apples. Whatever you decide stick with it.
My very loose personal method is that if I am not getting my HR to 100 at some point I don’t bother recording it. Lawn mowing, dog walks, art crawl on first Friday don’t go in TP. I do think that easy walks can actually aid recovery and therefore are beneficial.