I have worked as a Glacier guide for several years and when I’m guiding, my HR is normally around 80-100bpm. For reference, my AeT is 170 and AnT is 185.
My experience has definitely been in line with what you describe. Guiding would leave me tired but I’m not gaining any fitness. I suppose you get good at walking at a very slow leisurely pace(3-4km/hr) but there are no speed gains to be made at higher heart rates.
I was in decent shape in summer 2019 after spending a lot of time in the Alps with somewhat structured training. Then I was Guiding all winter doing ice cave trips in Iceland and because I was working long days which would leave me tired I did not train much outside of work. Walking around 6hrs/day and driving 3-4hrs and various other physical tasks.
I lost a lot of fitness during the winter.
Now in 2020, there is not much guiding work due to Covid-19 so I haven’t been guiding at all and have resumed proper structured training and my form is improving a lot. Running pace at AeT was 6:00 min/km in March and now in August, it is down to 4:50 min/km.
I’m 90% sure that if I had been guiding all this time I would not have been able to make these fitness gains due to the tiredness you get from this kind of work.