I suggest shutting off the recording feature on your watch when sitting on the summit so as to not record that. There is certainly diminishing returns when the HR is far below your AeT. BUT believe it or not accumulating huge volumes of very low intensity aerobic work does add to your aerobic base. I generally find that one hour at AeT gives a hrTSS of about 60. SO your 3:20 hike with a hrTSS of 166 seems pretty reasonable to me.
Another way to make these girlfriend hike more productive for you is to carry a loaded pack. This will push your HR up closer to your AeT on the uphills and provide a better strength training effect on the downs.