Within Training Peaks there is the option to create different zones and thresholds for different activities, so it possible that there is something off on your settings, leading to a higher hrTSS than you would expect. However, I think by default there is only one box to enter your thresholds in and it is applied to all activities, so this is perhaps unlikely.
It seems to me (as a punter, not a Coach) that the TSS is pretty high for the effort you descibe. But how does the TSS compare to other activities (run, hike) at that level of effort and/or HR? If the calculated TSS is comparable to the other activities, but there is no correlation for how much effort you feel you put in then perhaps consider cutting the TSS by an amount to reflect that. Take notes on perceived effort, vert, and how much you reduced the TSS and then refine your calculations over time. As well as the impact from the descents, I imagine the algorithms behind the TSS are not able to predict the influence of good technique particularly well.
To answer your specific question, I am a long-suffering Suunto HR strap user and have stopped using a HRM for most of my workouts. I estimate TSS based on perceived effort for the activity, so address any easy skiing as part of that calculation. I have a fairly narrow score range that I apply to XC as my limited skills prevent me from going too hard….