Total amateur here and also a desk jockey as a worker. From my point of view, yes, your job is base training, as I would think you are way fitter than I am as I struggle to do my 5 or 6 hours per week of training! (And as I also struggle to adapt to even that and so will need to over this year before I increase the volume for next year.) I also see it as a more specific and appropriate than, for example, what I do—Z1/2 jogging and Z1/2 cycling, doing strength work, and now entering the part of my base phase where Z3 is prescribed. From my point of view, carrying and tossing around trees, working a chain saw, and in general the climbing and carrying of stuff you do, is more appropriate training for general mountaineering than what I do, for sure! Your overall work capacity is bound to be miles above my own.
Or at least the job WAS base training for you, up until the point where it no longer provided a training stimulus, given that you are now adapted to the amount of work (and have been for a long time), as the coaches in this thread say. So the point also holds that the job does not have progression and so is exercise instead of training, to put it glibly. And the work capacity may need to be sharpened and re-stimulated for your larger goals.
Maybe what you need is some data on where you are with your fitness. Have you done the “alpine combine” test in TftNA, and where do you stand with it? Have you tested your AeT and AnT? In terms of heart rate, are they within 10% of each other as TftNA prescribes reaching before doing more AnT training? Maybe that is one way to gauge whether your after-work training should be only more Z1/2 base or is OK to be Z3/4 intervals or other high intensity work?
Maybe lactate testing in a lab would help making that determination. Maybe you could consider ways to make your work progressive in terms of training. I don’t know how that could work, maybe adding a weight vest with just a little bit of weight (5 lbs?) and then increasing that–over the course of, what, a year?—in small increments, monitoring HR and sleep/fatigue to avoid overtraining and to keep it Z1 instead of turning into muscular endurance work? Of course, the job likely keeps you at some level fatigued and beat up all the time that a long-time desk jockey (such as myself, ahem) can’t imagine, and makes this kind of suggestion naïve and ridiculous. Which is partly their point of not considering work like this training but more like work capacity maintenance that may be fatiguing enough that makes other, more specific training difficult to do.
An opposite tack would be to consider ways you could make your job LESS physical work. Are there any efficiencies you have been avoiding because you are trying to use the job as training? I don’t know what these would be; something like having a young(er) hired hand doing more of the lifting, delegating more (maybe you work alone), cutting trees in the back yard before moving to the front to lift lower weight. I don’t mean shirking duties! Just to consider ways to make it less fatiguing so you can put that energy to training outside of work. Maybe it’s obvious that you would have or can’t do this already, and I’m mostly showing my own ignorance of real physical labor here.
Again, I have very little experience in what I am blabbering on about! These are just some things that occurred to me when I was reading the thread as someone on the opposite end of the spectrum who wishes he were as fit as you likely are just from your job. And who mis-spent his youth not exercising and lifting enough when the hormonal environment was such that the gains would still be with me to this day (50 years old now).