Good question. Fasted workouts can be deceptively tiring, often after-the-fact the way that you’ve described.
I think the simplest thing to do is to back off on the duration of your fasted workouts if they’re negatively impacting other important work. Then you can gradually build them up as tolerated.
One of the big mistakes that I made early on (and, embarrassingly, for years) was trying to optimize everything at once. As a result, I blew up several times. I’ve had much better success by dialing back my ambition for each workout and slowly moving things forward. Over the long-term, it’s been much more effective.