Agility is a bit like speed and power……some people have it and other’s don’t. Unlike endurance, to which humans are genetically predisposed (hence highly trainable), speed and agility are very much heritable traits (you need to have chosen you parents well).
In working with top class XC skiers I have found it much easier to take a Fast Twitch (those generally better endowed with speed/power and agility) and turn him/her into an endurance athlete than the other way around.
Running coaches have an old saying: Sprinters are BORN…..Distance runners are MADE.
Not all is lost however. You can train this stuff. But, it is going to require you get out of your comfort zone A LOT and learn to move your feet quickly, develop better balance and ankle and leg strength.
Start with single legged balance training on a Bosu ball. Progress to doing hill sprint workouts on steep scree. I suck at this same thing and I have nasty scree hill I do hill sprints on. The scree needs to be big enough to be stable (not gravel).
Or just generally spend more time on unstable footing. Depending upon your genetics this will either come quickly or maybe not ever. But you won’t know till you train it.