Should I modulate and do an easy week anyway?

Yes. If a training regime is at a near-constant load, you’ll start to feel stale. Adaptation is better triggered by changes in load. Also, not everything we’re training builds and recovers on the same timeline. From what I’ve read, fight-or-flight adrenal response in particular needs downtime. If you’re caught in a chronic stress environment, eventually you’ll start breaking down.

Will my fitness progress faster if I back off even though I can handle my current mileage just fine?

Over what timeline? IN the short-term, feel free to extend a mesocycle by a week here and there if you’re feeling good. But within a macrocycle, you’ll need modulation to keep your adaptation going strong.

As a general rule of thumb, ask yourself questions like:

“Is what I’m about to do a strong-like-bull-smart-like-tractor approach? Am I being greedy when I should be fearful? Am I prioritizing short-term gratification over slow-and-steady progress?”

Be honest with yourself and, if the answer is yes, then back off.

Also ask:

“Have I done as little as required (to get a response) or am I trying to do as much as possible (so I feel good)?”

Or would it be better to use the time to build more aerobic capacity/strength.

It depends. See above.