With your car-free life style especially pushing a 100lb stroller and carrying 20kg often you are in a great position to use any of our ski touring plans. I think the Grand Traverse plan would be an excellent choice for you. Random unstructured exercise does improve fitness up to a point. You’re probably at that point now where switching to a plan will take you to a higher fitness level.
I hope this helps,