Muscles are muscles, forearms or legs. Doesn’t matter. Ice baths are good and bad. The ice will reduce inflammation and so speed recovery. But the inflammatory response if one of the principle signaling mechanisms that triggers some critical adaptations to training. So do you really want to train hard in order to start that signaling pathway only to dampening it by using an ice bath that inhibits the thing you are trying to achieve?
The same thing for NSAIDs. Do you want to dampen or enhance the training effect?
If the inflammation is severe and finger joints in a lot of pain then localized icing might be a good idea. But you probably over did it in this case and are risking injury. But regular, generalized icing may not be the bets thing for long term improvement.