Hmm, I've never heard of permanent damage from epoxy. The west systems guide seems to indicate that it is will resolve once contact with epoxy is avoided:
https://www.westsystem.com/ss/health-effects-from-overexposure-to-epoxy/
This CDC brochure talks about epoxy and fiberglass being temporary irritants:
https://www.cdc.gov/niosh/docs/2011-199/pdfs/2011-199.pdf
But since the OPs question has nothing to do with epoxy or fiberglass that's really not the point.
For the OP it seems much more likely to be a form a contact dermatitis:
https://en.wikipedia.org/wiki/Contact_dermatitis
Which is can be caused by just about anything. Some folks get it from prolonged contact with their own sweat. The best thing is simple avoidance - wear gloves (nitrile are better than latex) when handling your rocket gear. The main residues from BP are potassium salts which are corrosive. It's unlikely it's the APCP residue that is bothering you. AP does not leave a lot of particulate residue behind. For ejection charges you could switch from BP to something like Triple7 which does not use sulfur in it's mix, but you would need to test it with your setup to make sure it works for you.
You can also use a barrier hand cream before you start working with your rocket gear if you don't want to wear gloves. But gloves are the answer to your problem. It's the same thing a dermatologist will tell you.
Tony