I don't have a humidity problem in this dry climate, but I've used this on MDF items I've built to toughen their surfaces and water-proof them (from spills, not immersion):
https://www.homedepot.com/p/Deft-1-...-Wood-Finish-Brushing-Lacquer-01004/100142803
It easily soaks right into the MDF producing a very evenly shaded darker surface that looks great and is fully sealed against water spills.
It would probably soak even more easily into kraft paper tubing, used on the inside surface of a built rocket, at least as far as your arm can reach into a 3.9" ID airframe since this stuff is brushed on. Added weight would certainly be a result, but it might have an added benefit beyond humidity-proofing of strengthening the tubing and producing a lower friction surface for recovery system ejection.
Coating FLAMMABILITY due to ejection charge gases and embers might be a problem. I'll be testing the weight gain and coating flammability on a short (scrap) length of 4" paper tubing.