This is normally the result of overly vigorous sanding of the edges of the body (can also happen at any sharp edge in the design. You can either try to avoid it or deal with it afterwards... I try to avoid it but only succeed mostly, and often have to deal with it a bit. But if you say "fibrous and ratty looking" that probably means you went... well, somewhat too far.
Anyway: two ways to avoid it:
1) Simply be careful at the edges. Don't sand off the edge: rather, sand up to the edge. When everything else is done, sand around the edges carefully, trying not to lean on the edge with the sandpaper when you're half on/half off.
2) At the top end of the BT (where the nose or coupler goes, you can insert a coupler with a piece of sacrificial body tube. This will protect the edge.
Once it's already been oversanded, I have found that a good application of thin CA applied with a Q-tip will fix it. After doing that you'll need to sand again, this time being more gentle and careful. You may need additional application of CWF or filler/primer to finish it up.
This varies very widely among modelers; you will find no precise consensus here.
After doing filler/primer, I sometimes use no further primer, and go straight to the color coats. If I apply primer, I'll usually two two light coats, and usually sand after the second coat.
Painting is usually 2 or 3 coats depending on the paint. Usually your first coats will be light, and the last coat will be heavier to achieve gloss.
@hcmbanjo recommends two light coats and one final heavy coat; I use that as a guide, although sometimes I get away with one light and one heavy, and other times I need a fourth.
If the above steps do not yield a satisfactorily smooth finish, and I'm not totally sick of painting, I'll sand after completing the above steps and then apply one additional heavy coat.
View attachment 483832