My thought was to use GLUE as I always use then aply a paintable caulk over that to make a nicer fillet.
That's the right idea.
The glue at the base of the fin is what makes the structural joint. If you don't have a good tight fit of fin-to-BT then adding gobs more glue on the outside of the root is not going to restore the joint quality. In the mean time, excessive amounts of glue can warp the parts (especially model rocket cardboard-type BT materials), can dry into ugly shapes (especially most white glues and some yellow glues, when applied thick), and adds weight at the wrong end of the rocket.
Make a good structural joint and then (if you want) cover with anything you like that works well with your primers and paints. But keep in mind that unless you are super-polishing the entire rest of the airframe (for laminar flow) and airfoiling your fins and polishing them smooth also, and removing the launch lug (or button) and using a tower, then adding fillets is NOT GOING TO DO A DARN THING for streamlining or any practical improvement in performance.
It might look cool, that's a matter of personal choice, but don't fool yourself that fillets are "necessary"