You don't tighten them down too closely, plus sort of picture the rocket 'gently' bouncing off the rods. It will seek the path of least resistance and center out quickly.
As far as performance improvement, you get it both from the elimination of friction between the rod/rail and lug/buttons AND from the greatly decreased drag of nothing protruding off the side of your rocket.
The drag introduced by lugs or buttons is much more than you might realize. I had a discussion with someone who had actually seen double-digit percentage improvement in altitude with the elimination of launch lugs, although I don't remember the exact numbers.
It would be pretty inexpensive to do your own tests. Try building 2 models about like LOC Lil Nukes exactly the same, one with a launch lug, one without. Include a place to put a simple 'altitude only' altimeter. Make a simple tower out of 3 pieces of 5' long electrical conduit, a bucket, and some ready-mix concrete. Fly each model 3-4 times on an E-16 or similar motor and record the altitudes. I think you will see consistent performance improvement from the tower launcher.
I might just do this myself and report back in a few months.