That gets heavy surprisingly quickly. I've started not only building with a gram scale, but painting with one, too.
Would be an interesting science project to compare the performance of a lightest-possible finish vs. a smoothest-possible finish and the respective weights. Of course, the result will differ greatly depending on the rocket design. A minimum diameter MPR or greater often comes out below optimum mass, so adding weight to reduce drag is a win-win. On the other hand, a non-MD BP-powered rocket is almost always somewhat above optimum mass.
Maybe a bulk pack of Alphas would be the ideal starting point for such a test:
Two with 18mm mounts, built as light as possible.
Two built MD for 95mm motors
In both cases, one built as light as reasonably possible with "acceptable" finish and letting the mass creep up to get a polished-smooth finish. For the 95mm motor MD versions, I'd run the polished version both however it comes out and ballasted as necessary to reach optimum mass.