You can speed up the process a bit by only using "coarse" sandpaper (30, 40, 50 grit) for the most severely rough surfaces. When you use these grades of sandpaper (and continue using them), you will only get a surface as smooth as the size of the grit, which (for these grades) is nasty. These sandpapers will take down extreme irregularities in the surface but will leave it quite rough. I don't know anyone who uses these grades of sandpaper in rocketry.
The next step is to use a "medium" sandpaper (60, 80, 100 grit) to make the surface progressively smoother. These grades will erase part of the roughness left by the coarse sandpaper, but only down to the point where these medium grades are still leaving their own scratches and marks. This is still pretty rough stuff for any rocketry finishing applications.
The next step is down to "fine" sandpaper (120, 150, 180, 200-ish grit) to make the surface progressively smoother again. (See a pattern here?) These grades will (yada, yada...) These sandpapers will do a pretty good preliminary job of smoothing raw balsa and surfaces covered with primers.
Then (on the same project) you get to the "very fine" sandpapers (200 to 300 grit). You will probably spend twice as much time performing sanding at this stage as you did at the previous level. This is about as far as I like to go on the primer coats, but you might choose to go even farther and deeper into the realm of repetitive, monotonous, mindless, incredibly boring physical labor.
Then you get to the "extra fine" sandpapers (300 to 500 grit). You might have some paint on the rocket by now, and if you have made a painting mistake, this is probably the range of grit you need to gently remove the paint you don't want. Otherwise, I would not recommend using this grit on much more than the first coat of base color, as this grit will quite visibly scratch and damage any nice outer layer of finish paint.
Then you get to the "super fine" sandpapers (500 to 700 grit). Somewhere in here (certainly by now) you have switched over to wet-sanding, to help reduce the amount of waste that packs into your sandpaper and to help carry the waste material off the work surface you are sanding. You also have to wet-wipe (no, not those little perfumed tissues, but you do have to wipe the work surface frequently with a wet soft towel, then continue sanding with a well-wetted piece of super-fine-grit sandpaper.
Some of the guys here (who obviously don't have much to do in the way of household chores and errands) continue this nonsense down to 800, 1000, 1500 grit, and beyond. Myself, if I want an aerodynamically smooth surface for a competition model, I use car wax or Future floor polish and buff most of it back off again. For most model rockets, I don't care if I can see myself in the finish (or if anyone else can), even if it's going to be seen by many many other people (yeah, I know, that's a poor attitude).
There is a saying that "perfect" is the enemy of "good enough" and I agree. I figure that I can have 99% of the fun with 50% of the labor spent smoothing, painting, and finishing.
(Whispered: Sometimes I don't even sand or paint them at all!)