Please contact your local Tripoli Rocketry prefecture (aka club) and see if they can set you up with a mentor. I see a couple of things that you are doing that aren't very good ideas. I don't do much with sugar motors, so there are probably more. You definitely want to re-think PVC pipe. If you get your nozzle too small, it will turn into shrapnel that can't be seen on X-rays. 2" motors are also getting pretty big, into the area where the FAA probably would take an interest in what you're doing, and not in a good way.
Sounds like I need to downsize to keep the FAA off me. I've given some thought about whether or not I've built a pipe bomb or a rocket. So far we've been ducking behind whoever showed up with the oldest car with plywood covering the area under the car. (safety first) I know, sounds kinda hillbilly-ish but I don't want anyone to get hurt on my account.
Sounds like you have some stability issues. Normally, you want the center of pressure (aka center of drag) to be at least one body tube diameter (aka one caliber) towards the nose from the center of gravity. You'll need a little more if you go above Mach. OpenRocket is a free download and it can help you out here.
I've got it but haven't really played with it much. I will now.
The first flight may have taken the turn because you didn't have enough thrust so it wasn't going fast enough once it left the launch rail/rod. That's another thing that OpenRocket can help you with, though you will need to have a good sense of what kind of thrust your motor is putting out.