Handeman is absolutely correct.
Just using my own experience as an example, my L1 rocket was topped with a long payload section, which was joined to the main airframe with a long(ish) coupler (which will be the e-bay when I finally get an altimeter). This left around 8"-10" of airframe immediately forward of the motor in which to house the shock cord and 36" parachute. So the recovery system was housed aft of the rocket's midpoint, in the space just forward of the motor. (Granted, my airframe was much narrower -- 2.25" as opposed to your 4".) The flight went off without a hitch, nice and straight with perfect deployment and recovery.
I used a Nomex protector and nothing else to shield the chute. I typically use Nomex in my mid-power rockets as well. If you have a decent length of Kevlar for the shock cord then you will normally insert that first so that it will sit in the rocket below the parachute and will help to take up some of the volume.
Handeman is right -- unless you are using a large, heavy-duty BFR-type parachute, the parachute in an H or I powered rocket won't have enough mass relative to the rest of the rocket to have any critical effect on the CG if it shifts rearward during acceleration. So this shouldn't be of any concern if your rocket has a normal margin of stability.
You can, of course, check this prior to your anticipated launch date by fully loading the rocket as it would be when it is on the pad, with the recovery device all the way back and sitting on top of the motor mount. Stuff a ziplock bag containing an equivalent amount of weight as the motor you plan to use into the mount to take the place of the motor, and then check the location of the balance point. If it is indeed too far back then you can trim the balance by adding a small amount of weight to the forward end of the rocket. Once you have it correctly trimmed in this worst case scenario you will no longer have to worry about this issue.