The cold temperature characteristics of an alkaline cell will not show up unless it's under load. Open voltage (or the light load of the altimeter electronics only) will not show the voltage when firing a charge. So, the "beeped out" voltage of the Perfectflite will not show the whole story. Below 40F (cold soaked into the battery, not just ambient) is when the voltage vs temperature tanks for alkalines. Non-rechargeable Lithium-ion cells do much better down to -20F. They also have more capacity in the same 9V size. But all are not created equal, so look at the manufacturer's specs and verify by testing.
Considering the overall cost of the rocket, electronics, and motor, there's no reason to skimp on the battery, or make any assumptions. Duracell alkaline are great for most circumstances (that's what I use). For projects that are going to be outside <50F for a while, I use the 9V Lithiums... and never the Lipo rechargeables in the cold.
Here's the link to Duracell Coppertop 9V data sheet https://d2ei442zrkqy2u.cloudfront.net/wp-content/uploads/2016/03/MN1604_US_CT1.pdf
The operation temp. range of -4 F to 130 F.
Again, I use these for altimeters which are low load and ematches which are also low draw. Even cold soaked, I've never had an issue in the last 10 years.