When looking at these fins in person (which can be held in the palm of my hand), compared to the size of the rocket (which is taller than two people), they do not look big enough for a rocket of that size.

I've been looking into the following theories:

(1). For a skinny rocket, the CP moves upwards much faster than usual as the angle-of-attack increases, so the rule of 1.5-2.5 stability is not enough. The stability number of 3.25 is the stability at the OTRS at 0 degrees AOA. Due to CP shift, we should not rely on this number. However, OpenRocket can account for the CP shift in simulations if you set the correct wind speed and launch rail parameters; then you need to graph stability vs time and look at it off the rail, which is what we did for the PDR. If we assume wind speeds of 4.45 mph, the stability decreases to ~2 off the rail. If we assume 20 mph, the stability decreases to ~0.9. This theory explains only some of the issue. I'm unsure what stability number we should aim for here, but we know how to calculate it with the component analysis in OpenRocket.

(2). Dynamic instability: Fins that are this small cannot produce enough of a lift force to actually correct the rocket in time. The CP might be in the right spot, but the magnitude of its force is not actually high enough compared to the moment of inertia of the rocket. Most of what I've read suggests that dynamic stability issues are not much of a concern for long rockets.

(3). The CG we're estimating is just incorrect because of unknown mistakes in OpenRocket. Our masses for propulsion components were definitely off, such as missing the mass of the oxidizer and fuel grain. But once these were added, the issue actually got worse. Our CG is approximately ~80 inches from the nosecone. It changed to ~72 inches when I audited the sources for the propulsion masses, but I'm still in the process of auditing these masses. I was told it was typical for a Rocket's CG to be approximately in the center.

(4). For a long rocket, the turbulent boundary layer/wake will grow to a significant size, resulting in little or none of the fin span sticking into the laminar region where they can produce enough lift, and OpenRocket doesn't account for this. I have attempted running an ANSYS Fluent CFD simulation to investigate this theory, however, it is giving me a CP that is even further back then what OpenRocket tells us for an angle-of-attack of 11 degrees. This 11 degree aoa is the effective aoa for a wind speed of 20 mph and and otrs of 100 ft/s; the goal with using this number is to check for wind induced instability. The CFD does have some issues with mesh quality so it is still a work in progress.

(5). The fins aren't actually too small, our intuition is wrong and OpenRocket is right. We are trying to find a COTS rocket that matches our rocket dimensions but have so far been unsuccessful. Another thing we are considering is 3d printing a small scale version of our rocket with a similar reynold's number off the rail and a similar mass distribution.

I'm looking for advice on the best way to approach this problem; are the fins really too small?