You definitely do not want a high pass filter. This would simply give you all of the noise and none of the true signal.
Depending on your goal, it is useful to divide filters into two categories: causal, and non-causal. I suspect you want non-causal to compute drag coefficient after the fact.
A causal filter is based only upon past and present time and is useful for something happening in real time. If you have a need to know at this instant what is the best estimate of acceleration, you have only the data up to the present instant to filter. Both the Low pass filter and Kalman filter are examples of causal filters. If you want to do things with data in real time, such as decide to deploy parachutes at apogee, you must have a causal filter because you don't know the future.
There are many ways to implement a low-pass filter depending on the properties that you want: what frequency you want to roll off at and how aggressive you want to roll off.
The Kalman filter is an interesting creature. Essentially it starts with a dynamic model of what you are interested in. In this case it would be F = mA applied to a rocket. Maybe with some kind of linearized atmospheric drag model. The Kalman filter assumes that there are two noises: in this case there is a random force noise applied to the rocket, which I suppose is thrust noise and air turbulence noise. There is also a random measurement noise in the acceleration sensor. If you know these two noises and the dynamic equations governing the rocket motion you can come up with a filter that determines the absolute best guess for acceleration at that moment.
A low-pass causal filter will always produce a result with a phase lag, also known as time delay, compared to the original signal.
A non-causal filter is based upon past, present, and future time. It is useful when you recorded something and then want to filter it afterwards. I think this is what you want. An example is an FIR filter, finite impulse response. For example: at an instance in time you may take 0.25 * previous sample value, 0.5 * present sample, 0.25 * future sample. This is a nice smoothing filter. The cool thing is that there can be no time lag. The filter can produce a filtered version with no phase delay so you can have a good estimate of filtered acceleration at that instant, even with a very aggressive filter.
There are many different FIR filter implementations based upon what frequency you want to roll off at and how aggressively you want to roll off.
Another common non-causal filter is called a filt-filt algorithm. You filter the data with a regular causal like a low-pass filter. Then you flip the data end-for-end and filter it again backwards.
Things like least squared fitting and many different types of nonlinear fits based on the full time record would fall into the category of non-causal filters.