As far as CATO data, all we have as rocketeers is anecdotal info. The recipients of the MESS reports don't share.
There is a reason for this. The MESS reports only give us the numerator of the relevant fraction. The denominator - the actual number of motors of that type that were flown in the same time period - is not available to us. For this reason, the MESS reports cannot give a statistically valid answer to the question "which reload has the most CATO's?"
For example, if I were to tell you that motor A has had 1 CATO since 2013, and motor B had 50 CATO's in the same time period, then, unfortunately, most rocket hobbyists would assume that motor B has a reliability problem. If we subsequently found that only 45 A motors were sold and flown in that time period, and 10,000 B motors were used in the same time period, then some people would realize that the picture is actually quite different. Unfortunately, most amateur rocket hobbyists would still think that motor B had a problem, because 50 is greater than 1, right?
Only a trained statistician would realize that 1/50 is such a small sample size that valid statistical comparisons are impossible, and that the data in the previous paragraph actually tells you nothing about the relative reliability of motor A and motor B. When we add to this known weakness of small sample sizes the fact that so few rocket enthusiasts actually bother to fill out MESS reports after a motor failure, we can see that MESS report data is actually a poor way to make motor-to-motor comparisons.
We use the MESS report data mainly to 1. Determine if there are specific lot or "day stamp" numbers that are having more problems than others, and 2. Spot specific problem areas with particular motors that show the same mode of failure. This is why it is important to save the packaging for any motor, reload or single-use, to be able to accurately report the lot number, and describe the nature of the motor failure in as much detail as possible in the MESS report.
Please be aware that people are looking at every MESS report in great detail and that the information is very important to us. If everybody who experiences a motor failure were to fill out a MESS report, it would become a much more useful tool for all of us.
Alan Whitmore
Chair, Tripoli Motor Testing[/QUOTE
As far as CATO data, all we have as rocketeers is anecdotal info. The recipients of the MESS reports don't share.
There is a reason for this. The MESS reports only give us the numerator of the relevant fraction. The denominator - the actual number of motors of that type that were flown in the same time period - is not available to us. For this reason, the MESS reports cannot give a statistically valid answer to the question "which reload has the most CATO's?"
For example, if I were to tell you that motor A has had 1 CATO since 2013, and motor B had 50 CATO's in the same time period, then, unfortunately, most rocket hobbyists would assume that motor B has a reliability problem. If we subsequently found that only 45 A motors were sold and flown in that time period, and 10,000 B motors were used in the same time period, then some people would realize that the picture is actually quite different. Unfortunately, most amateur rocket hobbyists would still think that motor B had a problem, because 50 is greater than 1, right?
Only a trained statistician would realize that 1/50 is such a small sample size that valid statistical comparisons are impossible, and that the data in the previous paragraph actually tells you nothing about the relative reliability of motor A and motor B. When we add to this known weakness of small sample sizes the fact that so few rocket enthusiasts actually bother to fill out MESS reports after a motor failure, we can see that MESS report data is actually a poor way to make motor-to-motor comparisons.
We use the MESS report data mainly to 1. Determine if there are specific lot or "day stamp" numbers that are having more problems than others, and 2. Spot specific problem areas with particular motors that show the same mode of failure. This is why it is important to save the packaging for any motor, reload or single-use, to be able to accurately report the lot number, and describe the nature of the motor failure in as much detail as possible in the MESS report.
Please be aware that people are looking at every MESS report in great detail and that the information is very important to us. If everybody who experiences a motor failure were to fill out a MESS report, it would become a much more useful tool for all of us.
Alan Whitmore
Chair, Tripoli Motor Testing[/QUOTE]
I have submitted MESS reports.