UhClem
Well-Known Member
- Joined
- Feb 6, 2009
- Messages
- 2,052
- Reaction score
- 443
While at NARAM 46 I made a point of attending the B division R&D oral reports because one title caught my attention:
"All-Fire/No-Fire Tests Estes Igniters using the Neyer D-Optimal Test" by Cody Steele
At the time I was fairly certain that I was likely the only person in the
audience who had even heard of this test let alone read the report:
https://neyersoftware.com/Papers/D-Optimal/D-Optimal.htm
(Note: Cody's report does not reference this report but another one which is
only a summary of different methods and does not detail the methods.
https://neyersoftware.com/Papers/EandP99/ISO14304.htm )
It had been a while since I read this document (from Mr. Neyer's web site) so I was a little fuzzy on the details. But after listening to Cody's oral
presentation I was skeptical that he had followed the Neyer D-Optimal method. Now that Cody's written report is available through NARTS I have confirmed that not only did Cody not use the D-Optimal method but the method he did use is deeply flawed and the results highly suspect at best but more probably totally worthless.
Some background.
The D-Optimal test is a variation on the Bruceton sensitivity test. The
variation being in how the exposure levels are determined. This type of test is used on igniters because exposure to currents lower than that required to fire them may alter their characteristics. Thus after one exposure the test article is thrown away regardless of if it fired or not.
The Bruceton test requires that you provide an estimate of the average and
standard deviation of firing current. The first test is conducted at the average
firing current. If the igniter fires, the current is reduced by one standard
deviation and the test repeated. If the igniter doesn't fire then the current is
increased by one standard deviation and the test repeated. This continues until an adequate sample has been gathered (forty or more).
The result of this test is a set of data points which consist of exposure levels
(currents) and a binary pass/fail. Obviously you cannot compute the sample
average and standard deviation from these by the normal methods. The method most commonly used is called Maximum Likelihood Estimate and the math is not simple.
The D-Optimal method uses a different algorithm (see the report) to determine the exposure levels. If you have estimated the average and standard deviation badly the Bruceton test does not work well. The D-Optimal test is insensitive to this sort of problem.
Did Cody follow this method? No.
The report states that an igniter was exposed to a given current and this
current was increased until the igniter fired. A procedure completely at odds
with any sensitivity test.
How were the computations done? I asked and was told that Microsoft Excel was used which cannot perform the required math.
Let's take a look at one of the data sets. This is the first set which had the power supply set to six volts (Yet another problem with the report. Not understanding how current limited power supplies and Ohms law work.)
1.21 pass
1.15 pass
1.13 pass
1.02 pass
1.01 pass
0.97 pass
0.94 fail
0.94 fail
0.94 fail
0.89 fail
0.89 fail
0.89 fail
0.80 fail
0.90 average
0.05 standard deviation
If the Neyer method had been used, the test results would have shown the passes and failures mixed up rather than grouped as we see here. But the data could have simply been sorted for presentation. This data also shows another problem. The Neyer test sequence is chosen to create data where the successes and failures overlap. In other words there should be a success (pass) at a level lower than at least one of the failures. This data set does not have this property so the Neyer method could not have been used. I duplicated the average and standard deviation from the report by computing the average and standard deviation of just the failures. An odd result.
Because the NARAM 46 R&D judges were not familiar with the Neyer D-Optimal method not only was Cody awarded first place in B division but he also received a large cash prize.
For work that is basically junk. Worse, the report shows that the author had at least read about the D-Optimal method and then knowingly presented test results that did not even come close to following that method while claiming that they did.
"All-Fire/No-Fire Tests Estes Igniters using the Neyer D-Optimal Test" by Cody Steele
At the time I was fairly certain that I was likely the only person in the
audience who had even heard of this test let alone read the report:
https://neyersoftware.com/Papers/D-Optimal/D-Optimal.htm
(Note: Cody's report does not reference this report but another one which is
only a summary of different methods and does not detail the methods.
https://neyersoftware.com/Papers/EandP99/ISO14304.htm )
It had been a while since I read this document (from Mr. Neyer's web site) so I was a little fuzzy on the details. But after listening to Cody's oral
presentation I was skeptical that he had followed the Neyer D-Optimal method. Now that Cody's written report is available through NARTS I have confirmed that not only did Cody not use the D-Optimal method but the method he did use is deeply flawed and the results highly suspect at best but more probably totally worthless.
Some background.
The D-Optimal test is a variation on the Bruceton sensitivity test. The
variation being in how the exposure levels are determined. This type of test is used on igniters because exposure to currents lower than that required to fire them may alter their characteristics. Thus after one exposure the test article is thrown away regardless of if it fired or not.
The Bruceton test requires that you provide an estimate of the average and
standard deviation of firing current. The first test is conducted at the average
firing current. If the igniter fires, the current is reduced by one standard
deviation and the test repeated. If the igniter doesn't fire then the current is
increased by one standard deviation and the test repeated. This continues until an adequate sample has been gathered (forty or more).
The result of this test is a set of data points which consist of exposure levels
(currents) and a binary pass/fail. Obviously you cannot compute the sample
average and standard deviation from these by the normal methods. The method most commonly used is called Maximum Likelihood Estimate and the math is not simple.
The D-Optimal method uses a different algorithm (see the report) to determine the exposure levels. If you have estimated the average and standard deviation badly the Bruceton test does not work well. The D-Optimal test is insensitive to this sort of problem.
Did Cody follow this method? No.
The report states that an igniter was exposed to a given current and this
current was increased until the igniter fired. A procedure completely at odds
with any sensitivity test.
How were the computations done? I asked and was told that Microsoft Excel was used which cannot perform the required math.
Let's take a look at one of the data sets. This is the first set which had the power supply set to six volts (Yet another problem with the report. Not understanding how current limited power supplies and Ohms law work.)
1.21 pass
1.15 pass
1.13 pass
1.02 pass
1.01 pass
0.97 pass
0.94 fail
0.94 fail
0.94 fail
0.89 fail
0.89 fail
0.89 fail
0.80 fail
0.90 average
0.05 standard deviation
If the Neyer method had been used, the test results would have shown the passes and failures mixed up rather than grouped as we see here. But the data could have simply been sorted for presentation. This data also shows another problem. The Neyer test sequence is chosen to create data where the successes and failures overlap. In other words there should be a success (pass) at a level lower than at least one of the failures. This data set does not have this property so the Neyer method could not have been used. I duplicated the average and standard deviation from the report by computing the average and standard deviation of just the failures. An odd result.
Because the NARAM 46 R&D judges were not familiar with the Neyer D-Optimal method not only was Cody awarded first place in B division but he also received a large cash prize.
For work that is basically junk. Worse, the report shows that the author had at least read about the D-Optimal method and then knowingly presented test results that did not even come close to following that method while claiming that they did.