Suspected Bad Motor List

Discussion in 'Propulsion' started by lakeroadster, Jan 22, 2019.

Help Support The Rocketry Forum by donating:

  1. Jul 8, 2019 #61

    lakeroadster

    lakeroadster

    lakeroadster

    Lone Wolf... No Club TRF Supporter

    Joined:
    Mar 3, 2018
    Messages:
    1,304
    Likes Received:
    286
    Gender:
    Male
    Location:
    Central Colorado
    Updated list 2019-07-08: see post #1
     
  2. Jul 9, 2019 #62

    cbrarick

    cbrarick

    cbrarick

    Wildman CT

    Joined:
    Jan 23, 2009
    Messages:
    2,252
    Likes Received:
    118
    I disagree with making MESS available. There are way too many problems with this.
    1. We don't know ABSOLUTELY what happened with the motor....if black powder, was it thermal cycled? If composite and single use, was it dropped?
    2. We don't know ABSOLUTELY how the motor was built (reloadables) I was at a launch, the motor cato's, the LCO tells the flyer to get another motor and reload from me (I am a vendor). I got to the rocket and the problem was obvious - it was a AT motor, where they put the spacer on the nozzle side of the forward closure. This was absolutely operator error. Now let's talk about ungreased O rings, missing phenolic discs, cross-threaded CTI 38's.
    3. We don't know ABSOLUTELY why the motor failed. I've been doing it for a long while, but I bet I haven't seen everything.....and sometimes there's not enough to do a post mortum.
    4. We don't ABSOLUTELY know there's a issue with a lot, date of manufacture or type. It could have just been a one-off problem, such as a bubble in the propellant, a bad liner, crappy O ring or just bad luck. Don't forget NASA lost a few space shuttles. Anecdotal reports are generally not statistically significant. We also haven't talked about what is significant...5% of a lot? If that were the case a lot of 1000 would need 51 failures, only attributable to the motor, to have some significance.
    5. If we get to blaming every one-off or any of the above failures on the motor, it costs someone money. Either dealers eat the motors or the manufacture would have to. Either way is unfair....
    6. No other American system is set up to report singular failures (or for that matter, few are set up for multiple failures). Go ahead and try to find manufacture data on how many microwaves of a certain model failed, or how many washing machines failed. I feel fairly confident in stating that everyone's vehicle will fail eventually, in some way. The battery in my F350 died after 5 years outside....expected, but a failure none the same. Anyone gonna stop buying some sort of vehicle? Anyone yelling to find out how many F350 batteries died in 5 years?


    ps JMHO but if you work that hard to make a beautiful model, then you accept the risk when you go to fly it. my Decim8te got wrecked hitting a tree limb, under full chute. Stuff happens. I admire the work of the space modelers in our group and a good friend is one. I don't have 10% his skill, and admire his work. He's done a lot of re-do's because the finish got scratched on a nominal flight....never complains......
     
  3. Jul 9, 2019 #63

    dhbarr

    dhbarr

    dhbarr

    Amateur Professional

    Joined:
    Jan 30, 2016
    Messages:
    5,375
    Likes Received:
    471
    I'm never going to be a fan of asking people to GIVE information but then keeping it private for the benefit of private associations or corporations, even ones I happen to like.

    We want crowdsourcing, we should be willing to share that information back. I'd be an even bigger fan of an annual report summarizing our inputs, but it's very easy for me to spend other people's time.

    My opinions are mine, etc.
     
    Constant Leon, heada and lakeroadster like this.
  4. Jul 9, 2019 #64

    eljefe

    eljefe

    eljefe

    Well-Known Member

    Joined:
    Jun 12, 2014
    Messages:
    81
    Likes Received:
    1
    I think it's worthwhile to observe trends. If a date code has very few failures, it could be attributed to an operator error like one of the reasons you suggest. If there are many reports for that code, it suggests a potential manufacturing defect or perhaps an issue with how the motors ages over time. Those are certainly worth knowing for a rocketeer to make an educated decision on the risk of buying or flying suspect motors.
     
  5. Jul 10, 2019 #65

    lakeroadster

    lakeroadster

    lakeroadster

    Lone Wolf... No Club TRF Supporter

    Joined:
    Mar 3, 2018
    Messages:
    1,304
    Likes Received:
    286
    Gender:
    Male
    Location:
    Central Colorado
    You could use this same "don't allow the free sharing of data" for pretty much everything in life.

    I'd rather have the data and make up my own mind.

    E9 and E12 motors are the perfect example of why users should share data. We ABSOLUTELY know they are more prone to CATO than the other variants of BP motors.
     
    Last edited: Jul 10, 2019
  6. Jul 10, 2019 #66

    cbrarick

    cbrarick

    cbrarick

    Wildman CT

    Joined:
    Jan 23, 2009
    Messages:
    2,252
    Likes Received:
    118
    Trend analysis is an important part of statistics. I'm glad we have qualified people who will do that for us. Unfortunately, it's not a beginner concept. Rumor and innuendo are "I blew up a x12 and my friend blew up a x12 so the whole batch must be bad." Public release of that sort of stuff could open up the S&T to a liable law-suit. I'm not sure what statistics they use,but I think it may be a good place for MANCOVA. I did my PhD in instrument development and a post doc in pharmacoeconomics and think it would be fun to do.......but I'm also sure I'm not the best one for the job, either.....I may (probably) be completely wrong with my analysis of the issue. Stats are fickle, a specific function for a specific problem, understand the problem wrong and it's garbage in equals garbage out...... (sigh)
     
    Titan II likes this.
  7. Jul 10, 2019 #67

    cbrarick

    cbrarick

    cbrarick

    Wildman CT

    Joined:
    Jan 23, 2009
    Messages:
    2,252
    Likes Received:
    118
    hmm... let's crowd source another decision...

    cars kill people, directly due to failures https://www.ranker.com/list/catastrophic-car-failures/amandatullos
    bicycles safe? https://www.rospa.com/rospaweb/docs...fety/cyclists/cycling-accidents-factsheet.pdf
    guess not
    guess we're to horses....
    http://mentalfloss.com/article/91277/australia-horses-kill-more-people-venomous-creatures-do
    walking?
    https://www.nsc.org/home-safety/safety-topics/distracted-walking

    guess we should all stay home. Isn't that one way to look at the data? Isn't everything else NOT SAFE because you risk DEATH?

    ps I absolutely fly my loc viper 4 on D12-7's, every time.....never been a fan of the E9, mostly because it burns so long and doesn't have a whole lot of thrust.
    It would also be interesting to have a lawyer speak on liable issues from accusing a motor to be bad...Not that they would, but could. I'm too old to mess with that gimble lock. I want to retire SOON.

    pss. now show me one other consumer product where crowd sourced data is absolutely shared at a per incident / per person level.......
     
    Titan II likes this.
  8. Jul 10, 2019 #68

    dhbarr

    dhbarr

    dhbarr

    Amateur Professional

    Joined:
    Jan 30, 2016
    Messages:
    5,375
    Likes Received:
    471
    This comparison is silly. Show me another consumer product that cooperatively self-certifies viz. TMT/S&T. Internationally, no less.
     
    lakeroadster likes this.
  9. Jul 10, 2019 #69

    Steve Shannon

    Steve Shannon

    Steve Shannon

    Well-Known Member

    Joined:
    Jul 23, 2011
    Messages:
    4,879
    Likes Received:
    1,165
    Location:
    Butte, Montana
    I think the comments about public access to MESS reporting data would be more useful in the thread about the beta site of the new MESS reporting system. I also think that it’s important to realize that the MESS system has suffered over the years because people didn’t contribute. The most common reason we heard was that it was like a black hole; information went in but users never saw any results.
    It’s not going to be perfect but without participation it cannot even rise to useful.
    Here’s a link to the thread about the beta site:
    New motorcato.org site preview
    https://www.rocketryforum.com/index.php?threads/New-motorcato.org-site-preview.153660/
     
    timbucktoo likes this.
  10. Jul 10, 2019 #70

    lakeroadster

    lakeroadster

    lakeroadster

    Lone Wolf... No Club TRF Supporter

    Joined:
    Mar 3, 2018
    Messages:
    1,304
    Likes Received:
    286
    Gender:
    Male
    Location:
    Central Colorado
    Your comments are condescending. As if the only people smart enough to sort through data are folks with PhD's and statisticians.

    What is being done here is a simple list of CATO's. If a motor blows up, it's added to the list. If you think your fellow rocketeers aren't qualified to evaluate why the motor blew up, don't pay attention to the list.

    I like to make my own decisions instead of having them dictated to me.
     
  11. Jul 10, 2019 #71

    cerving

    cerving

    cerving

    Owner, Eggtimer Rocketry TRF Supporter TRF Sponsor

    Joined:
    Feb 3, 2012
    Messages:
    3,081
    Likes Received:
    250
    I agree with cbrarick, unless you know how many motors were produced from each lot the reported failures are only anecdotal. Even then, it may not be statistically significant... if you have one cato on a 98mm N motor and the manufacturer only produced 100 of them, is that any different than 1,000 catos on a motor where 100,000 were produced? Generally, only the manufacturers have that data, and that fact that vendors have instituted recalls and remediations for some of their products (i.e. CTI 38mm forward closure issue, AT DMS L1000 case issue, etc.) based on user-submitted incidents shows that the current system is working.
     
    Titan II likes this.
  12. Jul 10, 2019 #72

    lakeroadster

    lakeroadster

    lakeroadster

    Lone Wolf... No Club TRF Supporter

    Joined:
    Mar 3, 2018
    Messages:
    1,304
    Likes Received:
    286
    Gender:
    Male
    Location:
    Central Colorado
    If the motor CATO's in your rocket... do you care how many motors were made? I don't, I just want to pick a motor that others have found to be reliable.

    If your friend tells you he got food poisoning yesterday at the local Denny's, do you go there to do a statistical analysis of the food and the number of customers served, or do you just say "I'm not going there to eat"?

    We are sharing data about LPR BP motors so users can evaluate it any way they see fit.
     
  13. Jul 10, 2019 #73

    heada

    heada

    heada

    Well-Known Member

    Joined:
    Jan 18, 2009
    Messages:
    2,523
    Likes Received:
    124
    Gender:
    Male
    Location:
    Indianapolis, Indiana
    I don't think anyone is asking for statistical analysis of the data or absolutes, only high level trending data. If a motor has been on the market for 15 years with no reported failures and suddenly there are 10 reported failures in the span of a few months, all in the same manner by different people and the commonality is that they all came from a certain batch, then why would you not support adding that batch to a "suspected bad motor" list?
     
  14. Jul 10, 2019 #74

    Charles_McG

    Charles_McG

    Charles_McG

    Ciderwright

    Joined:
    Sep 12, 2013
    Messages:
    1,881
    Likes Received:
    216
    Location:
    SE Wisconsin
    You don’t need to know the denominator to tell that the numerator is non-zero. Knowing the denominator helps in assessing relative risk.

    These reports are owned by the owners of motorcato.org, not the motor manufacturers. Yes, it’s really easy to misinterpret the data, or over react. But increasing reporting is probably worth it.

    I actually like mining rocketreviews for data because people put in good flights, too. So you can get a feel for relative rates. Adding fields for CATO (Boolean) and lot code (string) would be great features over there. And showing them in the motor summary page.

    I found my 2017 data from rocket reviews:
    E9 : 8/118
    E12 : 3/43
    C6 : 0/107
    The motor summary page at the time would only display about 100 flights, max.
     
    Last edited: Jul 10, 2019
  15. Jul 10, 2019 #75

    Steve Shannon

    Steve Shannon

    Steve Shannon

    Well-Known Member

    Joined:
    Jul 23, 2011
    Messages:
    4,879
    Likes Received:
    1,165
    Location:
    Butte, Montana
    This is exactly what happened with the CTI one grain Pro38s. We spotted a spike in anecdotes here on TRF and I asked CTI about it. They issued a remedy. I never knew the denominator and it’s possible the denominator wouldn’t have made a n immediate difference. In this case it was the rapid rise in reports.
     
  16. Jul 11, 2019 #76

    cbrarick

    cbrarick

    cbrarick

    Wildman CT

    Joined:
    Jan 23, 2009
    Messages:
    2,252
    Likes Received:
    118
    geez didn't mean to be condescending..I actually said I wasn't qualified to do the analysis. You can be a math savant and not understand the question, therefore not be qualified to perform the analysis. I"m really glad that we have the two organizations doing it for us. They are immersed in a way that they probably ask questions we haven't even considered. The problem is that counts of motor blowing up don't do much good.
    Even Charle's data is limited in it's use. His review found 6.78% of E9s and 6.98% of E12s Catoe'd...out of a convenience sample of I don't know how many.....
    Plus, there's a participation bias. I rarely post any flights and never post commercial CATOs. You? Isn't there a selection bias as to who posts: Those who enjoy that and those that had a bad experience...Others? I don't know. Plus there's another selection bias - you gotta know about the site to do your rocket review, and have a machine that allows you to do so. Apparently C6's never CATO on rocketreviews, but I've seen it happen. then you have to throw in the independent variable of lot/manufacture date. Perhaps (I really don't know) the E9's only blew in one batch, making that one higher and every other lot zero, absolutely safe..... I think, after a long time going to lots of launches, that I've seen close to one of everything CATO. It's the risks we take when we fly.

    I, for one, hope that before either cato site releases data to the general public, they consult their lawyers. i'd be curious what they say. They may not like the idea.

    Anyhow, peace out everyone. For those that don't like my opinion, please forgive me and forget it...cause I'm out of here....
     
    lakeroadster and Titan II like this.
  17. Jul 11, 2019 #77

    Steve Shannon

    Steve Shannon

    Steve Shannon

    Well-Known Member

    Joined:
    Jul 23, 2011
    Messages:
    4,879
    Likes Received:
    1,165
    Location:
    Butte, Montana
    NAR, CAR, TRA and all of the motor manufacturers met face to face in April of this year. What we’re doing was generally agreed upon by all. Yes, it’s valuable to understand what these data can (and can’t) tell us. If you review what every one of us have said I think you’ll detect a desire not to have the new system be used to bludgeon or even threaten the manufacturers. Not only that but in certain circumstances the manufacturers might even be willing to privately communicate production numbers to the certification organizations.
    I don’t think we need to ask a lawyer for permission to disseminate information back to the people who provide it.
     
    lakeroadster and dhbarr like this.
  18. Jul 11, 2019 #78

    Charles_McG

    Charles_McG

    Charles_McG

    Ciderwright

    Joined:
    Sep 12, 2013
    Messages:
    1,881
    Likes Received:
    216
    Location:
    SE Wisconsin
    Peacing out with whiskey in hand.

    Data collection, organization, and analysis is my bread and butter - I have a greater than average appreciation for it's limits, and my own. I also have a really narrow definition of 'fact', and don't believe in 'Truth' at all.

    For instance, my search found 0/107 C6 CATOs. 107 was the maximum number of records I could retrieve using the public facing motor summary on rocket reviews. All it says was that no CATOs were reported in the most recent 107 reports. That's it. No more. I recall that at the time, I double checked how far back in time each query went - how long it took to collect about 100 reports. That gives an idea of how popular motors are. But I didn't write it down. I did find a note here on TRF that I posted at about the same time that there were no CATOs reported for 29mm Estes motors - but that there were only 18 records.

    All data has one or more biases. It doesn't invalidate the data. But it does require thought and caution in interpreting it.

    I've thought about trying to hijack the Annual Newton-Seconds Burned thread to ask people to self-report CATOs - but I haven't thought of an elegant, simple, unobtrusive format that would blend in and be mine-able.

    I've thought about asking people to self report lot numbers for the motors discussed in this thread that -didn't- CATO. A suspected GOOD batch list, as it were. That way we would collectively build an idea of how many batches are out there, and thus how unusual the reported ones are.

    In the internet age, we the consumer, can act collectively to gather data about products. Look at amazon reviews - they facilitate it. I can understand that that scrutiny might be uncomfortable for the manufacturers. And that some consumers will jump to premature conclusions because they don't understand the limits of the information. That's a broader problem in modern society. But I doubt they can stop us. It's not like Fedex or UPS that put 'thou shall not use our tracking tools to monitor our performance' right in the terms of use.
     
    Steve Shannon and lakeroadster like this.
  19. Jul 22, 2019 #79

    gna

    gna

    gna

    Well-Known Member

    Joined:
    Sep 10, 2014
    Messages:
    250
    Likes Received:
    1
    E12-6 Cato. Lot D 19 06 19. Didn't get a shot of the CATO, unfortunately. IMG_0635.JPG IMG_0636.JPG IMG_0637.JPG
     
    lakeroadster likes this.
  20. Jul 23, 2019 #80

    JohnCoker

    JohnCoker

    JohnCoker

    Well-Known Member

    Joined:
    Apr 13, 2013
    Messages:
    1,529
    Likes Received:
    162
    MESS reports are not useless. There is far more data in the MESS database than anywhere else. We are in the process of making it more widely available.

    I posted this in another thread, but here's a demo site where you can see the current status:
    https://motorcato.herokuapp.com/

    Here's the search result for the first date code in your list:
    https://motorcato.herokuapp.com/search?serial_no=A+06+28+11

    Please do not discourage people from submitting failure reports.

    Please continue to submit failure reports to the official site:
    http://motorcato.org/
     
  21. Jul 23, 2019 #81

    Steve Shannon

    Steve Shannon

    Steve Shannon

    Well-Known Member

    Joined:
    Jul 23, 2011
    Messages:
    4,879
    Likes Received:
    1,165
    Location:
    Butte, Montana
    Thank you, John!
     
  22. Jul 23, 2019 #82

    Charles_McG

    Charles_McG

    Charles_McG

    Ciderwright

    Joined:
    Sep 12, 2013
    Messages:
    1,881
    Likes Received:
    216
    Location:
    SE Wisconsin
    Thanks John. I can see my own reports!

    I'd like to offer some feedback and make some requests.
    • The searches come up with date order scrambled. It would be nice to sort newest-oldest by default.
    • I'd like a motor diameter sort column or filter option. That might not be in the database (at least without a join) so it might not be an easy request, I'm guessing. I asked because I was looking at the Pro38 1G failures, and I recognized my G78BS - but the related motors (like G69SK) didn't pop to me. I had to look at Pro38.com to check the codes and see the commonality.
    • If there is only one failure of a motor designation, clicking the link takes you to that mess - full text description, categorical entries, etc. Otherwise, I don't see a way to get to a specific report. The motor designation links pull up all the failures for that motor. I'm thinking one or the other is a mistake. Try 'P38'. I also tried a date search to get to a single motor listing, but it still pulled up list of all reports for that motor.
    • Speaking of which, is there a handy way to validate the motor designation? Maybe against the Thrustcurve database? The P38 above is actually a J530. Some quick browsing and searching highlights the errors in data entry. SU v reload seems to be very common.
    • Search by location?
    • Search/order by propellant?
     
    Last edited: Jul 23, 2019
    Steve Shannon likes this.
  23. Jul 23, 2019 #83

    BDB

    BDB

    BDB

    Absent Minded Professor

    Joined:
    Aug 22, 2015
    Messages:
    1,847
    Likes Received:
    210
    Gender:
    Male
    This is great! Thanks, John.
     
    Last edited: Jul 23, 2019
  24. Jul 24, 2019 #84

    JohnCoker

    JohnCoker

    JohnCoker

    Well-Known Member

    Joined:
    Apr 13, 2013
    Messages:
    1,529
    Likes Received:
    162
    Thanks for the suggestions; I'll think more about them. Sorting by date should be easy. The database doesn't have info other than what is entered, but could be enriched through ThrustCurve.org.

    The first column in search results (the clipboard) will take you to that single report. Clicking on a manufacturer will take you to all reports for that manufacturer and clicking on a common name will take you to all reports for that motor. (As you noticed, if there is only one report, it takes you to the detailed data for that report.) This seems logical to me, but you're the second person to mention that the behavior isn't intuitive, so something needs to be done here.
     
  25. Jul 24, 2019 #85

    Steve Shannon

    Steve Shannon

    Steve Shannon

    Well-Known Member

    Joined:
    Jul 23, 2011
    Messages:
    4,879
    Likes Received:
    1,165
    Location:
    Butte, Montana
    Great input Charles. That’s exactly why John is making the beta version available. Thanks for taking the time.
     
  26. Jul 24, 2019 #86

    Charles_McG

    Charles_McG

    Charles_McG

    Ciderwright

    Joined:
    Sep 12, 2013
    Messages:
    1,881
    Likes Received:
    216
    Location:
    SE Wisconsin
    I didn't recognize the clipboard as an active element, not did I hover over it to see the text. Maybe if the column header was 'Report', rather than the symbol? And maybe last instead of first?
     
  27. Jul 26, 2019 #87

    tomsteve

    tomsteve

    tomsteve

    Well-Known Member

    Joined:
    Feb 24, 2014
    Messages:
    730
    Likes Received:
    85
    i had an E12-6 date code D 19 06 19 blow its innards last saturday-7/20.
    glad it was in an art applewhite flying saucer so no damage
     
  28. Aug 8, 2019 #88

    djmartins

    djmartins

    djmartins

    Active Member

    Joined:
    Jun 12, 2019
    Messages:
    26
    Likes Received:
    4
    Gender:
    Male
    Ok, help me out here.
    Isn't the problem due to the black powder propellant cracking and this is far more likely
    on the larger sizes?
    Is this a problem with composite propellant?
     
  29. Aug 8, 2019 #89

    Charles_McG

    Charles_McG

    Charles_McG

    Ciderwright

    Joined:
    Sep 12, 2013
    Messages:
    1,881
    Likes Received:
    216
    Location:
    SE Wisconsin
    The common CATOs I've seen in the BP 24mm motors are spitting the nozzle aft, and/or spitting the whole fuel grain forward. So nozzle problems and grain to casing adhesion problems.

    Composite motors have their own issues. Nothing is perfect.
     
  30. Aug 8, 2019 #90

    djmartins

    djmartins

    djmartins

    Active Member

    Joined:
    Jun 12, 2019
    Messages:
    26
    Likes Received:
    4
    Gender:
    Male
    Thanks for clarifying as I had read cracked cores were a problem.

    As the energy involved increases I would expect there to be more problems cropping up.
    Isn't "rocket science" always about trying not to explode with failures far from unknown?
     

Share This Page