Wilson F/X failure at at Mini-Mid West Power. What happened?

The Rocketry Forum

Help Support The Rocketry Forum:

This site may earn a commission from merchant affiliate links, including eBay, Amazon, and others.

RocketRev

Well-Known Member
TRF Supporter
Joined
Jul 25, 2012
Messages
480
Reaction score
360
Hello all,

Those of you in attendance at TQC/QCRS’s Mini-Mid-West-Power launch back in May will probably recall that something went wrong with our Wilson F/X launch system. We were in the middle of a hot afternoon of rocket flying when the launch system suddenly stopped operating with no warning. AARRGGHH!!!!!

I did a quick check on the field and found two immediate possible problems. But when I eliminated both of them, the system continued to not operate. So we did the next best thing and removed all the wireless units and went back to hard-wired and finished the rest of the launch hardwired.

Mini-Mid West Power is our last launch till Mid-West-Power so we had some time to figure out what went wrong. And we did. Dan Fox, my cousin and the other half of Wilson F/X, came up last Friday. He and I went and picked up the whole launch system. I know we weren’t using the whole system at Mini-MWP, but it seemed to make sense to check out the whole system as long as we were at it. The system has been operating since 2003, so we like to keep on top of it.

As I said, there were two possible problems that I was thinking it might be: a new wireless communications plug and the new FET PBU-8w that the new plug was on. You all might as well know that I’ve been using TQC/QCRS as the beta testers for every new item that Wilson F/X has come up with since the very beginning of Wilson F/X. I became the founding Prefect of TQC way back in 1995 coinciding with the creation of the very first Wilson F/X launch system. Anybody remember that four pad system besides Dan and me? Well TQC has tested every system and every component since those earliest days of this club.

As I said there were two problems that I thought might be “it.” The first was a brand new PBU-8w that Dan and I had only finished putting together the night before Mini-MWP. We’re looking at replacing all the relays in our pad-boxes with Field Effect Transistors or FETs. This PBU-8w (FET) is the prototype and we were testing it out at Mini-MWP. As it turns out, it was operating flawlessly, but we didn’t know it when the system crashed.

The second problem was the communications line coming out of the PBU-8w itself. Again as many of you know, TQC owns eight of the original PBU-8 pad boxes with the original double stack circuit board design. There’s no room inside the NEMA 8x8x4 enclosure to fit the wireless unit so the external wireless unit’s female 16/3 plug simply plugs into the male 16/3 communications plug on the PBU-8. But the new design for the PBU-8 is a single circuit board layout which does allow plenty of room in the PBU-8 enclosure to add the wireless unit inside the same enclosure. But this would leave a male 16/3 plug hanging out just waiting to short 12V across the 16/3 plug and pop the fuse or short communications. So I replaced the 16/3 male communications plug with a female 16/3 plug to eliminate the 12 volts shorting problem. This prototype PBU-8w with FETs, worked great except for the fact that somebody grabbed the 16/3 female communications line and pulled hard enough to pull the plug end off of the wires themselves. I’m replacing all of these defective plugs.

When I saw that the plug had been pulled away from the wiring, I figured that the wires had shorted out on the inside of the plug causing the crossed polarity of the communications line and ground which would have shut down the whole system. Later when I inspected this plug it became apparent that while the wires were pulled out some from the plug, there was no grounding in the communications plug. So the mystery remained. What caused the system to shut down? Turns out it had nothing at all to do with the new PBU-8w with FETs that we were testing.

Anybody remember LDRS in 2016, put on by Rocketry of California on Lucerne Dry Lake? Dan and I drove out for that LDRS. We were expecting to have a great time as they showed off their brand new 72 pad Wilson F/X all wireless launch system with their LCU-64x Custom controller with all the bells and whistles.

If you were there, you will probably remember the extreme heat and that their launch system was having problems with intermittent operations. Dan figured out what the problem was by heating up a wireless unit with a heat gun in order to determine what component was failing. Turns out it was a redundant diode (redundant from an earlier design and no longer needed) that was failing in the extreme heat and shutting down the system. Without the extreme heat the problem would never show itself.

That was the first system wide Wilson F/X recall. I contacted every club (and all the individuals) that had Wilson F/X wireless systems and made arrangements to remove that unnecessary diode and it fixed the problem completely. End of problem right? Well of course not. I also had to go thru all my then current stock of wireless units and wireless unit circuit boards in order to remove that diode. And I did that too. So what’s the problem?

The problem is simply that I did not think to send that recall notice to my own club. For years now, my own club has been flying from the first of November when the crops come out of the field, thru till the end of May when we can no longer walk thru the fields. And the last few Mini-MWP launches in May were never above about 85 degrees. But as you recall this year’s Mini-MWP got hot.

With the heat at Mini-MWP, the wireless unit on the controller got hot enough that it shut down the whole wireless system. I know, because when I got home and started opening all the wireless units, the LCU-64’s wireless unit had the redundant diode still in place as did three other of TQC’s wireless units. Once removed, the system went back to operating flawlessly. How embarrassing….. I forgot to do the recall check on my own club’s wireless units. Oh well, live and learn as they say.

So that’s what happened at Mini-MWP. The problem was solved two years ago, but it just didn’t get applied to TQC’s launch system till July 20, 2018. And that’s my fault. Sorry about that. I figured everybody ought to know what actually happened and h ow it got fixed.

Brad Wilson of Wilson F/X Digital Control Systems
 
Must've been a crappy issue to trouble shoot... system shuts down... open the box, system cools down, starts working, close box, system stopped working. I've had this issue where it turns out a component had a tiny piece of metal touching the front panel, grounding itself out. When you took the circuit board out of the panel, it worked fine of course.


Strange question- but does the diode's datasheet show any operating temperature information? Granted, I'd personally never check, as the stuff I build (music synthesizers) usually are operated at room temperature. Also, I'd just assume the diode would function the way it was designed to.


Interesting write up- I learned something :)
 
I'm going from memory, but I think the data sheet said that some reverse leakage could occur above 145 degrees F. Unfortunately we didn't think about the fact that these components were sitting in a closed enclosure sitting in the hot sun which is where we first realized we had a problem at the ROC LDRS on the Lucerne Dry Lake bed.

The Wilson F/X communications protocols are such that if the communications line gets it's polarity backwards it shuts the system down. No damage is done, but the system will not operate with reverse polarity.

And you are right, it was very tricky to diagnose. ROC had sent us back a wireless unit a few months before LDRS, that they said wasn't working. I tested it in my shop back here at home in Illinois and everything was working fine. I even sent it to Dan in Indiana to test in his shop and again they worked perfectly. But then again both of our "shops" are in our A/C basements. So with the problem turning out to be a heat related issue, its no wonder we didn't figure it out before the launch. But we drove out to LDRS with about a dozen extra wireless units to sell at LDRS if we got the chance. So we had plenty of extras with us, and that helped a lot.

We did buy an infrared thermometer and a heat gun at a local Menards close by the hotel at that LDRS and that's how we figured out for sure that it had to be a heat related component failure. But we had to get the units back home on Dan's test bench to determine which component was failing.

In testing we did find out that some of the diodes were failing as low as 90 degrees F, and a very few were failing at higher than the rated temp. But as it turned out to be a redundant component we just removed that diode and that fixed the problem.

Oh yeah, remember the wireless unit they shipped back to us before LRDS because it wasn't working? We marked it before we sent it back to them. It turned out to be one of the units that failed at 90 degrees F.

Some problem only appear in actual use which is why I've been Beta testing every new component in my home prefecture Tripoli Quad Cities. But Illinois in the Spring, Fall, and Winter don't get hot enough to have produced this problem. But hey, we all have things to learn. And here at Wilson F/X we want nothing but satisfied customers. If its not working we're not happy!

Live and Learn as they say.

Brad
 
But Illinois in the Spring, Fall, and Winter don't get hot enough to have produced this problem.

On the other hand- you'll find conditions that are cold based really quickly! :)

Question- what was the manufacturer of the questionable diodes?
 
Hi Brad, that is a good piece of troubleshooting. The question is what to do about it going forward? A diode with a better thermal rating?

I do not own a system myself, but do fly with clubs that do.
 
Hello All,

OK, I have to do an update here as I stated a couple of things badly if not plain wrong.

1) I stated from memory that the diodes were rated up to 145 degrees F. My memory was bad. The diodes were only good to go till 125 degrees.... CENTIGRADE! I can't believe I forgot the Centigrade part. Oh well, I chalk it up to old-timers syndrome. And anybody who wants to argue with me will have to remind me... what were we talking about again? SQUIRRELL! (points out the window....)

2) When I stated that it was a "bad diode" I was speaking colloquially. Like, "Did you see the sun-set last night?" The sun neither sets nor rises, but we still use the language to describe what we see in terms of a local phenomenon. Same thing goes for the "bad diode." There never actually anything wrong with the diodes themselves. They were working exactly as the manufacturer stated that they should. The problem was an erroneous placement, but that needs some explaining. WFX didn't sell its first pair of wireless units until the year 2013, five years ago. But we had the first units built and being beta tested by TQC for a couple years previous to that. We hadn't even looked at the schematic for probably 5 years when the problem with ROC's system arrived at LDRS 35 back in 2016 as we were still working with the first set of 25 circuit boards. It was simply easier to say we had to remove a "bad diode" than to talk about removing a redundant high impedance trace that would allow for a .035 milliamp (35 microamps) reverse polarity voltage bleed that would shut down the return communications protocols on the pad-box's wireless unit. So I went with "bad diode" to simplify communications.

3) So what happened? Between the first prototype boards that we built with extra safety diodes because they were the first, we forgot to remove one of those "safety" diodes that was on a high impedance trace that basically shut down the communications line so that the pad-could not talk back to the controller when the temp inside the enclosure got over about 160 f degrees or so because of a 35 micro-amps bleed thru. The diode was working just fine, its just that it allowed those 35 micro amps of juice to run the wrong way at even lower temperatures. Remember 1000 micro amps is the same as 1 milliamp. That was still enough to shut down communications back to the controller from that pad box. That whole trace is gone now as it was not needed in production units. That was our mistake. My mistake was in missing doing the "recall" on my own club's wireless units. In the heat of Saturday of the Memorial Day week-end's Mini-Mid West Power launch

4) WHAT NOW? Well actually its all done. There's nothing more to do. There's no need to change diodes as we do no longer use that high impedance trace on production units. And if it hadn't been about four or five years between when we designed our first wireless units until they started actually selling, we would have noticed the prototype trace was still there with its vulnerable redundant diode and removed it.

Live and learn as they say...….

Oooooohhhh, I just realized that I gave a way a prototyping tip: build in safeties on prototypes to keep the unit from frying while you work out the bugs. There's a freebee!

And as to finding those cold temperatures here in the Mid West.... "Winter Is Coming!"

Brad, the "Stark" "Rocket Rev.," Wilson

ps: I'm sorry.... Game of What?
 
This is a model of how any manufacturer should handle these types of events.

I'm not a customer (of any launch system), but were I in the market, this type of attitude of disclosing issues and resolutions would go a LONG way towards me choosing Wilson F/X.
 
Back
Top