Excel Data Streamer with Eggtimer Telemetry Unit and Eggfinder RX USB Dongle

[Kane]

I recently picked up an Eggtimer Telemetry Unit and am interested in using the RX USB Dongle with my laptop for a ground station in lieu of using the Eggfinder LCD Receiver. As I understand it, this will work but you have to create your own GUI that can parse the ASCII data stream. I have heard through the grapevine that someone may be in the process of developing a GUI for this application, but because I'm impatient and perhaps looking for a challenge (and/or a headache), I figured I might try and give it a whirl myself.
Interestingly enough as I was researching how to write applications using Python or a platform like Processing, I found that Excel actually has a cool little plugin called Data Streamer. It is literally set up to read sensor data via a micro-controller. I've tested it using some downloaded flight data and it works pretty well.
The one thing I'm not clear on is whether it will be able to read the data that comes from the Eggtimer products. The Excel Data Streamer is set up with Arduino type sensors and microcontrollers in mind and reads CSV files. I've looked over the sample telemetry data format guide that Cris has on the Eggtimer website. It's described as "human readable ASCII" but the formatting is not a type that I'm accustomed to (I'm used to comma or tab separated formatting typical in CSV files which, interestingly enough, is the format that you get when you download flight data from an ET Quantum).
I'm decidedly NOT a programmer. While learning a coding language and attempting to build my GUI might be a noble pursuit, I am WAY better versed in Excel and would love to use this if it works.
Short of purchasing the RX USB Dongle, assembling it, and just seeing if it works (which may just be the way this has to go), I'm curious if any here that are more knowledgeable about these things than me might have some thoughts or insights.

 

[waltr]

Which Eggtimer product is sending the Data Stream?

This document:
http://eggtimerrocketry.com/wp-content/uploads/2021/05/Eggtimer-Telemetry-Data-Format.pdfShows the format of several Altimeters.

The Eggfinder GPS sends a NMEA formated data stream, that is comma seperated.
https://www.sparkfun.com/datasheets/GPS/NMEA Reference Manual-Rev2.1-Dec07.pdf" in lieu of using the Eggfinder LCD Receiver "

The NMEA data is easy to bring into Excel but not sure of Cris' Telemetry format.

"Terminator byte –always an ASCII “>” character "
I would try setting the '>' character as the delimiter.

Another way to to simply capture the data stream into a text file. Then read this data file into an editor or other program to parse.

What is it you want to see in the GUI???
Or would simple Text of status work?

 

[Kane]

"Which Eggtimer product is sending the Data Stream?"
I generally use Quantums.

"This document"
I have looked over the document you reference and noted it in my original post. I am just not familiar with files using the ">" as the delimiter. I don't think that Excel's Data Streamer understands that as a delimiter not do I know how or if this could be changed.

"The Eggfinder GPS sends a NMEA formated data stream, that is comma seperated."
The Eggfinder GPS products use a different format than the altimeters as I understand it. This is intentional and why you have to use one LCD receiver for GPS and a second for telemetry.

"Another way to to simply capture the data stream into a text file. Then read this data file into an editor or other program to parse.

What is it you want to see in the GUI???
Or would simple Text of status work?"

I don't technically need anything fancy. I set up the Excel worksheet that accepts the data stream to show "current" and "historical" data streams and wrote a macro to capture the greatest value in the altitude, velocity, drogue and main columns so I get those events and they stay visible. Of course, with Excel, I'm also able to set up charts to graphically display the data too which is nice, but not necessary. Outside of Excel, I wouldn't even know where to start in terms of capturing the data stream and parsing it. That will take some research.

 

[waltr]

Ok, Quantum data stream.
I know when importing a data file into Excel you can select or set a custom delimiter. Maybe this can be done with the extension.

The USB dongle (serial to USB adapter) should show up as a 'COM port'. Then in your code you need to open the com port, check for data available then read the data. There is a Python library to do serial ports. Other languages should also have a way to open and read serial data.

There is nothing magically about Eggtimer's USB dongle. It is a Serial to USB adapter. did you get the Data cable from Eggtimer when you bought the Quantum? If so then you can use it for testing.
First thing to do is connect it to Quantum and the computer. Open a 'terminal' program as Cris suggests in the Quantum manual, Appendix B & E.
You do need to simulate a launch to get it to output data. Make a simple vacuum test chamber for this.

A real time data stream is good for knowing Status, Drogue, Main deployment, etc, when the rocket is out of sight (very high). Otherwise not very helpful.
The Quantum saves the flight details and is downloaded after the flight for analysis. This is what I do with Quantum data. It is a csv file I import into Excel then graph the data.

 

[Kane]

Ok, Quantum data stream.
I know when importing a data file into Excel you can select or set a custom delimiter. Maybe this can be done with the extension.

The USB dongle (serial to USB adapter) should show up as a 'COM port'. Then in your code you need to open the com port, check for data available then read the data. There is a Python library to do serial ports. Other languages should also have a way to open and read serial data.

There is nothing magically about Eggtimer's USB dongle. It is a Serial to USB adapter. did you get the Data cable from Eggtimer when you bought the Quantum? If so then you can use it for testing.
First thing to do is connect it to Quantum and the computer. Open a 'terminal' program as Cris suggests in the Quantum manual, Appendix B & E.
You do need to simulate a launch to get it to output data. Make a simple vacuum test chamber for this.

A real time data stream is good for knowing Status, Drogue, Main deployment, etc, when the rocket is out of sight (very high). Otherwise not very helpful.
The Quantum saves the flight details and is downloaded after the flight for analysis. This is what I do with Quantum data. It is a csv file I import into Excel then graph the data.

I do have the data cable and use PuTty for programming an Eggtimer Classic so I should just be able to use that. Or, maybe Excel Data Streamer will be able to connect to the Quantum directly as it would a micro controller to sensor. I may give that a shot. Great idea, thanks.
As far as the "need" for telemetry I am planning for future flights upward of 12k to 14k feet so figure that this feature will be pretty nice to have.

 

[waltr]

12k plus flight is a good reason to want telemetry on the deployment events.
The program only needs to show the user basic info.
Altitude, Drogue and Main status would be important.

Please do post back here what you learn.

 

[cerving]

The Eggtimer Telemetry format was designed so that it would minimize the amount of data transmitted while providing redundancy, a CSV format would have required sending more data. It also had to be compatible with legacy transmissions from the Quark and TRS. I have an outstanding request to add a CSV option to the Quantum and Proton to send altitude and deployment status instead of the current format... however, it wouldn't be decoded by the LCD receiver, it would be there strictly for custom software. Maybe later this Spring...

 

[Kane]

The Eggtimer Telemetry format was designed so that it would minimize the amount of data transmitted while providing redundancy, a CSV format would have required sending more data. It also had to be compatible with legacy transmissions from the Quark and TRS. I have an outstanding request to add a CSV option to the Quantum and Proton to send altitude and deployment status instead of the current format... however, it wouldn't be decoded by the LCD receiver, it would be there strictly for custom software. Maybe later this Spring...

Cris,
So does the ETM modify the format of the data that comes from the Quantum? The flight data download from the Quantum is in CSV. If so, then simply plugging a Quantum into a laptop and checking to see if the Excel Data Streamer plugin works would not be a useful experiment.

 

[cerving]

The ETM data format is not the same as the download. The download data is computed from the saved data in the EEPROM so there's plenty of time to do whatever calculations and/or formatting that needs to be done, the ETM data is generated in real-time from the internal values during flight. so doing it quickly matters. It comes over the serial port at 9600 baud, so if you connect the USB-Serial cable to the serial port of the Quantum and do a vacuum test you'll see the data stream out. You can also do this with the RX or LCD receiver if you have an ETM connected to the Quantum, of course.

You can't convert the ETM format directly to a CSV, because each of the data values are sent and interpreted individually, and they are sent on an as-created basis so the number of data elements between samples (interpreted by the time of flight data element) varies depending on where you are in the flight, and in the case of the Proton and Quantum which modes the deployment channels have been set to.

 

[frogglicker]

This thread is over a year old but I found a site that will graph data with little fuss.

https://getcurve.io
Here is a snapshot of my Big Red Max on a G74-9T a few weeks back. I kind of struggled with Excel and stopped messing with it b/c I couldn't get the views I was after. With Curve, it pulls it in and splits different graphs based on the event. This is what I was looking for. When it parsed the data, it said the 'O' in N-O was a zero. Doesn't like numbers in the headers. I just replaced it with the letter 'O' and it was done. This is a game changer for me.




Here, see for yourself.

https://getcurve.io/dashboard/snapshot/zKINY3CCQf2gW6XlH2piTUSIaaD9sGL7

 

[Kane]

This thread is over a year old but I found a site that will graph data with little fuss.

https://getcurve.io
Here is a snapshot of my Big Red Max on a G74-9T a few weeks back. I kind of struggled with Excel and stopped messing with it b/c I couldn't get the views I was after. With Curve, it pulls it in and splits different graphs based on the event. This is what I was looking for. When it parsed the data, it said the 'O' in N-O was a zero. Doesn't like numbers in the headers. I just replaced it with the letter 'O' and it was done. This is a game changer for me.


View attachment 577660
View attachment 577661
Here, see for yourself.

https://getcurve.io/dashboard/snapshot/zKINY3CCQf2gW6XlH2piTUSIaaD9sGL7

Pretty cool! I'm decent with Excel and have created templates to extract what I want from my flight data and visualize it. Nonetheless, this app is interesting. I'll give it a whirl. Thanks for sharing.

 

[Charles_McG]

My project:
https://www.rocketryforum.com/threa...-ground-station-software-build-thread.178464/

 

[Kane]

My project:
https://www.rocketryforum.com/threa...-ground-station-software-build-thread.178464/

I've been following!
I think we may have been at Red Glare on the same day.

 

[Charles_McG]

I've been following!
I think we may have been at Red Glare on the same day.

Not me - I only fly at Bong. But one of my testing partners.