Recommendations for Software for Real Time Telemetry Plotting (Python is too slow?)

The Rocketry Forum

Help Support The Rocketry Forum:

This site may earn a commission from merchant affiliate links, including eBay, Amazon, and others.

ghuber

Greg Huber
Joined
Oct 14, 2019
Messages
44
Reaction score
30
Location
Connecticut
Hi. I've been working on writing real time telemetry plotting software for my LORA-based GPS/flight computer. I've been using python to plot the serial port output from my arduino base station, but it can't keep up even at 1HZ. Does anyone have any advice? Thank you.
 
Python is compiled to byte code and then run interpreted by the python virtual machine like Java, although Java will be much faster. Interpreted is always going to be slower than something compiled to run directly on your computers CPU and OS. The fastest and smallest code would be assembler if you have the time to write that. For higher level languages, the fastest would be C, then C++ . You might want to also look into Ada.

There are usually two solutions to a slow program, use a faster language, or faster hardware. Hardware is usually pretty straight forward. Faster language has many layers to it. Assembler written and compiled for your CPU without using the OS will scream, but is not portable and certainly not fast or easy to use. Languages compiled for a certain OS and hardware is very fast, but is limited for use on the hardware on OS it is compiled for. Interpreted languages are always going to be slower than compiled ones. Interpreted are the languages that can run on any OS and hardware because they use a virtual machine to interpret between the code and the OS/hardware. Of those types, Java is the fastest.

You might want to dig deeper on doing real time programming. Real time is not the same thing as real fast. Many times real fast will work, but real time will usually use a real time clock interrupt along with hardware interrupts to force critical code to run so real world events are not missed and the code doesn't lose track of what's happening in the real world. Real time programming is a whole branch of engineering by itself and usually also requires fault tolerance and recovery.
 
Python on any x64 CPU is plenty fast! What is your host system, specs, etc?

How do you know the Arduino is actually sending the data fast enough? 300 baud isn't fast enough at 1Hz for more than 30 bytes of data.

Could be a lot of things.
 
Assuming you are using a desktop class PC to do the plotting, #3 is most likely the problem. As a first step, try just printing the serial data to a console window and see if it keeps up. We would need more information to be more specific help.
 
It's not Python, in a previous job we ran a vehicle instrument cluster tester that would test 24 instrument clusters and log the 5 UART outputs from each cluster, that's 120 Serial ports. The scripts for the tester were written in python, running on a 6 core xeon from 2012, it had zero issues keeping up while also running 24 CAN channels and looking though 3 web cams to verify the screens were turning on.

Understand Python's strength is calling libraries from compiled languages. It is the de facto language for running machine learning and AI tasks (Though the computationally complex part of those are written in CUDA, C, or ROCm, and just called from Python).

Even if you were doing all the computing in python without calling external libraries, I doubt it would be so slow as to not handle serial output speeds that have been in use for 40 years. Chances are you're doing something in a very slow way in the code. Limit the number of loops you are nesting when you try to read from the UART, or use a library to do the work for you.
 
I agree, it could simply be the Python library you are using fro Uart, processing or Display.
I have found some Python display libraries very slow.

What are you using for the graphic display?

I like Python due to easy to write code that does a lot but hate Python in tring to figure out what is the best library for the functionality I desire.
 
Hi. I've been working on writing real time telemetry plotting software for my LORA-based GPS/flight computer. I've been using python to plot the serial port output from my arduino base station, but it can't keep up even at 1HZ. Does anyone have any advice? Thank you.
If it won't keep up at 1Hz something is seriously wrong, or else you're doing a *lot* more than just plotting.
 
Thank you everyone for your feedback. I'm using matplotlib and drawnow in python to create a 4 panel graph, an example of which is attached.

When I run it on my work PC workstation in debugging mode, it takes .51 seconds or so for each update, and on my 2018 era Lenovo laptop running in the field on battery, it can take as much as 1.2 seconds.

The below is a plot from a saved SD card file (of a bike ride), so there is no input/output speed issue. (It reads the whole file in before starting the plotting). If I simply turn off the graph drawing, it executes each loop in .002 seconds, so it is the drawing that is taxing it, not the calculations/etc. done before the calls to the plotting routines.

I assume this is a matplotlib or drawnow issue, but I wasn't sure what the better (faster) options were/are. Thank you as always.


Screenshot 2023-08-28 123235.jpg
 
Thank you everyone for your feedback. I'm using matplotlib and drawnow in python to create a 4 panel graph, an example of which is attached.

When I run it on my work PC workstation in debugging mode, it takes .51 seconds or so for each update, and on my 2018 era Lenovo laptop running in the field on battery, it can take as much as 1.2 seconds.

The below is a plot from a saved SD card file (of a bike ride), so there is no input/output speed issue. (It reads the whole file in before starting the plotting). If I simply turn off the graph drawing, it executes each loop in .002 seconds, so it is the drawing that is taxing it, not the calculations/etc. done before the calls to the plotting routines.

I assume this is a matplotlib or drawnow issue, but I wasn't sure what the better (faster) options were/are. Thank you as always.


View attachment 600550
Matlabplot and equivalent are geared to operate on datasets and make pretty plots.

For live data you want the software functionality of a strip chart recorder or an oscilloscope. Except you want to emulate the pen moving left to right instead of the "paper" moving right to left. Google for those and you will find better choices for your application.
 
matplotlib.animation works reasonably well on my system:

Python:
import matplotlib.pyplot as plt
from matplotlib.animation import FuncAnimation
from random import randrange
from math import sin, pi

fig = plt.figure(figsize=(6, 3))
x = [0]
y = [0]

ln, = plt.plot(x, y, '-')

def update(frame):
    x.append(x[-1] + (pi / 50))
    y.append(sin(x[-1]))

    ln.set_data(x, y)
    fig.gca().relim()
    fig.gca().autoscale_view()
    return ln,

animation = FuncAnimation(fig, update, interval=50)
plt.show()

Slightly modified from this source to replace the jittery random curve with a smooth sine.

The result is not buttery smooth, but should be more then enough for plotting real time data.

Depending on your needs, there are many more plotting libraries, with their own strengths and weaknesses. I stumbled across this podcast episode a while ago that touches the topic of plotting libraries if your interested, although it doesn't really touch animation or real time plots if I remember correctly.

Reinhard
 
Hi. I've been working on writing real time telemetry plotting software for my LORA-based GPS/flight computer. I've been using python to plot the serial port output from my arduino base station, but it can't keep up even at 1HZ. Does anyone have any advice? Thank you.
Python should be more than sufficient. My GUI uses python and can get 35datapoints/second (1500Bps) with usb serial or about 15dp/second (600Bps) with LoRa. I can probably squeeze more, but the least of my problems is the python GUI. The only thing that doesn't seem to keep up is pyqt refreshing widgets, and that's an intermitten issue that I think is because I'm not using pyqt correctly.

I'm using pglive for my active plotting and have several threads, but only two are used for the plotting: the main GUI thread, and a thread dedicated to reading input from serial and handing it to the GUI. The threading divides the work so that you see less lag per loop from it processing. If your widgets are responsive, but you're still getting slow data input, you might want to slim down what you have going on per loop. If you're familiar with multithreading (which is super easy in python, just don't forget to put locks around data that gets handled by multiple threads), divide up the tasks, and that should show some improvement.

Another thing: how are you formatting transmitted data? If it's in plaintext, JSON, XML, CVS, or any plain text format, it's going to be incredibly slow. You need to use the byte data the numbers are stored as and ctypes in python to decode it. My data is 40bytes big, and that stores a "magic identifier" (prevents interference from other LoRa devices), runtime, altitude, internal temperature, magnometer (3 integers), accelerometer (3 integers), and the rest is a bit map for statuses/booleans (for example, when my relay is active, it will be indicated in the bit map). The same data in JSON format (a web optimized format), it would probably be in the order of 150bytes, give or take, which would seriously reduce the amount of data I would get because that's almost 4 times bigger, and it has to be parsed. Byte data can be mem copied and then the ctypes struct already has an understand of how the data is formatted so there is WAY less work to do per datapoint.

(My GUI: )
 
Last edited:
Python is compiled to byte code and then run interpreted by the python virtual machine like Java, although Java will be much faster. Interpreted is always going to be slower than something compiled to run directly on your computers CPU and OS. The fastest and smallest code would be assembler if you have the time to write that. For higher level languages, the fastest would be C, then C++ . You might want to also look into Ada.

There are usually two solutions to a slow program, use a faster language, or faster hardware. Hardware is usually pretty straight forward. Faster language has many layers to it. Assembler written and compiled for your CPU without using the OS will scream, but is not portable and certainly not fast or easy to use. Languages compiled for a certain OS and hardware is very fast, but is limited for use on the hardware on OS it is compiled for. Interpreted languages are always going to be slower than compiled ones. Interpreted are the languages that can run on any OS and hardware because they use a virtual machine to interpret between the code and the OS/hardware. Of those types, Java is the fastest.

You might want to dig deeper on doing real time programming. Real time is not the same thing as real fast. Many times real fast will work, but real time will usually use a real time clock interrupt along with hardware interrupts to force critical code to run so real world events are not missed and the code doesn't lose track of what's happening in the real world. Real time programming is a whole branch of engineering by itself and usually also requires fault tolerance and recovery.
Python modules are often written in C/C++, so you don't necessarily gain advantage by using C languages over python nowadays because python is already leveraging those advantages. Any gains will really come at the expense of debugging an executable which can be complicated if you're not good at managing memory. For casual things or hobbies, python is probably the way to go, but certainly not the only way to do it.
 
just don't forget to put locks around data that gets handled by multiple thread
The Python global interpreter lock (GIL) makes this (mostly) unnecessary but also means that Python threads aren't unique processes and your performance improvements are heavily dependent upon the application. For something like this project, a thread will often help because the GIL will be released during waits, like when waiting for serial data.

When live plotting data like this, I often use Bokeh as the display or will setup a Flask app and stream to a webpage. This allows me to cleanly separate the data collection/logging from the view. Both are more complicated than matplotlib but open up a lot of possibilities.
 
Python should be more than sufficient. My GUI uses python and can get 35datapoints/second (1500Bps) with usb serial or about 15dp/second (600Bps) with LoRa. I can probably squeeze more, but the least of my problems is the python GUI. The only thing that doesn't seem to keep up is pyqt refreshing widgets, and that's an intermitten issue that I think is because I'm not using pyqt correctly.

I'm using pglive for my active plotting and have several threads, but only two are used for the plotting: the main GUI thread, and a thread dedicated to reading input from serial and handing it to the GUI. The threading divides the work so that you see less lag per loop from it processing. If your widgets are responsive, but you're still getting slow data input, you might want to slim down what you have going on per loop. If you're familiar with multithreading (which is super easy in python, just don't forget to put locks around data that gets handled by multiple threads), divide up the tasks, and that should show some improvement.

Another thing: how are you formatting transmitted data? If it's in plaintext, JSON, XML, CVS, or any plain text format, it's going to be incredibly slow. You need to use the byte data the numbers are stored as and ctypes in python to decode it. My data is 40bytes big, and that stores a "magic identifier" (prevents interference from other LoRa devices), runtime, altitude, internal temperature, magnometer (3 integers), accelerometer (3 integers), and the rest is a bit map for statuses/booleans (for example, when my relay is active, it will be indicated in the bit map). The same data in JSON format (a web optimized format), it would probably be in the order of 150bytes, give or take, which would seriously reduce the amount of data I would get because that's almost 4 times bigger, and it has to be parsed. Byte data can be mem copied and then the ctypes struct already has an understand of how the data is formatted so there is WAY less work to do per datapoint.

(My GUI: )

Do you have a posted public codebase anywhere? I'd be curious to see the details about the multithreaded implementation of pglive.

My LORA transmitter has an old public codebase here: https://github.com/gregoryhuber/GPSRocketTracker. (This is pre-integration of Baro/receiver calculation of angle to rocket above horizon/etc.). I've managed to pack everything into a small packet using structures and have no problem getting 10HZ+ for transmission, although I generally slow it down to 5HZ, with successful tracking to landing >2 miles away.
 
Do you have a posted public codebase anywhere? I'd be curious to see the details about the multithreaded implementation of pglive.

My LORA transmitter has an old public codebase here: https://github.com/gregoryhuber/GPSRocketTracker. (This is pre-integration of Baro/receiver calculation of angle to rocket above horizon/etc.). I've managed to pack everything into a small packet using structures and have no problem getting 10HZ+ for transmission, although I generally slow it down to 5HZ, with successful tracking to landing >2 miles away.
No, unfortunately I don't like sharing my code publicly. Only my friends have access to it... sorry.

But also, the pglive runs on a single thread in my GUI, and my "SerialInput" is on a second thread. Division of labor helps there so that pglive doesn't stall if the serial input stops receiving.

Did you tune LoRa for long range? The amount of data you can get from it tends to be significantly less with longer range settings. Mine is currently configured <2km to maximize the data I can pull from it. I'll probably make it more automatic eventually, but that's another thing you may want to consider: is range more important, or data acquisition? From a development standpoint, I tend to lean towards the latter because my range is pretty limited by launch sites here.
 
Python should be more than sufficient. My GUI uses python and can get 35datapoints/second (1500Bps) with usb serial or about 15dp/second (600Bps) with LoRa. I can probably squeeze more, but the least of my problems is the python GUI. The only thing that doesn't seem to keep up is pyqt refreshing widgets, and that's an intermitten issue that I think is because I'm not using pyqt correctly.

I'm using pglive for my active plotting and have several threads, but only two are used for the plotting: the main GUI thread, and a thread dedicated to reading input from serial and handing it to the GUI. The threading divides the work so that you see less lag per loop from it processing. If your widgets are responsive, but you're still getting slow data input, you might want to slim down what you have going on per loop. If you're familiar with multithreading (which is super easy in python, just don't forget to put locks around data that gets handled by multiple threads), divide up the tasks, and that should show some improvement.

Another thing: how are you formatting transmitted data? If it's in plaintext, JSON, XML, CVS, or any plain text format, it's going to be incredibly slow. You need to use the byte data the numbers are stored as and ctypes in python to decode it. My data is 40bytes big, and that stores a "magic identifier" (prevents interference from other LoRa devices), runtime, altitude, internal temperature, magnometer (3 integers), accelerometer (3 integers), and the rest is a bit map for statuses/booleans (for example, when my relay is active, it will be indicated in the bit map). The same data in JSON format (a web optimized format), it would probably be in the order of 150bytes, give or take, which would seriously reduce the amount of data I would get because that's almost 4 times bigger, and it has to be parsed. Byte data can be mem copied and then the ctypes struct already has an understand of how the data is formatted so there is WAY less work to do per datapoint.

(My GUI: )

This is from a Data Science oriented article "Multi-threading and Multi-processing in Python". DS tends to crunch lots of data and most PhDs hate waiting for a computer, so performance is always an issue.

"To recap, threading in Python allows multiple threads to be created within a single process, but due to GIL, none of them will ever run at the exact same time. Threading is still a very good option when it comes to running multiple I/O bound tasks concurrently. Now if you want to take advantage of computational resources on multi-core machines, then multi-processing is the way to go."
 
This is from a Data Science oriented article "Multi-threading and Multi-processing in Python". DS tends to crunch lots of data and most PhDs hate waiting for a computer, so performance is always an issue.

"To recap, threading in Python allows multiple threads to be created within a single process, but due to GIL, none of them will ever run at the exact same time. Threading is still a very good option when it comes to running multiple I/O bound tasks concurrently. Now if you want to take advantage of computational resources on multi-core machines, then multi-processing is the way to go."
tHoagland beat yah to it lol, but thank you. I definitely need to look into this, because I didn't know about the GIL. It seems like multithreading meets my usecase, but I'd be interested to see the performance difference if I implement multiprocessing. My GUI is not cpu bound, so its probably not going to improve anything, but it will be handy as I continue developing it. (EDIT) thinking about it, it might be useful for my voiceover thread. My software can do the altitude and stat announcements (apogee, max speed, etc), but it tends to be slow because of network issues (absolutely no guarantee of a good connection at a launch site, and you can only cache some of the voice over because the stats tend to vary from my simulations by at least a few meters). I couldn't get my GUI working with an offline text-to-speech library because it seemed to close the application after playing one voice message, even with multithreading. I bet you I can implement multiprocessing with that and the GUI will continue running with the offline tts voice overs.

Also heck yeah: another EZI-65 builder lol
 
Last edited:
Did you tune LoRa for long range? The amount of data you can get from it tends to be significantly less with longer range settings. Mine is currently configured <2km to maximize the data I can pull from it. I'll probably make it more automatic eventually, but that's another thing you may want to consider: is range more important, or data acquisition? From a development standpoint, I tend to lean towards the latter because my range is pretty limited by launch sites here.
LORA seems plenty fast and I've been designing for small packet sizes. I have had no problem when using the <2KM setup (which empirically still works at 2 miles with a line of site to the rocket).

I also experimented with the longer range packets (slower transmit speeds) and found both that they were too slow and also that when the rocket was moving quickly, they tended to arrive corrupted. (Someone told me this was due to doppler shift, but I'm not sure if that explanation is correct.)
 
No, unfortunately I don't like sharing my code publicly. Only my friends have access to it... sorry.
Do you have a pointer to a basic template/tutorial for the basic windowed setup you are using? I'd like to give something like that a try.
 
Do you have a pointer to a basic template/tutorial for the basic windowed setup you are using? I'd like to give something like that a try.
You can look up tutorials for pyqt5. Most of of my widgets are from common libraries like pglive and the built in “qwidgets”. However, there are two custom widgets: the magnometer and altimeter tape. The magnometer needs calibration for that (normalizing the max/min guass to 1 and -1 for each axis), otherwise it’s difficult to understand the meaning of the data. The widget just always views it from the “north” vector, so your perspective would resemble what you would see if you were right next to it. The altimeter tape isn’t really following how they normally do it, but the premise is to show vertical movement. How I did it shows motion from a static position like the magnometer widget I made. For those, I used opengl to render the displays. The “lights” are just text labels and I change the BG color to show a change in status, and you can easily reproduce those.

Basically, I would create a row using QWidget, set the layout to QHBoxLayout, and then add my widgets. Grouped things like telemetry, statistics, etc were first added to a QGroupBox so they have their own panel, and that was added to the layout for the row.

I used pyqt mostly because it’s compatible with osx, and I am still fairly inexperienced with it. Pyqt can make way more robust GUI with rearrangeable panels, tabs or other useful space saving widgets.
 
Another thing I want to bring up: pyqt does not seem to have a completely offline mapping widget that is completely python (and libraries used by it). It can be offline compatible, but you will basically be using open street maps or something similar in a small browser window opening a html/javascript window that loads the map. I don't like that, so I'm gonna try and make something on my own (I say boldly lol) when I finally assemble my flight computer with gps (next version I'm making for my EZI).
 
I'm impressed that you're getting the performance out of python/PyQt code. My experience with PyQt would make me hesitate to try it for this type of an interface. It's always seemed a little sluggish to me.
 
I'm impressed that you're getting the performance out of python/PyQt code. My experience with PyQt would make me hesitate to try it for this type of an interface. It's always seemed a little sluggish to me.
The trick is that pyqt doesn't really handle any of the plotting, it's just displaying a qt canvas which matplotlib is driving. Things get sketchier when you add significant qt-driven plot interactions. It's a shame PythonQwt isn't (wasn't) well maintained, it was a very nice, well performing plotting package.

I'm not sure folks here are using "pyqt" generically but of you're unaware there are two sets of python qt bindings. A couple of years ago, I transitioned from pyqt to pyside (pyside6 is the latest). The pyside package is the "officially " supported binding that is developed by The Qt Company (yes, that's the actual name). I find pyside to be more pythonic on some places and more comfortable to use. Your results may vary.
 
Back
Top