When reading data from UART, fetching data is not correct

The Rocketry Forum

Help Support The Rocketry Forum:

This site may earn a commission from merchant affiliate links, including eBay, Amazon, and others.

deepfreeze

Well-Known Member
TRF Supporter
Joined
Dec 3, 2022
Messages
55
Reaction score
1
Location
New Jersey, US
I am sending data from Xbee 3 pro (router) connected to Teensy 4.0 microcontroller, to another Xbee 3 pro (coordinator). I am successfully getting data,but the data i am getting is not structured correctly as lines. Lines are mostly combined.

printed data:
Code:
1.00,4096.00
,-0.21,-161,-0.79,1.03,23.13,1020.02,-0.21,-16.52,0,0,0,0,0.00,0.00,0.90,0.00,131.00,4096.00
,-190.00,-225.00,119.00,-0.51,-0.79,1.03,23.13,1020.02,-0.23,-16.52,0,0,0,0,0.00,0.0,1020.02,-0.23,-16.52,0,0,0,0,0.00,0.00,0.89,0.00,131.00,4096.00
.52,0,0,0,0,0.00,0.00,0.8932,-16.52,0,0,0,0,0.00,0.00,0.88,0.00,131.00,4096.00
020.03,-0.43,-16.52,0,0,0,0,0.00,0.00,0.87,00.79,1.03,23.15,1020.06,-0.50,-16.52,0,0,0,0,0.00,0.00,0.86,0.00,131.00,4096.00
214.00,115.00,-0.52,-0.80,1.03,23.13,10,0.06,0.26,0.98,-182.00,-214.00,115.00,-0.52,-0.80,1.03,23.13,1020.03,-0.27,-16.52,00.80,1.03,23.13,1020.03,-0.27,-16.52,0,0,0,0,0.00,0.00,0.83,0.00,131.00,4096.00


Sending data from Teensy:

C++:
void sendDataTotelemetry(){
 
  Serial3.print(millis());
  Serial3.print(",");
  Serial3.print(data.Accel_X);
  Serial3.print(",");
  Serial3.print(data.Accel_Y);
  Serial3.print(",");
  Serial3.print(data.Accel_Z);
  Serial3.print(",");
  Serial3.print(data.Gyro_X_Raw);
  Serial3.print(",");
  Serial3.print(data.Gyro_Y_Raw);
  Serial3.print(",");
  Serial3.print(data.Gyro_Z_Raw);
  Serial3.print(",");
  Serial3.print(data.Gyro_Yaw);
  Serial3.print(",");
  Serial3.print(data.Gyro_Roll);
  Serial3.print(",");
  Serial3.print(data.Gyro_Pitch);
  Serial3.print(",");
  Serial3.print(data.Baro_Temp);
  Serial3.print(",");
  Serial3.print(data.Baro_Pressure);
  Serial3.print(",");
  Serial3.print(data.Baro_Altitude);
  Serial3.print(",");
  Serial3.print(data.Baro_Altitude_Error);
  Serial3.print(",");
  Serial3.print(data.ParachuteOn);
  Serial3.print(",");
  Serial3.print(data.LegsDeployed);
  Serial3.print(",");
  Serial3.print(data.MotorFired);
  Serial3.print(",");
  Serial3.print(data.RocketState);
  Serial3.print(",");
  Serial3.print(data.VoltageValue);
  Serial3.print(",");
  Serial3.print(data.Voltage);
  Serial3.print(",");
  Serial3.print(data.Kal_X_Pos);
  Serial3.print(",");
  Serial3.print(data.Kal_X_PosP);
  Serial3.print(",");
  Serial3.print(data.Gyro_Sens);
  Serial3.print(",");
  Serial3.println(data.Acc_Sens);
}
Getting data from C++ program:
C++:
void readTelemetryData(){


    data.telemetry_conn_port = serial_t_port;

    serial_t.open(serial_t_port);

    if (!serial_t.is_open()){
        data.telemetry_conn_status = "N/A";

    } else{
        
        serial_t.ignore();
        getline(serial_t,incoming_data);
        data.telemetry_conn_status = "SUCCESS";
        if (incoming_data.find(',') != std::string::npos){
            
            data_seperated_t = split_data(incoming_data, ',');

            data.time = data_seperated_t[0];
            data.Accel_X = data_seperated_t[1];
            data.Accel_Y = data_seperated_t[2];
            data.Accel_Z = data_seperated_t[3];
            data.Gyro_X_Raw = data_seperated_t[4];
            data.Gyro_Y_Raw = data_seperated_t[5];
            data.Gyro_Z_Raw = data_seperated_t[6];
            data.Gyro_Yaw = data_seperated_t[7];
            data.Gyro_Roll = data_seperated_t[8];
            data.Gyro_Pitch = data_seperated_t[9];
            data.Baro_Temp = data_seperated_t[10];
            data.Baro_Pressure = data_seperated_t[11];
            data.Baro_Altitude = data_seperated_t[12];
            data.Baro_Altitude_Error = data_seperated_t[13];
            data.ParachuteOn = data_seperated_t[14];
            data.LegsDeployed = data_seperated_t[15];
            data.MotorFired = data_seperated_t[16];
            data.RocketState = data_seperated_t[17];
            data.VoltageValue = data_seperated_t[18];
            data.Voltage = data_seperated_t[19];
            data.Kal_X_Pos = data_seperated_t[20];
            data.Kal_X_PosP = data_seperated_t[21];
            data.Gyro_Sens = data_seperated_t[22];
            data.Acc_Sens = data_seperated_t[23];
            

        } else {

            data.message.push_back(incoming_data) ;
        }

        
        
    serial_t.close();
    }

}
 
Assuming you're trying to create a .CSV file, you need a carriage return Serial.write(0x0d) at the end of your last data element (instead of sending a comma).
 
Assuming you're trying to create a .CSV file, you need a carriage return Serial.write(0x0d) at the end of your last data element (instead of sending a comma).
Correct, not sending a 'line terminator'. A 'Carriage Return' (0x0D) works or send a 'new line'
Serial3.print("\n");
 
Guys i think i am dropping some data when sending with Xbee. I think this is the problem here other than line. Becuse when i send a small string with println. I am losing some of the characters also. My baud rate for both xbees are 115200. I am in AT mode for both coordinator and router.

DEEPFREEZEEEE DEEPFREEZEEEE EEPFREEZEEEE EEE E EEPFREEZEEEE EEPFREEZEEEE EEPFREEZEEEE EEPFREEZEEEE EEPFREEZEEEE EEPFREEZEEEE EZEEEE ZEEEE EEPFREEZEEEE EEPFREEZEEEE EEPFREEZEEEE EEE FREEZEEEE EEE EZEEEE EEPFREEZEEEE EPFREEZEEEE EE EEPFREEZEEEE
 
Lower your baud rate to 9600 and see if your comes through clear. Then, up the data rate incrementally to see if there is a point where dropoff happens. Thats a decently sized data burst.
 
Guys i think i am dropping some data when sending with Xbee. I think this is the problem here other than line. Becuse when i send a small string with println. I am losing some of the characters also. My baud rate for both xbees are 115200. I am in AT mode for both coordinator and router.

DEEPFREEZEEEE DEEPFREEZEEEE EEPFREEZEEEE EEE E EEPFREEZEEEE EEPFREEZEEEE EEPFREEZEEEE EEPFREEZEEEE EEPFREEZEEEE EEPFREEZEEEE EZEEEE ZEEEE EEPFREEZEEEE EEPFREEZEEEE EEPFREEZEEEE EEE FREEZEEEE EEE EZEEEE EEPFREEZEEEE EPFREEZEEEE EE EEPFREEZEEEE
That’s not unusual in any serial communications, especially wireless at high speeds. That’s why they usually use a communications protocol with error correction and the ability to request messages to be resent.
 
Last edited:
All correct, there can always be data lose on RF links.
A protocol for at least error detection and a method to detect beginning of a new data packet is required for reliable data link.

Go to Cris' Eggtimer website and look at his protocol for his RF telemetry.
http://eggtimerrocketry.com/wp-content/uploads/2021/05/Eggtimer-Telemetry-Data-Format.pdf
This is just one way to do this.

Also look up NMEA which is the protocol for GPS data. They have a similar protocol with characters indicating start & stop of each data packet with a checksum in the data for error detection.

Minimum is to use a unique charater or charater sequence to indice start of data. Else if a data byte drops then the receiving code get confused and the data becomes garbage.
 
Okay so basically each data is represent by a special char or bunch of chars. But isn’t it too much for my code to write too many if and elses? I always avoid too much if Elses. What do you think ?
 
I think bullet proofing your code against real world effects, only make it better. Figuring our how to do that efficiently and and effectively is what makes a good programmer great. Reducing the If/Else can certainly help, but maybe it isn't avoidable. Make it great.

Good Luck.
 
No, Each Packet of data has a special Start character. Like this:

Code:
void sendDataTotelemetry(){
 
   Serial3.print("<");    // send start char
  Serial3.print(millis());
  Serial3.print(",");
.
.
.
.
  Serial3.print(",");
  Serial3.print(data.Acc_Sens);
  Serial3.print(">");    // send end char
  Serial3.print("\n");  // send line terminator
}

Then the Receive code looks for the "<" char to start saving data and ensures there is a ">" char that is the end of data. Else if no end )">" before next start data was lost. Reset save data to beginning and start saving again.
 
You need to add a checksum, as suggested above, to protect the receiver from processing broken frames. Anything will do, like 1s complement sum.

BTW, XBee is a secret code word that means "doesn't work very well, if at all."

Agree with above post, slow down to 9600 baud. Or even 4800. With a little simple arithmetic you can calculate the rate actually needed.
 
BTW, XBee is a secret code word that means "doesn't work very well, if at all."
I've found they are very reliable once you get them properly set up, but they are tricky and poorly documented.

Which module are you using? If you are sending data into the UART faster than actual RF data rate, then dropping data is obviously inevitable.
 
Okay so basically each data is represent by a special char or bunch of chars. But isn’t it too much for my code to write too many if and elses? I always avoid too much if Elses. What do you think ?
Your code should be as simple as possible, but it shouldn't be simpler than that. You can still be extra careful with your code review or maybe refactor it, but you can't get rid of transmission errors so you need to deal with them even if it makes your code more complex.
If you think you've got too many if-else statements on your hand, don't forget to take a look at the switch statement.

Reinhard
 
Read the XBee datasheet. If using XBeePro you can enable CRC at the module itself. I wouldn't be surprised if most modern XBee's implement this. If it fails the CRC then nothing gets sent to the UART. You may drop packets, but the packets that gets sent to output you can be sure is valid.
 
Last edited:
If you can get by with lower data rates you will improve the reliability of your transmissions, because the bit cell lengths are longer and therefore less prone to noise glitches. The more data you try to send at once, the harder it gets to reliably receive it all, which is why the larger-faster packetized schemes usually have some kind of error detection/correction, and may use a bidirectional req-ack protocol with retransmissions in the event of an error.
 
No, Each Packet of data has a special Start character. Like this:

Code:
void sendDataTotelemetry(){
 
   Serial3.print("<");    // send start char
  Serial3.print(millis());
  Serial3.print(",");
.
.
.
.
  Serial3.print(",");
  Serial3.print(data.Acc_Sens);
  Serial3.print(">");    // send end char
  Serial3.print("\n");  // send line terminator
}

Then the Receive code looks for the "<" char to start saving data and ensures there is a ">" char that is the end of data. Else if no end )">" before next start data was lost. Reset save data to beginning and start saving again.
oh okay then it saves me for not having so many if elses. Thank you walter.
 
You need to add a checksum, as suggested above, to protect the receiver from processing broken frames. Anything will do, like 1s complement sum.

BTW, XBee is a secret code word that means "doesn't work very well, if at all."

Agree with above post, slow down to 9600 baud. Or even 4800. With a little simple arithmetic you can calculate the rate actually needed.
Okay will do that. Thank you.
 
Your code should be as simple as possible, but it shouldn't be simpler than that. You can still be extra careful with your code review or maybe refactor it, but you can't get rid of transmission errors so you need to deal with them even if it makes your code more complex.
If you think you've got too many if-else statements on your hand, don't forget to take a look at the switch statement.

Reinhard
yes yes it's faster but still like having 15 different data will effect. But it's okay, i figured it out thanks to waltr. Thank you too for your suggestion.
 
Read the XBee datasheet. If using XBeePro you can enable CRC at the module itself. I wouldn't be surprised if most modern XBee's implement this. If it fails the CRC then nothing gets sent to the UART. You may drop packets, but the packets that gets sent to output you can be sure is valid.
Oh that's actually is so useful. Thanks for teaching me that.
 
If you can get by with lower data rates you will improve the reliability of your transmissions, because the bit cell lengths are longer and therefore less prone to noise glitches. The more data you try to send at once, the harder it gets to reliably receive it all, which is why the larger-faster packetized schemes usually have some kind of error detection/correction, and may use a bidirectional req-ack protocol with retransmissions in the event of an error.
Will do, thank you
 
I’m just curious. Do you know what a Case statement does?
As a 25 years of computer engineer. You'd be suprised. I am just too much obsessive about coding and making it better so i try to consume all the experiences by throwing whatever comes to my mind. I don't care how i look from outside and respect other's experiences even if i am sure most of the people doesn't have that much experience like me. Long story short.. It doesn't mean i'm a noob. Keep your arroga..-curiosity to yourself.
 
As a 25 years of computer engineer. You'd be suprised. I am just too much obsessive about coding and making it better so i try to consume all the experiences by throwing whatever comes to my mind. I don't care how i look from outside and respect other's experiences even if i am sure most of the people doesn't have that much experience like me. Long story short.. It doesn't mean i'm a noob. Keep your arroga..-curiosity to yourself.
I’m sorry if my question came across as arrogant. That wasn’t my intention. It really was a serious inquiry to try and understand your knowledge of C programming so I would know what I could say that might help. Anyway, it looks like you’ve got some good help.
 
Good thread so far since the OP has gotten the help needed.

Deepfreeze,
I see you are in NJ. I'm across the river in Bucks County and my local club flys in upper Bucks.
You are welcome to come to our launches if the drive is not too far for you.
http://www.para520.com/
 
I’m sorry if my question came across as arrogant. That wasn’t my intention. It really was a serious inquiry to try and understand your knowledge of C programming so I would know what I could say that might help. Anyway, it looks like you’ve got some good help.
Then it's my bad. I apologize if i offended you. Yes i got good help. Thank you :)
 
Good thread so far since the OP has gotten the help needed.

Deepfreeze,
I see you are in NJ. I'm across the river in Bucks County and my local club flys in upper Bucks.
You are welcome to come to our launches if the drive is not too far for you.
http://www.para520.com/
Oh thank you so much waltr. I would love to come and see. It is like 1.5 hours driving. It's nothing for me. But i won't be able to launch anything yet. I will be coming for learning and meeting with you and some people who has experience. I think i need a push to launch my first and then after my first crash, i will be relaxed :) Because right now i am trying so much to make it perfect as if i really can do it in my first time. It would be great for me to experience something like that.
 
Back
Top