When reading data from UART, fetching data is not correct

Advertise with us
eggtimer rocketry
Wildman Rocketry
deepfreeze

deepfreeze

Well-Known Member
TRF Supporter
Joined
Dec 3, 2022
Messages
52
Reaction score
1
Location
New Jersey, US
I am sending data from Xbee 3 pro (router) connected to Teensy 4.0 microcontroller, to another Xbee 3 pro (coordinator). I am successfully getting data,but the data i am getting is not structured correctly as lines. Lines are mostly combined.

printed data:
Code: 
1.00,4096.00
,-0.21,-161,-0.79,1.03,23.13,1020.02,-0.21,-16.52,0,0,0,0,0.00,0.00,0.90,0.00,131.00,4096.00
,-190.00,-225.00,119.00,-0.51,-0.79,1.03,23.13,1020.02,-0.23,-16.52,0,0,0,0,0.00,0.0,1020.02,-0.23,-16.52,0,0,0,0,0.00,0.00,0.89,0.00,131.00,4096.00
.52,0,0,0,0,0.00,0.00,0.8932,-16.52,0,0,0,0,0.00,0.00,0.88,0.00,131.00,4096.00
020.03,-0.43,-16.52,0,0,0,0,0.00,0.00,0.87,00.79,1.03,23.15,1020.06,-0.50,-16.52,0,0,0,0,0.00,0.00,0.86,0.00,131.00,4096.00
214.00,115.00,-0.52,-0.80,1.03,23.13,10,0.06,0.26,0.98,-182.00,-214.00,115.00,-0.52,-0.80,1.03,23.13,1020.03,-0.27,-16.52,00.80,1.03,23.13,1020.03,-0.27,-16.52,0,0,0,0,0.00,0.00,0.83,0.00,131.00,4096.00


Sending data from Teensy:

C++: 
void sendDataTotelemetry(){
 
  Serial3.print(millis());
  Serial3.print(",");
  Serial3.print(data.Accel_X);
  Serial3.print(",");
  Serial3.print(data.Accel_Y);
  Serial3.print(",");
  Serial3.print(data.Accel_Z);
  Serial3.print(",");
  Serial3.print(data.Gyro_X_Raw);
  Serial3.print(",");
  Serial3.print(data.Gyro_Y_Raw);
  Serial3.print(",");
  Serial3.print(data.Gyro_Z_Raw);
  Serial3.print(",");
  Serial3.print(data.Gyro_Yaw);
  Serial3.print(",");
  Serial3.print(data.Gyro_Roll);
  Serial3.print(",");
  Serial3.print(data.Gyro_Pitch);
  Serial3.print(",");
  Serial3.print(data.Baro_Temp);
  Serial3.print(",");
  Serial3.print(data.Baro_Pressure);
  Serial3.print(",");
  Serial3.print(data.Baro_Altitude);
  Serial3.print(",");
  Serial3.print(data.Baro_Altitude_Error);
  Serial3.print(",");
  Serial3.print(data.ParachuteOn);
  Serial3.print(",");
  Serial3.print(data.LegsDeployed);
  Serial3.print(",");
  Serial3.print(data.MotorFired);
  Serial3.print(",");
  Serial3.print(data.RocketState);
  Serial3.print(",");
  Serial3.print(data.VoltageValue);
  Serial3.print(",");
  Serial3.print(data.Voltage);
  Serial3.print(",");
  Serial3.print(data.Kal_X_Pos);
  Serial3.print(",");
  Serial3.print(data.Kal_X_PosP);
  Serial3.print(",");
  Serial3.print(data.Gyro_Sens);
  Serial3.print(",");
  Serial3.println(data.Acc_Sens);
}
Getting data from C++ program:
C++: 
void readTelemetryData(){


    data.telemetry_conn_port = serial_t_port;

    serial_t.open(serial_t_port);

    if (!serial_t.is_open()){
        data.telemetry_conn_status = "N/A";

    } else{
        
        serial_t.ignore();
        getline(serial_t,incoming_data);
        data.telemetry_conn_status = "SUCCESS";
        if (incoming_data.find(',') != std::string::npos){
            
            data_seperated_t = split_data(incoming_data, ',');

            data.time = data_seperated_t[0];
            data.Accel_X = data_seperated_t[1];
            data.Accel_Y = data_seperated_t[2];
            data.Accel_Z = data_seperated_t[3];
            data.Gyro_X_Raw = data_seperated_t[4];
            data.Gyro_Y_Raw = data_seperated_t[5];
            data.Gyro_Z_Raw = data_seperated_t[6];
            data.Gyro_Yaw = data_seperated_t[7];
            data.Gyro_Roll = data_seperated_t[8];
            data.Gyro_Pitch = data_seperated_t[9];
            data.Baro_Temp = data_seperated_t[10];
            data.Baro_Pressure = data_seperated_t[11];
            data.Baro_Altitude = data_seperated_t[12];
            data.Baro_Altitude_Error = data_seperated_t[13];
            data.ParachuteOn = data_seperated_t[14];
            data.LegsDeployed = data_seperated_t[15];
            data.MotorFired = data_seperated_t[16];
            data.RocketState = data_seperated_t[17];
            data.VoltageValue = data_seperated_t[18];
            data.Voltage = data_seperated_t[19];
            data.Kal_X_Pos = data_seperated_t[20];
            data.Kal_X_PosP = data_seperated_t[21];
            data.Gyro_Sens = data_seperated_t[22];
            data.Acc_Sens = data_seperated_t[23];
            

        } else {

            data.message.push_back(incoming_data) ;
        }

        
        
    serial_t.close();
    }

}
 
cerving

cerving

Owner, Eggtimer Rocketry
TRF Sponsor
TRF Supporter
Joined
Feb 3, 2012
Messages
5,564
Reaction score
3,733
Assuming you're trying to create a .CSV file, you need a carriage return Serial.write(0x0d) at the end of your last data element (instead of sending a comma).
 
W

waltr

Well-Known Member
Joined
Jul 17, 2021
Messages
1,290
Reaction score
772
Location
SE Pennsylvania
cerving said:
Assuming you're trying to create a .CSV file, you need a carriage return Serial.write(0x0d) at the end of your last data element (instead of sending a comma).
Click to expand...
Correct, not sending a 'line terminator'. A 'Carriage Return' (0x0D) works or send a 'new line'
Serial3.print("\n");
 
deepfreeze

deepfreeze

Well-Known Member
TRF Supporter
Joined
Dec 3, 2022
Messages
52
Reaction score
1
Location
New Jersey, US
Guys i think i am dropping some data when sending with Xbee. I think this is the problem here other than line. Becuse when i send a small string with println. I am losing some of the characters also. My baud rate for both xbees are 115200. I am in AT mode for both coordinator and router.

DEEPFREEZEEEE DEEPFREEZEEEE EEPFREEZEEEE EEE E EEPFREEZEEEE EEPFREEZEEEE EEPFREEZEEEE EEPFREEZEEEE EEPFREEZEEEE EEPFREEZEEEE EZEEEE ZEEEE EEPFREEZEEEE EEPFREEZEEEE EEPFREEZEEEE EEE FREEZEEEE EEE EZEEEE EEPFREEZEEEE EPFREEZEEEE EE EEPFREEZEEEE
 
StreuB1

StreuB1

Well-Known Member
Joined
Aug 28, 2007
Messages
1,068
Reaction score
1,242
Location
Illinois
Lower your baud rate to 9600 and see if your comes through clear. Then, up the data rate incrementally to see if there is a point where dropoff happens. Thats a decently sized data burst.
 
Steve Shannon

Steve Shannon

Well-Known Member
TRF Supporter
Joined
Jul 23, 2011
Messages
8,643
Reaction score
7,193
Location
Butte, Montana
deepfreeze said:
Guys i think i am dropping some data when sending with Xbee. I think this is the problem here other than line. Becuse when i send a small string with println. I am losing some of the characters also. My baud rate for both xbees are 115200. I am in AT mode for both coordinator and router.

DEEPFREEZEEEE DEEPFREEZEEEE EEPFREEZEEEE EEE E EEPFREEZEEEE EEPFREEZEEEE EEPFREEZEEEE EEPFREEZEEEE EEPFREEZEEEE EEPFREEZEEEE EZEEEE ZEEEE EEPFREEZEEEE EEPFREEZEEEE EEPFREEZEEEE EEE FREEZEEEE EEE EZEEEE EEPFREEZEEEE EPFREEZEEEE EE EEPFREEZEEEE
Click to expand...
That’s not unusual in any serial communications, especially wireless at high speeds. That’s why they usually use a communications protocol with error correction and the ability to request messages to be resent.
 
Last edited:
W

waltr

Well-Known Member
Joined
Jul 17, 2021
Messages
1,290
Reaction score
772
Location
SE Pennsylvania
All correct, there can always be data lose on RF links.
A protocol for at least error detection and a method to detect beginning of a new data packet is required for reliable data link.

Go to Cris' Eggtimer website and look at his protocol for his RF telemetry.

This is just one way to do this.

Also look up NMEA which is the protocol for GPS data. They have a similar protocol with characters indicating start & stop of each data packet with a checksum in the data for error detection.

Minimum is to use a unique charater or charater sequence to indice start of data. Else if a data byte drops then the receiving code get confused and the data becomes garbage.
 
deepfreeze

deepfreeze

Well-Known Member
TRF Supporter
Joined
Dec 3, 2022
Messages
52
Reaction score
1
Location
New Jersey, US
Okay so basically each data is represent by a special char or bunch of chars. But isn’t it too much for my code to write too many if and elses? I always avoid too much if Elses. What do you think ?
 
Handeman

Handeman

Well-Known Member
TRF Supporter
Joined
Jan 18, 2009
Messages
8,411
Reaction score
1,006
Location
Stafford, VA
I think bullet proofing your code against real world effects, only make it better. Figuring our how to do that efficiently and and effectively is what makes a good programmer great. Reducing the If/Else can certainly help, but maybe it isn't avoidable. Make it great.

Good Luck.
 
W

waltr

Well-Known Member
Joined
Jul 17, 2021
Messages
1,290
Reaction score
772
Location
SE Pennsylvania
No, Each Packet of data has a special Start character. Like this:

Code: 
void sendDataTotelemetry(){
 
   Serial3.print("<");    // send start char
  Serial3.print(millis());
  Serial3.print(",");
.
.
.
.
  Serial3.print(",");
  Serial3.print(data.Acc_Sens);
  Serial3.print(">");    // send end char
  Serial3.print("\n");  // send line terminator
}

Then the Receive code looks for the "<" char to start saving data and ensures there is a ">" char that is the end of data. Else if no end )">" before next start data was lost. Reset save data to beginning and start saving again.
 
C

cls

Well-Known Member
Joined
Jan 21, 2009
Messages
3,016
Reaction score
657
You need to add a checksum, as suggested above, to protect the receiver from processing broken frames. Anything will do, like 1s complement sum.

BTW, XBee is a secret code word that means "doesn't work very well, if at all."

Agree with above post, slow down to 9600 baud. Or even 4800. With a little simple arithmetic you can calculate the rate actually needed.
 
mikec

mikec

Well-Known Member
Joined
May 9, 2009
Messages
2,859
Reaction score
643
cls said:
BTW, XBee is a secret code word that means "doesn't work very well, if at all."
Click to expand...
I've found they are very reliable once you get them properly set up, but they are tricky and poorly documented.

Which module are you using? If you are sending data into the UART faster than actual RF data rate, then dropping data is obviously inevitable.
 
R

Reinhard

Well-Known Member
TRF Supporter
Joined
Jan 18, 2009
Messages
1,329
Reaction score
634
Location
Austria
deepfreeze said:
Okay so basically each data is represent by a special char or bunch of chars. But isn’t it too much for my code to write too many if and elses? I always avoid too much if Elses. What do you think ?
Click to expand...
Your code should be as simple as possible, but it shouldn't be simpler than that. You can still be extra careful with your code review or maybe refactor it, but you can't get rid of transmission errors so you need to deal with them even if it makes your code more complex.
If you think you've got too many if-else statements on your hand, don't forget to take a look at the switch statement.

Reinhard
 
jderimig

jderimig

Well-Known Member
TRF Sponsor
Joined
Jan 23, 2009
Messages
5,086
Reaction score
2,992
Read the XBee datasheet. If using XBeePro you can enable CRC at the module itself. I wouldn't be surprised if most modern XBee's implement this. If it fails the CRC then nothing gets sent to the UART. You may drop packets, but the packets that gets sent to output you can be sure is valid.
 
Last edited:
cerving

cerving

Owner, Eggtimer Rocketry
TRF Sponsor
TRF Supporter
Joined
Feb 3, 2012
Messages
5,564
Reaction score
3,733
If you can get by with lower data rates you will improve the reliability of your transmissions, because the bit cell lengths are longer and therefore less prone to noise glitches. The more data you try to send at once, the harder it gets to reliably receive it all, which is why the larger-faster packetized schemes usually have some kind of error detection/correction, and may use a bidirectional req-ack protocol with retransmissions in the event of an error.
 
deepfreeze

deepfreeze

Well-Known Member
TRF Supporter
Joined
Dec 3, 2022
Messages
52
Reaction score
1
Location
New Jersey, US
waltr said:
No, Each Packet of data has a special Start character. Like this:

Code: 
void sendDataTotelemetry(){
 
   Serial3.print("<");    // send start char
  Serial3.print(millis());
  Serial3.print(",");
.
.
.
.
  Serial3.print(",");
  Serial3.print(data.Acc_Sens);
  Serial3.print(">");    // send end char
  Serial3.print("\n");  // send line terminator
}

Then the Receive code looks for the "<" char to start saving data and ensures there is a ">" char that is the end of data. Else if no end )">" before next start data was lost. Reset save data to beginning and start saving again.
Click to expand...
oh okay then it saves me for not having so many if elses. Thank you walter.
 
deepfreeze

deepfreeze

Well-Known Member
TRF Supporter
Joined
Dec 3, 2022
Messages
52
Reaction score
1
Location
New Jersey, US
cls said:
You need to add a checksum, as suggested above, to protect the receiver from processing broken frames. Anything will do, like 1s complement sum.

BTW, XBee is a secret code word that means "doesn't work very well, if at all."

Agree with above post, slow down to 9600 baud. Or even 4800. With a little simple arithmetic you can calculate the rate actually needed.
Click to expand...
Okay will do that. Thank you.
 
deepfreeze

deepfreeze

Well-Known Member
TRF Supporter
Joined
Dec 3, 2022
Messages
52
Reaction score
1
Location
New Jersey, US
Reinhard said:
Your code should be as simple as possible, but it shouldn't be simpler than that. You can still be extra careful with your code review or maybe refactor it, but you can't get rid of transmission errors so you need to deal with them even if it makes your code more complex.
If you think you've got too many if-else statements on your hand, don't forget to take a look at the switch statement.

Reinhard
Click to expand...
yes yes it's faster but still like having 15 different data will effect. But it's okay, i figured it out thanks to waltr. Thank you too for your suggestion.
 
deepfreeze

deepfreeze

Well-Known Member
TRF Supporter
Joined
Dec 3, 2022
Messages
52
Reaction score
1
Location
New Jersey, US
jderimig said:
Read the XBee datasheet. If using XBeePro you can enable CRC at the module itself. I wouldn't be surprised if most modern XBee's implement this. If it fails the CRC then nothing gets sent to the UART. You may drop packets, but the packets that gets sent to output you can be sure is valid.
Click to expand...
Oh that's actually is so useful. Thanks for teaching me that.
 
deepfreeze

deepfreeze

Well-Known Member
TRF Supporter
Joined
Dec 3, 2022
Messages
52
Reaction score
1
Location
New Jersey, US
cerving said:
If you can get by with lower data rates you will improve the reliability of your transmissions, because the bit cell lengths are longer and therefore less prone to noise glitches. The more data you try to send at once, the harder it gets to reliably receive it all, which is why the larger-faster packetized schemes usually have some kind of error detection/correction, and may use a bidirectional req-ack protocol with retransmissions in the event of an error.
Click to expand...
Will do, thank you
 
deepfreeze

deepfreeze

Well-Known Member
TRF Supporter
Joined
Dec 3, 2022
Messages
52
Reaction score
1
Location
New Jersey, US
Steve Shannon said:
I’m just curious. Do you know what a Case statement does?
Click to expand...
As a 25 years of computer engineer. You'd be suprised. I am just too much obsessive about coding and making it better so i try to consume all the experiences by throwing whatever comes to my mind. I don't care how i look from outside and respect other's experiences even if i am sure most of the people doesn't have that much experience like me. Long story short.. It doesn't mean i'm a noob. Keep your arroga..-curiosity to yourself.
 
Steve Shannon

Steve Shannon

Well-Known Member
TRF Supporter
Joined
Jul 23, 2011
Messages
8,643
Reaction score
7,193
Location
Butte, Montana
deepfreeze said:
As a 25 years of computer engineer. You'd be suprised. I am just too much obsessive about coding and making it better so i try to consume all the experiences by throwing whatever comes to my mind. I don't care how i look from outside and respect other's experiences even if i am sure most of the people doesn't have that much experience like me. Long story short.. It doesn't mean i'm a noob. Keep your arroga..-curiosity to yourself.
Click to expand...
I’m sorry if my question came across as arrogant. That wasn’t my intention. It really was a serious inquiry to try and understand your knowledge of C programming so I would know what I could say that might help. Anyway, it looks like you’ve got some good help.
 
W

waltr

Well-Known Member
Joined
Jul 17, 2021
Messages
1,290
Reaction score
772
Location
SE Pennsylvania
Good thread so far since the OP has gotten the help needed.

Deepfreeze,
I see you are in NJ. I'm across the river in Bucks County and my local club flys in upper Bucks.
You are welcome to come to our launches if the drive is not too far for you.

PARA520 Philadelphia Area Rocketry Association

Philadelphia Area Rocketry Association Section 520
www.para520.com www.para520.com
 
deepfreeze

deepfreeze

Well-Known Member
TRF Supporter
Joined
Dec 3, 2022
Messages
52
Reaction score
1
Location
New Jersey, US
Steve Shannon said:
I’m sorry if my question came across as arrogant. That wasn’t my intention. It really was a serious inquiry to try and understand your knowledge of C programming so I would know what I could say that might help. Anyway, it looks like you’ve got some good help.
Click to expand...
Then it's my bad. I apologize if i offended you. Yes i got good help. Thank you :)
 
deepfreeze

deepfreeze

Well-Known Member
TRF Supporter
Joined
Dec 3, 2022
Messages
52
Reaction score
1
Location
New Jersey, US
waltr said:
Good thread so far since the OP has gotten the help needed.

Deepfreeze,
I see you are in NJ. I'm across the river in Bucks County and my local club flys in upper Bucks.
You are welcome to come to our launches if the drive is not too far for you.

PARA520 Philadelphia Area Rocketry Association

Philadelphia Area Rocketry Association Section 520
www.para520.com www.para520.com
Click to expand...
Oh thank you so much waltr. I would love to come and see. It is like 1.5 hours driving. It's nothing for me. But i won't be able to launch anything yet. I will be coming for learning and meeting with you and some people who has experience. I think i need a push to launch my first and then after my first crash, i will be relaxed :) Because right now i am trying so much to make it perfect as if i really can do it in my first time. It would be great for me to experience something like that.
 

Latest posts

Top