how to structure flight data

[deepfreeze]

Guys i am building a model rocket that has MPU6050, BMP280, a voltage divider to read batt voltage, flash chip and microSD card reader.

Everything is working fine. But when i get the data from all the sensors and also passing between functions, i am using ArduinoJson Library. So i structure all data into Json and pass it thorough functions and also to the outside to my simulator.
As a programmer, i have a urge to structure everythin in Json. But is it a good structure to have ?

I have a simulator that i wrote in python to show rocket position in 3D and other values. But i feel like it is making it slow little bit. Does this effect the performance?

For example instead of structuring the MPU6050's data as below, should i just give it as ypr value next to each other or some other way. Could you please illluminate me on that ?

  DynamicJsonDocument doc(1024);
  doc["CODE"] = "MPU_DATA";
  doc["yaw"] = 0;
  doc["pitch"] = pitch;
  doc["roll"] = roll;
  doc["x_accel"] = 0;
  doc["y_accel"] = 0;
  doc["z_accel"] = 0;

 

[waltr]

The way I do this is a simple comma separated values. First I write a header then at each time step write the data.

Example of the first few lines of a data file. The columns are:
time(usec), accel X, y, z, Gyro X, y, z, Magnet X, y, z, Baro Temperature, pressure, Altitude

t,AX,AY,AZ,GX,GY,GZ,T,mX,mY,mZ,BT,BP,BA
412817,1.638,9.697,-1.743,0.005,0.047,0.013,29.089,-0.263,-43.628,14.104,27.412,1012.666,88.219
412837,1.987,9.697,-1.935,-0.001,0.02,0.002,29.109,-0.132,-43.671,14.82,27.412,1012.666,88.288
412857,1.614,9.687,-2.145,-0.003,-0.004,0.003,29.098,-0.526,-43.54,15.054,27.419,1012.658,88.218

I use the file extension .csv which then opens in Excel and can be easily read with Python a text editor, etc.
The data is ASCII text but each data doesn't have the extra 'name' which takes time to write, read and uses memory.

In Python open the file then tot read it:

import csv

j = 0
with open(in_name) as csvfile:
    readCSV = csv.reader(csvfile, delimiter=',')
    for row in readCSV:
      try:
          float(row[0])         # is frist data a number value or text?
      except ValueError:
          print(row[0])         # not a value
      else:
            time_stamp[j] = ( float(row[0]))

            # Acc data in m/s/s, Read and calibrate Accelerometer data
            acc_data[j] =   ( (float(row[1]) -Acc_offset[0]) *Acc_gain[0],
                              (float(row[2]) -Acc_offset[1] -9.81) *Acc_gain[1],   # subtract Earth from vertical axis
                              (float(row[3]) -Acc_offset[2]) *Acc_gain[2])
   ....... and the other data values

    j = j+1   #next time index

Json is good when the information is entered into the file in any order. The Json functions then lnow how to parse out of the file the desired data. this takes code time to do.

Since you know the exact order the data is written Json is not needed plus it makes the files much larger.
The .csv data files I get during a flight can be over 300kB. I write to an on board flash memory that is shared so do have limited storage space. Since you have a huge SD card space is not a concern
Another point is the time it takes to Write the data. This is important to you since you really want the data sampled and written 50 to 100 times a second.


Was this helpful?

 

[tHoagland]

^^^ Everything he said. I love JSON files on a PC with "unlimited " RAM and no time requirements. For a time sensitive application like yours, I'd try a CSV file and fall back on a raw binary format if needed (and then write a desktop utility to convert that to JSON or CSV)

 

[deepfreeze]

Okay i will be recording the data into flash and sd card as CSV. Thank you guys. It is working better now.