fw2kml - A Tool to Convert Featherweight GPS Data to .kml files for Google Earth!

[kjhambrick]

lol, I'm a mac user luckily. I've been using PyQT5...

@Aeva --

I've not dug around yet but ...

Q: Do setup.py and your new GUI interface run on OSX ?

If so, is this everything I need from github ? github > kayla-tokash / fw2kml ( forked from pirate21213/fw2kml )

Thanks !

-- kjh

 

[Aeva]

Q: Do setup.py and your new GUI interface run on OSX ?

If so, is this everything I need from github ? github > kayla-tokash / fw2kml ( forked from pirate21213/fw2kml )

The new GUI runs on OSX, linux and windows. "setup.py" is for the py2exe building, which I updated but haven't tested. That's strictly for windows. Eventually I'd like to see how to put the script in an app package so it can be installed for osx.

To set it up, you can do "pip install -r requirements.txt" and from then on just run "python gui.py" from the terminal in the checkout directory and that will start the gui.

github is up to date with the GUI changes; make sure you pull the "DragDropGui" branch. The readme needs updates, but the only thing I haven't uploaded was refactoring the xml writing to use the element tree class instead of file read/writes.

E: Added my changes for the elementtree stuff. I gotta gut out the old fileIO, but it works now anyway. Also made it so it randomly generates the color rather than using predefined styles.

Another idea I had was to merge csv's into one kml and leave a checkbox to enable/disable that option.

 

[Aeva]

I haven't flown in a while, and I've had no problem with this conversion tool in the past. However, I just tried to convert a file from a friend of mine that flew last weekend and am having no luck. Comparing his recent file to one of my older files, it appears the number of columns and headers have changed. The newer file contains additional information as well. I believe this file may have been transfered from the newer Android Blue Raven app rather than the iFIP app.

The newer file is attached.

Updated to support blue raven [app] exported files. Let us know if there are issues. I'm gonna put in a pull request to @Arpak's repo tonight so that the main track is updated. The changes are on my branch for now.

I need to double check it supports multiple flights on blue raven [app] data. There is also a bug with the colors because it's not padded with 0's lol... so I'll fix that.

E: fixed the colors, and it does support multiple flights from the look of it


I'm assuming that the pink track was the walk to the launch pad.

 

[kjhambrick]

@Aeva --

When you say Blue Raven, do you mean GPS Data downloaded via the Featherweight Blue Raven Android / iPhone App ?

Or are you doing magic with the high rate quaternion data from the Blue Raven Altimeter ?

I want THAT !

Thanks !

-- kjh

 

[Aeva]

@Aeva --

WHen you say Blue Raven, do you mean GPS Data downloaded via the Featherweight Blue Raven Android / iPhone App ?

Or are you doing magic with the high rate quaternion data from the Blue Raven Altimeter ?

I want THAT !

Thanks !

-- kjh

Downloaded via app.

Heh, wouldn't even know where to begin with the quaternion data. At the very least, I'll look into that because I probably should have some understanding of it for my own flight computer.

 

[kjhambrick]

@Aeva --

If you want to look ...

I was thinking that if one recorded the GPS coordinates, (lat,lon) of the pad and then the (lat,lon) where the rocket landed.

One might be able to process the Blue Raven Altimeter high rate quaternions from launch to apogee and then MAYBE take a stab at the recovery phase track from the (lat, lon, altitude) calculated for apogee and altitude -vs- time back down to the landing spot.

Maybe ?

These the positions of the pad and the tree where I landed:


The Blue Raven files are attached if you want to play with them ( quaternions are in the high_rate file )

Have fun !

-- kjh

EDIT: appended launch parameters file

 

[Aeva]

Nice! these will help a ton!

I haven't worked with quaternions yet; it looks like a lot linear algerbra and vector math. We'll see how much I remember from college lol.... might have to take a crash course refresher.

 

[kjhambrick]

Nice! these will help a ton!

I haven't worked with quaternions yet; it looks like a lot linear algerbra and vector math. We'll see how much I remember from college lol.... might have to take a crash course refresher.

I've wanted to work on this 'little' problem myself ( as in a math problem ) but my job is is my way.

Unlike you, I would need more than a refresher ... I would be starting over after all these years ( use it or lose it ).

Anyhow, there are quite a few quaternion libraries out there for the taking ...

If you need more data sets, I've got four more of them like the one I posted.

One thing ... in this rocket, the Blue Raven is 'inverted' in the Av-Bay so the raw x-axis acceleration and gyro data in the high_rate file are inverted ( up is down ) and tilt and roll in the low_rate file appear to follow the signs of the gyro readings.

The Blue Raven auto-detects the orientation and compensates so that it does all the right stuff, and then it stores raw value in the files ( as it should ).

-- kjh

EDIT: p.s. I have to say that the data from the Blue Raven is first class. Anyone working on a flight computer would do well to try to match it's capabilities ...

 

[Aeva]

Guess what? now you get pins for several event locations. Changes committed to the DragAndDrop branch.

~Stealth edit~


This was hard to crack, but now it shows pretty much all significant events for single stage rockets (experimentally). Truthfully, it depends on the quality of the data and the configurations in the library... for example, less impressively:


It's really just a best guess. I'll have to check what data is available in the csv and use that to improve it.

 

[thzero]

I haven't flown in a while, and I've had no problem with this conversion tool in the past. However, I just tried to convert a file from a friend of mine that flew last weekend and am having no luck. Comparing his recent file to one of my older files, it appears the number of columns and headers have changed. The newer file contains additional information as well. I believe this file may have been transfered from the newer Android Blue Raven app rather than the iFIP app.

The newer file is attached.

This looks like its got repeated set of data in it. This is how it came out of the Blue Raven app?

 

[thzero]

This looks like its got repeated set of data in it. This is how it came out of the Blue Raven app?

@Adrian A
Looks like this has a repeating set of data at row 2044?

 

[Aeva]

@Adrian A
Looks like this has a repeating set of data at row 2044?

there were a lot of repeats. I ended up adding a filter to check for duplicate time indexes

? num_lines=$(cat ~/Downloads/test\ data/Log_03-20-2024_07_18.csv | wc -l); num_lines_uniq=$(cat ~/Downloads/test\ data/Log_03-20-2024_07_18.csv | sort | uniq | wc -l); echo "$num_lines lines in file, and $num_lines_uniq after uniq filter"
    4086 lines in file, and     1961 after uniq filter

Using bash, it looks like nearly every datapoint was repeated.

Maybe these were retransmits?

 

[shockie]

View attachment 555198

Hi all,

It's been a bit since I posted on TRF, been catching up on life (I graduated and got a big boy engineering job!). I had a discussion last night with my buddy Adam and he mentioned to me that he wished he could view his featherweight GPS data in Google Earth, apparently that's not a released feature yet. Turns out its not too difficult to convert it, but it would be incredibly tedious by hand.

So I made a python script! I've been calling it fw2kml (FeatherWeight To(2) KML), and it seems to work fairly well. It turns this....
View attachment 555201

Into this!
View attachment 555200

Right now its feature set is brief but as follows;

- Drag a featherweight .csv data file onto the script to generate a [filename]_fw2kml.kml file of that data
- Automatically filters out "bad" data (altitude is below ground)
- Nifty icon

I'd like to maintain this for a bit until it gets stable enough to work in practically every scenario with the featherweight, but I need your help for this. If you own a featherweight GPS, please try this tool out and let me know if you hit any snags.

How to Use

1. Download the "fw2kmldrag.exe" file from the link below
2. Drag a Featherweight .csv data file onto the .exe, it will automatically generate the output .kml and place it in the same location as the .exe
3. Open Google Earth Pro (its free)
4. Go to File>Open
5. Select the generated .kml file
6. Gawk and Awe
7. Upload a screenshot of your flight here!

Download

I've made the source code and built .exe file available on github, for you code-savvy folks out there its written in python and generated into a .exe using the py2exe library. To download the tool, go to the following link and download "fw2kmldrag.exe".

https://github.com/pirate21213/fw2kml/releases/tag/Latest

Note: You may/will probably get a warning that the file is malicious. This is because it doesn't have proper certificates and whatnot and for all intents and purposes looks to windows like a virus. I can assure you, its not. The source code is up. Its basically a highschool CompSci project. Don't believe me? I guess don't use it... This is the bulk of the logic.

    outfile_name = str(droppedFile).replace('.csv', '_fw2kml.kml')
    coords = []
    for row in rows:
        if int(row[5]) <= 0:
            print("Bad data, skipping", row[5])
        else:
            coords.append("{},{},{}".format(row[4], row[3], (int(row[5]) * 0.3048)))

    coordstring = ' '.join(coords)

If you run into an issue using the file, let me know what happened! I'd love to catch all the strange edge cases and make this fairly straight forward.

Happy flying!
-Patrick

This is fantastic Patrick!

 

[Adrian A]

there were a lot of repeats. I ended up adding a filter to check for duplicate time indexes

? num_lines=$(cat ~/Downloads/test\ data/Log_03-20-2024_07_18.csv | wc -l); num_lines_uniq=$(cat ~/Downloads/test\ data/Log_03-20-2024_07_18.csv | sort | uniq | wc -l); echo "$num_lines lines in file, and $num_lines_uniq after uniq filter"
    4086 lines in file, and     1961 after uniq filter

Using bash, it looks like nearly every datapoint was repeated.

Maybe these were retransmits?

The latest version of the app that I'm testing now, downloads the tracker data cleanly even when it has to re-transmit or the download is interrupted for any reason. It should be available in the next few days after we clean up a couple more bugs in other areas.

 

[Adrian A]

The latest version of the app that I'm testing now, downloads the tracker data cleanly even when it has to re-transmit or the download is interrupted for any reason. It should be available in the next few days after we clean up a couple more bugs in other areas.

The new and improved version 1.08 build 300 is available now through the Apple app store (search for Featherweight UI) and corrects a number of bugs, particularly in the tracker recorded data downloading. The Android version is almost ready also, but there's an Android-specific voice selection issue we're working on.

 

[CameronMakesRockets]

So I made a python script! I've been calling it fw2kml (FeatherWeight To(2) KML), and it seems to work fairly well. It turns this....

Are these the CSV headers needed for fw2kml to work? I am working on my own data logger and this tool would be a great way to visualize the launch later!
If there are any more headers needed, what are they? Are they all required?

 

[Aeva]

Are these the CSV headers needed for fw2kml to work? I am working on my own data logger and this tool would be a great way to visualize the launch later!
If there are any more headers needed, what are they? Are they all required?

It won't work because of that date field. I can modify it later to be more compatible. The headers should work fine otherwise.

If the "time" field is not in UNIX time format, it tries to create the UNIX time string using the date field and time field. It would try and fail on this file. What you could do to "hack" it is just set the date field to a random but consistent date for every entry (01/02/3004). The actual time is not reflected by this tool, only relative time. This should be a viable workaround at least for my branch of fw2kml.

E: If you want to export data to CSV then run it into fw2kml, it supports data formatted with "LAT", "LON", "ALT", and "UNIXTIME" or "DATE" + "TIME" fields; it also supports "TRACKER Lat", "TRACKER Lon", "TRACKER Alt asl", and "DATE" with "TIME".

@Arpak you still around? I put in the pull request a little bit back but never heard from you or saw a notification it was merged into your repo.

 

[Arpak]

It won't work because of that date field. I can modify it later to be more compatible. The headers should work fine otherwise.

If the "time" field is not in UNIX time format, it tries to create the UNIX time string using the date field and time field. It would try and fail on this file. What you could do to "hack" it is just set the date field to a random but consistent date for every entry (01/02/3004). The actual time is not reflected by this tool, only relative time. This should be a viable workaround at least for my branch of fw2kml.

@Arpak you still around? I put in the pull request a little bit back but never heard from you or saw a notification it was merged into your repo.

Still around, real life just grabbed hold of all of my time. I saw the PR, just need to open a slice of time to review and merge it.