Immersive RC Oculus Rift FPV Cockpit for my Cruise Rockets

Not too long ago I posted a description thread for my "twin" (big sis and little sis) cruise rockets detailing how they operate.
This weekend, weather permitting, (fingers, arms, next and bones crossed!!) I will be trying out something quite interesting, myself serving as a guinea pig in my own experiment.

The larger of the two cruise rockets, Condore X11E, is already flight proven for FPV controls. From liftoff to parachute deployment, I am basically piloting it using a first person view video feed.
At my very first time using FPV, joystick & rudder I noticed it's about five times better than line of sight RC - at least for me - feet on rudders and hands on a stick is way more intuitive than the video-game-like RC controllers. I kinda suck flying with my thumbs.


First a shot of the old ground control setup (will be almost the same, just add me with an Oculus Rift on my face), then the bird, a shot of the cockpit during testing, where I used my webcam as video input, just to test and demonstrate and finally one where I was trying out the cockpit with an FPV recording played back on the VR screen.

Now, from above pic, I want more immersion, and I want to AVOID the downsides that come with a screen on the field: glare and reflections, requiring this huge shroud over the laptop to prevent such things.
Given I have an Oculus Rift dev kit and I'm a programmer, I figured I'd expand my skills beyond web & c++, venturing into 3D environment development using Unity3D.

I've seen other Oculus demos for FPV, but most are terrible: Why? Because "slapping the video onto the display" is not the way to adapt content for the Oculus Rift. If the content you present the player/user is not coordinating with head movement, said user will vomit after 10 minutes of use, not to mention fall into a long VR sickness stint that can last hours. The best VR demos I've seen so far are one where your movement fully interacts and fully coordinates with the VR 3D environment.

My FPV, given the fast motion of a rocket, would need to offer two things: Since the FPV image is not stereoscopic 3D, which is a must, must, must in the Rift, again to prevent motion sickness, I needed a 3D environment within which a large screen would exist. On this screen, the FPV video would be projected. Think of it as a single person movie theater. Now I would also add elements within the VR environment to provide the user a separation between themselves and the FPV feed. This separating environment would then be fully coordinated with head movement just like any properly developed VR app.

The result is so far extremely encouraging. Watch the video below. At first I open with a quick explanation of the app, show a very incomplete use scenario, and by the halfway mark, I show the candy: The VR cockpit plays back some recorded FPV video from the very cruise rocket I intend to fly with this. The result is a comfortable viewing environment that will inherently eliminate all reflections and glare issues, provide maximum distraction free immersion and allow me full authority over the aircraft.
Not seeing the controls themselves will not be an issue since I have rehearsed this scenario in actual flying games (War Thunder) and driving simulators (iRacing) and have been surprised by how I simply always have my hands in the right place and press the correct buttons without seeing said controllers.
[video=youtube;LOkYWC8Ks2I]https://www.youtube.com/watch?v=LOkYWC8Ks2I[/video]

The only catch? And I am indeed rehearsing this. I need to look directly at the rocket at liftoff. I keep her straight well that way. I'll hold the rift away from my face, extending the head strap to see the rocket directly at ignition, and as soon as I pass 1s after burnout, I'll slap the Rift on and get to business. All this will be recorded, so I should be able to report the performance here. One thing I know is the laptop's GPU is being stretched to its very upper limits by this. VR requires a card capable of rolling out 120 monoscopic frames per second at 720p ... the very minimum (resulting in 60fps stereoscopic). Any less, and your user gradually feels sick.

Down the line, this is SCREAMING for a 180 degree stereoscopic camera. Then I can allow the VR pilot to "VR-look-left-and-right" and the stereo 3D would go a long way into immersing the pilot ten times more.

If the video comes back with me leaning over a puke bucket, time to re-think the cockpit or get a better GPU. I'm able to spend 4 hours straight in iRacing - I'm not too worried, but who knows.

Is it legal ?

I doubt the stereoscopic view from glider altitudes would give you much depth perception.

You could save gpu frames by sharing the video stream between the two eyes since the stereo separation at the distances you'd be looking at would be less than a pixel. I think that's been done before.

...................

MarkedOne said:

Is it legal ?

Click to expand...

If the vehicle weighs not more than 1500 grams and the rocket motors contain not more than 125 grams of propellant, then he's probably legal.

If it's heavier than 1500 grams or has more than 125 grams of propellant, he needs a written FAA waiver and must activate the waiver before he launches to be legal.

Bob

..................................

TopRamen said:

... Section 922r, which states that under Active Guidance, a Rocket of any size is classified as a Guided Missile, and thus is not lawful, regardless of whether or not it carries a Payload or Warhead.

Click to expand...

You're joking, right?

-- Roger

degreaser said:

Bottom line, I'd talk to a lawyer

Click to expand...

There's no need to talk to a lawyer when there are so many people on the internet willing to offer legal opinions.

Rocketry hobbyists and model airplane enthusiasts have been flying radio-controlled, rocket-powered aircraft for years. The "active guidance makes it a weapon" stuff is pure nonsense.

-- Roger

degreaser said:

I'm not one of them. I said, "talk to a lawyer".

The problem is there's so much wiggle room that it can be interpreted either way.

Click to expand...

No, there isn't. Here's a post from the last discussion like this:

I searched the US Code for the word "rocket" and found 18 hits. Only two of the matches seemed remotely applicable. First there is the definition of a "Missile System Designed to Destroy Aircraft":

The second is the definition of a "Destructive Device":

The term “destructive device” means

(1) any explosive, incendiary, or poison gas

(A) bomb,
(B) grenade,
(C) rocket having a propellent charge of more than four ounces,
(D) missile having an explosive or incendiary charge of more than one-quarter ounce,
(E) mine, or
(F) similar device;

(2) any type of weapon by whatever name known which will, or which may be readily converted to, expel a projectile by the action of an explosive or other propellant, the barrel or barrels of which have a bore of more than one-half inch in diameter, except a shotgun or shotgun shell which the Secretary finds is generally recognized as particularly suitable for sporting purposes; and

(3) any combination of parts either designed or intended for use in converting any device into a destructive device as defined in subparagraphs (1) and (2) and from which a destructive device may be readily assembled.

The term “destructive device” shall not include any device which is neither designed nor redesigned for use as a weapon; any device, although originally designed for use as a weapon, which is redesigned for use as a signaling, pyrotechnic, line throwing, safety, or similar device; surplus ordnance sold, loaned, or given by the Secretary of the Army pursuant to the provisions of section 4684 (2), 4685, or 4686 of title 10 of the United States Code; or any other device which the Secretary finds is not likely to be used as a weapon, or is an antique or is a rifle which the owner intends to use solely for sporting purposes.
​

Note, in both cases, devices that are not intended to be weapons are excluded from the definitions.

-- Roger

.............................

degreaser said:

Yes, I mentioned this paragraph in post 8.

Your interpretation may be correct. I'd still CmYA though and get some legal advice before pursuing this further. It's very cool but with all the morons flying FPV quads out of control between skyscrapers, it would be wise to be cautious.

Click to expand...

Consulting a lawyer to ask about something hobbyists have been doing for years seems silly.

NAR even has a special version of the safety code for rocket gliders and contests for them.

jadebox said:

Consulting a lawyer to ask about something hobbyists have been doing for years seems silly.

NAR even has a special version of the safety code for rocket gliders and contests for them.

Click to expand...

Jadebox, TopRamen - Let me make a suggestion. Let's collectlvely pull our posts so this poor guy can have his thread back. We can carry this conversation elsewhere. What do you think?

Actually this is all up to speculation: There are full lines of aircraft that lift off, fly and operate under full control from end to end.
RC groups have fully RC rocket powered aircraft.

This is not a guidance system it's manual piloting. The word "guidance" involves a rocket with a brain. A gimbal. Autopiloted surfaces.
Furthermore the goal during ascent is to remain straight. Horizontal flight is not forced, it's simply controlled after the apogee.
Let's not forget that the "guided" law also involves a TARGET; my only "target" is to bring her down in one piece and not have 500$ in damages, so I have no target.

This is a stable rocket even in its uncontrolled form primarily, with the ability to be flown thus improving stability during ascent and allowing control for the recovery.
Not to mention touch down is still on a parachute.

I will likely leave my post up. I don't have any problems with speculation like this, it's false. I had a TRA RSO approve these birds today once more.

MarkedOne said:

Is it legal ?

Click to expand...

Yes. 100% approved by one TRA RSO and one CAR RSO so far. I am a longstanding member with both clubs and know both their rules sets, none of which these birds break. More to come, always a little explaining to do but both birds always end up getting right to pad... so to be the legality discussion is a lot of white noise in the background.

jadebox said:

You're joking, right?

-- Roger

Click to expand...

Yeah he is. He has to be

jadebox said:

Note, in both cases, devices that are not intended to be weapons are excluded from the definitions.

-- Roger

Click to expand...

Every RSO I've faced with this, I asked about the whole "wiggle room" thing and they giggle at me and go "So are you going to point at someone you don't like and charge them?" Obviously a burst of laughter explodes, I go "No" and the RSO goes "So you're not a guided missile"... and then follows a lot of explaining, because the aircraft does have a few unique but fully legal aspects to be inspected in a particular way.

I just need to put a cherry on top this sundae and show how ridiculous it even is to have this legality discussion:
https://www.rocketryforum.com/forumdisplay.php?44-Rocket-Boosted-Gliders
There's a bloody Rocket Booster Glider section on this forum. They're at least half RC, most of them without chutes. Rocket power end to end with control.
My assembly is no different, just it bares the profile and caliber of a rocket while having RC capabilities, thus joining the two best of both worlds. She uses a standard launch rail instead of those crazy double rail launchers for traditional rocket gliders. She flies straight up, which is safer and she's structurally sound enough for HPR. This conversation is pure ridiculous, absurd and a total waste of time.

If we want to have a conversation, let's have a more fun one, PLEASE.

It isn't a waste time. Among other things, a TRA BoD member indicated that they had a lengthy discussion regarding guided flights at the last BoD meeting and said it's "dicey" and that TRA lacks formal rules on this. I think we need some rules in place just to be on the right side of things.

Anyway, in your PM you said you had some flights. Let's see some data and video!

You said you have a dev kit. Any idea when the production versions will be available and if they are going to do something with the size of the goggles? Fatshark Dominator HDs will be available next year sometime but their resolution won't come anywhere near that of Oculus Rift.

degreaser said:

You said you have a dev kit. Any idea when the production versions will be available and if they are going to do something with the size of the goggles? Fatshark Dominator HDs will be available next year sometime but their resolution won't come anywhere near that of Oculus Rift.

Click to expand...

Heh, Fatshark claims "best resolution" and "best field of view"...of production HMD's. Hah. 45 degree diagonal won't give anywhere near the immersion of an Oculus Rift.

Nobody has a clue when Oculus Rift production versions will come out. Apparently the panels they planned to use got phased out as cellphone screen size and resolution changes. Hopefully that settles down soon.

Also, the games need to reach a certain level of implementation maturity before the general public will actually enjoy using VR goggles over a standard TV or monitor.

degreaser said:

It isn't a waste time. Among other things, a TRA BoD member indicated that they had a lengthy discussion regarding guided flights at the last BoD meeting and said it's "dicey" and that TRA lacks formal rules on this.

Click to expand...

As far as I know, TRA doesn't have a separate code for rocket gliders like NAR does. While the OP's rocket isn't a glider, it does share a common trait - active remote control during boost. I can understand that the TRA board might be concerned about the safety aspect of that. After all, a more traditional rocket is likely to be moving a lot faster than a typical rocket glider during boost and afterwards. But, that's a far cry from suggesting that the OP is doing something illegal.

-- Roger

CarVac said:

Also, the games need to reach a certain level of implementation maturity before the general public will actually enjoy using VR goggles over a standard TV or monitor.

Click to expand...

And the size needs to be reduced a lot. I don't want whiplash when moving my head

HD over a radio link is not easy or cheap. Those are two hurdles to overcome before something like OR will be practical for FPV. As it is, we're still looking at standard def 4:3 coming from the FPV feed and the FOV of that feed probably isn't much more than 45 degrees. I suppose you could use the space on the sides to display telemetry until the technical and $$ issues are resolved.

There's also the direct retinal projection systems which show promise and they aren't as cumbersome. I'm afraid to see the $$$$ pricetag though.

https://techreport.com/news/25505/virtual-retinal-display-beams-images-onto-your-eyeballs

jadebox said:

As far as I know, TRA doesn't have a separate code for rocket gliders like NAR does. While the OP's rocket isn't a glider, it does share a common trait - active remote control during boost. I can understand that the TRA board might be concerned about the safety aspect of that. After all, a more traditional rocket is likely to be moving a lot faster than a typical rocket glider during boost and afterwards. But, that's a far cry from suggesting that the OP is doing something illegal.

-- Roger

Click to expand...

I think it won't be long until we have some rules...

degreaser said:

HD over a radio link is not easy or cheap. Those are two hurdles to overcome before something like OR will be practical for FPV. As it is, we're still looking at standard def 4:3 coming from the FPV feed and the FOV of that feed probably isn't much more than 45 degrees. I suppose you could use the space on the sides to display telemetry until the technical and $$ issues are resolved.

Click to expand...

It'd be perfectly simple to start with a fisheye image from the aircraft and have the computer defish it into a 180 degree hemispherical view (although with reduced resolution at the sides).

CarVac said:

It'd be perfectly simple to start with a fisheye image from the aircraft and have the computer defish it into a 180 degree hemispherical view (although with reduced resolution at the sides).

Click to expand...

It's still standard def though. Times 2 for stereoscopic vision. I'm curious what Bud is using for his FPV radio link. You could interlace to save bandwidth.

degreaser said:

It's still standard def though. Times 2 for stereoscopic vision. I'm curious what Bud is using for his FPV radio link. You could interlace to save bandwidth.

Click to expand...

I pointed out that unless you do your flying really close to the ground, stereo won't do much. Especially with low resolution. A single video feed would save bandwidth, and also allows tricks to reduce the rendering effort for the GPU.

CarVac said:

I pointed out that unless you do your flying really close to the ground,

Click to expand...

You mean, like landing?

Yes, I know he's using parachutes but thinking beyond to real gliders (or other RC aircraft for that matter) that actually land, having depth perception would be of great benefit.

That is pretty cool. Nice job. I keep thinking something like google glass would be really cool to use to track your rocket.

degreaser said:

I think it won't be long until we have some rules...

Click to expand...

That normally means it won't be long until it's illegal. I hope the people behind these rules aren't the uninformed "knee jerk reaction of refusing when servos are spotted" kind of people.

Derek said:

That is pretty cool. Nice job. I keep thinking something like google glass would be really cool to use to track your rocket.

Click to expand...

Would be cool as an AR HUD, but this is fully immersive VR; I don't think wearing both sets would be possible LOL
I do plan on having the display panels in the VR environment eventually display relevant telemetry.

As for tracking in this case it's more like not flying too far, staying above the killzone (or returning to it if you're on a high, long loitering flight) and making sure you're recording your FPV video stream in case you need the last known GPS coords.

GG could be used to pinpoint your rocket in the sky as you try to find it with your eyes (HUD to mark where to look if you're not turned toward it, and a reticule when you do spot it)