PTZ (Pan tlilt zoom) DSLR camera tracker with liftoff detection project

Discussion in 'Rocketry Electronics and Software' started by curtisheisey, May 17, 2019.

Help Support The Rocketry Forum by donating:

  1. May 17, 2019 #1

    curtisheisey

    curtisheisey

    curtisheisey

    Forum Supporter TRF Supporter

    Joined:
    Jan 27, 2010
    Messages:
    98
    Likes Received:
    17
    In my quest for ever better photos of launches, I have decided to take on a gimbal DSLR camera mount and rocket tracker project. It would have standoff detection of rocket launch and track the rocket for the first 1000 feet. I plan to initially use a 28mm lens at about 50 feet standoff. Eventually I would like to work automated track and slew. Finally, tie in zoom control of a DSLR with a telephoto, but that is a ways down the road.

    I would use a raspberri pi for the computer/controller.

    I figure this would be in several phases.

    1) Build camera gimbal
    I found this wonderful gimbal on thinguniverse https://www.thingiverse.com/thing:3375167
    I probably need to beef up the plastic gears with metal ones. I estimate the initial vertial slew rate is going to be 30 deg/se to 60 deg/s.

    2) Create liftoff detection and use canned slew profiles. Collect data and develop tracking algorithms.
    I could use the Lightware SF30 range finder. This has a high update rate and a 50 meter range. I could either point it at a fin, or just about the launch rail.
    Adafruit has an 8x8 pixel IR camera, but I don't know if it has has the range. I could use a blow torch at 50 feet as a surrogate. Also, the update rate is not great (10/s).

    3) Incorporate tracking algorithms.

    There are some good open source video analytics packages - openCV, Processing, etc.

    4) Work on DSLR zoom


    This will keep me out of trouble for the next year. Ideas and comments welcome.

    upload_2019-5-17_7-58-4.png
    upload_2019-5-17_7-58-21.png
     

    Attached Files:

  2. May 17, 2019 #2

    dhkaiser

    dhkaiser

    dhkaiser

    Well-Known Member

    Joined:
    Nov 27, 2014
    Messages:
    406
    Likes Received:
    5
    I would guess that a distance of 50 ft a pan rate of 60 deg/sec would be insufficient.
     
  3. May 17, 2019 #3

    rharshberger

    rharshberger

    rharshberger

    Well-Known Member

    Joined:
    Oct 13, 2014
    Messages:
    8,083
    Likes Received:
    663
    Gender:
    Male
    Location:
    Pasco, WA
    +1, it would be better to use a telephoto lens at a greater distance, but a shorter lens should work well enough for testing purposes. I am definitely interested in following your progress on this project.
     
    anbhtblr likes this.
  4. May 18, 2019 #4

    Xrain

    Xrain

    Xrain

    Well-Known Member

    Joined:
    Nov 26, 2013
    Messages:
    279
    Likes Received:
    16
    Honestly, I'm not sure the IR tracking method would be the best. You could easily do the same function using a regular camera, and just looking for a bright spot in the frame which would be the plume. This would also let you use much less expensive higher resolution cameras. The only 8x8 pixels of that IR sensor is going to make it pretty hard to keep the tracking object in frame. Better yet, you can just use the video output of the DSLR itself to point the tracker.

    You basically have two main choices on how to track an object.
    The first and straightforward method, is to have a tracker on the rocket itself, which sends the position of the rocket over RF down to your optics mount. Then all you need is a GPS coordinate of the mount itself, and then its pretty simple math to get pointing angles for your mount. Then some basic PID or other form of closed loop control and then you just need to tune things in to get it working right.

    The other method would be by computer vision. There has been a huge amount of development in this area in recent years. So there are very likely some very good well behaved programs around that will do a vast majority of the work on this for you. I know at the very least there was an optically tracking paintball turret program around 10 years ago. The computer vision solution would let you just track based off an outline, the plume, or a bright paintjob on the rocket. With very good results.

    If you get fancy you can combine the two options I mentioned. Using the RF telemetry to get a general pointing angle, then using the computer vision to center the rocket in the view.
     
  5. May 18, 2019 #5

    rharshberger

    rharshberger

    rharshberger

    Well-Known Member

    Joined:
    Oct 13, 2014
    Messages:
    8,083
    Likes Received:
    663
    Gender:
    Male
    Location:
    Pasco, WA
    There is/was a camera tracking system that used a RF fob to track something like a runner, cant remember the brand name, but it was determined in an old thread it would be most likely to slow in pan/tilt to work for rockets.
     
  6. May 18, 2019 #6

    curtisheisey

    curtisheisey

    curtisheisey

    Forum Supporter TRF Supporter

    Joined:
    Jan 27, 2010
    Messages:
    98
    Likes Received:
    17
    FYFRODGI7IJ9TU6.LARGE.jpg The beacon tracking has been suggested to me by several people. I would like some configuration that just uses video tracking, so that I can point it to anyone's rocket. Maybe use the beacon for my own when video gets out of range.

    I was thinking that the thermal camera could be used for liftoff detection. However, the update rate is not great (10/s). Simple video processing might suffice, even just looking at frame to frame differences, and the having some threshold algorithm. There are several good open source video processing packages out there.

    I'll probably need two set-ups eventually. Wide-angle and close-up, and telephoto long range.

    I did find another pan/tilt design that may be more suitable. https://www.instructables.com/id/Motorized-panorama-and-timelapse-pan-tilt-sytem/ . I could connect the drive of the servo right to the camera mount. For the close-up verision, I probably don't need a pan axis. Just tilt.
     
  7. May 18, 2019 #7

    cwbullet

    cwbullet

    cwbullet

    Obsessed with Rocketry Staff Member Administrator TRF Lifetime Supporter Global Mod

    Joined:
    Jan 24, 2009
    Messages:
    21,672
    Likes Received:
    1,132
    Gender:
    Male
    Location:
    Glennville, GA
    I am eager to see if this works. This could a great project.
     
  8. May 18, 2019 #8

    UhClem

    UhClem

    UhClem

    Well-Known Member

    Joined:
    Feb 6, 2009
    Messages:
    1,415
    Likes Received:
    45
  9. May 18, 2019 #9
  10. May 19, 2019 #10

    Xrain

    Xrain

    Xrain

    Well-Known Member

    Joined:
    Nov 26, 2013
    Messages:
    279
    Likes Received:
    16
    The GPS TM over RF is nice because there is a lower programming threshold. But on the other hand, it takes a lot more set-up for it to work. As well as you lose pointing angles if the GPS's max velocity is exceeded or you lose TM lock.

    The thermal camera I am guessing will not save you much work from implementing with a normal camera. If it is hot out, it's likely the heat from the motor will get washed out by the ambient temps, especially with the low resolution of that IR sensor. If you had one that was like 320x280 or in that neighborhood you stand a better chance of seeing it. Other issue is, when the plume starts, there is a very large ball of hot gas that goes everywhere after the plume reflects off the ground. So if you are trying to track off of it, you will have a hard time of it.

    So I think the launch detect with optical means than with IR. You could always trigger it manually if it is a big problem, and just have the camera pan up 5 or so degrees at a fixed rate until it starts the video processing and searches for the rocket.

    I don't really like that second mount that much. Seems like it would have bad wobble. If you can, you generally also want a counterweight on your mounts. Which will balance out the mass of your camera/lens/mount. This makes things easier on your motors, and in tuning your PIDs, as the required force isn't varying as much with angle. You can always gain schedule it with respect to pointing angle I suppose; to make up for that type of variance.
     
  11. May 19, 2019 #11

    jadebox

    jadebox

    jadebox

    Roger Smith

    Joined:
    Jan 18, 2009
    Messages:
    5,398
    Likes Received:
    41
    This is another of those projects that I have been brainstorming for many years. As with most of the other grand ideas, I decided that this is beyond by knowledge and skill levels. But, here are some thoughts I had that may be useful.

    I was considering processing the real-time video output. That has gotten easier than when I first considered the idea, not just because of better computing hardware and software, but because of most DSLRs supporting 4K video. You can set the camera to record 4K, but produce the final video at 1024 resolution. That gives you 1/4 of the image to publish and 3/4 to use for motion tracking. Of course, the video won't be as nice as if you recorded in 4K and zoomed in so that the rocket filled the frame. But, it certainly will be watchable.

    In addition to providing more area to process in real time, the system only has to keep the rocket within the large 4K frame since you can center the rocket in the 1024 frame in post.

    The software should be easier than most feedback-based control software. ("Easier," but not necessarily easy!)

    Rather than having the feedback directly control the pointing of the camera, you would have it tune a model of the rocket's flight and use that model to drive the camera's movement. From lift-off to ejection, the model should be simple - the rocket follows an arc Transitioning from flight to recovery would be a challenge, but doable I would think. But, even if tracking was lost at ejection, it would still be cool.

    Processing the video in real time is not something I have any experience with. So, I am not sure how hard that will be from either the hardware or software side. It is something a lot of things do, though, so references should be available online to help.
     
  12. May 19, 2019 #12

    jadebox

    jadebox

    jadebox

    Roger Smith

    Joined:
    Jan 18, 2009
    Messages:
    5,398
    Likes Received:
    41
    I haven't thought about how to control the zoom and focus during the flight.

    Theoretically, if the tracking is smooth enough, the camera's autofocus should work. But, does your DSLR support autofocus of video? Mine doesn't.

    A mirror-less camera might be a better option. In addition to supporting autofocus for video they are usually lighter and, if you design the system to take stills instead of video, they won't shake when taking a photo.

    Also, some mirror-less (or just plain video) cameras are controllable through USB, Bluetooth, or whatever which would offer a way to control the zoom programmatically.
     
  13. May 19, 2019 #13

    Handeman

    Handeman

    Handeman

    Forum Supporter TRF Supporter

    Joined:
    Jan 18, 2009
    Messages:
    7,423
    Likes Received:
    84
    Location:
    Stafford, VA
    I haven't tried to do anything like this, although I have done some embedded programming of motor drive devices, but I have followed a few others that have started projects like this.

    If I recall correctly, the biggest issue was getting the feedback loop fast enough to drive the camera angle to follow a fast rocket during the first several hundred feet of flight. The closer you are to the rocket the faster the Deg/s movement of the camera has to be. Of course the faster the rocket the faster the Deg/s has to be too. The other end of that is that the closer you are, the quicker the Deg/s drive required slows down as you are looking close to vertical much sooner than if you were further away, but that means your acceleration and deceleration rates will be higher.

    Your overall Deg/sec rate won't exceed about 60 deg/sec, but your actual instantaneous drive rate might need to be several hundred deg/sec at different times during the flight. I agree with dhkaiser, I don't think 60 deg/s will be enough.

    Since the motor profile will be all about acceleration and deceleration, do you know what current loads your drive circuits will need to handle? They could be surprisingly high with the required accel/decel rates. Your current limits in the drive circuits could limit your accel/decel more than the motor or drive line.

    If you use video processing, what kind/size of processor do you need to get a feedback loop that can update as fast as your video frames?

    Thanks for the thread, this will be interesting and please post the engineering/calculations behind the design.
     
  14. May 19, 2019 #14

    curtisheisey

    curtisheisey

    curtisheisey

    Forum Supporter TRF Supporter

    Joined:
    Jan 27, 2010
    Messages:
    98
    Likes Received:
    17
    I plan to use a raspberri pi B+. It has a 1.3Ghz Arm processor on it. Nonetheless, the video processing and control loop latency will be a challenge.

    I plan to use a separate video camera for the tracking and just set the DSLR for stills. I have a Sony A65 mirrorless that does about 5 frames/s.

    I figure the very first part of liftoff can use a fixed altitude/time profile, then the video can acquire track as it goes up.

    Good point about very high initial accelerations and pulling a lot of current on the stepper. I was going to use a NEMA23 stepper and just direct drive it to the tilt.
     
  15. May 19, 2019 #15

    heada

    heada

    heada

    Well-Known Member

    Joined:
    Jan 18, 2009
    Messages:
    2,496
    Likes Received:
    115
    Gender:
    Male
    Location:
    Indianapolis, Indiana

Share This Page