Any thoughts on ht's to consider and which features I need to look for would be much appreciated.
One big benefit of learning RDF is just how much range this offers on the ground over APRS or digital telemetry. I've been able to hear RDF tones from miles away, while APRS or digital telemetry is limited to half a mile or less. Find a bit of elevation and the range is even greater; even a few dozen feet can be dramatic. That's why all of our devices offer RDF as a backup to GPS telemetry; it can provide that critical piece of information to get you close enough to receive the GPS data and locate the rocket, in cases where you failed to get GPS data as the rocket approached the ground.
As far as HT features go, there are two potentially useful ones. The first is a "real" signal strength meter. As others have noted, some less expensive receivers have a meter which is essentially all-or-nothing. That makes it harder to find the peak signal strength when sweeping across the signal. The second is an RF attenuator. That allows you to reduce the sensitivity of the receiver so that when you are close to the transmitter, you can still get some variation in signal strength instead of having the signal 'really loud' in all directions. If you can receive GPS data, that's less important, but if you're just RDF'ing, it's pretty necessary in any kind of brush.
I've done RDF with inexpensive radios, with those, you have to learn how to distinguish signal strength by the relative amount of tone to noise you're hearing. It's not as accurate as a signal meter, but it is generally sufficient to get an idea of which direction to head; certainly within 45 degrees or so. That has generally been in combination with a TeleBT and android device to get GPS data once we got close enough. I think it'd be hard to get all the way to the transmitter without a reasonable meter.
What I really like to carry is a radio with good RDF performance and APRS reception. This provides a backup to my TeleBT/Android setup for GPS data (in case the phone dies) while also offering the ability to do RDF in case I need to. That's not a cheap option though; the FT1D that I bought to do APRS development is the most expensive radio I've ever owned.
Finally, RDF requires a directional antenna. I use a 3 element Arrow yagi, but you can easily build one too. It's nice to have at least a little gain to improve the radio system range. I swap the antenna between the TeleBT and the FT1D if I want to RDF as I want to capture as much telemetry data as I can during flight, and if I can get GPS data from the rocket to my android device, having live maps with that presented makes it possible to navigate for recovery.