Disclaimer: I am not a vibrations person, so take this with a block of salt...
I double dog disclaim you. I'm also not a vibration expert. Also, as they say, a little knowledge is a dangerous thing, and a little knowledge is exactly what I have. So all this is what I'm presenting for consideration, not anything I'm planning or suggesting to be done.
1. If the elastomer core is significantly "stretchier" than the skins, then the skins will be bearing most of the load...
There are other practices with, if not stretchy then weak cores, e.g. FG over balsa. The balsa (or other light but weak material) provides a form to lay the FG over. Also, some thickness is necessary for stiffness (as of course you know) and the core provides that.
Hard rubber, in my purely qualitative experience, is not nearly as stretchy as it is bendy, and I know it's very lossy when bent. (Bend it back and forth a few times and it gets hot.) SMRD is, at least as far as I can test by hand, not stretchy at all and barely bendy, and maybe other materials like that are available. Now, either of these is a lot denser than balsa or birch. The only way this saves weight or bulk is if the vibration damping allows for thinner, i.e. less strong and less stiff skin layers to be acceptable. Slightly thinner skin layers would save bulk and quite possibly cost weight if the core dimension remains the same. If the damping allows for a thinner core by allowing less stiffness (because vibration is damped through this other method) then it might be possible, maybe, to save weight. I think.
2. A damping layer would definitely take out resonant behavior, so when you tap the fins, they wouldn't ring like a bell. However, most of our common fin materials are pretty good at that already. I'm willing to listen to anyone's fin-ringing concert, though.
The elastomer core might change the resonant frequency, but I'm not sure by how much in the grander scheme of things and it might change in unpredictable ways. The fins are still going to be excited into vibration by the flight forces (mumble mumble von Karmann vortex street). The question to me is whether fin flutter failures are a resonance problem or a simple structural problem from exciting forces that are higher than the materials can withstand.
Fin shredding is surely the latter, but most oscillations of this sort are completely resonance driven. Think of the stop signs you see waggling behind the poor TV reporters who get sent outside storms just to say "It's really windy out here, Steve!" A flat plate wants to turn broadside to the wind. It only oscillates because the post has torsional springiness and a resonance is struck between that and the inertia of the sign and post system. It has long been my assumption that fin flutter is essentially the same. I could be wrong in this, but I doubt it.
When a lossy element (a.k.a an energy dissipater) is added to such a system, it not only alters the resonant frequency but also reduces the Q of the system. In other words it (pardon me) flattens the curve, so the amplitude of the oscillation at resonance is reduced. Circling back, when the amplitude is reduced it
may be possible to get by with a less stiff and less strong fin.
As always, I'll sit behind a folding table with a "Prove me wrong" banner.
I will leave proving you or me (or even both of us) wrong to greater minds than mine.