The material of the hammer would probably go at the bottom of the list in order of importance, but in my humble opinion it still makes a tiny difference. And yes the tip must have much more impact on the end result. But let's imagine an extreme example, where the hammer body would be made of cardboard, or styrofoam... I can only guess it would make a difference. More than wood / plastic, but still. I've only worked on a few Rhodes in my life, mostly my own MkI and MkV, and between these two I definitely hear a difference in sound, and they have different hammers.
Now that you mention it Steveo, the hammer pivot system does have its importance too. It might be more felt in the action, but it's also heard in the sound to some extent. I've converted one of the hammers of my MkV, for which the comb was broken, to "metal center pin and wooden flange with felt bushing", and I can definitely tell the difference, mostly in the action. (Whatever difference in sound may be psycholgical side-effect...)