First, I can't rule anything out until that fan is on there to better than 0.001" . I personally won't fly a fan with more than 1/2 thou of runout.
Recheck the fan's balance. It is not acceptable to have the fan just hold still on the balancer at several different places. You need to be able to spin the fan and have it stop in completely random places around its entire clockface.
Installation of the engine/fan/driver: No more than 0.001" gap difference measured from front and back AND left and right. This is less likely to cause your problem, but it still ought to be right.
Engine: I don't care if that engine is brand new and out of the box.........STILL check the rear bearing. I've had brand new engines with bad bearings. Trust nothing when you're trouble shooting.
Clutch bell: Watch it closely while you spin the maingear it make sure it doesn't wobble. An occasional bad bell with wobble has been seen by me.
Gear Mesh: As mentioned above, make sure the mesh isn't not tight when checked WITH THE MODEL HAVING JUST BEEN FLOWN. Tight'ish gear meshes get too tight when running. The sole time it is acceptable to run a mesh with a bind is with brand new gear that have not yet bedded in.
Tail rotor: Check the balance of the tail hub on a shaft on a balancer. Then add the grips. Repeat. Then add the tail blades. Repeat. Never underestimate what a crappy set of supposedly "balanced" carbon blades can contribute to vibration. Span and chordwise CG can be royally effed up on bldes with the SAME mass.
Tail box: I'm not sure which tail box you have, but check the bearings on both the input and output shafts. Double check the straightness of the output shaft.
Head: An unlikely source of the high frequency vibration you describe. Most of the time tweaked head blocks will cause shakes in the canopy and landing gear. Rotor speed is not high enough to generate significant high frequency vibration, but you CAN apparently affect the vibration by changing headspeed because that changes engine and driveline speed.
Before you go completely nuts and if all else fails, please try to find a friend with a proven smooth MA model that you can rob the ENTIRE tail boom and rotor assembly from to fly on yours. This is by far the quickest way to isolate a vibration such as this. If it appears on a proven assembly, then we can then focus on the main mechanic.