Hello!
Sorry to hear you are having issues, vibs and questionable trust in the FBL can haunt any airframe for sure!
Reading through your post, sounds like the heli was flying well both times, just bail out caused issues. Now the thing to keep in mind here, all rescue, bailout, pick your term, systems use accelerometers to control the auto level portion, not the gyros. These Accelerometers are EXTREMELY sensitive to vibrations, of any kind. Even the smallest vibration that you and I would consider normal gear train can wreck these systems if the FBL isn't properly isolated from them. Nitros add to that issue by default with the addition of the engine and it's normal vibs. Even the best tuned, balanced, and broken in engine/drive train on a nitro will make more nominal vibrations than an electric.
That said, the noise in the tail could be a clue too. Was this frame used? Could the tail have been touched to the dirt at any time? Even a 0.001mm tweak on a shaft, really hard to see by eye, will cause the tail to buzz or vibrate at 9,500+ RPM.
I know you are frustrated, and I know this post doesn't help much. But I encourage you to tune the heli to the FBL make it fly well, then consult your FBL OEM suggestions for vibration dampening on a nitro. Every FBL recommends something different.