Audio Compressor vs Voice Isolation

So I have this voice over clip that was initially recorded with a little bit of background noise. And Final Cut Pro's Voice Isolation feature is amazing in that it completely removes that background noise. The problem is that when I then add an audio Compressor to level everything out, it reintroduces that background noise. Just curious as to what I may be doing wrong. Please see:

Initial audio:

After applying Voice Isolation:

Noise returns after applying the audio Compressor and my levels are now clipping:

Should I be reducing the sound on the Input Gain first? Even if I turn on "Limiter" the sound is still distorted, too hot.