FCP X not ejecting SD Card

I'm having this issue where after I import footage, I get a notification that the import is complete with a button to eject the SD card. I click the eject button, but when I go to the finder I notice the drive is still mounted. I go back to the import window of FCP X and click the eject button there, after a BBoD for a few seconds nothing happens and finder still shows the card is mounted. When I try and eject it from the Finder, I get a message saying that it cannot be ejected because FCP X is using it. The only way I can get it to eject is if I completely quite FCP X, which is a huge problem for workflow! This happens even when FCP X is the only app running!