'Initialize Audio' causes Logic Pro to crash, probably due to a deleted plugin

I've been trying to use an Audio Interface with Logic, but everytime I try to switch devices and it starts to initialize audio and everything freezes. There is an error message that pops up (which is frozen too) that leads me to believe it has to do with a Native Instrument plug-in that I've already deleted. I've removed the Audio Unit cache, did a clean un-install of the plugins through the components folder but Logic Pro still goes and tries to access it. The deleted plug-ins are even still shown on the Plug-In manager. Is there a way to solve this? Thanks.