Why not just turn it off in your system preferences?
And if you really don't want accessing the bluetooth preferences after turning it off remove the preference panel from /System/Library/PreferencePanes.
But if you insist, and really want to remove the drivers, then like any other driver they are in /System/Library/Extensions. Based solely on the name I assume they are IOBluetoothFamily.kext and IOBluetoothHIDDriver.kext. It's not clear whether removing those will break your system or not. So you better keep the originals as backups and be prepared to boot off some other boot disk to restore them (observe ownership and permissions).