Yes you can use other cards. You can either get a second-hand Mac specific card or you can get a brand new PC specific card. PC specific cards will work in your Mac Pro but will not show anything on screen until the Mac has completely finished booting and loaded the drivers.
It is possible to flash the firmware of a PC specific card to make it have Mac firmware and this is fairly straight forward for AMD cards and older Nvidia cards like the GTX 680. Newer Nvidia cards however currently cannot be done by yourself and currently can only be flashed by a company called MacVidCards. Flashing a card is done from Windows and while it could be done in a 'real' PC is often done using Boot Camp on a Mac.
Apple include AMD drivers as standard in OS X, but you would have to download and install Nvidia drivers. If you currently don't have a working video card then that could present a catch-22 situation.
The AMD Sapphire HD 7950, or the Nvidia GTX 680 were cards with last Mac specific versions released. Newer cards would only come as PC models but as mentioned can still be flashed to Mac firmware, this includes the AMD R-290x, and the Nvidia GTX 980.