Font book is showing quite a few fonts that have issues though, should I clear them or leave them?
That's kind of a "watch and wait" thing. If your cache files being corrupted again in a rather short amount of time (causing garbled text or error messages from Office), then you should consider removing them since they are damaged. It does also depend on now serious Font Book thinks the errors with the fonts are.
Bad fonts can actually cause a number of system errors since each enabled font does take up a small amount of residence in RAM. A bad font can then overflow the area in RAM it's suppose to be contained to and intrude on another application's space.
Replace any fonts you have that with fresh copies from the original disks they came from. Any others, download them again if they were obtained from sites that host free fonts. Also note that most free fonts are major league junk. Avoid them if you can. There are good ones out there, but they represent a small percentage compared to the ones that no one should ever use.