Starting up in Safe Mode (among other services the Mac performs when doing this) is all cache files are removed from the user account you logged into in Safe Mode, and Font Book's database is reset.
Some items need to be handled separately.
Close any apps you have running. Open the Library folder of your user account. One way to get there is to click anywhere on the desktop so Finder is the foreground app listed at the top left by the Apple logo. Hold down the Option key and on the top menu bar, choose Go > Library.
Now that the Library folder is visible, open the Preferences folder. Put these two items in the trash.
com.apple.FontBook.plist
com.apple.FontRegistry.user.plist
Next, from an administrator account, open the Terminal app (in the /Applications/Utilities/ folder) and enter the following command. You can also copy/paste it from here into the Terminal window:
sudo atsutil databases -remove
This removes all font cache files. Both for the system and the current logged in user. After running the command, close Terminal and immediately restart your Mac.
If that still doesn't help, try these next.
Back in the Library folder of your user account, open the FontCollections folder. Put all items within that folder in the trash. Except for any sets you many have created, these are not even slightly important. They're just examples of how to use sets in Font Book. The reason for removing them is damaged sets can cause Font Book to behave very badly.
BDAqua has already mentioned the other item I was going to. It can be a very useful test. No problem in the new account? Then the issue is confined to your normal account. Same problem? It's a system wide issue that typically means you need to reinstall the OS.