Same here. There were employer sites in Top Sites that required a VPN to access. When the VPN was disconnected, Safari could not contact them. It proposed a solution (signing) that would not have helped (so you may as well click Deny).
Removing the "unhappy" sites from Top Sites (as Youraveragemo did) partially solved it. But when the number of Top Sites shown (say, 6 or 12) is less than the number of Top Sites known (up to 24), unseen unhappy sites could lurk or creep in later.
Solution: In Safari > Preferences, change the number of Top Sites shown to 24. In History > Top Sites, remove every unhappy site (click its X icon). Pin every "happy" site (click its white pin icon to make it blue). If this leaves fewer than 24 sites, you may need to repeat the process a few times during the next several weeks until 24 Top Sites are pinned and none of them are unhappy.
For general instructions for using Top Sites and Favorites, see this Apple article.