My Mac mini has been running very slow due to something running in Activity Monitor

What year model Mac mini...?

What display(s) are you using...?

Restart in Safe Mode followed by a Normal Restart and then recheck the Activity Monitor.

see > Use safe mode on your Mac - Apple Support

Then if WindowServer activity come back and stays over 10%

Download and run the free version of EtreCheckPro, from > https://etrecheck.com/en/index.html

Then port the Report, as per > How to use the Add Text Feature When Post… - Apple Community

FWIW I run dual displays on my 2018 Mac mini with multiple windows open on both displays and the WindowServer bounces between 3.0 and 10%

If you post an EtreCheck report as den.thed has wisely suggested, we may be able to see what is causing WindowSever to work so hard. You cannot get rid of WindowServer but you can rid yourself of things that make it so "piggy."

Anti-virus software, so-called "cleaning" apps, clients-side VPNs ( the ones the spend $millions$ on TV adverts) can all overwork your system and interfere with elegant automated protection you paid Apple to build onto macOS.

That is a key process in macOS:

https://www.howtogeek.com/312755/what-is-the-process-windowserver-and-why-is-it-running-on-my-mac/

👍