Does the HomePod mini have to be plugged in at all times?

If you want it to work, yes. The Home pods don’t have batteries. If they aren’t plugged in to power, then, they are just fancy paperweights.

You can of course unplug it if you aren’t planning on using it.

Yes. It won't work otherwise.