Hotspot always visible for others

Yes, when Personal Hotspot is enabled, your iPhone's hotspot will be visible to any device within range. The password feature is what will allow you to limit access to your Personal Hotspot.

For more information, see Apple's relevant Support article.

How to set up a Personal Hotspot on your iPhone or iPad - Apple Support

Cheers.

Are you sure this applies to all nearby devices, or only devices that have been previous paired?

Devices that have been previously paired will often display the iPhone in the list of Wi-Fi networks or Bluetooth devices, but attempting to connect will fail, if Personal Hotspot is turned off.