See the Change AirPlay settings section of Change settings for HomePod - Apple Support. If you set it up with everyone and no password then there is the opportunity for those nearby to take control of your HomePod. If not then a mistaken response to some ambient noise is the most likely explanation.
tt2