No, anyone CANNOT track your AirTag. You misinterpreted that. Without the chirping a stalker could place an AirTag in your personal item and you would not know it’s there. The chirping is there to alert you that there’s an AirTag on your person or possessions without your knowledge. And that's why the chirping cannot be turned off. And you certainly don’t want to approach the thief in any way, shape, or form.
AirTags were NOT designed to be theft deterrents even though people started using them for that purpose immediately. In fact there are several class action lawsuits alleging AirTags are being used by stalkers. It’s the reason Apple added the chirping.
AirTags were not designed so a husband or boyfriend can track you without your knowledge. But bad people started using them for that anyway.