That is not a feature of iOS, and requiring a passcode would not solve your problem. How do you know the phone was turned off? You also couldn’t track it if the thief put it in a faraday shielded bag (the most likely, as it’s the easiest), or a metal box, wrapped it in aluminum foil, let the battery run down, and other methods. Find My is primarily for lost devices, not stolen. Anyway, most likely you will be able to track it when it arrives in its destination, Shenzhen, China, where there are factories that repurpose iPhones and other stolen electronics for resale.
Now suppose you could track it. What would you do? Confront the thief? People have been murdered doing that. Tell the police? They would have to get a search warrant, and they aren’t going to bother for just one of thousands of iPhones stolen every day.
Beware of a common phishing scam: You may get a text or email from “Apple” or “Apple Security” (or sometimes “Police”) saying your phone has been found and to click on a link to see its location. This is NEVER from Apple or from law enforcement; it is from the thief or fence to steal your Apple ID and identity. Delete any such message without responding to it. Apple never gets involved in lost or stolen devices→Avoid phishing emails, fake ‘virus‘ alerts, phony support calls, and other scams