If I remove stolen iPhone from Find My, does that make things easier for the theif?

Yes, removing it from find my disables Activation Lock. The thief gets 10 guesses at your passcode. Is that enough to guess correctly? If so, cancel all credit cards you have in the Wallet and your Apple ID account, and change all passwords you have ever used.

Beware of a common phishing scam: You may get a text or email from “Apple” or “Apple Security” (or sometimes “Police”) saying your phone has been found and to click on a link to see its location. This is NEVER from Apple or from law enforcement; it is from the thief or fence to steal your Apple ID and identity. Delete any such message without responding to it. Apple never gets involved in lost or stolen devices→Avoid phishing emails, fake ‘virus‘ alerts, phony support calls, and other scams

Removing the device from Find My will allow the user to reset the device and use it as their own. When they reset the device, it will also delete your photos. If your Passcode is easy to guess and they have used it without triggering the lock by too many guesses, then they already have access to your photos if not on a network to process your erase request first.

Two things:

• If they were able to use the Passcode, they can get into the device whether you have removed it from your account or not. When not using a easily guessed passcode, that is all that is needed to protect your data as the device will lock before they can guess it.
• Removing the device from Find My does not give them access to your phone, it just allows them to reset the device and that action will erase all of your data.