Hacking/remote access

If someone may have downloaded an app on my phone while unlocked and as a fix I restore my phone to factory settings, will it not just reload the same app back onto my phone with all of my other apps?

I have so many apps & I don’t know that I downloaded all of them. I’ve cleaned out some. But I’m almost positive my ex is still accessing my phone. I saw restoring the phone as a possible fix.. but is it?