If you restore your phone back to factory settings do you lose all your photos

That’s true. However, it would be helpful to understand iCloud better:

iCloud+ has 2 independent functions. iCloud backups, and iCloud sync. It’s confusing because they both have the same name prefix. But they have no connection with each other. Go to Settings/[your name]/iCloud. You will see a bunch of switches (with iOS 16 also tap Show All). When you turn on a switch that data type will sync to iCloud. If you do this on multiple devices that share an Apple ID the selected data will sync to all of those devices. 

You can also turn on iCloud Backups. This will back up your phone every night if the phone is plugged in, connected to Wi-Fi and locked.

But note that anything that you sync by the first method will be excluded from the iCloud backups.

Yes, restoring to factory settings will wipe out everything you have on your phone. Music, pictures, video, etc