Quick Start is all or nothing, and so is restoring a backup of the old phone to the new phone.
If you sync your photos and videos to iCloud they will not be contained in a backup, so that is one way to exclude them, but you would then have to not sync photos and videos via iCloud to the new phone, but do sync them from the old phone to iCloud.
Here’s one scenario:
- Sync your photos to iCloud from the old phone (Settings/[your name]/iCloud/Photos ON) - you may have to “rent” additional iCloud space to do this.
- Back up the old phone to your computer or iCloud (use encrypted backup if its to your computer)
- Don’t use Quick Start
- Follow the setup procedure starting with the Hello screen from the new phone
- When prompted, restore the backup you have made to the new phone
- When everything else has set up on the new phone, go to Settings/[your name]/iCloud and turn off Photos
- When prompted, say to remove the content from the new phone
- If you don’t want the photos on the old phone either, delete the photos while still connected to iCloud, from both the library and the Recently Deleted folder - this will also delete them from iCloud
You will then have a new phone with no photos or videos, and an empty iCloud if you want to sync new photos to it.