My old iPhone 8 is locked because failed password attempt but when I plug it into my Mac it doesn't show up on the finder area. I tried to reset and update part but I don't want to restore it and loose the photos I have on it

No. When you restore from a disabled phone it becomes a new phone; nothing on it can be saved.

Are you sure you didn’t do ANY of these ways of preserving photos and videos?

• emailed, texted or messaged the photos to friends or family
• iCloud photo sync
• iCloud backup
• iTunes backup
• Transfer your photos to your computer via USB cable
• Apple Photo Stream
• Sync to Google Photos
• Sync to some other photo archiving service such as Shutterfly
• Sync to Dropbox
• Sync to box.com
• Sync to Microsoft OneDrive
• Sync to Amazon Photos
• back up to Livedrive
• Backup to any other backup or archiving service

If the answer is no to all of the above your photos are gone forever.

Unfortunately, you'll need to erase the device. Fortunately, you can restore it from your backup.

Learn more: If you forgot the passcode on your iPhone, or your iPhone is disabled