Ethernet port on iMac M1 takes too long to initialize after sleep

My wife has a new 24" M1 iMac and I have noticed that it takes a good 10+ seconds for the Ethernet port to wake-up after the computer comes out of sleep. You have to sit there and wait to browse the web, check email, etc.

Any suggestions to force the Ethernet port to wake-up faster or not disable at all when the computer goes into sleep mode?