Unless your Mac is acting sluggishly in other iMovie projects or in other apps, I don't believe that you have a hardware issue. It might just be that your original Mp4 has some complex rendering that is causing the export process to take some time. Or perhaps there are other computer intensive processes running at the same time that are slowing down the process.
What are your Mac's specs?
You can try doing the conversion over again in another project and see if it renders faster.
Also here are a couple of diagnotic procedures that you can try if you are using an intel Mac versus an M1 or M2 silicon chip Mac.
Resetting the System Management
Controller
Try resetting the System Management Controller (but no longer available for the new
M1 or M2 computers). See the link below for instructions.
Reset the SMC of your Mac - Apple Support
Resetting the NVRAM or PRAM
Try resetting the NVRAM or PRAM, although this issue would be fairly rare:
Reset NVRAM on your Mac - Apple Support
-- Rich