Fastest data transfer from old Mac mini to new Mac mini M4

Hello all. I have bought myself the new Mac Mini M4 (2024) and want to transfer all data from my old Mac Mini which runs macOS Catalina (this is the latest OS I can get on old one). I started using the Migration Assistant over wifi but it was taking a ridiculous amount of time (over 24 hours) so I cancelled it. I have a Seagate Backup Plus Portable Drive connected to my old Mac that has Time Machine backups on.

What's the fastest way to transfer the data? Can I just connect the Seagate drive to the new mac and transfer it that way or do I need to connect the two Macs by a cable and use the Migration Assistant? I have enough free space on new Mac Mini to take everything from the old one. I have just read not to use wifi as its too slow and can corrupt data.

Any help much appreciated.

[Re-Titled by Moderator]