Hello Joseph Wiedermann,
In answer to your first question, yes, it would be beneficial to reformat your external HD to APFS, as that is what Time Machine uses as of Big Sur. Prior to Big Sur, it was Apple Extended Journaled (HFS+). So any Time Machine Backups created with Big Sur, will not be compatible with any other OS, including Catalina.
In answer to your second question, since you did not reformat the HD to which you were creating the backups, Big Sur would have created backups in its APFS format, if there had been room for it to do so, with the older OS' backups still installed on your Time Machine HD. So, your new Time Machine backups under Big Sur would be compatible with Big Sur that you have installed.
The previous OS' backups are now superfluous, and if you do not need them any longer, it would be best to remove them. It would also free up space on your external HD for Big Sur's Time Machine needs.
If you are quite certain that you can do without having a backup for a little while, I would do as I first suggested, and reformat the drive to APFS. The advantage is that the drive will be formatted to what is necessary anyway.
Also, the reason that the backups may be taking longer, is because APFS does not play that well with mechanical HDs, but seems to be more efficient, a little faster, with Big Sur, but to my understanding SSDs work best for this file format.
I hope this helps!