I can confirm that you’re certainly not alone.
Right from update to the production release of iPadOS14, I have experienced the problem myself - and have noted and replied to multiple reports of this issue within this community. I initially spent many hours with the belief that this was some clever new feature of the OS that I simply couldn’t fathom, searching documentation for an explantation - before concluding that this is one of many functional bugs.
The inconsistent Home Screen App icon sizing also extends to folder icons and the new Widgets. As you swipe between Home Screens - and launch/close Apps, different icons and widgets resize. I’ve also seen a couple of occurrences of a vertically elongated App icon. A forced-restart of the iPad (to resolve other issues) appeared to stabilise the problem - only for the issue to later return.
Sorry, despite many hours of effort, I can’t offer you a lasting workaround.