Thanks for your reply, but I/m afraid it's not helpful.
Step 1 - Check/Done
Step 2 - Check/Done
Step 3 - Check/Done
Step 4 - Check/Done (This step is not considering the fact that the same cable and same USB port works for other iPhones that are running iOS 15.3 or older) So why should the accessory or USB port matter if I connect another iPhone with iOS 15.4 ?
Step 5 - Check/Done (even though it does not matter logically)
Step 6 - Check/Done (even though it does not matter logically)
Step 7 - Check/Done (even though it does not matter logically)
Step 8 - Check/Done (even though it does not matter logically. Another PC running Windows11 - Same problem)
Unfortunately the Apple senior level support (they probably call it tier 2 or 3) could not provide any solution. All they concluded is that during the upgrade of iOS they could be some data corruption if you do it via Wifi and that my Wifi may not be strong or reliable enough. (I hate to dismiss a senior level support, but in this case they are clueless, I am sorry to say)
iOS 15.4 has some issues that Apple has not tested for. That's all I can conclude.