Apple Leather Wallet not working with Apple Leather Case

I have an iPhone 13 Pro and bought an apple leather case and apple leather wallet together. I initially connected the wallet to my iPhone and it worked fine. After about a month now the phone no longer recognizes when the wallet is connected or disconnected unless I take the leather case off. I am updated to the latest iOS 15.4 and it is still an issue.