Shared With You

I’m experiencing a problem with the ‘Shared With You’ in Messages. When I receive a message with a photo, I can see the photo in my ‘Shared With You’ section in my photos app on my iPhone but not my iPad but sometimes it’s the other way round, I see it in my ‘Shared With You’ section on my iPad but not my iPhone. I never see a photo that someone has sent in my Shared With You on BOTH my iPad and my iPhone. It is either on one or the other? Shared With You is switched on on both devices as evidenced by the fact I receive the photo on one of the devices but never both. Should I get them through on both? Can anyone explain why? Thanks