Apple Calendars contact invite not working.

When I invite someone to my event on Apple Calendars via Contact the recipient doesn't receive any notification or option to join the event. Though when I tested it by inviting via their iCloud email it goes straight to their email and allows them to join. Is there a solution to an invite a contact without their iCloud email to my event? I created the event on all 3 of my devices to see if inviting from there would change anything.

I'm using an iPhone 8 Plus

Macbook Pro (2019)

and an iPad Pro 11 Inch