I figured out the problem.
icloud uses the apple id as the sender email address. But this is not allowed on many mail servers. Because if the target mail server receives a icloud invitation, they ask the mail server from sender email address, if the server who sends this email is allowed to do that. This will be done by the SPF record.
as an example: your apple id is myappleid@gmx.net, now you invite one person: oneperson@hisdomain.com
no the email invitation will be sent from the icloud server (badger1302.apple.com) but the sender address is myappleid@gmx.net. now on the target side, the mail server hisdomain.com asks gmx.net, if mail server badger1302.apple.com is allowed to send email with sender address myappleid@gmx.net. But gmx.net doesn allow that. So most target mail server then decides to drop such emails. if email server ignore this SPF record, then the email will be acceppted.
So to bring this running, it should be possible to configure, which email should be used to send such email invitations (good would be a @me.com address), but currently the apple id will be used and there no @me.com is allowed.
an other solution is to use a apple id where you have control of the SPF record (if you have your own domain)
I hope that helps