I would not go so far as to call it “unsafe.”
I would however opine that the fewer entities who know - or can otherwise deduce - that any particular e-mail address IS your AppleID …
… the less prone you are to targeted phishing attacks based on your known “Apple’ness.”
Sort of of security through obscurity.
That said, multiple email address brings it’s own “costs” … the technical ability and personal discipline to securely manage and access multiple accounts is definitely NOT for “everybody.”
And poor security practices spread across multiple e-mail address might actually make matters worse.
There’s no one “silver bullet” for security, it’s all about defensive layers.
As another member here recently posted; security is not a fixed, static defense but rather a dynamic “Arms Race.”