So I figured it out, it took editing the com.apple.Music.plist file to fix it. Something to be careful doing.
First, I made a backup copy of com.apple.Music.plist from the ~/Library/Preferences folder. Then deleted it from there. Opened the Music app and found that the issue went away. Great except all my other settings in the app were reset to default.
Hovering over the green button it says "Full Screen" but when hovering over with the option key selected it says "Zoom". I thought maybe these are references in the .plist file.
Next I created another backup copy of the com.apple.Music.plist file. Opened it up and did some digging. I eventually found there were two keys named maximized and zoom.
<key>maximized</key>
<true/>
<key>zoomed</key>
<true/>
I found all references to these two values and deleted them. Reopened the Music app and the app opened in a normal window size.
Why the app wasn't saving the window size setting not being maximized or zoomed when closing I don't know but this did the trick for me.
Thanks