It only shows occasionally, making usage very frustrating. I actually have a solution to this problem which many before me have pointed out, but this stupid platform didn't let me respond to those and instead prompted me to start a new post. So here it is!
MacBook Pro 15″, macOS 10.15
Find "/~/Library/Preferences/com.apple.Music.plist" and delete it (or rename it). Then restart the Apple Music app. The back button should now be back. This is actually the solution to a lot of issues created by upgrades. If any app behaves odd, just find its preference file and remove it. It will be replaced by a new default file as soon as the app is re-opened resetting any odd behavior.
Apple Music back button has disappeared in MacOS Catalina
