Although the Apple Music subscription service allows you to download "offline listening" copies, these are just for convenience, so that you can listen to music even when not continuously connected to the Internet.
They are still rental songs, and if you stop paying for a subscription, you lose access to them. A "subscription" to some streaming service like Apple Music, Spotify, or Netflix is more like paying for cable TV (stop paying and they cut off the cable feed) than it is like a magazine subscription (where you just stop receiving new issues – but can keep all of the ones you have received).
It's also my understanding that if you let your Apple Music subscription lapse, all of your "offline listening" copies and all of your playlists that contain subscription songs get deleted quickly. If you resubscribe, you may find that you have to re-download "offline listening" copies and re-build playlists all over.