Slight clarification, to avoid any confusion: Unless I'm very mistaken, all of this data is stored entirely on-device.
So Apple doesn't have this information, in that it's not stored or able to be used by Apple on a server elsewhere. Your phone has the information stored on it--locally--and is able to use it--again, locally--to determine patterns for Frequent Locations, which it will then use for suggestions.
This of course doesn't help at all for people who want access to the data, but there is a non-trivial distinction between Apple itself being able to access this data, and your phone being able to do internal processing on data that is not accessible from anywhere else. I do still believe that, since the data is being stored, the user should be able to access it, but it's not the same as the case of Apple having direct access to data that the user does not.
If you give a specific app permission to use this same data (such as the great suggestion to use DayOne as a log), that app does get access to the same data, but only from the point that you give it permission onward. This problem could be effectively mitigated if there was a method for an app to request access to all location data, not just "from now on" data, but that doesn't seem to be a thing that's currently allowed. I'm also not sure whether a separate database is generated for an app that is granted access to location data, or if the app is using the same central system database but with a time restriction on requests.