How does Time Machine work between different macOS versions?

Since Migration Assistant won't work - too much old data w not enough new storage - you don't have a lot of options. You will have to do this manually by dragging and dropping, but it's easy to drag entire folders to an external drive. If you've kept your files fairly well organized then it's easy to do by dragging entire folders at a time. You could drag your entire home folder to an external drive and have captured most, if not all, of your personal data.

Or use a utility such as Carbon Copy Cloner on your old Mac to copy the home folder and everything in it to an external drive. Then manually copy just the files I want to my new home folder. You can take your time if there's no hurry to sift through your stuff.

Either way, you process really can't be automated. You'll have to manage it one way or another.