2011 imac to new imac migration tips?

In all my Mac years, I've always done the long slog of connecting old mac > new mac and just copying everything over.

Which seems time consuming and perhaps unnecessary and probably copying over a lot of stuff I don't need anymore.

What I DO need is my documents folder (easy enough) and my photos (due to a buggy Photos app, I have most of my photos in Photos but a 2 year chunk that only opens in iPhoto because of metadata issues). Everything else I could start from zero (downloading all I need from work software pages or reinstalls). I feel like I've copied that same "stuff" over and over from 1 desktop and three laptops ago and it might be good to start fresh (excluding docs and photos).

Good idea? Bad idea?

Am slightly paranoid about losing the pictures so what do I need to copy to ensure this doesn't turn out poorly? We have two users on the computer but with the other user it's only the docs/photos that needs saving as well.

(Itunes backed up with whatever itunes match is now so can start from scratch there yes)? AM looking at the new 27 inch imac that's online at the moment.

any tips/suggestions/warnings would be appreciated.