A difference in the counts would suggest you either have orphaned music files in the original folder, e.g. content removed from the library but not sent to the trash, or legacy iTunes media that isn't part of the Music library, e.g. audiobooks, podcasts, movies, TV shows etc. Compare some sample folders on the original location and on the external drive to see if you can make sense of the difference. Neither Music nor iTunes actively scan the media folder. You can place content inside and add the whole folder to the library, or make use of the Automatically Add to Music folder inside the media folder to add tracks have have them organized within the media folder.
The exclamation marks are for entries in the library where the original file has been removed, or renamed in a way macOS cannot detect automatically.
tt2