You have the repeat function on. Tap the album artwork while a song is playing. The scrubber bar with the repeat and shuffle icon will appear. Turn off repeat - blue is on, white is off.
Play a track, in portrait orientation (upright) tap the album artwork to bring up the scrubber bar. Tap the oval repeat arrows on the left side of the scrubber bar:
white = repeat off
blue = repeat playlist
blue with a "1" = repeat song