Which shuffle model is it? This document shows the four types
For any model, make sure you have it set to shuffle and not play-in-order. On the 2nd gen shuffle, there is a separate switch for play mode. On the other shuffle models, the power switch has a middle position, which is play-in-order. Make sure it's set to shuffle, because play-in-order always starts at the top of the list of songs.
If it's a 3rd or 4th gen shuffle, it can play from multiple playlists; use the VoiceOver feature and select the All Songs list from the playlist menu. This is a document for the 4th gen shuffle about setting up and using VoiceOver; the 3rd gen shuffle should be similar.
1st and 2nd gen shuffle does not have VoiceOver, and only has one list that it plays from.