It does not matter if the shuffle's power switch is in the OFF position or one of the ON positions. However, to be consistent while trouble-shooting, put it in the OFF position (because the OFF position "resets" the shuffle).
If Manually manage music is not checked, you are using automatic syncing. You somehow used automatic syncing to load that first album ("CD") to the shuffle. But depending no how you did this specifically, it may not automatically load the second album.
Since you are new to the shuffle, you may want to use the Manually manage music setting instead of automatic syncing. If you want to try it that way, checkmark Manually manage music and click Apply. Songs already on the shuffle should remain on the shuffle. Here's how to add additional songs manually.
Show your music library in the iTunes window (click the Music button at left end of horizontal bar). Near middle of horizontal bar, click Playlists. A sidebar appears along left side, with your shuffle listed under devices. In the main part of the iTunes window, you can control the view used to show your music library, using the control at the right end of horizontal bar. I normally like using the Songs view, but if you want to load a complete album on the shuffle, you can use the Albums view. Drag the album you want to add to the shuffle and drop it on the shuffle in the sidebar. Alternately, instead of dragging/dropping, you can right-click the album and Add to Playlist -> [name of shuffle].
NOTE: To add songs (not complete albums), use the Songs view. Find the song you want on the iPod and drag/drop it on the iPod in the sidebar. You can select multiple songs at the same time, and drag/drop the selection. The right-click and Add to Playlist method works too. If you want to add something else, like an audiobook, show your audiobooks library (instead of music library) and do the same thing.
When using Manually manage music, the iPod's light blinks continuously when connected to iTunes. Blinking light means Do not disconnect - Eject iPod before disconnecting. On the iPod's settings screen in iTunes, the Eject button is next to the iPod's name. It looks like a triangle with a line below it. Click that button to eject iPod. The iPod's light become solid (not blinking). Solid light means OK to disconnect. And its color (when not blinking) indicates battery status; solid green means it's fully charged.
There is another technique called Autofill. For using my shuffle with my iTunes library, I find that Autofill is the most convenient way to use a shuffle. You can post back for more info.
Also, if you DO want to use automatic syncing instead if the manual method, I can describe how to set that up.