You need to attach the album art to the songs in iTunes first. The art is part of the song file on your computer, it is not something that is only on the Nano. In other words, you can only display art on your Nano if the song file in your iTunes library has the art attached first.
First, open iTunes and select a song. If it already has album art attached, it will display in the box at the lower left of the screen. If there is no art attached, the box will say "drag album artwork here". In that case, you need to find a picture of the album art (note, iTunes will not automatically go and find the album art for you when you put a CD in - you have to find the art and attach it yourself or find a program that does it for you). You could scan your CD artwork, or find pictures of album covers on the web at places like Amazon.com. Once you find an image you want, then attach it to the song in iTunes by dragging it to the box, or by right clicking the song, selecting "Get Info", and adding it in the "Artwork" tab. You have to add the artwork to each song (you can group add for songs from the same album).
Once you have the artwork, sync to the Nano and the artwork should be transfered. Also make sure in your iPod preferences that you have the "Display album artwork on your iPod" box checked. That should be it. The artwork should show up next to the song info on your Nano's screen, and you can view the art full screen whila a song is selected by pressing the center select button a couple of times.