That is a really good question that I'm not sure I know the answer to. I have a similar setup on my library, but it works fine. I can think of 2 possible reasons for this,
1. You have the limit set to 100 songs but you have more than that in your library. If you say have 140 songs in your library that haven't been played, but you set the playlist to limit to 100, then you will have 40 more songs in the playlist that were not there before.
2. Try turning off repeat.
3. Maybe you need to also check the live updating box. Is it still full when you sync to ITunes?
The reason the songs don't delete themselves after playing from the playlist on the iPod is because your iPod is not smart enough to do that. It's only when you re-sync your iPod to your iTunes library does the smart playlist in your iTunes library delete the played songs. Then the playlist on your iPod will adjust to the number left unplayed in that playlist.
So as long as you don't sync back to your library the number of songs in that playlist will never reduce to zero.