If you want to disable the Previous Recipients feature, you can do it with this command in Terminal:
defaults write com.apple.mail SuppressAddressHistory -bool true
Neither Mail nor Address Book should be running at the time you do this. As the OP stated they don't want anyone in that list except people who are in their Address Book, if they add someone to their address book, they will need to reverse this command before that person could be added to the Previous Recipients list.
Mulder