Unfortunately itunes won't play videos in .vob or .avi format, they are both (essentially) Windows formats. That means Quicktime hates them as well. Quicktime will play .avi files but it's usually a hassle (see the first link below).
The solution is to convert the files to mp4 format. Handbrake is a an app that many community users recommend. I use it to convert DVDs and it works great.
Here are some reference articles that may help:
http://support.apple.com/kb/ht1536 - Quicktime playing .avi files
http://support.apple.com/kb/HT3526 - Adding media formats to Quicktime so they will play
http://handbrake.fr/downloads.php - Handbrake download