Start by dragging the files concerned into the Songs list in iTunes if they aren't already there.
To convert files to M4a ('AAC'):
In iTunes Preferences>General click the 'Import Settings' button and choose 'AAC encoder' (it may be the default) and use the next drop-down menu to choose your settings - you can choose 'Custom' and set the bitrate as suits you.
Click OK, then OK again to close the Preferences.
Now go to the library, select your original files and from the from the 'File' menu choose 'Create new version' then 'Create AAC version' (iTunes 11 - in earlier versions choose 'Create AAC version' from the 'Advanced' menu).. This will make duplicate AAC versions and you can delete the original versions in iTunes. (Never tamper with the actual library in the Finder).
To convert files to MP3:
Firstly, go to iTunes Preferences>General and click the 'Import Settings' button. Set 'Import Using' to 'MP3 Encoder' and use the next drop-down menu to choose your settings - you can choose 'Custom' and set the bitrate as suits you.
Click OK, then OK again to close the Preferences.
Go to the Music list; select the files you want to convert and from the 'File' menu choose 'Create new version' then 'Create MP3 version' (iTunes 11 - in earlier versions choose 'Create MP3 version' from the 'Advanced' menu).
MP3 versions will be created and placed in the music list (the original versions will remain there). You can drag them out if you want to save them elsewhere (doing this will not remove them from the Library). You can select and delete any files as required in the Music list. (Never tamper with the actual library in the Finder).
It has been suggested that converting files to WAV or AIFF will improve the quality - it won't. AAC and MP3 compression works by leaving out bits of the sound it is assumed you cannot hear because they are masked by other sounds. Copying to an uncompressed format such as WAV cannot restore the missing sounds, so you are maintaining the same quality and just making the files a lot larger.