Could you have more than one entry for imap.mail.yahoo.com in Keychain Access?
Open Keychain Access in Utilities, use Keychain First Aid under the Window Menu item, then either check the Password under that item, change it, or delete it and start over.
Then See if this helps...
Mac OS X 10.4 Help, I forgot a password in my Keychain
Resetting your keychain in Mac OS X...
If Keychain First Aid finds an issue that it cannot repair, or if you do not know your keychain password, you may need to reset your keychain.
I was having the same problem....I read up on a few suggestions and found that you might have to go to your YAHOO MAIL in YAHOO and change your password. Mine was an 7 word/number password...Once I changed in to an 8 word/number password in YAHOO, I went to (perferences) in mail, then to (accounts), and changed the old 7 password to the new 8 password. I exited the tab and saved it.....the mail open and no more rejection of password. Has been working since then.
Give it a try.