Hi,
The Messages app takes a Picture and manipulates it for the Buddy List.
It takes whatever size the original pic is and creates a thumbnail about 72 pixels square. (in reality only a few kbs of data)
This is then sent to the relevant Server (AIM, Jabber or Yahoo) and then it is sent to your Buddies.
With Yahoo you can add yourself to the Buddy List so you can see what the pic looks like.
Incoming pics are stored in ~/Library/Caches/com.apple.Messages/Pictures
Even the Apple default ones get reduced in Quality.

However the result can vary depending which Service the thumbnail is going through

The "Front" one here is from the same Buddy as the next to the right and despite the second being larger the quality is different.
I would check the Yahoo site and see what size of pic they suggest is used to start with.
Using this in Messages will reduce the number of compressions it goes through.

9:26 pm Thursday; November 26, 2015
iMac 2.5Ghz i5 2011 (Mavericks 10.9)
G4/1GhzDual MDD (Leopard 10.5.8)
MacBookPro 2Gb (Snow Leopard 10.6.8)
Mac OS X (10.6.8),
Couple of iPhones and an iPad