I'd like to add that restoring the contacts from iCloud backup did give me back all contacts, but it wouldn't help, since it didn't change the fact that my contacts weren't syncing. What really solved it was the following steps:
1 - I quit both the Contacts and Messages apps on my Mac. (I QUIT them, not closed them)
2 - I deleted (renamed) the ~/Library/Application Support/Adress Book folder as suggested
3 - I checked that both my iPad and iPhone were syncing over wifi on iTunes.
4 - In System Preferences, I signed out of iCloud entirely and deleted all iCloud information on my Mac
5 - Shutdown/Restarted the mac
6 - I logged back into iCloud
7 - I opened Photos and waited till my most recent pictures showed up. (Mainly to make sure the iCloud services were working somewhere other than Contacts)
8 - I opened Contacts
when I finally opened my Contacts, it showed the little wheel that shows it's syncing, but to be honest, the contacts didn't show up immediately. I left it open in the background and THEN everything was back after 2 to 3 minutes. The thing is... after waiting the Messages app, I didn't touch it again until the whole process was done. Maybe something that Messages does (or did) was preventing from syncing properly? Like keeping a file locked somewhere? It's a wild guess, but it worked.
I have no idea how related it is too, but while ****** it wasn't working, after opening the Contacts app and seeing it didn't immediately sync, I fooled around with Siri a little. Talking to Siri may be entirely useless to fix the Contacts syncing problem, but since it's all iCloud based, who the heck knows. I asked her why my Contacts wouldn't sync. Then, they did! So no real idea what voodoo was it. But it's all working again.