I struggled with this for six hours and finally decided to call my router manufacturer (TP Link), and got a solution. It was a problem with the settings on my router. The combination of the signal width and security type on my wifi settings didn't seem to work with my iMac even though they worked with a MacBooks, iPad etc. I'm not sure if this was due to the age of my system or not but here's what I was told to do:
Reset the router to factory settings (you can probably try without doing this, but I'd recommend it to isolate this from any other issues)
Go to System Preferences - network - WiFi - advanced, click the minus next to your old network
Set the IPv4 option to DHCP
Try and sign into the router login page (usually 192.168.0.1 or 192.168.1.1), to test if you can connect to it, if you can:
Change the router signal width from 40mhz to 20mhz
Change the wireless channel to 11.
Rename your network.
Change the wireless security type to WEP and set a password of ten numbers
On your mac:
Remove your network from network settings again, then reconnect with the password you just set
Try and connect to the internet
It should work now, but change back the security to WPA/PSK again and set a password, as this is much more secure than WEP.
I believe that random 169..... IP address is because for some reason there is a failure in connection between the router and mac, and it is essentially a 'wild guess' built into OS. So I think for many people this is likely to be a problem with your router settings rather than your Mac.
Hopefully this helps somebody, I felt like it would be looking for a needle in a hay stack otherwise.