How do I deauthorize a computer that no longer works?

I am trying to convert all of my music from my iPhone to my new Macbook, however it keeps telling me that my Apple ID is authorized to 5 computers already. But my Apple ID is technically only connected to one other computer that works, as the other four are past computers that we have sold or traded in or just plain don't work anymore. How do I deauthorize those so I can have my music on my new computer?