I forgot my Apple ID and the site is telling me it does not exist.

I tried signing into an old Apple ID and when I clicked on the forgot my password and typed in the email it said that it does not exist. Then when I tried to make a new account with the same email address, it tells me that the email is already being used. How can I get back onto the account or just delete it and start over?