First time setup problem

I just got a new iPhone today and when setting up my phone for the first time I used an email address which is no longer in use. But I couldn't activate my account as the email was sent to this account which is no longer active. I tried to delete the account from the phone but it's asking for a password before I do that but I can't remember what the password is. I thought a reset would have asked me to create a new iCloud account but it doesn't. It's asking me for the password for the account I set the phone up with which I can't access and now the phone is stuck on setup after restore.

Any ideas how I can get around that or is it a case of having to visit an Apple store?