Connect airport extreme by ethernet to router to improve wireless upstairs

If I just want to use it to improve my wireless coverage upstairs, do I choose Create a new wireless network, or extend a wireless network?

"Create a wireless network" is the correct setting. Use the same network name and password as your ISP router....and....make sure that the AirPort Extreme is set up to operate in Bridge Mode, which Apple calls "Off (Bridge Mode)".

When I choose the former, I have to manually connect to it on my iPhone to get the full wireless coverage - that doesn't seem right, I just want to extend the same network from downstairs.

This is not surprising since iPhones and iPads and most other mobile devices will not usually automatically "switch" over to another router if you move them from an area near one router to another area near another router.

For example, I normally keep my iPhone at my desk near the router in the office. If I move the iPhone to the den, where there is another router, the iPhone will usually stay connected to the router in the office, even if there is a stronger signal in the den.

When you move the iPhone near the AirPort router, temporarily turn off the WiFi on the iPhone and then turn it back on.....or.....temporarily turn Airplane Mode on and then switch it back off again. The iPhone will then usually......but not always.......pick up the signal from the closer router.

But if I choose the latter, it doesn't work and says something about the device needing to be in range of the network - but it's actually connected by ethernet, so the range shouldn't be an issue.

"Extend a wireless network" is used only if the AirPort is connecting to another Apple AirPort router using a wireless connection.....not an Ethernet connection. This setting is not compatible with a third party router, so it would never be used with a 3rd party router.