Can I use apple watch ultra 2 purchased in USA in New Zealand?

usa, mexico, canada and porto rico use a diffrent eSim system than the rest of the world so if you get an usa version then it will only work as a non cellular gps only apple watch in new zealand

You would not have a warranty, as warranties are country specific because of different laws with respect to warranties.

While unlikely to be an issue, some countries impose restrictions on features Apple is allowed to enable in specific countries. It would be best to cross check U.S. vs New Zealand. Again, I do not expect there will be any issues.

https://www.apple.com/watchos/feature-availability

In theory the Apple Watch Ultra 2 is the same model in both countries, so it may work. Feel free to test this and report back so volunteers in this forum have some confirmation.