Is it worth it to just get an Apple Watch Ultra?

You're right IF it would be worth it to you. Before Apple Watch Ultra, I ONLY bought Apple Watch in Stainless, which also has a Sapphire screen. So in your scenario, I wouldn't have likely caused much of any damage to my screen. But this year when it was time to upgrade my Apple Watch, I went with the Ultra, which also has a Sapphire screen. Do not have a misconceived impression that Titanium can't scratch, however. Titanium can scratch for sure. But in both the Ultra and Stainless models, the screen is made of the same materials and very difficult to scratch.

I've had the aluminum GPS only Series 4 since it came out four+ years ago, and it is still working very nicely.

If you are the type to constantly get your watch arm into situations where you can seriously scratch or break the watch face, then sure, get the better watch face. For example someone that works with their hands in a tight spot, like vehicle machanics, plumers, HVAC, etc...

If you are an office worker, that is less likely. Yes, I'm an office worker.

While people do break their Apple Watch, it is statistally low.

How you spend your money is totally your decision.