Does it hurt the battery to leave a brand new phone in its box for months without charging it up?

Note that your lithium battery degrades whether used or not. Every lithium battery has a finite life span, and if left truly static on a shelf for a many months or a year or so will show substantial degradation or be useless.

Lithium batteries, once assembled at the factory, will provide the longest useful lifespan if they experience regular use (partial discharge/charge cycles). So buying a “spare” and just letting it sit idle for many months or longer is basically counter productive. Rechargeable Lithium batteries are best bought when needed, and then used for as long as they provide useable capacity.

At least once a week or so it would be best then to turn that phone on for awhile and then charge it some so it gets some active current flow through it.