Is it safe to take my iPhone with me if it's below freezing outside or will it damage the battery?

Straight from Apple:

Operating ambient temperature: 32° to 95° F (0° to 35° C)

Non‑operating temperature: −4° to 113° F (−20° to 45° C)

if you use it in cold temperatures, yes the battery will drain faster.

it shouldn’t damage the battery unless you completely drain it.

Just keep it in your pocket and it will be fine.

To expand a bit on what KiltedTim said, keep it in an inside pocket, as close to your body as possible. The battery won't be damaged by being out in the cold (unless you leave it out in the cold for days or weeks) but it will drain very quickly the colder the phone gets.