How can I get the SUM formula to output “0h 0m” or “0h” instead of just “0”

Not sure I follow the screenshots, but 0 indicates the cell contains a Number value while “0h 0m” and “0h” indicate that the cell contains a Duration value.

Durations (spans of time as opposed to a point in time) result from subtracting one Date & Time from another or by adding or subtracting a number (which Numbers interprets as days) to or from a Date & Time. Adding a Duration to a Duration also results in a Duration.

Durations can be input using w d h m ms shorthand.

Documentation on Number vs Duration data formats is terse, but worth a read:

Format dates, currency, and more in Numbers on Mac - Apple Support

SG

Assuming the cell value is something that Numbers can interpret as a duration, use the Cell Format inspector:

Set the Cell Format to Duration, then under Custom Units you can choose which units (weeks, days, hours, minutes, seconds, milliseconds) to display.

If all the cells included in the SUM are empty, the sum will be the number 0 and you cannot format it as a duration. But you can include DURATION(0) in the SUM, or added to it, to force it to be a duration. Format the cell to show it as 0h. Or you can use DURATION(,,0) and use automatic formatting.

Examples:

=SUM(B6:O6,DURATION(0))

or

=DURATION(0)+SUM(B6:O6)