TL;DR: Yes it will work just fine.
Long answer:
96W is fine just make sure to use a high capacity USB-C cable that can handle it.
I use iStat Pro to monitor the input the MacBook Pro 16" sees - I have the M1 but that also came with a 140W charger.
The only time you will exceed 100W is when the battery is nearly empty and the computer is running on full capacity, using CPU and GPU. That's a pretty rare case.
Once the battery reaches 50% it can no longer fast charge anyway - this is a limitation on the battery, all batteries work like that. So the power use for recharging also goes down a lot, it's now slow charging.
The laptop itself, when charged, won't use a lot of power. For example my M1 Pro draws 12W right now, with 100% Battery and plugged in, doing not much. M4 Max TDP is 50W - 80W. Most of the time it will be way less.
As far as I can tell the 140W is mostly for fast-charging. If you have very intense workflows, raytracing, rendering movies for hours, or training AI models, then ... maybe.
I checked my statistics, My M1 Pro never used more than 22W in the past 2 weeks. Max uses more energy but not that much more.
