I had similar issue. One part is getting it working, the second is keeping it stable.
my primary use is through hdmi cable - this gets 4K working fine and every time (through a small USB-C hub). I have got used to this and the resolution is fine (I am mostly working on it and the 5k seems to be not much better)
my previous intel based Mac only seemed to for 4K and one of the reasons to get the M1 Mac was to use the full 5k but ironically the old Mac now works fine but the new one doesn’t (it works every time I plug it into that one but this one takes some playing around)
i haven’t found a consistent method yet but it’s a matter of cable swapping between old Mac and new Mac, having the display working on one and then switch it over to the other and/or switching between the hub (with hdmi attached) and/or rebooting etc... it’s a mess.
so far, as above, I have got it working by switching to display port version 1.2 but if I power down and on again, often it stops working... I think it’s to do with how the usbc’s 2 way interaction is not being recognised... I think that the latest/next macOS patch is the key. You can have a second device, plug it into that and switch over, option click in the display settings to detect external monitor, boot with it plugged in or not, turn it on and off and other combinations and it works after an indeterminate sequence but I am now just going to wait a few weeks hoping that the next macOS patch will have the fix