Is anything close to the cold-air intake vents on the left and right side where the chassis meets the bottom plate?
The critical hot-air exhaust area is along the hinge. I make sure my M4 Pro's hinge is at least four inches (100mm) from any object on the work surface. Any blockage of that area can run the temps fast.
Are you using an external monitor? They commonly increase the workload. If you use an external and you close the M4's lid and place it vertically, such as on a shelf, the hinge side must be UP for proper cooling.
Have you installed any unneeded utilities like anti-virus or so-called "cleaning" apps? They generally cause more problems than they cure due to their interfering with Apple's built-in self-maintenance and security routines.