Hue and Homekit

Hi, I have numerous Hue bulb and other Homekit enabled devices on my network. For the most part they all work well with Homekit and I can control them using Siri om my HomePod mini. However up until recently I could control lamps in the kitchen and kids room fine, but after I bought a Hue Dimmer Switch for each room and connected it to those bulbs I can no loger control them from HomeKit, the are simply show as No response. I have tried adding the Dimmer in the HUE app as an accessory and mapped it to the corresponding room, this made no difference except no I cannot use it to control the bulb without having to pair it with the lamp again.

So the issue started when I paired each of the bulbs with a Dimmer Switch (latest model).

Is it possible to control the HUE bulbs using the Dimmer as well as HomeKit?

AS far as I know I'm on latest Software version, although one bulb is 1.53.3 and another is 1.90.1, but the result is the same