27" Late 2009 Macs support Target Display Mode using Mini DisplayPort input.
Their Technical Specifications say that they support DisplayPort input, and there are no sneaky footnotes in those specifications saying that the input must be only from other old Macs. The implication was that those iMacs could act as displays for any DisplayPort-compliant video source that could present a signal with appropriate resolution. However, Apple later added restrictions such that the video source has to be a Mac with a hardware model year of 2019 or earlier, running Catalina or earlier. The M4 Mac mini is too new on both counts.
See the bad news here: Use your iMac as a display with target display mode – Apple Support
Thus, according to Apple's Support document, you can't use either of your 27" iMacs as Target Displays for a M4 Mac mini. Officially, you need a real display of some sort, such as a monitor, HDTV / UHDTV, or projector.
----------
That said, I have seen one or two threads where people reported success in using a 27" 2009 iMac, or a 27" 2010 iMac (I forget which), as a Target Display. Those were the years before iMacs got Thunderbolt (and thus, the years when TDM only required DisplayPort input instead of the harder-to-obtain Thunderbolt input).
So you might be able to use a 27" Late 2009 iMac that has a good graphics card as a Target Display, with the aid of a USB-C to Mini DisplayPort adapter cable (or equivalent). If it works, it will be unsupported, and you will need to be prepared for the chance that a macOS update could break your setup at any time, forcing you to buy a real monitor. But who knows, you might get lucky, and it might be "Christmas."
I do not think there is any chance of this working with the 27" Late 2009 iMac that has a bad graphics card. All of the video data that comes into an iMac in Target Display Mode will be going through that iMac's GPU on the way to the screen. The M4 Mac mini's GPU may have done all the work of compositing multiple objects into a series of single, large, bitmaps; but if the iMac's faulty GPU cannot render those bitmaps on the iMac's screen, all of the M4 mini's GPU's work will have gone for naught.