One approach may be to move the Xcode application to the Trash and empty the Trash. Then press shift+cmd+G (e.g. go to folder) and enter the following:
/Library/Developer
and then drag and drop the CommandLineTools folder to the Trash, emptying it. This will likely challenge you for your admin password before you can do the drag/drop.
Locate Xcode v16.4 and download Xcode 16.4.xip (direct download link).
This may require you to sign in with your Apple ID unless you are already signed into the Apple Support Community in another tab. In your Downloads folder, just double-click the Xcode xip file and wait while it unpacks it in place. Then simply drag and drop Xcode to your /Application folder as before. Then run it to finish its installation.
This entire process should dispense with another 3 GB Commandline tools for Xcode 16.4 appearing in Software Update.