Unable to remove tags

I had a similar issue with some really stuck Finder tags in MacOS 10.15.5. Looked all over for a solution that worked with no success, including deleting all my .ds_store files with a terminal command, bit finally figured out the solution:

As it turns out Tags in Finder preferences are not stored in a file but exist as metadata in the actual files or in .ds_store (not sure which as it does not matter for the fix). One of the places it gets this info is from Spotlight. Just using a tool to rebuild the index does not seem to work. What is needed is to actually delete the entire index and then rebuild it. This is done the following way:

• Go into Finder Preferences/Tags and attempt to delete the stuck tag several times. If unsuccessful proceed to next step.
• Close Finder Preferences if open.
• Go into System Preferences and open Spotlight preferences.
• Click on Privacy
• Drag or Add your boot drive to the exclude list. This deletes the index for the drive.
• Now remove the boot drive from the exclude list. This starts the rebuilding process.
• Give it time to finish. You can check this by opening Spotlight from the menu bar it see if it is still indexing. The time needed will depend on the speed or your machine, drives, amount of content and drive size.
• When it is done, and not until, open Finder Preferences
• You will likely find the stuck Tags are gone.

Log into another admin user account, Set up users, guests, and groups on Mac,  and see if the problem persists. (This tells us if the problem is limited to your user account or is system wide).

Then boot into Safe Mode (How to use safe mode on your Mac - Apple Support) by booting with the Shift key held down and test there to see if the problem persists.  Reboot normally and test again.

NOTE: Safe Mode boot can take up to 10 minutes as it's doing some system cache cleaning, volume verifying and directory repairing. 

Have you tried Control (right) - clicking on the item and unchecking the tag in the resulting window?