Select the colour by clicking on its cell:
Press Delete:
Selection, in this case moved from the now-empty third cell in row 2 to the (also empty) cell to its left.
Pages 11.2
macOS 12.1
If the two steps above do not result in the colour being deleted from this palette, an alternative would be to go for appearance without actually emptying the cell:
Select an object.
Set its Fill to "White Fill."
In the color picker, this will make the larger square left of the user palette show the same colour (White) as the selected object.
Click in that larger box to select the fill colour it contains, then drag that colour from the large square to the palette cell you want to 'erase' and drop the white fill there.
Downside: The palette cell now looks empty, but actually contains 'white', making it possible to accidentally fill an object (or text) with that colour.
Regards,
Barry