What did you do guys after all!? I found I have the same issue like you guys, iPhone Xs Gold 64 GB. Is my 3rd replacement, First 2 have physical damage out of the box, firs on the back side of the glass on the beautiful gold colour have a black dot, and second out of the box was display cracked in the middle, with the third all perfect just this issue with brightness, I found by me I experienced this just when is combined with reduce white point from accessibility , if reduce white point is disabled don’t happens , I call the support and they said is 100% a hardware issue, hmm and they said I am the only one with this issue, but how I said? I give you the thread number from apple discussions, after 15 min of waiting the answer was send the info to engineers and my phone they send me a box to send it for repair, after almost 2 months, 1149 € paid and 2 replacement after first Xs, I send my iPhone to repair, is very convincing. I believe in a change of this kind of problems with the first units almost every year with different issues , new gates.....And who we have same issue we must to call support to tell this, if are more same complains will be a fast solution available, maybe just a new update!