Phone Brightness keep changing

Turn off auto-brightness. Settings> Display & Brightness. Here you’ll find the option to disable auto-brightness. I don’t think True Tone is the issue. You could also try disabling the “Night Shift” option.

Any screen protection installed on the screen, or case around the phone, that might interfere with the brightness sensors on top the phone?

Try to force restart the phone first, before you set it back to factory settings, to make sure, that this is not software related.

Force restart iPhone - Apple Support

Restore your iPhone, iPad or iPod touch to factory settings

Is this your corporate iPhone? Do you have any profile installed by your corporate / organisation / enterprise / School Management then please contact the system administrator of your corporate / organisation / enterprise / School Management

If you have installed it on your own --> Install or remove configuration profiles on iPhone - Apple ...

You can turn auto-brightness on or off in Settings > Accessibility > Display & Text Size.