Thank you so much for taking time to try and help out.
I like to have each kamera angle in each a layer and it has never been a problem before.
If I should go and do what you suggest, then I would also have to re-grade everything, right?
Is there no solution where I can keep the work I have done?
If two layers came back from Color as you wanted, just re-send the one layer that needs work.
The "normal" work flow is to edit in a ProRes timeline and when you finish your cut, duplicate the sequence.
In the duplicate, collapse everything to a single track, remove all transitions, fades, slow motion, pans, zooms, etc so that all you have is a single ProRes track with full scale clips playing at the native frame rate. Send that to Color and you will never have issues.
When you get the clips back, reconnect them to the original sequence and life is good.