Latency with Track folders in Logic Pro X

Hello all. Just came across this issue.

Issue is when using take folders in Logic x 10.2.4, the first 2 or 3 takes are in time, then all heck breaks loose and the others are out of time.

I am using Apollo 8 Thunderbolt. OS 10.11.3

I am using low latency mode during recording. I do have plugs on the master buss but they are turned off during recording. Curious if that is enough as I know when I work in Protools just bypassing them is not enough. They have to be made inactive.

Anyone have any ideas? This does not happen in Protools 12 as I use both DAW's so it is Logic issue.

Takes 1 and 2 are on the dime...however after that the other takes are all off.. but by the same amount... This is the confusing part. This is even with Low latency enabled. I read elsewhere someone else having the exact same issue. He said he had to go into the Devices, and basically change the buffer to another setting, then set it back to where he was working. The prob goes away but does come back.

I trashed the Pref in Logic and re initalized my Apollo 8 and have done some testing. Seems ok at the moment, but this is like the 3rd or 4th time of having to trash the pref in Logic. I have settings that I like to have that are personal so it is a drag every time to reset Pro X back to my working environment. I do have a screen shot of this.. Please view how takes 1 and 2 are in time, and the others are out. Thank you