ICloud freezes when syncing and gives errors. fileproviderd takes 100% of cpu. EtreCheck provided

I just switched 3 days ago from a 2019 MacBook Air with an Intel chip to an M3 Air with 24gb ram and 1Tb storage. I used migration assistant from the old mac with a thunderbolt 3 cable. I used icloud drive on the old mac with syncing on for Desktop and Documents, pretty much only use Desktop. I only have about 2.5 gb in icloud drive. I also just upgraded from 50gbs icloud to 200gb. As soon as I switched over, I noticed that it is incredibly slow to move files, open files, rename, or move to trash on my Desktop. I mean like 1-5 minutes or it would crash, but if I used the terminal it was instant. These aren't big files either, like 150kb images I had just download couldn't even open. I spent like 5-6 hours yesterday with Apple support, I updated to the latest 14.4.1, from 14.3, booted into safe mode to check the disk which was all good, and I reinstalled my OS. Nothing worked.

ICloud in finder says it is trying to upload 1.2 gbs, it was at 787 mb when I noticed yesterday. It was going at about 5.5 mb per hour which would take about 72-80 hours. My internet isn't the problem. I have tried on 200 mbs upload and 11 mbs upload internet. Doesn't make a difference. fileproviderd was taking up 100% of my cpu. I ran fileproviderctl check -v and had some major problems, like thousands of files. After all of this and checking that in non icloud folders, on my hard drive, I had no problems. I unsynced and removed the files, I signed out of Icloud and restarted my mac. I went back in, turned it back in and it downloaded the files. Then there were 8 problem files with fileproviderctl check -v. They were from a project on github and were just the binaries that are in the .gitignore so that shouldn't be a major issue, I think. I moved the github projects to a hard drive file. fileproviderctl check -v said all good.

But then I accidentally moved a small 905 kb from on Desktop folder to the Desktop, and the problems continued, it instantly synced the 905kb, but there was an additional 45 kb of I assume metadata. It took 20 minutes then crashed and in finder, it said "ICloud is not syncing, there are pending changes to sync".

I have resorted to turning off Icloud sync with Desktop and Documents because it makes my computer unusable. Should I take it to the apple store or what should I do?