Time Machine keeps stalling during back up

I have 2 different time machine backups. I will plug one in do a time machine back up, and then later connect the other one and do a back up. I have had both stall at different times.

The drives I use are partitioned, so I have 250GB partition for time machine on both drives.

Sometimes it does work. When I format the partition and do a brand new first time backup, it does it fine.

What keeps screwing it up?