I've been having the same issue lately. It seemed to start when I installed Server on ML.
But here's something I also noticed. I added a new disk to TM and removed the old one from the list. The new disk was completely blank. Initially, the backup was going to take 4 hours. I let it run for 20 minutes, then I ran the main server app, which reserves a lot of system memory.
Once the system memory got reserved, the TM progress came down dramatically. The estimated backup time went to 6 hours. After a few hours, the estimated time went to multiple days.
Stopping the server app that takes a lot of memory doesn't help. It's as if TM sees the free memory and decides since nobody's using it, TM takes it all. Then, when a regular app starts to take all of the memory, TM decides to "be a good citizen" and let the other apps have priority.
Seems like Apple is trying to balance TM's priority with other application priority. But this attempt seems to overcompensate away from TM. It'll just never finish at this rate. My server writes more data than TM can ever archive.