Why do snapshots take so long for First Aid to process?

One more thing about this. Spotlight seems to be keeping the backup disks really busy for long periods of time. I haven't yet obtained a tool that can prove this, but one or the other of my Time Machine drives frequently makes noise and keeps its light flashing for long periods of time; while at the same time, the process mds is very active on the CPU page of Activity Monitor. So I'm guessing that macOS (Spotlight, or whatever) is indexing the Time Machine drives. I don't know why it needs to do that, but AFAIK it can't be avoided. NTL, in light of your post, I'm a lot less likely to run First Aid on them!

"Snapshots don't have any files at all. A snapshot is the entire disk

index. It is a pointer every single file that was on the disk when the

snapshot was made. Large files that have been fragmented and/or modified

will have multiple pointers."

Now that's the answer I was looking for!!! Finally!!!

Yes, I know. Those speeds are for large contiguous files without any folders within, hence, the "up to" part. There is all that file system overhead. The existence of lots of folders, like there are in many of my Final Cut libraries, doesn't help any! Yes, I know. But those speeds are better than nothing. I don't think they're irrelevant. They're much higher for the internal disk, just like you said. It's like a car in a race. Its top speed still tells you something, even when you have the overhead of all the curves in the road, traffic lights, etc. (Anh [rhymes with Poincare, or a little like "meh"], a rough analogy. But you know what I mean.)

As for the internal disk, even 2 TB would not be enough. All my footage and libraries would not fit. So I absolutely need the external drive. I can hear it now: "You should get a Mac Pro or Studio." Yeah, when I have mucho bucks, if that day ever comes, maybe then. Again, I was told by users in the Final Cut group that an external drive should be fine for Final Cut. So I went for it. Now, for half the bucks, I could get another SSD from OWC -- same size, much faster. Maybe FCLM will actually work right on that one. Maybe I'll get it -- maybe not.

I wish the cables fit better or more snugly. The "A-type" aren't too bad, but the B-type that looks like a square with a hump -- those are great! They are easy to plug in and make a nice solid connection. I think for at least one of my drives, the power cord connections are a little too "sensitive" or "fragile." I need to shut down and redo everything. I wish I had bought 1m cables instead of the 2m ones! Oh, the worst are the other B-type, the thin wide ones with two parts. Pain in the butt and don't have a real snug feel to them.

After checking for certain files on an old backup disk I retrieved and hooked up, I must have bumped something, as one or both of my backup disks went offline. I just plugged everything back in, and things were wacky for a while! The first backup appeared to be close to stuck at one point, and there were many md* processes, and they were CPU-consuming. But I didn't run First Aid, and in a few hours, the ship righted itself (!), as they say. I'm getting short backups about 1 hour apart as before.

BTW, Spotlight keeps the backup disks pretty busy at times! I think it's Spotlight. I have a small portable fan in my office, and it helps mask the constant "buzzing" (not quite the right word). Ohp! There are a lot of mdworker_shared processes right now, and mds is using roughly 20% CPU. And one of the backup drives is buzzing away and flashing. Ughhh.

"Snapshots don't have any files at all. A snapshot is the entire disk

index. It is a pointer every single file that was on the disk when the

snapshot was made. Large files that have been fragmented and/or modified

will have multiple pointers."

Now that's the answer I was looking for!!! Finally!!!

Yes, I know. Those speeds are for large contiguous files without any folders within, hence, the "up to" part. There is all that file system overhead. The existence of lots of folders, like there are in many of my Final Cut libraries, doesn't help any! Yes, I know. But those speeds are better than nothing. I don't think they're irrelevant. They're much higher for the internal disk, just like you said. It's like a car in a race. Its top speed still tells you something, even when you have the overhead of all the curves in the road, traffic lights, etc. (Anh [rhymes with Poincare, or a little like "meh"], a rough analogy. But you know what I mean.) Actually, if you have a car that can do 100 mph, it can go, say, 60 mph on certain curves. But another car that has a top speed of 30, cannot go 60 on those same curves. So it does help.

As for the internal disk, even 2 TB would not be enough. All my footage and libraries would not fit. So I absolutely need the external drive. I can hear it now: "You should get a Mac Pro or Studio." Yeah, when I have mucho bucks, if that day ever comes, maybe then. Again, I was told by users in the Final Cut group that an external SSD drive should be fine for Final Cut. So I went for it. Now, for half the bucks, I could get another SSD from OWC -- same size, much faster. Maybe FCLM will actually work right on that one. Maybe I'll get it -- maybe not.

I wish the cables fit better or more snugly. The "A-type" aren't too bad, but the B-type that looks like a square with a hump -- those are great! They are easy to plug in and make a nice, solid connection. I think for at least one of my drives, the power cord connections are a little too "sensitive" or "fragile." I need to shut down and redo everything. I wish I had bought 1m cables instead of the 2m ones! Oh, the worst are the other B-type, the thin wide ones with "two parts." Pain in the butt to put in and don't have a really good feel to them once they're in. They feel "sloppy," for lack of a better word that I can think of at the moment. Not robust.

After checking for certain files on an old backup disk I retrieved and hooked up, I must have bumped something, as one or both of my backup disks went offline. I just plugged everything back in, and things were wacky for a while! The first backup appeared to be close to stuck at one point, and there were many md* processes, and they were CPU-consuming. But I didn't run First Aid, and in a few hours, the ship righted itself (!), as they say. I'm getting short backups about 1 hour apart as before.

BTW, Spotlight keeps the backup disks pretty busy at times! I think it's Spotlight. I have a small portable fan in my office, and it helps mask the constant "buzzing" (not quite the right word). It's somewhere between buzzing and grinding or gravely. It's like irregular pulsing. Ohp! There are a lot of mdworker_shared processes right now, and mds is using roughly 20% CPU. And one of the backup drives is buzzing away and flashing. Ughhh. I don't remember this being the case. Maybe it's because there are now many more backups? No, sir; I don't like it!

The backup disks are finally quiet, and all those md* processes are gone! Good reason not to do regular reboots!

Is there anything in macOS that is not in the purview of the developers? There are users of this site who know things like this.

It's not stuck. I said it took several minutes per snapshot. I never said anything was stuck.

I am not erasing the backup drive. I'm pretty sure the backups are okay. Until I find out that the dataset is corrupt, only then will I erase the drive. Why do that when the odds are really high that all is okay? The First Aid pass will probably be done sometime later today, and if it comes up fine, there's no need to erase anything. I simply want to know why it takes so long, in particular, compared to everything else it does. (BTW, backups are very important! You might want to know what was in a file months ago, e.g. Backups are for more than just recovering from a sudden disk failure. For what you're talking about, you need a RAID array. Backups != data availability.)

"If it passes First Aid, and sometimes you do have to run it more than once, then you're home free. But then you have to consider it did take more than once to complete first Aid. That may or may not be significant."

I have no idea what this means.

This phenomenon where it takes several minutes to do a snapshot happens all the time on all disks that have snapshots. I'd be erasing everything by your advice. I'm just curious what it is that makes it take so long? Is it so terrible to ask such a question? I don't see why it's impossible that someone outside of the Apple dev population would know something about this. Maybe a dev person might even log in and answer it him- or herself! Apple once published how its old memory system worked, and if you didn't know, someone who read it might answer here.

I think I know the problem. There are times when I am just curious about something, and people assume there must be a problem. Well, actually there is a problem: why does it take so .... long?! Seems to me that this is the case for others, too. So what's the deal? It's a simple question.

I meant the problem with not getting a good answer. People always assume I need to fix something. I simply want to know what it is doing that takes so long. I am CURIOUS. Also, if there is a way to shorten the time it takes for the snapshots, or skip them altogether. I mean, really, it takes hours for the thing to run. I've got 77 snapshots on the disk I'm checking now. It will probably take 9 hours. And for what? That's what I'm asking. (So it's both curiosity and a question asking if I can shorten the elapsed time.)

"You can erase your backup drive with no harm to your system drive any time you want to. It is a backup, after all."

OK, I guess you meant if I found a problem. I thought you meant I should just go ahead and erase it as if it were no big deal. Even so, re "It is a backup after all." Backups are for restoring a failed disk, finding old versions of files, and recovering files you deleted by mistake. So they're not "just backups." What you seem to be talking about is data availability. And for that you use RAID (well, except for RAID 0, which I think is just striping), not backups. They're not the same. RAID means you can survive a disk failure or two without interrupting system operation. Useful in production environments. Backups mean you can recover lost data, which may or may not be from a disk failure.

Still, several minutes, like 5 to 10 or longer, for each snapshot? Still seems excessive. These are 16TB drives, so when they get full, it might take an entire 24 hours!

"People have an unnatural affinity for the history of their backups. It's a nice trick that Time Machine has."

Well, it's not just Time Machine. I used to do full and incremental backups on OpenVMS systems. One could easily go to an old tape and recover deleted or corrupted files. Oh, I did backups on the Stratus, too.

Well, one of the two backup disks is on UPS. But at some point you run out of outlets. I already have an extension power strip plugged into it, and those transformer blocks waste a lot of the outlets. Then I was plugging my UPS cable into the hub and two other disks gave me those ". . . eject first . . ." messages. And yes, the hub is on the UPS. Maybe it's another bad hub. It's been working much better than my last one. Time to review and perhaps redo the whole setup.

And yes, there was a brief power outage yesterday. Probably at about 21:xx EDT.

Thanks! (^_^(

"Never run Disk First Aid on a Time Machine backup."

Well, it's too late. I already ran it on the one that wasn't on the UPS when there was a brief power outage here. (It came up clean, BTW.) Then I noticed the UPS cable wasn't plugged into the hub. So I plugged it in and two other disks gave the ". . . eject first . . ." notifications. I'm presently running First Aid on the 2nd Time Machine drive. Unlike the other one that wasn't on the UPS, I had to use a manual command in Terminal:

% time sudo diskutil verifyVolume /dev/disk9s2

It's now checking snapshot 41 of 77. Should I ^C it and configure it back into Time Machine? Or just let it run its course?

There were 3 drives involved:

TM disk 1 - F.A. is running now. Had gotten the ". . . eject . . ." msg. when I was plugging the UPS comm cable into the hub. It's checking snapshot 41 of 77.

TM disk 2 was not on UPS, so I ran F.A. on it. Came up clean.

External hard drive for ordinary storage. Got the ". . . eject . . ." message on this when I plugged the UPS comm cable into the hub. Ran F.A. on it, ran really quickly, and came up clean.

A year or two ago, I once tried copying a snapshot. Yeah, that didn't go too well! Never tried to delete one, though! But what to do if the TM drive is failing, and you want to copy your data off fast? I guess you better have another backup going in parallel, which I do. Yeah, your backups are safe, unless the disk dies.

BONUS: As for wear and tear, isn't it good if the disk doesn't stop spinning for sleep? I hate waiting for a disk to spin up, so I wish I could turn this "energy saver" feature off. Yeah, how much energy do I save if I have to go to the store that much sooner?

TIA!

My Time Machine HDDs are definitely less than 5 Gbps. Apparently much less. I have downloaded the AJA app, and it won't work on Time Machine drives. Looks like nothing but Time Machine itself can do anything to them. If a TM drive is failing, you can't copy off the backups! Well, hopefully it's at least robust, ransomware-proof, and will restore files and folders correctly, unlike my last restore operation with an older version of TM, which I had to do due to a failed drive. It put the Movies and iTunes or Music folders in the wrong places, and missed some files, IIRC! Probably best to have two running in parallel, which is what I do. For really old backups, I guess the only way to be sure you can go as far back as you like is to buy that with an offsite/online backup service.

Anyway, the claimed speed on these TM drives (OWC), at least currently, is "up to 252MB/s real-world speed." But that's probably for a large file, not something like snapshots, which have many thousands of files and folders. I assume there's a lot of overhead with that structure!

The great thing about Time Machine is that it makes it easy to restore files that go into a database, like Mail, TV, Music and such. macOS does it for you!

Running AJA on my external SSD I get about 800 MB/s average. Not bad! The M1 Mac gets like 2300 MB/s!

I still think it takes an incredibly long time for "First Aid" to run if you have a fair number of snapshots. Wow, I just ran AJA on the internal drive and got 3139 MB/s write and 1898 MB/s read!

The external SSD: Just got 838 MB/s write and 766 MB/s read. I think the advertised value was "up to 1025 MB/s" or similar. 

I've had mixed success with WD. One drive was broken right out of the box! I couldn't even initialize it. I could hear the head assembly going back and forth in spurts maybe 1/2 second apart.

I had a SanDisk 2TB SSD that would occasionally brick on me, and now it's just dead. I've heard that a lot of them were going south!