Finder takes 3 minutes to open a folder with 600k files

Question

Level 1

8 points

Finder takes 3 minutes to open a folder with 600k files

My system is a MacBook Air running macOS Catalina on a 2.2Ghz Dual-core i7 with 8GB of 1600 Mhz DDR3 Ram with a 500GB Apple SSD using APFS.

I created a folder with 600k files. I then opened that folder in Finder.

I got a loading screen for 3 minutes.

It then came up:

I also wanted to try this in a folder with 6M files but didn't as I expected it to behave a lot worse.

Is out of the box support for this type of use case expected to happen?

I can make the system I'm working on more complex by doing things like sharding at the folder level but it would be cool if I didn't have to worry about those implementation details and could pass to my users a simple mental model :).

MacBook Air 13", OS X 10.11

Posted on Jan 30, 2020 7:32 AM

Reply

Answer 1

Top-ranking reply

BDAqua

Level 10

245,495 points

Jan 30, 2020 11:05 AM in response to kcerb

8GB isn't much these days.

First opening will cause icons to be made & such, how long does it take to open a second time or after a restart?

Can I ask why 600K files, especially with only 500 GB & 8 GB of RAM?

& where exactly path wise did you create them? Hopefully not the Desktop folder.

Reply

Answer 2

kcerb Author

Level 1

8 points

Jan 30, 2020 6:31 PM in response to BDAqua

> 8GB isn't much these days.

The files are very small. The contents of the files adds up to about 1GB.

> how long does it take to open a second time or after a restart?

I did not try it a second time, but good suggestion. Okay, it takes just 7 seconds or so the second time. So something is doing some good cacheing...The responsiveness isn't so bad. (If this works I'd like to repeat with 6M files and then 200M).

> Can I ask why 600K files, especially with only 500 GB & 8 GB of RAM?

It's for a new simple database idea I have. Context is here if you are interested.

> & where exactly path wise did you create them? Hopefully not the Desktop folder.

Not in the desktop folder, haha. Just nested in a folder on my home dir. "/Users/breck/imdb/600k/".

Reply

Answer 3

kcerb Author

Level 1

8 points

Feb 24, 2020 11:22 AM in response to BDAqua

It's going well! Months of work to do from the early experiment. Might just fall back to a folder sharding solution. Or perhaps a Finder iteration will come out that could gracefully handle millions of files in a folder.

Reply