Automator: get x number of files (1000s), run workflow in chunks/batches of y (~50) files at a time?

I think Automator itself has your answer.

You need to break this into a two-step process.

First, you save the existing workflow as-is.

Then you create a second workflow. This workflow accepts files, and you add the Run Workflow action.

This action has the ability to invoke another workflow (i.e. your existing workflow), in a loop, passing in batches of the current input.

You can indicate the size of the batches (e.g. 50 files at a time), and how many to run concurrently (I suggest 1, but you may live life more dangerously). This workflow does the heavy-lifting of breaking your input into manageable chunks for the existing workflow to manage.

It's been three years and I didn't remember writing that linked solution. Re-read my last entry there about the maximum number of filename bytes in arguments that are allowed in a UNIX command line such as Terminal, or an Automator Run Shell script. In Mojave, it is a restricted 262,144 bytes and that is why you cannot drag and drop 10,000 filenames on an Automator application, or select that many in the Finder (if you could) for a Quick Action.

I think before offering a link to a 3rd-party Automator action — in fact, software that was written in 2007, you should have first tested it for compatibility with Automator on macOS Mojave or later. Things change and this solution may not work.