Can a zsh script to batch zip files work recursivly? (VikingOSX script in post)

Last month VikingOSX graciously coded a script to batch process particular sets of files into archive zips that take on the name of the files. From this post. It rocks. I've zipped about 50,000 files since then.


I have recently learned a new vocabulary word: recursive. Yes, I'm new to the command line.


Could the script below be changed so that it works recursively on all sets of designated files (jpg, png, eps, svg, in this case) in the directory AND subdirectories of that directory? If so, would it work even if some directories had no files? And would it ignore (not archive) any other file type than the ones designated in the script ((jpg, png, eps, svg)?


#!/bin/zsh

: <<"COMMENT"
For a given folder containing filenames identified by product code and suffix,
aggregate the four image files associated with it into a zip container.

For example: Dzn-4-n-m-no.zip contains the four images: Dzn-4-n-m-no.{eps,jpg,png,svg}

Tested: macOS 14.1.2 (Zsh 5.9), macOS 10.14.6 (Zsh 5.3)
Revision: 2
Author: VikingOSX, 2023-12-03, Apple Support Communities, No warranties implied.

Usage: lab.zsh folder
COMMENT

# absolute path of folder passed as only argument to the script
ZIPDIR="${1:a:s/\~\//}"

# image types to process
EXTPAT="eps|jpg|png|svg"
last_ext=
last_fname=

setopt nocaseglob
for f in "${ZIPDIR}"/*.(${~EXTPAT})(.Non);
do
    # Two captures: product name and two or three character suffix
    [[ "${f:t}" =~ "(^.*\-)([[:alpha:]]{2,3})\..*$" ]] || continue
    [[ $last_fname != "${match[1]}"  || $last_ext != "${match[2]}" ]] || continue
    last_fname="${match[1]}"
    last_ext="${match[2]}"
    # capture unique suffixes to their respective zip file excluding dot files
    zip -qj -6 "${ZIPDIR}/${(j::)match[@]}".zip "${ZIPDIR}/${(j::)match[@]}".{eps,jpg,png,svg} -x ".*"
    # rm -f "${ZIPDIR}/${(j::)match[@]}".{eps,jpg,png,svg}
done
zipcnt=( "${ZIPDIR}"/*.zip )
echo "Zip archives created: $#zipcnt"
unset zipcnt
exit 0



Posted on Jan 14, 2024 1:10 AM

Reply

Similar questions

41 replies

Jan 15, 2024 12:35 PM in response to swapot

Did you launch the script as:


./golden.zsh ~/Desktop/1


That would have set the ZIPDIR variable to /Users/nyhomename/Desktop/1


and the for loop:


for f in "${ZIPDIR}"/**/*.(${~EXPTPAT})(.Non);


would have done recursion in that ZIPDIR and below to find all of the EXTPAT files. And, if there were no other errors, one or more zip files would have been written in ZIPDIR.


My line 36 based on the full script I posted above and your error:

zipcnt=( "${ZIPDIR}"/*.zip )


Jan 16, 2024 8:23 AM in response to swapot

You only need to make the script executable once. Edits don't affect that.


The error message above means that the script is not finding the files it needs to produce the .zip archive because no reference was made to the subdirectory in which the files were located. I have fixed this, assuming the following sample folder and filenaming convention:



And after processing, but before uncommenting individual folder file removal:


and the contents of the *tau.zip file:



and the revised script code to achieve the preceding:


#!/bin/zsh

: <<"COMMENT"
For a given folder containing filenames identified by product code and suffix,
aggregate the four image files associated with it into a zip container.

For example: Dzn-4-n-m-no.zip contains the four images: Dzn-4-n-m-no.{eps,jpg,png,svg}

Tested: macOS 14.1.2 (Zsh 5.9), macOS 10.14.6 (Zsh 5.3)
Revision: 2
Author: VikingOSX, 2023-12-03, Apple Support Communities, No warranties implied.

Usage: golden.zsh folder
COMMENT

# absolute path of folder passed as only argument to the script
ZIPDIR="${1:a:s/\~\//}"

# image types to process
EXTPAT="eps|jpg|png|svg"
last_ext=
last_fname=

setopt nocaseglob
for f in "${ZIPDIR}"/**/*.(${~EXTPAT})(.Non);
do
    # Two captures: product name and two or three character suffix
    [[ "${f:t}" =~ "(^.*\-)([[:alpha:]]{2,3})\..*$" ]] || continue
    [[ $last_fname != "${match[1]}"  || $last_ext != "${match[2]}" ]] || continue
    last_fname="${match[1]}"
    last_ext="${match[2]}"
    NAME="${(j::)match[@]}"
    # capture unique suffixes to their respective zip file excluding dot files
    zip -qj -6 "${ZIPDIR}/${NAME}".zip "${ZIPDIR}/${NAME}/${NAME}".{eps,jpg,png,svg} -x ".*"
    # rm -f "${ZIPDIR}/${NAME}/${NAME}".{eps,jpg,png,svg}
done
zipcnt=( "${ZIPDIR}"/*.zip )
echo "Zip archives created: $#zipcnt"
unset zipcnt
exit 0

Tested: macOS Sonoma 14.2.1



Jan 17, 2024 2:08 PM in response to swapot

Ok. Here is what I have now. I created your example folder structure and am generating *.zip files just inside the designated ZIPDIR.



And only filenames, no paths inside the zip containers:



and the much revised and reduced code to generate those zip files:


#!/bin/zsh

: <<"COMMENT"
For a given folder containing filenames identified by product code and suffix,
aggregate the four image files associated with it into a zip container without paths

Reference: https://discussions.apple.com/thread/255411922?sortBy=oldest_first
Tested: macOS 14.1.2 (Zsh 5.9), macOS 10.14.6 (Zsh 5.3)
Revision: 4 (2023-01-17)
Author: VikingOSX, 2023-12-03, Apple Support Communities, No warranties implied.

Usage: golden.zsh folder
COMMENT

# absolute path of folder passed as only argument to the script
ZIPDIR="${1:a:s/\~\//}"

# image types to process
EXTPAT="eps|jpg|png|svg"

# case-insensitive
setopt nocaseglob
for f in "${ZIPDIR}"/**/*.(${~EXTPAT})(.Non);
do
	ZIPNAME="${ZIPDIR}/${${f:h:r:t4}:gs/\//-}".zip
	zip -qj -6 "${ZIPNAME}" ${ZIPDIR}/${f:h:r:t3}/*.{eps,jpg,png,svg} -x ".*"
    # rm -f "${ZIPDIR}/${f:h:r:t3/*.{eps,jpg,png,svg}
done

zipcnt=( "${ZIPDIR}"/*.zip )
echo "Zip archives created: $#zipcnt"
unset zipcnt
exit 0


Feb 19, 2024 1:41 PM in response to swapot

The current script will capture all text up to the second "-" and the last three characters of the basename and join these to name the zip file. It will follow that pattern for any folder contents whatever the filename.


Try this:


#!/bin/zsh

: <<"COMMENT"
Collect all files ending in the EXTPAT extensions into zip containers based on 
(filename-name-)n-(xxx).ext captures () and name the zip file with those joined
capture strings written into the parent directory.

Reference: https://discussions.apple.com/thread/255411922?sortBy=oldest_first&page=1
Tested: macOS 14.3.1
Revision: 3
Author: VikingOSX, 2024-02-19, Apple Support Communities, no warranties.
Usage: mkzip.zsh folder
COMMENT

STARTDIR="${@:a:s/\~\//}"
EXTPAT="eps|jpg|png|svg"

setopt nocaseglob
for f in "${STARTDIR}"/**/*.(${~EXTPAT})(.N);
do
    # match against filename.ext for (Filename-word-)-x-(xxx).ext
    # no match get next file
    # [[ "${f:t}" =~ ".*-([0-9]+-[a-z]{3})\..{3}$" ]] || continue
    [[ "${f:t}" =~ "(.*-)[a-z]{1}-([a-z]{3})\..{3}$" ]] || continue
    /usr/bin/zip -jqu -6 -X "${STARTDIR}/${(j::)match[@]}".zip "${f:a}"
done
exit 0


Jan 17, 2024 12:13 PM in response to swapot

Your folder structure is not what the Jan 16 code expects or clearly that I have tested against. It may be a deal breaker as the zip invocation has to be hardcoded for the directory structure path to reach your current last folder's files. The other thing I observed is that there are a mixture of files ending with a different ending suffix (e.g. -brp,-brr,-ber, etc) in the same directory, so that will require a different approach too.


Let me recreate my test folder to your hierarchy and retest. This may take some fiddling… and more time to see if it is salvageable.


The tool that I use to show hierarchy trees is named ironically named tree.


brew info tree
brew install tree
tree -n ~/Desktop/1
man tree


With Mojave 10.14.6, the last compatible Xcode version is 11.3.1. Xcode has nothing to do with Zsh scripts.

Feb 17, 2024 1:13 PM in response to VikingOSX

The following will attempt to match filename-name-(n-xxx).* where n may be any number of digits. It will create zip files named 1-xxx.zip, 2-xxx.zip, nnn-xxx.zip with only those files in each.


#!/bin/zsh

: <<"COMMENT"
Collect all files ending in the EXTPAT extensions into zip containers based on 
filename-name-(n-xxx).ext where n-xxx is the match that creates the zip filename
that contains similar named filenames.

Reference: https://discussions.apple.com/thread/255411922?sortBy=oldest_first&page=1
Tested: macOS 14.3.1
Revision: 2
Author: VikingOSX, 2024-02-17, Apple Support Communities, no warranties.
Usage: mkzipx.zsh folder
COMMENT

STARTDIR="${@:a:s/\~\//}"
EXTPAT="eps|jpg|png|svg"

setopt nocaseglob
for f in "${STARTDIR}"/*.(${~EXTPAT})(.N);
do
    # match against filename.ext for nnn-xxx
    # no match get next file
    [[ "${f:t}" =~ ".*-([0-9]+-[a-z]{3})\..{3}$" ]] || continue
    
    /usr/bin/zip -jqu -6 -X "${STARTDIR}/${match}".zip "${STARTDIR}/${f:t}"
done
exit 0


with this result:


Jan 14, 2024 5:11 AM in response to swapot

In Zsh, recursion into sub-folders from a parent folder is done as follows using the script above as its source:


for f in "${ZIPDIR}"/**/*.(${~EXPTPAT})(.Non);
do
   ...
done

The "**" is Zsh's way of invoking recursion. The (.Non) is looking for regular (.) non-dot files sorted ascending by name (.on) order. Sorting is purely optional here, but the (.N) is not optional when chasing after files.


The script will only process files whose extension is in the pattern EXTPAT. It will skip over folders where no files matching that pattern are found.

Jan 14, 2024 10:03 AM in response to VikingOSX

I somewhat understand and totally appreciate the explanation. I learned of recursion this past week while working on files with exiftool by Phil Harvey. I'm not exactly sure of how to implement the above. Do I just incorporate that code into the script?


Guessing:


 #!/bin/zsh

: <<"COMMENT"
For a given folder containing filenames identified by product code and suffix,
aggregate the four image files associated with it into a zip container.

For example: Dzn-4-n-m-no.zip contains the four images: Dzn-4-n-m-no.{eps,jpg,png,svg}

Tested: macOS 14.1.2 (Zsh 5.9), macOS 10.14.6 (Zsh 5.3)
Revision: 2
Author: VikingOSX, 2023-12-03, Apple Support Communities, No warranties implied.

Usage: lab.zsh folder
COMMENT

# absolute path of folder passed as only argument to the script
ZIPDIR="${1:a:s/\~\//}"

# image types to process
EXTPAT="eps|jpg|png|svg"
last_ext=
last_fname=

setopt nocaseglob
for f in "${ZIPDIR}"/**/*.(${~EXPTPAT})(.Non);
do
    # Two captures: product name and two or three character suffix
    [[ "${f:t}" =~ "(^.*\-)([[:alpha:]]{2,3})\..*$" ]] || continue
    [[ $last_fname != "${match[1]}"  || $last_ext != "${match[2]}" ]] || continue
    last_fname="${match[1]}"
    last_ext="${match[2]}"
    # capture unique suffixes to their respective zip file excluding dot files
    zip -qj -6 "${ZIPDIR}/${(j::)match[@]}".zip "${ZIPDIR}/${(j::)match[@]}".{eps,jpg,png,svg} -x ".*"
    # rm -f "${ZIPDIR}/${(j::)match[@]}".{eps,jpg,png,svg}
done
zipcnt=( "${ZIPDIR}"/*.zip )
echo "Zip archives created: $#zipcnt"
unset zipcnt
exit 0


I'm guessing the

   ...

was a placeholder for what was already there? Pardon the question, I really haven't had experience with code before.


Thanks for the speedy response. This will really accelerate my workflow.

Jan 19, 2024 6:00 AM in response to swapot

As I mentioned earlier, the code as shown in your last Jan 18 8:03 pm post works fine on Sonoma 14.2.1 with Zsh 5.9, but fails with Zsh 5.8.1 on Mojave because of features not yet introduced in Zsh 5.8.1. I spent a couple of hours Thu evening trying to make the script work on Mojave — without success. It is also spewing an error line for each iteration of the loop for a reason not in the code, nor the line number for that condition. Fun.


I introduced branch logic testing for Zsh version older, or Zsh 5.9 or newer. The branch works but still struggling with the code to generate the zip files without having the Zsh 5.9 features to help.


I have prior commitments Friday AM, and then will visit the macOS 10.14.6 coding again this afternoon.

Feb 17, 2024 9:55 AM in response to swapot

For your Feb 16, 2024 request, this will generate a ber.zip or stb.zip, or any other three-letter code zip file from the files in the folder.


#!/bin/zsh

: <<"COMMENT"
Collect all files ending in the EXTPAT extensions into zip containers based on 
whether those files have a three-letter code at the end of the file basename. Like
files containing that three-letter code will be added to zip containers named after
that three-letter code (e.g. stb.zip, ber.zip, etc.)

Reference: https://discussions.apple.com/thread/255411922?sortBy=oldest_first&page=1
Tested: macOS 14.3.1
Author: VikingOSX, 2024-02-17, Apple Support Communities, no warranties.

Usage: mkzip.zsh folder
COMMENT

STARTDIR="${@:a:s/\~\//}"
EXTPAT="eps|jpg|png|svg"

setopt nocaseglob
for f in "${STARTDIR}"/*.(${~EXTPAT})(.N);
do
    # match against filename.ext for three-letter code (ber, stb, etc.)
    # no match get next file
    [[ "${f:t}" =~ ".*\-([a-z]{3})\..{3}$" ]] || continue
    
    /usr/bin/zip -jqu -6 -X "${STARTDIR}/${match}".zip "${STARTDIR}/${f:t}"
done
exit 0


And the contents of a ber.zip based on your generic filenames above:


Feb 19, 2024 8:13 AM in response to swapot

What I need to end up with from the real filename example further above is:


LOVE-Block-ber.zip (containing the 20 files ending in "ber")

LOVE-Block-blr.zip (containing the 20 files ending in "blr")

---

This is a contradiction to your earlier request of 1-ber.zip format that I changed in the last posted script solution on 2024-02-17.


Are there files in these folders that contain different characters in the -x- position that you want excluded from those that contain the (l) (o) (v) (e) (a) (m) (r) (h) (d) characters? If not, the capture should be the string "LOVE-Block-" and any of the last three "xxx" characters of the filename before the extension as the preceding character matches wouldn't be necessary to build the zip filename - but necessary if there were exclusions.


(l) (o) (v) (e) (a) (m) (r) (h) (d) are not alphanumeric characters, they are alpha only (e.g. a-z).


I have a revised version of the script that constructs the Filename-blr.zip or Filename-ber.zip from your examples above. I will post that when we are agreed on the changes necessary.

Jan 15, 2024 11:25 AM in response to VikingOSX

Hi VikingOSX,


I tried the "recursive" script for the first time today.


I got the following error:


./golden.zsh:36: no matches found: /Users/myhomename/Desktop/1/*.zip


I renamed the script "golden" for my purposes and changed line 13 on the script to:


Usage: golden.zsh folder


I do have sets of the four file types (eps, jpg, png, svg) in directory "1" on the desktop, but they are in sub-directories of "1" - none are in the top level directory named "1".


Does something else need to be added or tweaked for it to work recursively through a directory?

This thread has been closed by the system or the community team. You may vote for any posts you find helpful, or search the Community for additional answers.

Can a zsh script to batch zip files work recursivly? (VikingOSX script in post)

Welcome to Apple Support Community
A forum where Apple customers help each other with their products. Get started with your Apple Account.