FTP: how to get total number of files in directory

Try "pls" instead of "ls" in ftp:

% export PAGER='less -M'
% ftp ftp.remote.host
ftp> pls

This will list the contents of the remote current directory by using the "less" command. Hit "G" to go to the end of the list. Then prompt of less will show how many lines are there. (Type ctrl-G if the prompt shows no info).

Or you can use a local temporary file:

ftp> ls . tmpfile
output to local-file: tmpfile [anpqy?]? y
229 Entering Extended Passive Mode (|||54412|)
150 Here comes the directory listing.
 86851 50.55 KB/s 
226 Directory send OK.
ftp> !wc tmpfile
 1335 12015 85516 tmpfile

Here, "ls . tmpfile" indicates to save the listing of the current remote directory "." to the local file "tmpfile", and "!wc tmpfile" is to run "wc tmpfile" locally.

For times like this you may want consider using ( ssh ) and ( scp ).

Your local command [ ls | wc -l ] would translate to [ ssh user@host ls | wc -l ],
this way you don't even need ( ftp ). Once you get the remote list of files,
you can use ( scp ) to get them.


-Bruce-

Sorry about the strike-though. I didn't know my brackets would be treated as formatting delimiters.

So...

AS-LOCAL : ls -l | wc -l
AS-REMOTE: ssh username@hostname ls -l | wc -l

sorry if someone already suggested this.

curl -l -s -u username ftp://hostname/directory/ | wc -l

Message was edited by: Nils C. Anderson

I've done a lot of ftp scripts 10 years ago, mostly because I could not guarantee my scripted telnet would be available everywhere. The best answer today though is listed throughout this thread and that is to use the ssh family of commands that include ssh for secure remote shell, scp for secure remote copy and sftp which is the ftp interface to scp.

My point is neither telnet or ftp are considered safe or secure anymore because listening to communications is popular, it is referenced in most "wizard" magazines, and usernames and passwords are not encrypted.

Set up your environment first with ssh-keygen. Version 2 is more secure than version 1 if both are still offered. Then use "ssh user@hostname ls | wc -l" to get your number of files. It is simple execution with one command and returns a status.

If you are really looking for a solution using what you are familar with and want to ignore advice concerning security, here is a sample batch script (untested). You can change the filenames, usernames, and hostnames as desired.

rm dir.out 2> /dev/null
echo "user sampleuser samplepw" > script.ftp
echo "dir sampledirectory dir.out" >> script.ftp
echo "quit" >> script.ftp
ftp -n sample.hostname < script.ftp
if [ ! -r dir.out ] # file not created
then
echo login failed
elif [ ! -s dir.out ] # file created but empty
then
echo directory is empty
else
nf=`wc -l dir.out | cut -f1`
echo "There are $nf files in the directory"
fi

Ouch! The http:// is a translation error by the forum application and should be square brackets surrounding the test condition.

{quote:title=Doug Niven1 wrote:}
I have directories with 3000 or 4000 files and I'm trying to get the exact number in the parent directory, but I just don't think there's a way to do this, other than zip archive the directory and download, which is what I wanted to avoid.

Further brilliant ideas?
{quote}

Did you see Bruce's suggestion to run ls | wc -l on the remote machine via ssh? When running a remote command via ssh the results are printed to your console and can be saved locally. I think that would solve your problem.

--
Cole

Message was edited by: Cole Tierney

Is there a reason you need to use ftp? Just to be clear, if you run:

ssh username@hostname ls -l | wc -l > somefile

...somefile will be on your local machine and not on the remote host as it would if you were logged in.

--
Cole

Dear Doug,

If it was me, I would set up a .netrc file like this:

$echo machine isp.com login userID password secretPassword>.netrc

$chmod 400 .netrc

Then I would type a command like this:

$echo dir|ftp userID@isp.com|grep rw|wc

If ftp sees you have set up a .netrc file it will take the password from the .netrc file for ftp login. This means that ftp does not ask for password & you just bypass security. I am not certain if this works for sftp. But if you do use sftp there is a way to bypass security for sftp too. Just let me know & I will look up my notes.

The echo command feeds the request for directory information to ftp. It is about the same as typing dir at the ftp prompt.

The ftp command runs and sends its output to standard out.

The grep command filters out all of the ftp status junk & just lists the files.

The wc command counts up the number of lines or files.

If I can be of any further help, please feel free to post any further questions.

Your humble unix geek,


Kurt

the curl example that I gave above uses the ftp account that I have on the system that I tested it on.

{quote:title=Doug Niven1 wrote:}
Because I have ONLY an FTP account with this system.
{quote}

Thanks for that clarification. If Nils' suggestion doesn't pan out for some reason and you run out of things to try, you might try a cgi shell script:
<pre style="padding-left: .75ex; padding-top: .25em; padding-bottom: .25em; margin-top: .5em; margin-bottom: .5em; margin-left: 1ex; max-width: 80ex; overflow: auto; font-size: 10px; font-family: Monaco, 'Courier New', Courier, monospace; color: #222; background: #eee; line-height: normal">#!/bin/sh

echo Content-type: text/plain
echo

ls ${QUERY_STRING} | wc -l
</pre>
If you do try this, you'd probably want to password protect it or do some input validation.

--
Cole

If the ftp server is configured to limit the size of the directory listing to 2000, and if you can access the server only via ftp, then I guess the only way out is to divide 3000-4000 files into subdirectories.