This apparently is not an OS 8.6 problem, as much as it is a Tiger problem. I had the exact same networking problem this week. Larger file transfers (1.5MB+) cause my OS 8.6 G3 to disconnect from my OS 10.4.10 machine every time.
In case you or anyone else is still struggling with this problem, I tried a fix which Apple recommends for OS X Server, and it worked on my non-server Mac Pro.
Here's the link:
http://docs.info.apple.com/article.html?artnum=305420
A quick caveat: I know absolutely nothing about OSX server, and my skills in Terminal are limited at the very best. So, if anyone wants to jump in here and add some clarity, it would be much appreciated.
There are two approaches. The first approach only appears to work on OS X Server, as the "serveradmin" statement was not recognized when I typed it in terminal. The second approach worked fine for me.
Solution
On the server, change the TCPQuantum value used by the Apple File Service from 262144 to 327680. You can change this value using the serveradmin command, or by directly editing the preference file. You only need to use one of these methods:
*First approach:*
serveradmin method
1. Make sure all users are disconnected from the server.
2. Open Terminal (or ssh to the server from another computer) and execute these commands, each on a single line:
sudo serveradmin stop afp
sudo serveradmin settings afp:TCPQuantum = 327680
sudo serveradmin start afp
*Second Approach:*
defaults method
1. Make sure all users are disconnected from the server.
2. Stop Apple File Service.
3. Open Terminal (or ssh to the server from another computer) and execute this command on a single line:
sudo defaults write /Library/Preferences/com.apple.AppleFileServer TCPQuantum -int 327680
4. Start Apple File Service.
I stopped Apple File Service by opening Activity Monitor, listed "All Processes," selected AppleFileServer, and quit the process. (Activity Monitor and Terminal can be found in /Applications/Utilities). I wasn't as clear about how to restart AppleFileServer, so I did a fair amount of flailing in Terminal. After rebooting the computer I found that the TCPQuantum had either not changed, or had reverted to the original value. Repeating the process managed to get it to work.
I'm not sure how to verify the value of TCPQuantum in terminal, but I was able to view the /Library/Preferences/com.apple.AppleFileServer file using Text Wrangler.
Can someone with more Command Line experience spell out the steps for the second method? I know it should only amount to a few lines of typing in terminal.