Sorry for taking so long to get back to your question:
Unfortunately, from what I can see, the maxSize parameter does not limit the size of the Time Machine backup. In fact, I'm not sure what it does (if anything). I had set three Macs up with independent 200GB, 5GB and 1GB maxSize limits using the commands below (note the use of the '-integer' type coersion), but in my tests Time Machine consumed 316GB, 100GB and 63GB respectively on the server after two months of operation. I had assumed that the unit of measurement is "K", which may not be the case.
defaults write /Library/Preferences/com.apple.TimeMachine MaxSize -integer 209715200
defaults write /Library/Preferences/com.apple.TimeMachine MaxSize -integer 5242880
defaults write /Library/Preferences/com.apple.TimeMachine MaxSize -integer 1048576
It appears that in order to limit the size of the Time Machine backup you need to limit the size of the .sparsebundle associated with Time Machine. Left alone, Time Machine will eventually gobble up the entire volume it is hosted on, especially when you run several users off the same network share. The technique I'm experimenting with now for limiting the Time Machine size is described here, and in addition, I'm now setting MaxSize to the default state ("0" meaning "unlimited"):
http://www.macosxhints.com/article.php?story=20071108020121567
The article talks about SMB-based Time Machine vaults, but the limiting technique should work equally well for AFP-based Time Machine vaults. I'm finding that you also need to pay attention to permissions to prevent a security hole. After you've swapped the limited .sparsebundle for the default "unlimited" .sparsebundle that TimeMachine creates, I'm running "sudo chmod -R go= <thesparesebundle>" where <thesparebundle> is the name of the .sparsebundle on the server. This prevents one user from peering into another users Time Machine backup.
Lastly, I also associated an ACL with the network share in an effort from preventing one user from being able to delete another user's Time Machine .sparsebundle, but with OS X 10.5.6 I had to modfy the ACL to allow deletion for TimeMachine to work properly. Originally the ACL I used was:
chmod +a "group:timemachine allow list,add
file,search,add_subdirectory,readattr,writeattr,readextattr,writeextattr,readsec urity,limitinherit"
This was working from mid-October through December 18th 2008. I think it was the OS X 10.5.6 update I installed on my clients which started causing Time Machine to start producing an error which read "The backup volume is read only", and if I reselected the TimeMachine volume it said that the volume did not have write or append access (which it did). It was necessary to add delete+delete_child access to the ACL, and then Time Machine started purring along again, but this defeated part of the reason the ACL was in place. The ACL I'm now using on the TimeMachine share is:
chmod +a "group:timemachine allow list,add
file,search,delete,add_subdirectory,delete_child,readattr,writeattr,readextattr, writeextattr,readsecurity,limitinherit"
I'd be interested in what other folks have found works and doesn't work where limiting the Time Machine size is concerned.