10 users is the limit for Mac OS X (client), not Server. It seems odd that you've been running more than 10 users in the past and it's just switched out.
I'd check the license key to make sure that hasn't got munged somehow. Beyond that.. I'm not sure... IIRC 10.5.x had an option to set a concurrent user limit - that hasn't turned on by accident, has it?