Remote access to Mac Server

We have a Mac Server OSX 10.5.8. I have enabled screen sharing on it. It works on the LAN from any PC using RealVNC viewer, I just type in the server's IP, give it the password and I am on. When I forward port 5900 on the router, however, I get message "connection terminated unexpectedly". So what I have to do is first make an RDP session to one of the Windows servers and launch VNC client from it. So I know the port forwarding on the router works and I know I can access the internet from the Mac Server, what am I missing?