The > Mac mini (Early 2009) - Technical Specifications originally came with 10.5 Leopard.
If it is on 10.5, you will first need > Mac OS X 10.6 Snow Leopard and then update to 10.6.8.
If it is already on 10.6.8, you will need to setup a new account in your name at the App Store and then you can get > OS X Mavericks if the Mac Mini has enough RAM.
For more on upgrading to Mavericks, see > Upgrade your Mac to OS X Mavericks.
Mavericks does not require you to max out the RAM to 8GB, but you will notice a big performance hit with 4GB or less.
For quality RAM upgrades, go to > OWC macsales.com