Actually your Mac is up to date. 10.5.8 is the last version of OS X Leopard. You are confusing updating OS X with upgrading OS X.
You will need to buy & install the retail install disk for OS X 10.6 Snow Leopard and then upgrade to OS X 10.6 8 to have the public version of the Mac App Store (MAS) on your Mac. Then, based on which Mac you have, you might be able to use the MAS to upgrade to OS X 10.7 Lion, 10.8 Mt Lion or 10.9 Mavericks. Lion and Mt Lion will cost you $20, Mavericks is free.
OS X Snow Leopard 10.6.8 v1.1 update combo
http://support.apple.com/kb/DL1399
OS X Upgrades -
A Mac that can run OS X 10.6 Snow Leopard can usually run OS X 10.7 Lion. And a Mac that can run OS X 10.8 Mt Lion can also run OS X 10.9 Mavericks. Lion, Mt Lion and Mavericks can all be installed over OS X Snow Leopard 10.6.8, so you may upgrade from Snow Leopard directly to any of the other three. Both Lion and Mt Lion are paid upgrades, while Mavericks is a free upgrade. You may also upgrade directly to Mavericks from either Lion or Mt Lion.
System requirements for OS X Snow Leopard & OS X Lion
http://support.apple.com/kb/ht4949
System requirements for OS X Mountain Lion
http://support.apple.com/kb/HT5444
System requirements for OS X Mavericks
http://support.apple.com/kb/ht5842