This is not a problem. Inactive RAM can be used to speed up future tasks, if you happen to do some task that can re-use it, and can be dumped and turned into free RAM as needed. As I often tell people, free RAM is wasted RAM!
Now, if you're actually having performance issues, you need to seek out the causes of those issues. Those issues are unlikely to be RAM-related, since 4-5 GB of inactive RAM is basically the same as 4-5 GB of free RAM. Try looking at some of the suggestions in my Mac Performance Guide.
I keep having the same problem and have seen your explanation and apples for inactive memory which does make sense. However whenever I get the slow lagging problem and get the spinning ball , when I look at inactive memory it's gone high. A Purge clears it and then things run ok again. Then if I watch activity monitor I notice it slowly keeps creeping up again even if I am not opening anything new or doing anything.
This us getting very frustrating. It may be something other than inactive memory , maybe in combination with it ?
Just wanted to chime in that @mendes1 and @thomas a reed has given great answers to the question and add my own view on it.
OSX sometimes caches too zealously which results in the usage of too much ram leaving not enough ram for actual apps that the user is actively using. For example, downloading a 21gig file from the app store is taking me about 10 days to do so given the speed of the net where i'm at. I leave it running whenever I don't need the bandwidth for other stuff. To my surprise, leaving it downloading for over 30 mins would result in all my free ram gone and Free Memory App's auto-purge when Free Mem drops to 20 megs would occur which would then freeze the machine for a few seconds. Why would downloading a huge app from the app store require caching in ram at all? Save batter power? Seriously the basic idea is sound on how Apple implements caching and has resulted in a lot of performance improvements but sometimes there are just plain silly situations.
Lastly without knowledge of purge or Free Memory App, I've seen the macs of friend basically freeze and required force quitting apps they were actively using to get control of the system back. I've since taught them to buy the app or use purge periodically
I'm a researcher using a MBP 17 mid 2007 and after upgrading to Mavricks I discovered I had to up the memory fom 4GB to 6GB. This is just for having lots of Firefox windows open at the same time as well as pdfs in FireFox and/or Preview.
The documented maximum for my MBP is 4GB but many incliding OWC sell a 4GB module compatible with my MBP model (see OWC eshop at http://eshop.macsales.com/item/Other%20World%20Computing/5300DDR2S4GB/ ) which allows a 2GB+4GB=6GB the documented max for my MBP model.
I read about memory fragmentation and decided to install Memory Clean which is free on the AppStore (https://itunes.apple.com/us/app/memory-clean/id451444120?mt=12 .
However I noticed that Memory Clean rarely recognises all the RAM and it takes several Clean Memory button clicks to purge substantial chunks of RAM.
I've reported this to the developers FIPLAB and I'm awaiting their responce, which may shed more light on Mavricks handling of RAM.