Lion - Memory Usage Problems

Dperr23 wrote:

Although it was a bit painful to have to reset my preferences, I was able to do everything except the terminal to look at the third party stuff. Each time I inserted a command, I received an error message. It may be that I am copying and pasting the wrong text. Can you resend the terminal with a little more explanation of what actually needs to be inserted at the prompts and what I am supposed to see?

I don't understand. I copied and pasted now from the forum the commands in Terminal and they worked. Unless the lines get broken by a carriage return in the middle they should work. Anyway, they are 3 independent shell lines (I know, previously they were 4 but was redendunt:

That's what I get:

The list of kernel extensions:

kextstat -kl | awk ' !/apple/ { print $6 $7 } '

com.roxio.BluRaySupport(1.1.6)

com.vmware.kext.vmx86(3.1.3)

[...]

The list of jobs launched by launchd:

launchctl list | sed 1d | awk ' !/0x|apple|com\.vix|edu\.|org\./ { print $3 } '

com.prosofteng.DGMonitor

com.paragon.ntfs.upd

[...]

And the the last one is just a file listing of the processes that get started up at boot time. I actually modified it including the startup user processes:

ls -lArt {,~}/Library/{Launch,Start}* 2> /dev/null

/Library/StartupItems:

total 0

drwxr-xr-x 4 root wheel 136 27 Ott 2007 FanControlDaemon

drwxr-xr-x 5 root wheel 170 7 Giu 18:04 ChmodBPF

drwxr-xr-x 4 root wheel 136 16 Ago 14:50 rEFItBlesser

/Library/LaunchAgents:

total 40

-rw-r--r-- 1 root wheel 596 18 Mar 01:49 org.macosforge.xquartz.startx.plist

-rw-r--r-- 1 root wheel 786 9 Lug 10:40 com.google.keystone.agent.plist

-rwxr-xr-x@ 1 root wheel 456 23 Ago 20:04 com.bjango.istatlocal.plist

[...]

etc.

Looking at it now I have no clue what DGmonitor is for example. I will check in Google and see where it comes from. What you should check the most are the old entries. They may come from very old applications. Once you identifiy them and understand that they are not needed look for an uninstaller. Otherwise you should do it manually, but if you're not skilled don't touch anything. Especially the kernel extensions. You may end up with OS X not booting anymore.

Again, feel free to run the commands above, they are just a listing. And then you should be root to change anything. Also I repeat that those have nothing to do with the memory issues.

Am I doing something wrong? Here is what I got:

Last login: Tue Sep 13 21:27:17 on ttys000

[DPERRY23:~] demetriusperry% kextstat -kl | awk ' !/apple/ { print $6 $7 } '

/apple/: Event not found.

[DPERRY23:~] demetriusperry% launchctl list | sed 1d | awk ' !/0x|apple|com\.vix|edu\.|org\./ { print $3 } '

/0x: Event not found.

[DPERRY23:~] demetriusperry% ls -lArt {,~}/Library/{Launch,Start}* 2> /dev/null

ls: 2: No such file or directory

/Library/StartupItems:[DPERRY23:~] demetriusperry% /Library/StartupItems:ls -lArt {,~}/Library/{Launch,Start}* 2> /dev/null/Library/StartupItems:

/dev/null/Library/StartupItems:: Not a directory.

Tried different ones, but does anything stick out?

[DPERRY23:~] demetriusperry% kextstat | grep -v com.apple

Index Refs Address Size Wired Name (Version) <Linked Against>

95 0 0xffffff7f820c7000 0x8000 0x8000 com.AmbrosiaSW.AudioSupport (4.0) <94 5 4 3 1>

109 0 0xffffff7f81867000 0x50000 0x50000 com.digidesign.iokit.DigiDal (9.0.5f4) <9 5 4 3 1>

119 0 0xffffff7f80894000 0x3000 0x3000 com.serato.usb.kext (2.3.0) <33 4 3>

126 0 0xffffff7f808fc000 0x3000 0x3000 com.paceap.kext.pacesupport.snowleopard (5.8) <7 5 4 3 1>

127 0 0xffffff7f820e7000 0xd1000 0xd1000 com.vmware.kext.vmx86 (3.1.3) <7 5 4 3 1>

128 0 0xffffff7f821bb000 0xe000 0xe000 com.vmware.kext.vmci (3.1.3) <5 4 3 1>

129 0 0xffffff7f821c9000 0x7000 0x7000 com.vmware.kext.vmioplug (3.1.3) <33 25 5 4 3 1>

130 0 0xffffff7f821d2000 0xa000 0xa000 com.vmware.kext.vmnet (3.1.3) <5 4 3 1>

139 0 0xffffff7f807ee000 0x16000 0x16000 com.roxio.TDIXController (2.0) <55 54 12 7 5 4 3 1>

[DPERRY23:~] demetriusperry% launchctl list

PID Status Label

887 - 0x7fd08a505a60.anonymous.launchctl

881 - 0x7fd08a5029e0.anonymous.tcsh

880 - 0x7fd08a5026e0.anonymous.login

862 - 0x7fd08a426070.anonymous.tcsh

861 - 0x7fd08a426620.anonymous.login

839 - [0x0-0x7e07e].com.apple.Terminal

730 - 0x7fd08a504210.anonymous.VTDecoderXPCSer

680 - [0x0-0x65065].com.apple.Preview

650 - [0x0-0x56056].com.apple.ImageCaptureExtension2

546 - 0x7fd08a504b60.anonymous.EyeConnect

415 - [0x0-0x3e03e].com.apple.ActivityMonitor

380 - [0x0-0x35035].com.Growl.GrowlHelperApp

366 - 0x7fd08a503bb0.anonymous.DashboardClient

334 - 0x7fd08a425d60.anonymous.PluginProcess

330 - [0x0-0x22022].com.apple.iTunesHelper

297 - [0x0-0x1c01c].com.apple.AppleSpell

274 - 0x7fd08a423ac0.anonymous.com.apple.dock.

103 - 0x7fd08a5020b0.anonymous.WindowServer

265 - 0x7fd08a422fa0.anonymous.WebProcess

254 - [0x0-0x11011].com.apple.iTunes

249 - [0x0-0xe00e].com.apple.Safari

- 0 org.x.startx

- 0 org.openbsd.ssh-agent

- 0 com.apple.ZoomWindow

- 0 com.apple.xgridd

- 0 com.apple.WebKit.PluginAgent

315 - com.apple.warmd_agent

- 0 com.apple.VoiceOver

- 0 com.apple.UserNotificationCenterAgent

245 - com.apple.UserEventAgent-Aqua

- 0 com.apple.unmountassistant.useragent

- 0 com.apple.tiswitcher

256 - com.apple.talagent

257 - com.apple.SystemUIServer.agent

- 0 com.apple.systemprofiler

- 0 com.apple.syncservices.uihandler

- 0 com.apple.syncservices.SyncServer

- 0 com.apple.SubmitDiagInfo

- 0 com.apple.storeagent

- 0 com.apple.store_helper

- 0 com.apple.spindump_agent

- 0 com.apple.speech.voiceinstallerd

- 0 com.apple.speech.synthesisserver

- 0 com.apple.speech.recognitionserver

- 0 com.apple.speech.feedbackservicesserver

- 0 com.apple.softwareupdateagent

- 0 com.apple.ServiceManagement.LoginItems

- 0 com.apple.scrod

- 0 com.apple.screensharing.agent

- 0 com.apple.ScreenReaderUIServer

- 0 com.apple.safaridavclient

- 0 com.apple.ReportPanic

- 0 com.apple.ReportGPURestart

- 0 com.apple.ReportCrash.Self

- 0 com.apple.ReportCrash

- 0 com.apple.RemoteDesktop.agent

- 0 com.apple.reclaimspace

- 0 com.apple.rcd

- 0 com.apple.quicklook.ui.helper

- 0 com.apple.quicklook

- 0 com.apple.quicklook.config

- 0 com.apple.quicklook.32bit

- 0 com.apple.PubSub.Agent

- 0 com.apple.printuitool.agent

- 0 com.apple.printtool.agent

- 0 com.apple.PreferenceSyncAgent

- 0 com.apple.pictd

- 0 com.apple.PCIESlotCheck

- 0 com.apple.pbs

248 - com.apple.pboard

- 0 com.apple.parentalcontrols.check

- 0 com.apple.NetworkDiagnostics

- 0 com.apple.netauth.user.gui

- 0 com.apple.midiserver

- 0 com.apple.metadata.mdwrite

- 0 com.apple.mdmclient.agent

- 0 com.apple.marcoagent

- 0 com.apple.lookupd

- 0 com.apple.locationmenu

686 - com.apple.LaunchServices.lsboxd

- 0 com.apple.java.updateSharing

- 0 com.apple.java.InstallOnDemandAgent

- 0 com.apple.isst

- 0 com.apple.installd.user

- 0 com.apple.imklaunchagent

320 - com.apple.imagent

- 0 com.apple.iCalPush

- 0 com.apple.helpd

- 0 com.apple.FTCleanup

- 0 com.apple.FontWorker

- 0 com.apple.ATS.FontValidatorConduit

- 0 com.apple.ATS.FontValidator

- 0 com.apple.FontRegistryUIAgent

266 - com.apple.fontd

- 0 com.apple.findmymacmessenger

258 - com.apple.Finder

- 0 com.apple.FileSyncAgent.iDisk

- 0 com.apple.familycontrols.useragent

255 - com.apple.Dock.agent

- 0 com.apple.DiskArbitrationAgent

- 0 com.apple.DictionaryPanelHelper

- 0 com.apple.csuseragent

- 0 com.apple.coreservices.uiagent

- 0 com.apple.coreservices.appleid.authentication

- 0 com.apple.CoreLocationAgent

- 0 com.apple.coredata.externalrecordswriter

671 - com.apple.cookied

- 0 com.apple.cfnetwork.AuthBrokerAgent

- 0 com.apple.btsa

- 0 com.apple.bluetoothUIServer

- 0 com.apple.BezelUIServer

- 0 com.apple.AppStoreUpdateAgent

- 0 com.apple.AOSNotification

- 0 com.apple.aos.migrate

- 0 com.apple.alf.useragent

325 - com.apple.AirPortBaseStationAgent

- 0 com.apple.AddressBook.SourceSync

- 0 com.apple.AddressBook.abd

- 0 com.hp.help.tocgenerator

- 0 com.google.keystone.root.agent

64 - 0x7fd08a408f50.anonymous.loginwindow

- 0 com.apple.launchctl.Aqua

164 - 0x7fd08a4233c0.anonymous.CVMServer

64 - 0x7fd08a500ac0.anonymous.loginwindow

62 - 0x7fd08a408020.anonymous.mds

201 - 0x7fd08a5007b0.anonymous.compressord

205 - 0x7fd08a408c40.anonymous.qmastertaskd

87 - 0x7fd08a408940.anonymous.qmasterd

202 - 0x7fd08a408630.anonymous.qmastertaskd

91 - 0x7fd08a408320.anonymous.coreservicesd

- 0 com.apple.TrustEvaluationAgent

- 0 com.apple.netauth.user.auth

- 0 com.apple.mdworker.prescan.0

- 0 com.apple.mdworker.pool.framework.3

- 0 com.apple.mdworker.pool.framework.2

- 0 com.apple.mdworker.pool.framework.1

- 0 com.apple.mdworker.pool.framework.0

- 0 com.apple.mdworker.pool.3

- 0 com.apple.mdworker.pool.2

- 0 com.apple.mdworker.pool.1

765 - com.apple.mdworker.pool.0

- 0 com.apple.mdworker.lsb.framework.0

- 0 com.apple.mdworker.lsb.0

- 0 com.apple.mdworker.isolation.framework.0

- 0 com.apple.mdworker.isolation.0

- 0 com.apple.mdworker.i386.framework.0

- 0 com.apple.mdworker.i386.0

- 0 com.apple.KerberosHelper.LKDCHelper

- 0 com.apple.gssd-agent

- 0 com.apple.FileSyncAgent.PHD

227 - com.apple.distnoted.xpc.agent

- 0 com.apple.cvmsCompAgent_x86_64

- 0 com.apple.cvmsCompAgent_i386

- 0 com.apple.launchctl.Background

[DPERRY23:~] demetriusperry% ls -lArt {,~}/Library/

/Library/:

total 16

-rw-rw-r-- 1 demetriusperry admin 6148 Aug 24 2009 .DS_Store

drwxr-xr-x 2 root wheel 68 Jun 13 13:33 Spelling

drwxr-xr-x 8 root wheel 272 Jun 13 13:40 User Pictures

-rw-r--r-- 1 root wheel 0 Jun 13 13:46 .localized

drwxr-xr-x 2 root wheel 68 Jun 13 17:24 ScriptingAdditions

drwxr-xr-x 2 root wheel 68 Jun 13 19:02 Keyboard Layouts

drwxr-xr-x 3 root wheel 102 Jun 13 19:08 Compositions

drwxr-xr-x 3 root wheel 102 Jun 14 23:43 Video

drwxr-xr-x 2 root wheel 68 Jun 16 17:03 Components

drwxr-xr-x 2 root wheel 68 Jun 16 18:04 Input Methods

drwxr-xr-x 3 root wheel 102 Jun 16 19:21 iTunes

drwxr-xr-x 2 root wheel 68 Jun 16 19:23 Screen Savers

drwxr-xr-x 2 root wheel 68 Jun 18 14:44 SystemProfiler

drwxr-xr-x 3 root wheel 102 Jun 18 14:44 Sandbox

drwxr-xr-x 2 root wheel 68 Jun 18 14:44 Extensions

drwxr-xr-x 2 root wheel 68 Jun 18 14:44 ColorPickers

drwxr-xr-x 3 root wheel 102 Jun 24 22:03 DirectoryServices

drwxr-xr-x 3 root wheel 102 Jun 29 01:14 iChat

drwxr-xr-x 3 root wheel 102 Jun 29 03:42 Security

drwxr-xr-x 4 root wheel 136 Jul 21 12:41 Graphics

drwxr-xr-x 4 root wheel 136 Jul 21 12:45 Ruby

drwxr-xr-x 5 root wheel 170 Jul 21 12:45 WebServer

drwxr-xr-x 3 root wheel 102 Jul 21 12:45 Server

drwxr-xr-x 12 root wheel 408 Jul 21 12:45 Scripts

drwxr-xr-x 5 root wheel 170 Jul 21 12:45 PDF Services

drwxr-xr-x 4 root wheel 136 Jul 21 12:45 ColorSync

drwxr-xr-x 6 root wheel 204 Jul 21 12:46 Python

drwxr-xr-x 3 root wheel 102 Jul 21 12:52 Speech

drwxr-xr-x@ 3 root wheel 102 Jul 21 12:58 Google

drwxrwxr-x 3 root admin 102 Jul 21 12:58 Contextual Menu Items

drwxr-xr-x 6 root wheel 204 Jul 21 12:58 Perl

drwxr-xr-x 3 root admin 102 Jul 21 12:58 Driver Support

drwxr-xr-x 11 root wheel 374 Jul 21 12:58 Dictionaries

drwxr-xr-x 4 root admin 136 Jul 21 12:58 CFMSupport

drwxrwxr-x 124 Admin staff 4216 Jul 21 12:58 Automator

drwxr-xr-x 3 root admin 102 Jul 21 12:58 Managed Preferences

drwxr-xr-x 4 root admin 136 Jul 21 12:58 PhonePlugins

drwxr-xr-x 52 root wheel 1768 Jul 21 12:58 Desktop Pictures

drwxr-xr-x 20 root wheel 680 Jul 21 12:58 Widgets

drwxr-xr-x 6 root wheel 204 Jul 21 12:58 Spotlight

drwxr-xr-x 47 root wheel 1598 Jul 21 12:58 Modem Scripts

drwxr-xr-x 9 root wheel 306 Jul 21 12:58 Image Capture

drwxr-xr-x 4 root wheel 136 Jul 21 12:58 LaunchAgents

drwxrwxrwx 3 root admin 102 Jul 21 12:58 Qt

drwxr-xr-x 13 root wheel 442 Jul 21 12:58 Documentation

drwxr-xr-x 30 root admin 1020 Jul 21 12:58 Fonts Disabled

drwxr-xr-x 3 root admin 102 Jul 21 12:58 Mozilla

drwxrwxr-x 6 root admin 204 Jul 21 21:25 QuickLook

drwxrwxr-x 25 root admin 850 Jul 21 21:26 Receipts

drwxrwxr-x 4 root admin 136 Jul 29 21:37 Java

drwxr-xr-x 10 root wheel 340 Aug 10 21:07 Keychains

drwxrwxr-t 218 root admin 7412 Aug 25 13:20 Fonts

drwxr-xr-x 29 root wheel 986 Aug 25 13:21 QuickTime

drwxr-xr-x 11 root wheel 374 Aug 25 15:03 StartupItems

drwxr-xr-x 7 root wheel 238 Aug 26 10:14 Logs

drwxr-xr-x 28 root wheel 952 Aug 27 16:57 Internet Plug-Ins

drwxr-xr-t 8 root wheel 272 Sep 3 00:25 PrivilegedHelperTools

drwxr-xr-x 14 root wheel 476 Sep 3 00:25 PreferencePanes

drwxr-xr-x 15 root wheel 510 Sep 3 00:25 LaunchDaemons

drwxr-xr-x 11 root wheel 374 Sep 3 00:25 Audio

drwxr-xr-x 54 root wheel 1836 Sep 6 18:29 Frameworks

drwxr-xr-x 4 root wheel 136 Sep 6 18:29 Filesystems

drwxrwxr-x 68 root admin 2312 Sep 6 18:29 Application Support

drwxr-xr-x 10 root admin 340 Sep 12 17:43 Printers

drwxr-xr-x 3 root wheel 102 Sep 12 17:45 Updates

drwxr-xr-x 137 root wheel 4658 Sep 13 08:34 Preferences

drwxrwxrwt 10 root admin 340 Sep 13 13:31 Caches

/Users/demetriusperry/Library/:

total 16

drwx------ 3 demetriusperry staff 102 Sep 12 12:46 Colors

drwx------+ 2 demetriusperry staff 68 Sep 12 12:47 Fonts

drwxr-xr-x@ 3 demetriusperry staff 102 Sep 12 12:47 Google

drwxr-xr-x 5 demetriusperry staff 170 Sep 12 12:47 PubSub

drwxr-xr-x 3 demetriusperry staff 102 Sep 12 12:47 iTunes

-rw-r--r--@ 1 demetriusperry staff 6148 Sep 12 17:28 .DS_Store

drwxr-xr-x 6 demetriusperry staff 204 Sep 12 17:37 Address Book Plug-Ins

drwxr-xr-x 8 demetriusperry staff 272 Sep 12 17:38 Logs

drwxr-xr-x 3 demetriusperry staff 102 Sep 12 17:48 Mail

drwxr-xr-x 3 demetriusperry staff 102 Sep 13 08:36 Services

drwx------@ 25 demetriusperry staff 850 Sep 13 08:39 Saved Application State

drwxr-xr-x 7 demetriusperry staff 238 Sep 13 13:28 Calendars

drwx------ 3 demetriusperry staff 102 Sep 13 14:12 Containers

drwx------@ 28 demetriusperry staff 952 Sep 13 14:12 Application Support

drwxr-xr-x 3 demetriusperry staff 102 Sep 13 14:13 Frameworks

drwx------ 6 demetriusperry staff 204 Sep 13 14:17 Keychains

drwxr-xr-x@ 57 demetriusperry staff 1938 Sep 13 14:26 Caches

drwxr-xr-x 7 demetriusperry staff 238 Sep 13 14:37 TuneUpMedia

drwxr-xr-x@ 13 demetriusperry staff 442 Sep 13 22:05 Safari

drwx------+ 194 demetriusperry staff 6596 Sep 13 22:08 Preferences

drwxr-xr-x 4 demetriusperry staff 136 Sep 13 22:09 Cookies

Dperr23 wrote:

Am I doing something wrong? Here is what I got:

Last login: Tue Sep 13 21:27:17 on ttys000

[DPERRY23:~] demetriusperry% kextstat -kl | awk ' !/apple/ { print $6 $7 } '

/apple/: Event not found.

[...]

Ow... Now I see. I think you are using csh. You should use bash, which is the OS X default. Try to check he shell doing:

echo $SHELL

It very likely reports /bin/csh. To make those commands working anyway just type

bash

and then copy and paste the commans I gave you again. That will load the bash shell first and then all commands will run with its syntax.

The list you produced is too long and not informative. But still you may check this ones:

com.AmbrosiaSW.AudioSupport

com.digidesign.iokit.DigiDal

com.serato.usb.kext

com.paceap.kext.pacesupport.snowleopard

com.vmware.kext.vmx86

com.vmware.kext.vmci

com.vmware.kext.vmioplug

com.vmware.kext.vmnet

com.roxio.TDIXController

Google them and see if you can upgrade the applications first. Like VMware, if it is old it may mess up few things since it adds many virtual network interfaces at boot time. If it is the last release it is fine, I have it myself. The other kernel extensions are pretty suspicious too. Audio support, usb, something related to snow leopard. They may all create troubles if bugged in Lion. Just check what they are and either update or uninstall them.

You may want to install CNET TechTracker or a similar tool to keep all your applications updated. It isn't perfect (sometimes it reports that an application is outdated while it isn't) but it is pretty complete. It is also going to be updated to Premium for free soon (whatever that means). That requires a free registration.

I got them to work and achieved smaller list. Anything suspisious now?

DPERRY23:~ demetriusperry$ kextstat -kl | awk ' !/apple/ { print $6 $7 } '

com.AmbrosiaSW.AudioSupport(4.0)

com.digidesign.iokit.DigiDal(9.0.5f4)

com.serato.usb.kext(2.3.0)

com.paceap.kext.pacesupport.snowleopard(5.8)

com.vmware.kext.vmx86(3.1.3)

com.vmware.kext.vmci(3.1.3)

com.vmware.kext.vmioplug(3.1.3)

com.vmware.kext.vmnet(3.1.3)

DPERRY23:~ demetriusperry$ launchctl list | sed 1d | awk ' !/0x|apple|com\.vix|edu\.|org\./ { print $3 } '

com.hp.help.tocgenerator

com.google.keystone.root.agent

DPERRY23:~ demetriusperry$ ls -lArt {,~}/Library/{Launch,Start}* 2> /dev/null

/Library/LaunchAgents:

total 16

-rw-r--r-- 1 root admin 605 Jul 28 2009 com.hp.help.tocgenerator.plist

-rw-r--r-- 1 root wheel 786 Jul 9 21:53 com.google.keystone.agent.plist

/Library/StartupItems:

total 0

drwxr-xr-x@ 5 root wheel 170 Sep 16 2009 HP Trap Monitor

drwxr-xr-x 5 root wheel 170 Oct 10 2009 M-Audio Firmware Loader

drwxr-xr-x@ 5 root wheel 170 Oct 10 2009 HP IO

drwxr-xr-x@ 4 root wheel 136 Oct 10 2009 MxBtDaemon

drwxr-xr-x 5 root wheel 170 Oct 10 2009 MissingSyncListener

drwxr-xr-x@ 8 root wheel 272 Jan 30 2010 Digidesign Mbox 2

drwxr-xr-x 5 root wheel 170 Jul 16 11:27 EyeConnect

drwxr-xr-x@ 5 root wheel 170 Aug 25 13:21 DigidesignLoader

drwxr-xr-x 5 root wheel 170 Aug 27 16:59 PACESupport

/Library/LaunchDaemons:

total 104

-rw-r--r-- 1 root wheel 729 Jul 16 2008 com.memeo.Memeod.plist

-rw-r--r-- 1 root admin 392 Apr 25 2010 com.rane.sl3.daemon.plist

-rw-r--r-- 1 root admin 406 Apr 25 2010 com.rane.sixtyeight.daemon.plist

-rw-r--r-- 1 root wheel 568 Aug 25 2010 com.microsoft.office.licensing.helper.plist

-rw-r--r-- 1 root wheel 377 Aug 25 2010 com.apple.qmaster.qmasterd.plist

-rw-r--r-- 1 root admin 392 Nov 15 2010 com.rane.sl4.daemon.plist

-rw-r--r-- 1 root admin 392 Jan 4 2011 com.rane.sl2.daemon.plist

-rw-r--r-- 1 root wheel 520 May 21 19:17 com.vmware.launchd.vmware.plist

-rw-r--r-- 1 root wheel 543 Jun 16 23:08 com.apple.aelwriter.plist

-rw-r--r--@ 1 root wheel 450 Jul 7 02:00 PACESupport.plist

-rw-r--r-- 1 root wheel 976 Jul 9 02:35 com.paceap.eden.licensed.plist

-rw-r--r-- 1 root wheel 790 Jul 9 21:53 com.google.keystone.daemon.plist

-rw-r--r-- 1 root wheel 421 Jul 21 12:43 com.apple.remotepairtool.plist

It appears you have PACE anti-piracy software on your Mac. This is notorious for causing all kinds of problems. You might want to see if you can remove it as a test.

Removing PACE will cause the application that installed it to fail. If it removing it, fixes the problem then you must choose which way you wish to go.

Allan

You have relatively old stuff from 2009. I don't know what's the "HP Trap Monitor" for example. Unless it is some printer software. I'd check all of them anyway.

Also Allan is right. I'd avoid installing any third party antimalware sw in OSX as if it was the plague. Just forget about them, they cause more troubles than anything. You may have PACE's InterLock pro installed. See if you have already the tool to uninstall it. Otherwise contact their support.

Michelasso wrote:

I don't think it is possible for the kernel to identify to which process belong the pages that are either marked as modified, in the inactive list, or that have been paged out. That is also confirmed by the author of a tool similar to top, htop.

I have no idea what are you talking about. Kernel routines (along with a little help from the I/O Kit at the device level) are responsible for essentially all of memory management functions in OS X. If it can't identify these things, how on earth do you think VM could function?

BTW, the htop tool is written for Linux & is not applicable to OS X, which uses a hybrid kernel (XNU). Regardless of that, I don't see anything in your htop link to confirm this strange idea.

The paging system it is transparent to the processes after all.

It isn't transparent to the kernel, nor is it completely so for certain processes like device drivers that use memory mapped regions as buffers.

Indeed note that I didn't write "Safari paged out gigabytes of memory". I couldn't anyway, since the page choice for paging out is application independent.

That isn't strictly true either. The OS can't page out memory in active use by a process.

The issue is that any OS paging algorithm pages out only small amounts of memory per time.

Again, I have no idea what you mean by this, unless you are referring to the limitations imposed by the speed that data can be written to the file system.

All of this is covered in the two Apple developer docs that have been mentioned several times in this discussion. I don't understand how you could have read them & gotten so much of this so wrong. Free memory is memory not currently used for anything -- period. It is not used to "preload" pages or to optimize disk performance.

It has nothing to do with speculative execution or the memory it uses. That is a widely used, at least ten year old technology that exploits the multiple pipelines & cores in modern CPU's to process several threads speculatively in parallel. If you want to learn about the various ways this can use memory, check out this paper, but the essential point is it is uses memory in the same way anything else a process does.

Regarding your complaint about Apple not doing a better job with its documention, keep in mind that "Manual pages are intended as a quick reference for people who already understand a technology," a direct quote from the top of every online Apple man page, & that some of Apple's code is proprietary, also known as "closed source." Apple provides ample documentation for programmers that need to access the functionality that code provides, & support through the developer connection, but don't expect Apple to publicly reveal everything about how the OS works.

R C-R wrote:

I have no idea what are you talking about. Kernel routines (along with a little help from the I/O Kit at the device level) are responsible for essentially all of memory management functions in OS X. If it can't identify these things, how on earth do you think VM could function?

BTW, the htop tool is written for Linux & is not applicable to OS X, which uses a hybrid kernel (XNU). Regardless of that, I don't see anything in your htop link to confirm this strange idea.

The concept is simple: the memory mapping is unidirectional. The virtual memory is mapped into pages that may reside either in RAM (mapping managed by HW with the MMU) or into the disk. But the reverse is not implemented.

In our specific case the kernel always knows where a virtual page is physically stored. If it is not stored a page fault will be generated when trying to access it. But given a physical address not necessarily it is possible to identify to which virtual address it corresponds. And thus neither the application it belongs. A software reverse mapping might probably be possible but it should take too many resources and so it is usually not implemented. I never heard of it at least, not on Unix systems.

The kernel doesn't need to know about it because it keeps the mapping consistent no matter what. Its job is to make sure that any time a logical memory address must be accessed it can furnish the correspondent physical address in RAM. Allocating RAM pages for the requesting process if needed. Deallocating/releasing them when the process is killed. The last is done simply by furnishing the logical (virtual) addresses. The ones that have already been mapped get translated and the corresponding resource gets released. No need for reverse mapping.

Here is what the author of htop said about this:

Why doesn't htop feature a SWAP column, like top?

It is not possible to get the exact size of used swap space of a process. Top fakes this information by making SWAP = VIRT - RES, but that is not a good metric, because other stuff such as video memory counts on VIRT as well

That's for Linux. Top in OS X doesn't even show that value.

--

Regarding the speculative memory, in OS X it should act like an anticipatory paging. To discover this I had to dig into the OS X Open Source code for vm_stat:

/* added for rev2 */

/*
* NB: speculative pages are already accounted for in "free_count",
* so "speculative_count" is the number of "free" pages that are
* used to hold data that was read speculatively from disk but
* haven't actually been used by anyone so far.
*/

That is your famous cache for "free memory is wasted memory". Still, it shows as "free" memory in Active Monitor. Thus someone should explain me why an high inactive memory is a good sign, since it is stealing RAM space otherwise used by the speculative memory.

Now, since that information is public, I still would like to know why the vm_stat man page hasn't been updated accordingly (note the "rev2". It wasn't present before). Since only vm_stat seems to show the real value for "free memory" no wonder everyone gets confused. To say that Apple is right not making the man pages consistent, or even worse to ask $99 per year to get the correct information is apologetic the least.

Michelasso wrote:

But given a physical address not necessarily it is possible to identify to which virtual address it corresponds. And thus neither the application it belongs.

This is both confused & irrelevant to your original comment that you "don't think it is possible for the kernel to identify to which process belong the pages that are either marked as modified, in the inactive list, or that have been paged out."

We know from Page Lists in the Kernel that "the kernel maintains and queries three system-wide lists of physical memory pages." From About Virtual Memory we know that the CPU & MMU "maintain a page table to map pages in the program’s logical address space to hardware addresses in the computer’s RAM" & that "the MMU uses the page table to translate the specified logical address into the actual hardware memory address." The translation is automatic & transparent. We even know from the 5 step Paging Out Process that it is the kernel that moves pages from the active to inactive list & pages them out. But most importantly, we know (if not from common sense then again from About Virtual Memory) that when a page fault occurs the process that caused it must be halted until the appropriate kernel pager can load the needed page into physical memory, after which control is returned to the process.

Thus, it is clearly not just possible but necessary for the kernel to know at the process level the virtual address involved, its active or inactive status, if any of it should or can be paged out (or in), if so by which pager, & so on.

Beyond that, the only significant thing to consider in this context about hardware & logical RAM address space translations is discussed in the Address Spaces in the Kernel Programming Guide, & has to do with 32 vs. 64 bit drivers. It is not relevant for Lion's 64 bit only driver requirements. But also note (as previously mentioned) that there are provisions in the I/O Kit for a process to query physical memory directly, typically used only for memory-mapped driver processes. (Here, it is important to remember that OS X is not a conventional UNIX system, in part because the I/O Kit handles I/O tasks rather than the kernel.)

Thus someone should explain me why an high inactive memory is a good sign, since it is stealing RAM space otherwise used by the speculative memory.

From the same header file you referenced:

VM_FLAGS_NO_CACHE
* Pages brought in to this VM region are placed on the speculative
* queue instead of the active queue. In other words, they are not
* cached so that they will be stolen first if memory runs low.

IOW, since this memory holds data read speculatively but has not been used by anything, it is considered of less potential value than data that has been used by something & might be used again. It has no backing store; it is simply loaded on a space-available, opportunistic basis.

Now, since that information is public, I still would like to know why the vm_stat man page hasn't been updated accordingly

From the BSD manpage(5):

In general, however, manual pages are written to be as concise as possible. While this makes them a somewhat difficult place to start as a new programmer, this is intentional. They are not intended to replace conceptual documentation such as books on programming. They are intended primarily to provide a quick reference for people who are already somewhat familiar with the subject.

Clear enough? Manual pages don't cover everything you need to know about any topic to understand it; in fact, many are so concise that sometimes not even all the options are mentioned, much less explained.

Since only vm_stat seems to show the real value for "free memory" no wonder everyone gets confused.

Once again you are trying to oversimplify things far too much. There is no single "real value" that is contextually, conceptually, or computationally appropriate for the purpose of every monitoring or analysis tool, so of course it will be confusing if you expect them all to agree. The reasons for this have been discussed to death in this topic. There is nothing I can add to that.

To say that Apple is right not making the man pages consistent, or even worse to ask $99 per year to get the correct information is apologetic the least.

1. Man pages have never been consistent, whether provided by Apple or any other source -- again, see manpage(5). If Apple isn't "right" then neither is anybody else.

2. The paid Developer Connection subscriptions are intended for developers that need Apple's resources for product development, not for end users struggling to understand the intricacies of how the OS works.

3. Most of the publicly available 'overview' Apple developer documents include a section explaining who they are intended for & links to other relevant info & documentation to study. It isn't that Apple doesn't provide the "correct information," it is that there is far too much of it to condense into a few easy-to-understand documents.

I don't bloody believe this. I wrote a full answer, just pressed delete to correct a word and Safari went one page back erasing everything.

In any case, fast answer: it doesn't work like you say. The confusion lies elsewhere.

1. Check what it happens with the inactive memory in the kernel:

Page Out:

• If a page in the active list is not recently touched, it is moved to the inactive list.
• If a page in the inactive list is not recently touched, the kernel finds the page’s VM object.

Page In:

When any type of fault occurs, the kernel locates the map entry and VM object for the accessed region. The kernel then goes through the VM object’s list of resident pages. If the desired page is in the list of resident pages, the kernel generates a soft fault. If the page is not in the list of resident pages, it generates a hard fault.

The kernel has to search for the VM objects. The VM objects are not referenced by the physical pages. Even less the processes, then. Thus it knows nothing about the processes "owning" the inactive pages unless it goes looking for it. A process to know its inactive pages should make some system calls that through an API ask the kernel to do a job that isn't implemented. Those system calls are simply non existent.

2. The inactive memory is not a cache either. It is the physical memory that has been already used by the processes and got old. Which means sooner or later it will cause page outs which is undesireable or become active again keeping stealing RAM to the other processes. Unless the process decides to release some on its own, sure. The speculative memory instead is a kind of "pre-cache". The speculative pages are reassigned at no cost to the free list as soon as free memory is needed. While the inactive pages may require a disk page out Thus to have Safari or any other process raising constantly in memory isn't good for the whole performances of the system.

I don't really understand how anyone can say otherwise.

Oh, btw. The BSD's man page for man is stating that man can be used as a concise reference, especially when programming, but it can not substitute a manual. Nowhere it is stated that they are incomplete on purpose. Unless in English concise and incomplete are synonymous. But it wasn't so the last time I checked. And since they are words with a Latin root, I am sure that isn't the case anyway.

Well I was going to reply to the post you are replying to but I see you have done a magnificent job! Safari is NOT a professional app and there is no reason for it to, over time, suck almost all of my ram. I have to first restart safari and then after a while that does not work and so I have to restart the **** machine to bring it back from the brink of sluggishness beyond return!

While I understand the logic it seems to be defeated by the implementation. The problem here is that once the memory leak gets to a certain point not only is EVERYTHING in safari affected, but my entire system begins to VERY noticeably slow down. Lion is a piece of crap as it stands and Apple needs to fix it!

Michelasso wrote:

In any case, fast answer: it doesn't work like you say. The confusion lies elsewhere.

I suspect a lot of the confusion is the result of considering only part of the kernel environment & how it relates to memory management in OS X.

As explained here, in OS X the kernel is more than the Mach microkernel. Mach provides among other things low level VM support & memory protection. But the BSD & I/O Kit components can't be ignored since for instance the BSD component provides higher level support for both & the I/O Kit provides the kernel’s enforcement of protected memory spaces. All of these kernel components contribute to what the kernel "knows" about the use of real & virtual memory. The various monitoring, diagnostic, & performance measuring tools must use their programming interfaces to show users the appropriate system-wide or process-specific memory use info -- since user space programs can't access the protected kernel space directly, there is no other way to do it. These API's obviously do exist since the tools could not work without them.

Obviously, understanding the complexities of memory management in OS X requires studying more than just the overview documents but a careful read of them should provide some insights about why what applies to memory management in other OS's does not necessarily apply to OS X, why the different tools provide different & sometimes seemingly conflicting info, & why no one tool provides everything one needs to know to analyze how a process uses memory.

Besides, even if we just want to analyze Mach's contribution, we can't ignore the difference between a VM & memory object (the former of which may be backed by one or more of the latter & the latter of which may not be backed at all), the different "owners" of memory objects & the several ways those objects can be populated or shared among processes (as mentioned here), or that Mach of necessity must have access to the pmap of logical to physical addresses since it handles the machine-independent part of this mapping even though the CPU & MMU handle the machine-dependent part of it.

With three different but interdependent kernel components to consider, each of which alone has many complex & interrelated parts, is it any wonder that trying to understand how it all works together is confusing?