Skip navigation

Grep running extremely slow on Mountain Lion.  What has happened to it?

1038 Views 10 Replies Latest reply: Jan 2, 2013 7:25 AM by SadLion RSS
SadLion Calculating status...
Currently Being Moderated
Oct 9, 2012 4:28 AM

I upgraded to Mountain Lion (MacBook Pro Mid 2010, 4Gb Ram, 2.66GHz) recently.  The upgrade went fine and the system is running well in general.

 

However, I previously used grep (with -x or -w) in terminal to take a file of patterns and pull matching lines out of another file.  As in 'grep -F -x -f patternfile basefile > outputfile'.  Both files are admittedly very large (20,000 - 30,000 rows).  However, under Snow Leopard, this would take about a minute.  Under Mountain Lion, it is taking hours!  I mean literally hours!

 

Has anybody else had this problem?  Is there anyway to fix it?  I handle huge files like this all the time and I'd rather downgrade to Snow Leopard than have grep behave like this.

 

PS.  I know grep has been changed and the -p option removed, but I don't use that option so surely that shouldn't affect me.

MacBook Pro, OS X Mountain Lion (10.8.2)
  • VikingOSX Level 5 Level 5 (4,695 points)

    Are either or both files compressed?

     

    The grep(1) man page indicates that fgrep is faster than grep or eqrep on fixed patterns. Try the following to see if your elapsed time improves.

     

     

     

    Code:
    fgrep -x -f patternfile basefile > outputfile
    

     

  • VikingOSX Level 5 Level 5 (4,695 points)

    fgrep -x -f patternfile basefile > outfile

    Mac mini, OS X Mountain Lion (10.8.2), • 8GB • Vertex 4 128GB SSD • 500GB
  • VikingOSX Level 5 Level 5 (4,695 points)

    Arrrggh.

     

    Is it possible to restructure the pattern file in a highest to lowest probability of matches to the basefile?

     

    Mountain Lion has shed GNU grep (Snow Leopard) for FreeBSD grep due to licensing whims outside of Apple. The Perl-Regex (-P) code that made the Snow Leopard grep so fast was the sacrificial lamb. Even though you stated you were not explicitly using the -P option in Snow Leopard, it may have been implicitly used in the pattern matching process anyway.

     

    I have spent some time now with Google search attempting to find any optimization techniques for FreeBSD grep and so far --- mostly rhetoric. It is possible to download the source to GNU grep and build it locally with Xcode command line tools, but it also has some pre-requisites and poor build documentation that make this a migraine.

     

    If you have language skills, perhaps you can write something in Ruby, Python, or Perl that allows you to create a faster pattern matching solution than offered by FreeBSD grep.

     

    Good luck.

  • Linc Davis Level 10 Level 10 (107,735 points)

    Test in safe mode. Same?

  • VikingOSX Level 5 Level 5 (4,695 points)

    Short of downloading the latest stable GNU grep, compiling it, and installing it in /usr/local/bin so it doesn't step on the FreeBSD grep, there probably is no liveable grep solution for you on Mountain Lion. Depending on coding skill and energy, you may or may not have an AWK or other reqular expression solution working before the next paragraph.

     

    I just revisited 10.7.5 and it has GNU grep. If you had originally purchased Lion, then you could redownload it to Mountain Lion (but not install it), burn it to a USB stick, and then perform a clean 10.7.5  install on an external USB drive. Then, reboot from the 10.7.5 drive, and let fly with GNU grep as before. Based on the amount of time the FreeBSD grep is costing you, the Lion download and installation would probably finish first.

     

    Thoughts?

  • codinglamp Calculating status...

    I had the same problem on Moutain Lion.   Using grep -v -f patterns-file on my 24MB test data file took >10s.  I then used the grep from my Lion partition and it took only 0.2s!    

  • vogelw Calculating status...

    Here is my solution: using MacPorts, https://www.macports.org, install the GNU version of grep.

     

    $ sw_vers -productVersion

    10.8.2

     

    $ sudo port install grep

     

    $ which grep

    /opt/local/bin/grep

     

    $ grep --version

    grep (GNU grep) 2.14

    Copyright (C) 2012 Free Software Foundation, Inc.

Actions

More Like This

  • Retrieving data ...

Bookmarked By (0)

Legend

  • This solved my question - 10 points
  • This helped me - 5 points
This site contains user submitted content, comments and opinions and is for informational purposes only. Apple disclaims any and all liability for the acts, omissions and conduct of any third parties in connection with or related to your use of the site. All postings and use of the content on this site are subject to the Apple Support Communities Terms of Use.