You can make a difference in the Apple Support Community!

When you sign up with your Apple Account, you can provide valuable feedback to other community members by upvoting helpful replies and User Tips.

Looks like no one’s replied in a while. To start the conversation again, simply ask a new question.

Terminal.app crashing: Can I disable callbacks?

Five times in the last 48 hours, Terminal.app "quit unexpectedly". The Terminal_datestamp.crash files show

EXC_BAD_ACCESS (SIGSEGV) KERN_INVALID_ADDRESS at 0x0000000000000000

with a call stack that always includes:

com.apple.Terminal 0x000000010530bc94 0x105295000 + 486548 com.apple.Foundation 0x00007fff972ac5df __NSFireTimer + 83 com.apple.CoreFoundation 0x00007fff95837d74 __CFRUNLOOP_IS_CALLING_OUT_TO_A_TIMER_CALLBACK_FUNCTION__ + 20 com.apple.CoreFoundation 0x00007fff958379ff __CFRunLoopDoTimer + 1071 com.apple.CoreFoundation 0x00007fff9583755a __CFRunLoopDoTimers + 298 com.apple.CoreFoundation 0x00007fff9582ef81 __CFRunLoopRun + 2065

User impact

I cannot use the system: must find solution, even if the cost is to wipe disk and install previous OS (El Capitan).

The primary question

(q1) Is there some new feature of Sierra that integrates Terminal with other apps that can be turned off, perhaps a setting to disable callbacks?

Secondary questions

  • (q2) Can I potentially run a copy of the Terminal.app from a previous version of Mac OS X? Prior to my new laptop (and Sierra) these crashes never happened.
  • (q3) What additional detail would be useful, and how can I collect it?

    I am familiar with Unix application development, but not for Mac OS X. I am happy to collect additional detail by turning on debug tools, traces, event collectors, etc. but would need pointers to get started. Xcode is available, but I would be a beginner with it.

System summary

  • New MacBook Pro (Retina, 15-inch, Mid 2015).
    • Arrived 6 days ago.
    • (Yes, I intentionally bought a brand-new instance of the previous model.)
    • Surprisingly, it arrived with Sierra. It presently has 10.12.1
    • Xcode installed
    • GCC 6.2.0 available, built locally.
  • Previous history brought in by Migration Assistant
  • Not tried (yet, anyway)
    • I have briefly visited safe mode but have not stayed there. I do not know whether there would be crashes of Terminal.app if I spent all day in safe mode
    • I do not know whether there would be crashes if I created a fresh, different user account.
  • Tentative rule-outs:
    • Malware, adware, etc are reasonably unlikely given the usage pattern for this work system: firewalls in sw and hw, no visiting of dicey websites, primary application used all day long is vim.
    • However, if Terminal is now integrated with the world wide web then I suppose anything is possible.

MacBook Pro with Retina display, macOS Sierra (10.12.1)

Posted on Dec 8, 2016 7:06 AM

Reply
Question marked as Top-ranking reply

Posted on Feb 19, 2017 4:53 AM

Heya!


This problem's been bugging me as well and with the information from this thread I narrowed it down to the following:


Stuart's test case can be simplified to the following:

  1. Open vim an enter three lines of arbitrary text such as this:
    a
    a aaaaaaaaaaaaaaaaa
    a
    - First and third line just have to be present - no idea why.
    - The second line needs to have a space as the second character and wrap exactly one character to the next line.
    User uploaded file
  2. Delete the first character of the second line by pressing x so that now the line starts with a space and the last character of the line moves to the last column of the terminal.
    User uploaded file
  3. Wait about a second and Terminal.app will crash.


This suggests that Terminal.app has for some reason problems with lines that start with a space and end on the last column of the terminal.


It also only seems to relate with redrawing of the screen because I was not able to reproduce it with echo commands from the command line and I have only ever seen it with full-screen terminal applications like vim and screen, running both locally and remotely.


I had a suspicion it was related to the alternate screen which both vim and screen use by default. I was able to rule that out by setting TERM=vt100 to disable use of the alternate screen and still being able to trigger the crash as described above.


The problem seems to have been introduced with Terminal.app 387 of the 10.12.1 update. I extracted version 377 from my recovery partition (also present in the original 10.12.0 installer) which copes fine with above test case.


The obvious difference between both versions when diff'ing output of the strings command on both executables it the additon of Touchbar functionality mostly relating to man page interaction. I was not able to find any knob to disable this functionality to further narrow down the cause.


Thanks everyone for their research, providing a simple and reproducible test case and the workaround of going back to version 377 for now,

Michael

193 replies

Dec 20, 2016 8:59 AM in response to john henning

Hi Broomdodger, Yes, I have used MacVim, but not recently; the adventures documented here are strictly /usr/bin/vi

John

My intention is not to know if you previously tried MacVim, but to use the current version of MacVim with the Sierra terminal (or standalone) to see if it also caused the crashes. Reading your recent post I now understand vi[m] may not be the problem.

Dec 21, 2016 2:02 PM in response to mattj_io

mattj_io wrote:


Upgrading to latest vim via Homebrew seems to have fixed this issue for me.

How long have you been testing this setup?

Do you have 'Secure keyboard entry' enabled in the Terminal menu?



I understand everyone is looking for a solution to this but it appears people are trying many different ideas with nobody testing the work of others.


Please consider starting tests with the basic option 'Secure keyboard entry'. If that works it will not be required to make system wide changes.

If that fails then use the older Terminal app or other versions of vim/ vi. Sakuro's post suggests it is wider than just vi/vim (git fails too).


If anyone is willing to risk it please consider installing a key logger & disable secure keyboard entry. It would be very useful to have a set of instructions that can trigger this crash - which may be related to the keyboard input.

Dec 21, 2016 2:28 PM in response to john henning

So I had a day without crashes, which was nice, but was pleased to have reproduced the error again while running "script" on the remote system. This allowed me to capture the output that the shell on the remote machine was emitting. So I had a terminal window (one of many) up for around 4 hours, I was interacting solely with that window for 5 minutes or so when the terminal crashed. Interestingly it crashed as I was starting vim on the remote system.


I replayed the escape sequences just by issuing 'cat typescript.1' and also by writing a small program to print the characters one at a time interspersed with a small delay. Neither of those reproduced the crash. Given that the terminal went away, it is possible that ssh tore down the session and killed script before the final (buffered) characters were written, so I may have lost the really critical characters.


To be clear, on my local Mac after I opened terminal, I immediately used ssh to start a shell on a remote system and was working solely in that terminal on the remote system when Terminal crashed.

Dec 21, 2016 3:12 PM in response to channui

Hi all,


Just checking in with a (possibly meaningless) update. I have been using secure keyboard entry for 1-1/2 days now with no crashes, although my vi usage has only been light to moderate (15-30% of the time). I don't consider this remotely close to proof of a fix yet, just wanted you all to know the experiment continues. If I have still seen no crashes in a few more days, including at least a couple days of serious vi exercise, I'd perhaps be willing to believe secure entry might be a workaround. I don't know enough about secure entry to say how that is actually affecting the way in which keystrokes are delivered to Terminal, but it's hard to imagine there's a very big difference.


I don't think that I would expect crash frequency in general to be varied by working on a remote system inside your Terminal window -- in either case, your local Terminal is still processing all of your keystrokes as input and redrawing itself in response to your activities. It seems likely that it's one of these two things that is causing the crash, and we now have ample evidence that although vi usage may tickle the bug more than most Terminal children, it's not the root cause or even the only tickler.

Dec 21, 2016 8:31 PM in response to RogerDavis

  • Mr. Chan-nui: terrific! More below.
  • Roger: thank you for trying secure terminal entry, and for the report that it might have helped. (Instructions: this thread, entry by Drew, Dec 16, 2016 10:05 AM
  • John (me): at the moment, I am feeling very good about the El-Capitan-Terminal workaround. (recipe at entry from Dec 18 at 6:22pm)
  • Mattj_io: at the moment, you are feeling good about the vim 8.0 workaound. Interesting. Please can you provide a 'recipe'? The last time I built vim was years ago: I downloaded a tarball, unpacked, used make; but I guess the modern method is to use git, at http://www.vim.org/git.php; however you seem to be referencing yet another way to do it; please can you provide a pointer? And please tell us what options you picked for your build (especially what options you turned off - see the bit about a hypothetical 'xyzzy' below.)


Now, back to Mr. Chan-nui's results. I think you have said that you did manage to capture an output stream that was directed by remote vi toward your screen, which may have crashed it; and furthermore that you have parsed that stream into something readable. You are unsure whether the stream is complete; nevertheless, I think that this is Most Excellent.


Is the stream suitable to post somehow? Or, say, the last 150 bytes of it? If not, you and I know each other outside this forum, perhaps you could point me at the entrails privately.


From the first paragraph of the first post in this thread, we know that there is an invalid address of all zeros when the crash happens.


I am wondering if there might be any clues in the stream of characters that could lead us to thinking about something like "ah, it has just referenced the cached terminal state supporting unicode set xyzzy; the aforementioned cache is expected to be persistent; but perhaps on rare occasions the cache is invalidated by external event plover (which could lead to a null pointer); therefore if we just turn off the vim feature plugh, it will never look for that cache".

Dec 22, 2016 12:01 AM in response to john henning

Hi John


The easiest way is to use Homebrew ( http://brew.sh/ )


You'll need Xcode installed first, then install Homebrew, then just do :


brew install vim


That will probably pull down a pre-compiled version, but you can force it to recompile by doing :


brew install vim --build-from-source


I'm just using the standard compile flags from the Homebrew recipe. If you want python support in vi, you'll probably need to brew install python first.

Dec 22, 2016 9:45 AM in response to john henning

I've trimmed the tail off of the stream for two instances, I was not able to catch a third where vi was not involved. I'm including a base64 gzipped version of those. Decode by copying into your pasteboard and then run "pbpaste | base64 -D - | gzcat > output_file_here.txt"


This is from vi (typescript.1)


H4sICGsOXFgAA3R5cGVzY3JpcHQuMS1zbWFsbADNlsFqhDAQQO9CvyCXsKceaslMJlorS9mllIVC

fyD0VAUL2kMXttDDfnsn4lZd7K06K5gYMvLGiTxGg713qUaIH8u3GBJ1vVW+UZ6SHHbKQ07Jp/L8

RC4nx/MDYM0juiqMxhCvkHjvFFZE49eKNjpEwUb556tooeEsVajUepQpUtGHwvZSptXt974qPw4r

7V/KL/30XpevZ+eA4WS6mg6+J1TaUnPUC16MDT/K0sxUgHknwMyWZ1ojwAQBJgow7czMoSgYltnd

yecDUXQbwRVTOZJAXZwAU8BbVsBbVsBbVsBbJOAtEvAW/aO3Rn0F5ilO6aLbmNTFXznO7bkpJrUN

q804e+A7bbpGag6cuTEx9MtNXf8m0Fe0bTAHPWhb2ugHw5igV3ELAAA=


And this is from vi editing the previous dump file (typescript.2):


H4sICKMPXFgAA3R5cGVzY3JpcHQuMi1zbWFsbADtlt9LwzAQx98H+wvyUvYkSKV3+VmDODaRgmz/

QHAvtjBh9UFBwb/eLHU0V5yZ4JgP9qFNvnefS67pJWVnM+Za5lBYqJi7Btz4O8o1c2C5eWbOq1xb

AO0b4D11a4Kt7GzIa/YZYy+gCYBpQBEA0oCMAShnSUAQANIAjwFxhAGOlULZrxvkhqQBaA2vkhwU

PIdCDkivpFFV8txIIKix6gBSg8yN5gPS6D0kQj+m58SQMz2Hlqv+pTm9iGN5KQN+KXWGkN80Dzn4

bxHnW5Zju3L+UbbbMFFPqFA7kQJWbIeIFGoX0gpJpa7yYiEUIRGKQlAfX7VCfj9Q7RvFcH53XW8x

Hv2KcngKQPpXP0sGRX1YFrtVmH3l/m88kXE1/nNT2mOcXLy/rJun10nmls1bvJVwZZWpwgm5IVtQ

0LPbx01zT0OndwIcFGcXmxaGb8iuiEU7zU5+hYShmp56HmEm2mJRhTe1OwHcuV+EUs1h50PPyfgv

JxwKow+4RzqvBgkAAA==

Dec 26, 2016 1:19 AM in response to john henning

I tested 'Secure Keyboard Entry' with two cases.

In short, the crash *can happen with* Secure Keyboard Entry enabled.


In both cases I ssh-ed to a server ('remote') and 'tail -f'-ed a log file of nginx

which serves to decent amount of requests.


Case A (in tmux):

mac$ tmux

mac[tmux]$ ssh remote

remote$ tail -f /var/log/nginx/blahblah.access_log


Case B (not in tmux):

mac$ ssh remote

remote$ tail -f /var/log/nginx/blahblah.access_log


Case A frequently crashes, within seconds in the worst case.

Sometimes it immediately crashes again when I reattach the terminal to the crashed session with 'tmux a'

Case B does not seem to crash for 10 or more minutes.


I thought the ncurses library is the culprit becase both tmux and vim are linked to it,

but my tmux is linked to /opt/local/lib/libncurses.6.dylib from MacPorts

while /usr/bin/vim is (of course) linked to /usr/lib/libncurses.5.4.dylib :-/

Dec 27, 2016 2:36 PM in response to john henning

I'm running Sierra 10.12.2 and suffering terminal crashes daily since the ".2" update.


The crashes always happen when I'm editing code, often when saving with ":w".


I have a big external monitor pretty much paved with terminal windows, several of which are running vim on different parts of my code base. Several terminals windows are generally logged into remote production servers, and I'm pretty sure I've had crashes editing files on those ssh'd systems as well as on my local MacBookAir6,2.


I'm going to pay more attention to which system's vim causes the crash, and exactly what my "automatic" fingers are actually doing to provoke the crash. (I'm also not very conscious of what my vim fingers are doing)


It sure is painful to crash terminal when I have so many open files and ssh sessions at a time!


My laptop's /usr/local/bin/vim is symlinked to MacVim 8.0, but the servers are running 7.3


My laptop:


:version

VIM - Vi IMproved 8.0 (2016 Sep 12, compiled Dec 9 2016 08:03:38)

MacOS X (unix) version

Included patches: 1-124

Compiled by root@Traviss-Mac-592.local

Huge version with MacVim GUI. Features included ➕ or not (-):

+acl +diff +jumplist +mouse_urxvt +scrollbind +virtualedit

+arabic +digraphs +keymap +mouse_xterm +signs +visual

+autocmd +dnd +lambda +multi_byte +smartindent +visualextra

+balloon_eval -ebcdic +langmap +multi_lang +startuptime +viminfo

+browse +emacs_tags +libcall -mzscheme +statusline +vreplace

++builtin_terms +eval +linebreak +netbeans_intg -sun_workshop +wildignore

+byte_offset +ex_extra +lispindent +num64 +syntax +wildmenu

+channel +extra_search +listcmds +odbeditor +tag_binary +windows

+cindent +farsi +localmap +packages +tag_old_static +writebackup

+clientserver +file_in_path +lua/dyn +path_extra -tag_any_white -X11

+clipboard +find_in_path +menu +perl/dyn -tcl -xfontset

+cmdline_compl +float +mksession +persistent_undo +termguicolors +xim

+cmdline_hist +folding +modify_fname +postscript +terminfo -xpm

+cmdline_info -footer +mouse +printer +termresponse -xsmp

+comments +fork() +mouseshape +profile +textobjects -xterm_clipboard

+conceal +fullscreen +mouse_dec +python/dyn +timers -xterm_save

+cryptv -gettext -mouse_gpm +python3/dyn +title

+cscope -hangul_input -mouse_jsbterm +quickfix +toolbar

+cursorbind +iconv +mouse_netterm +reltime +transparency

+cursorshape +insert_expand +mouse_sgr +rightleft +user_commands

+dialog_con_gui +job -mouse_sysmouse +ruby/dyn +vertsplit

Dec 28, 2016 7:43 AM in response to randy_parker

Thread summary to date

8 users report crashes of Terminal.app

  • Crashes reported both with Sierra 10.12.1 and 10.12.2
  • Users: This thread: JohnHenning, RogerDavis, channui, mattj_io, sakuro, randy_parker; Related threads: dsandber, markwolgemuth

The 'vi' editor seems to be the victim, not the criminal

  • Three users report crashes while using remote systems
  • Two were using vi remotely; one is just using 'tail' (Dec 26, 2016 1:19 AM Sakuro-san)

Related threads

Reproducer progress: No reproducer yet. Progress toward one includes:

  • Dec 26, 2016 1:19 AM Sakuro has a case where 'tail -f' usually does it
  • Dec 22, 2016 9:45 AM Chan-nui posted 'script' capture when remote vi caused crash
    • I have been (casually) looking at the sequences (with od -abc)
    • There are a lot of 8-bit characters in use but I am having trouble guessing what 8-bit encoding is in use.
    • It does not seem to be https://en.wikipedia.org/wiki/UTF-8

Peeking inside?

  • It would be nice if there were truss or dtrace logs

    A link above (Dec 9, 6:19am) allegedly explains how to enable dtrace, but I have not dared.

  • The stack traces are not obviously insane when compared to 'normal' running app. (see entry Dec 8 at 7:19pm)
  • Maybe 'fs_usage' could provide clues?

Rejected workarounds (tried, did not solve)

  • Migrated account
  • Fresh new account
  • Safe mode
  • Wiped disk, fresh Sierra
  • Old Vim 7.3 from El Capitan
  • Update Cisco VPN (still saw crashes + subsequently excluded as a candidate by the wiped-disk-other-laptop test)


Remaining Workaround Candidates

  • Secure Text Entry Recipe: Dec 16, 2016 10:05 AM suggested by Drew.
    Status: conflicting reports: Dec 21, 2016 3:12 PM RogerDavis reports 1.5 days without crashes
    Dec 26, 2016 1:19 AM Sakuro-san says crashes can happen
  • Vim 8.0 Recipe: Dec 22, 2016 12:01 AM
    Status: conflicting reports: Dec 21, 2016 2:20 PM Mattj_io reports 2 days success
    Dec 27, 2016 2:36 PM randy_parker is 'pretty sure' he has had crashes both remote and local; local is MacVim 8.0
Old Terminal Recipe: Dec 18 at 6:22pm suggested by me
  • Status: Testing by 1 user (me). No crashes after use for 9 days
  • Request: Would someone else please try this, and confirm whether it helps you too?

Terminal.app crashing: Can I disable callbacks?

Welcome to Apple Support Community
A forum where Apple customers help each other with their products. Get started with your Apple Account.