Apple Event: May 7th at 7 am PT

Looks like no one’s replied in a while. To start the conversation again, simply ask a new question.

Terminal.app crashing: Can I disable callbacks?

Five times in the last 48 hours, Terminal.app "quit unexpectedly". The Terminal_datestamp.crash files show

EXC_BAD_ACCESS (SIGSEGV) KERN_INVALID_ADDRESS at 0x0000000000000000

with a call stack that always includes:

com.apple.Terminal 0x000000010530bc94 0x105295000 + 486548 com.apple.Foundation 0x00007fff972ac5df __NSFireTimer + 83 com.apple.CoreFoundation 0x00007fff95837d74 __CFRUNLOOP_IS_CALLING_OUT_TO_A_TIMER_CALLBACK_FUNCTION__ + 20 com.apple.CoreFoundation 0x00007fff958379ff __CFRunLoopDoTimer + 1071 com.apple.CoreFoundation 0x00007fff9583755a __CFRunLoopDoTimers + 298 com.apple.CoreFoundation 0x00007fff9582ef81 __CFRunLoopRun + 2065

User impact

I cannot use the system: must find solution, even if the cost is to wipe disk and install previous OS (El Capitan).

The primary question

(q1) Is there some new feature of Sierra that integrates Terminal with other apps that can be turned off, perhaps a setting to disable callbacks?

Secondary questions

  • (q2) Can I potentially run a copy of the Terminal.app from a previous version of Mac OS X? Prior to my new laptop (and Sierra) these crashes never happened.
  • (q3) What additional detail would be useful, and how can I collect it?

    I am familiar with Unix application development, but not for Mac OS X. I am happy to collect additional detail by turning on debug tools, traces, event collectors, etc. but would need pointers to get started. Xcode is available, but I would be a beginner with it.

System summary

  • New MacBook Pro (Retina, 15-inch, Mid 2015).
    • Arrived 6 days ago.
    • (Yes, I intentionally bought a brand-new instance of the previous model.)
    • Surprisingly, it arrived with Sierra. It presently has 10.12.1
    • Xcode installed
    • GCC 6.2.0 available, built locally.
  • Previous history brought in by Migration Assistant
  • Not tried (yet, anyway)
    • I have briefly visited safe mode but have not stayed there. I do not know whether there would be crashes of Terminal.app if I spent all day in safe mode
    • I do not know whether there would be crashes if I created a fresh, different user account.
  • Tentative rule-outs:
    • Malware, adware, etc are reasonably unlikely given the usage pattern for this work system: firewalls in sw and hw, no visiting of dicey websites, primary application used all day long is vim.
    • However, if Terminal is now integrated with the world wide web then I suppose anything is possible.

MacBook Pro with Retina display, macOS Sierra (10.12.1)

Posted on Dec 8, 2016 7:06 AM

Reply
Question marked as Best reply

Posted on Feb 19, 2017 4:53 AM

Heya!


This problem's been bugging me as well and with the information from this thread I narrowed it down to the following:


Stuart's test case can be simplified to the following:

  1. Open vim an enter three lines of arbitrary text such as this:
    a
    a aaaaaaaaaaaaaaaaa
    a
    - First and third line just have to be present - no idea why.
    - The second line needs to have a space as the second character and wrap exactly one character to the next line.
    User uploaded file
  2. Delete the first character of the second line by pressing x so that now the line starts with a space and the last character of the line moves to the last column of the terminal.
    User uploaded file
  3. Wait about a second and Terminal.app will crash.


This suggests that Terminal.app has for some reason problems with lines that start with a space and end on the last column of the terminal.


It also only seems to relate with redrawing of the screen because I was not able to reproduce it with echo commands from the command line and I have only ever seen it with full-screen terminal applications like vim and screen, running both locally and remotely.


I had a suspicion it was related to the alternate screen which both vim and screen use by default. I was able to rule that out by setting TERM=vt100 to disable use of the alternate screen and still being able to trigger the crash as described above.


The problem seems to have been introduced with Terminal.app 387 of the 10.12.1 update. I extracted version 377 from my recovery partition (also present in the original 10.12.0 installer) which copes fine with above test case.


The obvious difference between both versions when diff'ing output of the strings command on both executables it the additon of Touchbar functionality mostly relating to man page interaction. I was not able to find any knob to disable this functionality to further narrow down the cause.


Thanks everyone for their research, providing a simple and reproducible test case and the workaround of going back to version 377 for now,

Michael

193 replies

Jan 3, 2017 2:05 PM in response to john henning

Why do no Apple engineers post to this thread? Surely they are suffering from this annoying crash just like the rest of us, and could save us all a lot of trouble by telling us what they know.


For example, "we've isolated and fixed the problem, and it'll be fixed in 10.12.3 (or whatever)".

Or, "we've narrowed it to x, y, or z. Please post examples if this-or-that".


I used to have a lot of loyalty to Sun Microsystems because they so openly helped customers solve problems. Engineers freely collaborated with customers, and while I'm sure they didn't show us all their dirty laundry, it was clear that they were sincerely trying to solve problems. Once, after we bought one of their first $650,000 data center servers, they even flew a team of kernel engineers and a compiler engineer to our Atlanta data center to try to capture a kernel panic using custom kernels. It took a few days, but they found the bug and fixed it. We were pretty **** loyal after that commitment! Apple, on the other hand, leaves us utterly in the dark even when everyone knows they have a bug. Maybe they'll fix it, or maybe they have an internal workaround and just don't give **** about us.

Jan 3, 2017 2:09 PM in response to randy_parker

"Why do no Apple engineers post to this thread?"


Because Apple engineers would get inundated. Also, when I was an Apple engineer, I know we only had time to search these threads if we were looking for a repro case where we didn't already have one. Most of the time, that was fruitless. There is not enough information in this thread to build a repro case.


Essentially, this community a sandbox that is looked at (by engineering) less than 0.1% of the time. If that much.

Jan 3, 2017 2:21 PM in response to Baughn

Hi Baughn,


I'm trying to reproduce from your crash recipe, but no luck so far. I have never used either screen or irssi until today, so maybe I am doing something stupid. My hardware config is a bit different than yours, which may cause some obscure differences in the sleep/disconnect. I am using an iMac Sierra desktop (rather than a laptop) to connect to my remote server (also an iMac Sierra), as I do not have a Sierra laptop available. Here is exactly what I did:


(1) Configured client iMac to sleep after two minutes.


(2) Started Terminal on client, and using that Terminal window I ssh'd into server where I started screen (no arguments) and then, after screen started, started irssi (also no arguments) within the screen session. I then detached from screen with <CTRL>-a <CTRL>-d, and closed the ssh with a <CTRL>-d. (I did not engage in any activity whatsoever within irssi as I have no idea how to use it.)


(3) In the same original client Terminal window I reconnected to the server with ssh, and reattached to the screen session I started in the previous step with 'screen -r'. irssi was still running inside the screen session. Again, I typed nothing into the irssi program.


(4) I then walked away and allowed the system to go to sleep. When I returned, the client display was black, I moved the mouse to wake it up, and was shown the lockscreen password display. I logged back in, and then in the Terminal window I typed the up-arrow key (alone) followed by <RETURN>. I then saw the ssh disconnect message (but only after typing those two keystrokes).


I've tried this a few times, but alas no crash has resulted. Am I doing something wrong?


It would be very nice to be able to reliably reproduce this bug in a simple way. The only other suggested recipe which has been posted here involved tail'ing output from a specific web server engine that would be very difficult to replicate on another system, I think.


Thanks!


Roger

Jan 3, 2017 2:32 PM in response to Deirdre Saoirse Moen

Essentially, this community a sandbox that is looked at (by engineering) less than 0.1% of the time. If that much.


Another good reason for everyone to click that 'Send to Apple' button in the crash dump notification. Deirdre, do you know what threshold is required for Apple engineers to notice THAT?


I am sort of wondering if this bug may not really be isolated to Terminal. The stack backtraces look like there's a dangling null pointer somewhere in Apple's GUI code, so it could be this affects other apps, too. Maybe if enough iTunes users complain all us influence-less Terminal jockeys will get some wholly unintended satisfaction. Here's hoping!

Jan 3, 2017 2:58 PM in response to RogerDavis

It has to be a top crasher, probably in the top 25 or 50. And yes, it is based on reports sent to Apple, because that's all they have access to.


Even if it is a null pointer issue, it could still be a bug that only Terminal is hitting.


If you do have specific repro steps that always work, even on a new account, then file a bug report with Apple with those specific steps. If nothing else, that'll make it easier to verify when the bug is fixed.

Jan 3, 2017 10:42 PM in response to john henning

I just (seemingly) had this happen to me ☹️.


I was ssh'ed into a Linux machine (Ubuntu 16.04) and running screen there. No idea what triggered it (I *wasn't* in an editor, but I did have a very long command line) - but it killed Terminal several times.


At first I though it might be Touchbar related. I just removed the 'man page' widget from the TB in Terminal. My thought was perhaps this is still getting parsed and processed - but reading through the thread it seems like there's plenty of folks without TB in the equation having this issue.


I'm really hoping this gets fixed in the next update. I've been Mac daily driver since 10.7 in 2011. I don't think I've ever had Terminal up and crash until today.

Jan 4, 2017 10:16 AM in response to RogerDavis

BTW, secure keyboard entry is now thoroughly debunked as a workaround based on ProValid's test (in case anyone still had the slightest doubt given other recent failures).


The test is interesting in its requirement of a peculiar combination of circumstances. If you first delete either the first or last line, it doesn't crash -- in fact, if you then undo those line deletions and try again with xxxx it STILL doesn't crash. Also, other ways of shortening the long line do not crash, e.g., deleting the last half by moving to the center of the line and typing D. Great detective work, ProValid!


Maybe if each of us can play around for 10 minutes a day for the next week and send all the crash dumps to Apple someone over there will wake up and smell the rancid coffee. (Deirdre, what's it take to be a Billboard Top-50 crash dump poster?)

Jan 4, 2017 10:18 AM in response to RogerDavis

Sidebar to the impatient:

Sun Microsystems was praised in this thread earlier.

While the words were very kind, you might want to take into account the engineers' point of view from "the 12 bugs of Christmas": https://www.youtube.com/watch?v=_lZ7LMoITy0

Too long / didn't listen summary: bugs take time. Patience, that is why we need workarounds

Jan 4, 2017 10:22 AM in response to john henning

Reproducer with Apple Script

Here is a scripted version of the reproducer that was posted upthread.

(* This applescript demonstrates crashing of Sierra Terminal using simple 'vi' commands. WARNING: never run a random script from the internet without reading it and understanding it. Assuming that you have done so, the instructions are: (0) Save this as "crashTerminal.applescript" (1) Make sure that your Terminal default width is 80 (2) Close other terminal apps. (3) From terminal, say 'open crashTerminal.applescript' (4) Hit the "Play" (triangle) button. *) tell application "Terminal" activate delay 1 tell application "System Events" tell process "Terminal" set textBuffer to "cat > justover80.crashtest.tmp" repeat with i from 1 to count characters of textBuffer keystroke (character i of textBuffer) delay 0.1 end repeat keystroke return keystroke return delay 0.1 repeat with i from 1 to 83 keystroke "=" delay 0.02 end repeat keystroke return keystroke return keystroke "d" using control down delay 1 keystroke return delay 0.1 set textBuffer to "rm -f .justover80.crashtest.tmp.swp" repeat with i from 1 to count characters of textBuffer keystroke (character i of textBuffer) delay 0.1 end repeat keystroke return delay 1 set textBuffer to "vi -i NONE -u NONE justover80.crashtest.tmp" repeat with i from 1 to count characters of textBuffer keystroke (character i of textBuffer) delay 0.1 end repeat delay 2 keystroke return delay 1 keystroke "j" delay 1 keystroke "x" delay 1 keystroke "x" delay 1 keystroke "x" delay 1 keystroke "x" end tell end tell end tell

Jan 4, 2017 10:50 AM in response to john henning

summary: bugs take time. Patience, that is why we need workarounds


Good advice, John. (Although I can't help but note that half the verses of that song are the stereotypical blatant attempts at obfuscation of responsibility used by some tech support groups.)


Sun Microsystems was indeed an exceptional organization in its early days. I once sent Scott McNealy an e-mail when I had an issue with a local sales office concerning a single desktop workstation. He responded the same day and the problem was solved. I tried the same trick about 8 years later with Dell and you can imagine the (non-)result.

Jan 4, 2017 12:32 PM in response to john henning

I have just filed my own bug report on this (100106033554) and cross-referenced it to John's (100085116387). I would have done this much earlier but my work systems were not under support and I only recently discovered that my kid's iMac at home (where I have only just now been able to recreate the crash) is still under AppleCare.


While speaking with Apple phone support I uploaded all crash dumps resident on my system directly into their case files (or so I was told) using Apple's Capture Data tool. I also made them aware of this discussion thread and the recent foolproof repros submitted there by ProValid and John.

Jan 6, 2017 9:15 AM in response to channui

Summary: the problem might be related to 'scroll regions'


I had promised to try to interpret the output from Mr. Chan-nui, but failed to do so.

In partial recompense, here is an interpretation of the characters from one of my

crashes using the recent reproducer. In this case, the characters were captured using


script -F fifo


Unfortunately, various attempts at playing back the capture do not cause the

crash; therefore, the annotated output may be of limited value.


Sources used:

http://www.xfree86.org/4.5.0/ctlseqs.html

http://www.vt100.net/docs/vt510-rm/chapter4.html

http://vt100.net/docs/vt510-rm/DECRQM.html#T5-8

http://vt100.net/dec/ek-vt240-hr-002.pdf


esc[ = 'CSI', control sequence introducer


I think that the sequence Show cursor / Hide Cursor indicates

that user input happened in between those two.

The cursor is shown,

the user types 'x'

the cursor is hidden and work begins.


'esc[?25h' Show cursor

'esc[?25l' Hide cursor

'esc[79C==' Move 79 chars right, insert ==

'esc[3;1H=' Line 3, col 1 =

'esc[3;2H' Line 3, col 2

'esc[K' Erase to right

'esc[2;1H' Line 2, col 1

'esc[?12l' Stop blinking

'esc[?25h' Show cursor


At this point, we are at line 2 column 1 with a solid cursor.

The user presses 'x', and the world dies.

The sequence below gets rid of line 2 by *scrolling* it off a dynamically-defined scroll region,

then re-creates line 2 with its new content,

and creates a new empty line at the bottom with a blue tilde at the front.


'esc[?25l' Hide cursor

'esc[2;23r' Scroll region 2:23

'esc[23;1Hcrnl' Move to line 23 return, newline

'esc[1;24r' Scroll region 1:24

'esc[2;2H' Line 2, column 2

'=============================================================================== '

'esc[23;1H' Line 23, col 1

'esc[94m' Set color blue, next line paints it

'~ '

'esc[m' SGR off (SGR = graphics attributes)

'esc[24;1H' Line 24, col 1

'esc[K' Erase to right

'esc[2;1H' Line 2, col 1

'esc[?12l' Stop blinking

'esc[?25h' Show cursor


In a normal run, we are ready for more input. In Sierra 10.12.1 or .2, we don't exist.

Terminal.app crashing: Can I disable callbacks?

Welcome to Apple Support Community
A forum where Apple customers help each other with their products. Get started with your Apple ID.