hexdump -C outputs unreadable characters

Question

hexdump -C outputs unreadable characters

In previous versions of OS X (10.4 and older), the terminal command "hexdump -C" only displayed ASCII characters in the right column (i.e. when using the "%_p" format). All the others (control characters etc, and anything above 127) were replaced by a dot. This is also how hexdump behaves in Linux.
Since Leopard however, the characters above 127 are also displayed somehow, and the terminal seems to convert them to 4-byte sequences. This makes that type of display useless. A file that looks like this in the old hexdump (best viewed in a monospaced font):

00000000 ff d8 ff e0 00 10 4a 46 49 46 00 01 01 01 00 60 |......JFIF.....`|
00000010 00 60 00 00 ff e1 00 6e 45 78 69 66 00 00 49 49 |.`.....nExif..II|
00000020 2a 00 08 00 00 00 01 00 69 87 04 00 01 00 00 00 |*.......i.......|

... looks like this in Leopard's hexdump:

00000000 ff d8 ff e0 00 10 4a 46 49 46 00 01 01 01 00 60 |<FF><D8><FF><E0>..J
FIF.....`|
00000010 00 60 00 00 ff e1 00 6e 45 78 69 66 00 00 49 49 |.`..<FF><E1>.nExif.
.II|
00000020 2a 00 08 00 00 00 01 00 69 87 04 00 01 00 00 00 |*.......i.......|

According to the manpage, the "%_p" format +"displays nonprinting characters as a single ``.''"+. So it seems Apple changed its definition of "nonprinting characters" in Leopard.
Is there any way to get the old behavior back?

MacBook Pro C2D 2.33, Mac OS X (10.5.3)

Posted on Jun 5, 2008 7:02 AM

Reply

Answer 1

Top-ranking reply

orangekay

Level 5

4,085 points

Jun 5, 2008 8:25 AM in response to Alexander Thomas1

Seems to work for me.

$ hexdump -C Duke.jpg
00000000 ff d8 ff e0 00 10 4a 46 49 46 00 01 02 01 00 48 |????..JFIF.....H|
00000010 00 48 00 00 ff e1 11 fb 45 78 69 66 00 00 4d 4d |.H..??.?Exif..MM|
00000020 00 2a 00 00 00 08 00 07 01 12 00 03 00 00 00 01 |.*..............|
00000030 00 01 00 00 01 1a 00 05 00 00 00 01 00 00 00 62 |...............b|
00000040 01 1b 00 05 00 00 00 01 00 00 00 6a 01 28 00 03 |...........j.(..|
00000050 00 00 00 01 00 02 00 00 01 31 00 02 00 00 00 1e |.........1......|
00000060 00 00 00 72 01 32 00 02 00 00 00 14 00 00 00 90 |...r.2..........|
00000070 87 69 00 04 00 00 00 01 00 00 00 a4 00 00 00 d0 |.i.........?...?|
00000080 00 0a fc 80 00 00 27 10 00 0a fc 80 00 00 27 10 |..?...'...?...'.|
00000090 41 64 6f 62 65 20 50 68 6f 74 6f 73 68 6f 70 20 |Adobe Photoshop |
000000a0 43 53 32 20 4d 61 63 69 6e 74 6f 73 68 00 32 30 |CS2 Macintosh.20|
000000b0 30 36 3a 30 34 3a 32 31 20 31 30 3a 33 38 3a 30 |06:04:21 10:38:0|

Reply

Answer 2

Jun 5, 2008 12:31 PM in response to orangekay

OK, I didn't tell the full story: what I usually do, is +"hexdump -C filename | less"+, so I can easily scroll the output. I haven't tried it without the +"| less"+ yet and indeed, it doesn't produce the hex codes but question marks instead. The reason why it produces question marks, is because the terminal tries to interpret the characters as unicode. Codes like FFD8 are not valid utf-8, hence the question marks. "less" prints the raw hexadecimal values instead, producing the ugly output from my first post. It's possible to make "less" interpret the values as iso latin 1, which also produces the question marks. Even though it's not perfect, it does produce more usable output (garbage will still appear if there happens to be an accidental existing utf-code in a binary file).

I guess to get the old behavior (dots for non-ASCII characters), I'll have to either compile my own version or try to copy the one from a Tiger machine. I could try submitting a bug report but I'm pretty sure Apple will tell me that it complies to the man page.

Reply