UK user. Can I write my PhD thesis using Pages?

For:

Pages can produce text with advance typographical features which, when exported to pdf format, will have only limited searchability. If searchability is important for your purposes, you should make sure you use the following settings:

Read:

Apple Mac OS X supports typesetting with simple typographical features as well as with advanced typographical features. When exported to PDF 1.3 in Apple Mac OS X 10.4 and PDF 1.3 in Apple Mac OS X 10.5, only simple typographical features will be searchable. If searchability is important for your purposes, you should make sure you use the following settings:

Unicode imaging services are system-wide as are most aspects of activating the system-wide Unicode imaging services. As you suggest, in addition there should be per-application advise.

+Do not work with PostScript Type 1 or PostScript Type 1 Expert fonts. Searching may neither be supported before you save into PDF from Apple Mac OS X 10.4 and Apple Mac OS X 10.5.

/hh



/hh

+Do not work with PostScript Type 1 or PostScript Type 1 Expert fonts. Searching may neither be supported before you save into PDF from Apple Mac OS X 10.4 and Apple Mac OS X 10.5.

Read:

+Do not work with PostScript Type 1 or PostScript Type 1 Expert fonts. Searching may neither be supported before you save into PDF 1.3 nor after you save into PDF 1.3 from Apple Mac OS X 10.4 and Apple Mac OS X 10.5.

Politically, it is preferable that there be a public trusted font family; that the font family be free in simple TrueType, AAT and OT formats all three together; that this trio be a free font be available in the EU; and that the free trio be documented absolutely to death.

/hh

So do you agree that if an author is using Windows Word or Mac Word before version 2008, he does not have to be concerned about searchability when he exports his document to pdf format?

No, I don't agree.

Adam Twardoch, the technical writer for Fontlab (the most popular font production software) has posted that Adobe OpenType products are provided with Private Use Area drawing for supplementary appearance shapings is,

1. To allow versions of Microsoft Word that do not support supplementary advanced shaping through AAT MORX tables or OT GSUB tables nonetheless to draw these alternate glyphs - irrespective of whether the source character string is corrupted ('***********' is his term).

2. To allow versions of Microsoft Word that do support supplementary advanced shaping through OT GSUB tables to draw shapes when the application Word does not support the shaping feature but the OT GSUB tables of the specific font file do.

In other words, the way Adobe and Microsoft have set up the OT GSUB tables, the application has to have the internal support. Therefore, while a feature may be added by the type maker, the application cannot draw it until the application is updated.

By definition, this is avoided in the design of AAT as demonstrated at the start. Typesetting features unique to the individual font file can be programmed, and any and all applications that work with the system level services are able to take advantage of the added feature without upgrading.

Adam Twardoch posted in March 2004 that,

"Applications that support OpenType Layout features that are used to access these glyphs will naturally be able to access these glyphs via these features.

"Private Use Area mappings are devised for applications that support Unicode but do not support any OpenType Layout features or the particular features that you used in your font. In such case (e.g. Microsoft Word), the user can use an "Insert Symbol" feature that will bring up a Character Map-alike window, and there, insert the character. Of course, the text stored behind the character inserted that way will be "garbled". Meaning: if the user inserts an "a.init" or "a.smcp" using the PUA mapping, the spellchecker, the hyphenation algorithm or the search/replace feature will not recognize it as a proper "a" (while this would be the case if the character is inserted via the OpenType Layout features). So this really is the last resort option. OpenType Layout features are better than PUA mappings, but since there are still applications that don't support OTL, PUA may be the only option to allow the users inserting these characters.

"I know that John Hudson advises against PUA, I personally do the same, I also think that Microsoft has the same position. Adobe officially advises against the practice, but have been assigning PUA in their own fonts. My guess is, however, that they will stop doing so at some point."

/hh

Reference:
http://www.typophile.com/node/3536

So do you agree that if an author is using Windows Word or Mac Word before version 2008, he does not have to be concerned about searchability when he exports his document to pdf format?

No, I don't agree.

Thanks for the explanation.

Could you clarify a bit: I understand that PUA mappings will not be searchable, but are you saying that PUA mappings can be generated automatically from the keyboard when certain fonts are used? Or will these only be produced when the author resorts to Insert Symbol or similar means to create them (in Win Word and Mac Word before 2008)?

Thanks for your comments. After correcting some other errors in my text, I have this as a proposed warning for Pages users:

=========================

Warning Regarding PDF Searchability

Apple Mac OS X supports typesetting with simple typographical features as well as with advanced typographical features. When exported to PDF 1.3 in Apple Mac OS X 10.4 and PDF 1.3 in Apple Mac OS X 10.5, only simple typographical features will be searchable. If PDF searchability is important for your purposes, you should make sure you use the following settings in Pages:

+Format > Font > Ligatures: None

+Inspector > Document: Uncheck box for Use Ligatures

+Inspector > Text: Check box for Remove Ligatures

+Character Styles: Uncheck attribute box for Advanced Font Features and make sure Ligatures is set to None.

+Format > Fonts > Show Fonts (Font Panel): Do not check any boxes in Advanced (gear wheel) > Typography

In addition, do not work with PostScript Type 1 or PostScript Type 1 Expert fonts. Searching may neither be supported before you save into PDF 1.3 nor after you save into PDF 1.3 from Apple Mac OS X 10.4 and Apple Mac OS X 10.5. No such fonts are provided with OS X or with Pages.

Politically, it is preferable that there be a public trusted font family; that the font family be free in simple TrueType, AAT and OT formats all three together; that this trio be a free font be available in the EU; and that the free trio be documented absolutely to death.

Plus the font should presumably cover all the characters needed for the 23 official languages (Does the union of those characters have a name?).

Will we see the political dimension of this emerge sometime soon in the CDFG or another EU institution?

Plus the font should presumably cover all the characters needed for the 23 official languages. (Does the union of those characters have a name?). Will we see the political dimension of this emerge sometime soon in the CDFG or another EU institution?

CDFG and EFNIL have been asked to collaborate on a public podcast discussion of whether support for spelling, sorting and searching depends, as the Unicode composition model proposes, on a protective separation of character processing from glyph processing, and thus whether in teaching writing today we are concerned with characters at all. ISO-IEC Technical Report 15285: An Operational Model for Characters and Glyphs is clear that in the Unicode composition model, character codes are public and font-independent while glyph codes are private and font-dependent. Thus in teaching writing today we are concerned with character codes as content and with protective drawing of glyph codes as appearance.

Wrt whether CDFG and EFNIL will accept? You would have to ask the Technical Secretary of CDFG and the Director of EFNIL. I simply don't know.

Wrt Europa, the free and documented implementation of the intelligent composition model, implementations of PDF/A are pending.

Project Europa is a rendering specification for the intelligent composition model. A rendering specification speaks to the issue of how a graphic information processing model is to be implemented, in this context for the base model and both advanced models.

Clearly, the obligatory CMAP Character Map cannot be configured for drawing with incorrect public characters. For instance, Linotype Zapfino for simple TrueType draws ligatures from incorrect public characters ... the model from Adobe Type 1 'Expert'.

Clearly, the obligatory CMAP Character Map cannot be configured for drawing with incorrect private charactoids. For instance, Linotype Zapfino for OpenType programmed by Adam Twardoch in 2004 draws more than 75% of its glyph codes off Private Use Area code points.

In the archives of the Apple ColorSync Users List, you will find hundreds of post trashing private tables, which are provided for in the ICC Specification. You will also find hundreds of posts trashing deviceLink transforms, which are provided for in the ICC Specification.

/hh

Could you clarify a bit: I understand that PUA mappings will not be searchable, but are you saying that PUA mappings can be generated automatically from the keyboard when certain fonts are used? Or will these only be produced when the author resorts to Insert Symbol or similar means to create them (in Win Word and Mac Word before 2008)?

If you draw either by inserting incorrect public characters or PUA public code points intended for private characters, your source character string is no longer solely public or solely correct.

If and only if the glyph drawings have glyph identifiers that follow the Adobe glyph naming guidelines, your source character string can to some degree by synthesised.

The degree to which the source character string can be successfully synthesis depends on how linear the relationship is between character input and glyph output in the transform.

For Arabic and Indic, which are still contextually shaped calligraphic scripts, you will have less success than for Latin, Greek and Cyrillic, which are typographic scripts.

John Hudson writes, "Of course, the whole mechanism fails spectacularly as soon as you start dealing with complex scripts in which glyph order does not correspond to character order."

I would prefer not to post on what Linux users and Windows users would do in order to draw with Private Use Area code positions in the CMAP Character Map of their purchased type products.

Possibly they would work their typesetting by manually selecting default glyphs and manually substituting non-default glyphs using Insert Symbol, or possibly they would find ways of automating their typesetting if their applications were able to search public characters and replace them with public code points that are intend for private 'charactoids'.

/hh

Reference:
http://www.typophile.com/node/2994

CDFG and EFNIL have been asked to collaborate on a public podcast discussion of whether support for spelling, sorting and searching depends, as the Unicode composition model proposes, on a protective separation of character processing from glyph processing, and thus whether in teaching writing today we are concerned with characters at all.

Thanks for that info.

Project Europa is a rendering specification for the intelligent composition model.

Interesting! I can't seem to find any references to it on the web yet.

Don't see where the OP asks anything about pdf.

It appears that you are posting for Cambridge University. So, to repeat. I began by stating that if the original poster intended to distribute a page description model that persists on disk, PDF 1.3 from Mac OS X 10.4 and Mac OS X 10.5, then the original poster was advised that the accepted canons of academic composition would produce unsearchable softcopy.

I also answered that if the original poster intended to distribute printed paper rendered from a page description model that perishes in printer memory as the page emerges, Adobe PostScript, then there is no issue as the audience merely interacts with the glyph appearance and not with the character content.

I am not sure of conditions at Cambridge University, but I would hope that there are not many there who are careless about choosing the tools and technologies for authoring, archiving and accessing information in abstract archival space, whether that be through Apple Spotlight and Microsoft Live Search on a standalone computer, or intranet or internet search services.

With best wishes,
Henrik Holmegaard

Thanks for your comments. After correcting some other errors in my text, I have this as a proposed warning for Pages users:

Your copy is correct. The Adobe Type 1 composition model, and by implication the configurable name space model in PostScript, was disparaged in October 2005 when Adobe Systems was compelled to clarify that Microsoft Corporation would not support Adobe Type 1 under the Presentation Foundation of Microsoft Windows Longhorn, now Vista. Would it were that Apple had been in a position to do this in 1994-1995 under QuickDraw GX.

/hh

Ritch, I did my masters in Pages but my advisor was a Mac user who also preferred Pages. Had that not been the case I'd probably have used Pages as a text processor and then completed it in Word. At present I'm leaning toward that methodology for my dissertation but not sure whether I'll write in TextMate, Pages, or DevonThink (which is what I've started using for my research). I have no desire to write in Word but given the footnoting, TOC, and other formatting requirements (hoops I have to jump through) I feel I have little choice but finalize it in Word.

If the intelligence is in the application or the operating system, then the strong struggle with the strong for the advantage in appearance shaping.

If the intelligence is in the intelligent drawing transform, then there may be professional, semi-professional and simply unprofessional transforms, but the field is free.

If the field is free, the market decides the difference between the worthwhile and the worthless.

Good arguments for 10 years ago. I love AAT too, but since that fine White Paper in 1998, production and use of such fonts has been totally eclipsed by OpenType, support for which Apple is now providing in OS X as well. So I think realistic efforts to fix whatever problems exist will have to assume that as the basic framework.

Thanks for that reference link, I had lost it.

I would prefer not to post on what Linux users and Windows users would do ... Possibly they would work their typesetting by manually selecting default glyphs and manually substituting non-default glyphs using Insert Symbol, or possibly they would find ways of automating their typesetting

I suspect that in practice most Word users do neither of these and simply use the various "false" typographic features available in the normal menus, so that the pdf's they produce do turn out to be searchable. Which could be one reason why the problems you describe don't attract more attention.

I suspect that in practice most Word users do neither of these

It is not possible to determine unless and until a study is done. Meanwhile, it is possible to determine that the Adobe Type Library has declined to discontinue PUA drawing. Marketing manager Thomas Phinney declined at the Linotype Typotechnica in April 2007, "Things we're NOT doing: Not removing PUA." It is also possible to determine that the Adobe OpenType guide written by Thomas Phinney and David Lemon does not document the products in the Adobe Type Library. Similarly, it is possible to determine that the OpenType Guide published in HTML by the Linotype Type Library, until August 2006 owned and operated by Heidelberger Druckmaschinen, does not document the products in the Linotype Type Library. The latter can claim that the everyday enduser is able to use the Encoding preview provided in the HTML interface to the Linotype Type Library in order to determine that PUA drawing is provided, and the degree to which PUA drawing is provided for the particular product. But since the Linotype Type Library also does not document in its OpenType Guide what happens when the everyday enduser draws with PUA, how would the everyday enduser know?

The Adobe Type Library used direct mail marketing of the Adobe Font & Function magasine to inculcate into the Macintosh market the concept that the proper procedure in professional composition was to search correct character strings and replace them with incorrect character strings in order to set ligatures, small capitals, superscripts, subscripts and more. The concept inculcated in order to market Adobe Type 1 products is implicitly left in place, providing an analogy for the professional procedure for professional composition today. In teaching writing today, one would teach that a composition model is unsuitable that can draw a glyph if and only if the glyph is directly depicted onto a character, see ISO-IEC Technical Report 15285.

If the Adobe Type Library published that its procedure for Adobe Type 1 products was incorrect, this could call into question support for searching, spelling and sorting with the hundreds and hundres of millions of type products in the Adobe Type 1 font program dictionary format that have been sold. At the same time, it is not possible to deprecate Adobe Type 1 because layouts in PageMaker, QuarkXPress and InDesign that have corrupted the source character string in order to support drawing with incorrect public characters cannot be changed to another type product as the change would cause the composition to turn into random glyphs, e.g. Adobe OYces for Adobe Offices. This implication of the Adobe Type 1 market was specifically stated in developer discussions 1992-1995.

Private Use Area drawing for products in the Adobe Type Library today allows drawing even if the application is not able to draw without altering the source character string. This does not hurt Adobe PDF which for the Latin script falls back on synthesising the source character string from glyph identifiers, but this does hurt Microsoft XPS which revolve around persisting the source character string. Thus if there is no documentation and no discussion, the student and scholar may perceive that Adobe's solution is more successful than competing softcopy document solutions. Don't misread this, it neither advocates the one or the other softcopy document solution but simply says that technically PUA drawing is not necessarily a threat to Adobe PDF the way it is necessarily a threat to Microsoft XPS.

/hh

Reference:
http://blogs.adobe.com/typblography/typotechnica2007/Facelift.pdf
http://www.freepatentsonline.com/y2007/0136660.html