.notdef

Ray Larabie's picture

Have .notdef boxes gone out of style? I used to leave them blank because I didn't realize how critical they were for typographers in preventing expensive disasters. For example: going to press without spotting missing glyphs. Recently MyFonts (the new site) added a choice of multilingual pangrams. So give it a try. Switch your sample text to Japanese and browse. If my .notdefs are missing it's because I forgot to add them. But some font designers seem to always leave them out and most do it about half the time.

My point: does anyone deliberately leave .notdefs blank? If so, why?

Nick Shinn's picture

I leave them blank because I figure if you've included all the characters in the code page(s) supported, there's no need for them.

Besides, don't layout applications insert missing characters from large system fonts, e.g. Arial?

Ray Larabie's picture

So some apps swap it out with another font. OpenOffice writer does. Photoshop and Illustrator both show the notdef if a glyph isn't present in the font. I'm not sure about other Adobe apps.

So if I copy and paste

abcdǻabd

into Illustrator or Photoshop and the font doesn't contain aringacute, it's replaced by a notdef glyph.

The .notdef box is a safety mechanism. Is there a case for deliberately leaving them blank? Are they just uncool now? Clue me in, please.

John Hudson's picture

Nick, it's not a good idea to leave the .notdef glyph blank. The point of the glyph is to signal the presence of characters not supported by the font, and that includes characters beyond the codepages that the font aims to support. If your .notdef glyph is blank or, worse, blank and zero-width, those characters will simply disappear from the text in many applications. Consider what happens if someone is using one of your fine fonts, but is sent a text for typesetting that includes e.g. one Chinese character. There should be some visible indication that this character is present and that it requires a different font to be selected.

My preferred .notdef design is a box with a question mark in it. Recently, seeking to give this maximum visual impact on the page, I've taken to reversing the question mark out of a black box. I usually align the height of the box to the cap-height, because this producing a proportion that doesn't have an atypical impact on the line-layout, i.e. its width is representative of a typical missing character.

Si_Daniels's picture

I would put something in there, and a box is as good as a squiggle or a question mark. I’m surprised we don’t see more creative shapes – logos, alien heads etc.,

I do recall a customer complaint. One chap had been using a random Arial codepoint that resulted in a missing glyph. He used it as a check-box on all his forms. When we updated the font his check boxes were all replaced with the obscure character that belonged in that slot.

Also there are “blank” code-points in Unicode – all those different spaces for example – if you don’t populate them and have a blank missing glyph, expected spacing could be messed up.

Theunis de Jong's picture

I would put something in there, and a box is as good as a squiggle or a question mark ..

I like to make pretty sure I don't miss a single missing glyph:

Ray Larabie's picture

I like that, it really stands out.

Florian Hardwig's picture

Yes, please make sure it really stands out. Framing or reversing sounds like a good solution.
Related: Filling in empty slots with placeholder junk

cerulean's picture

I always kind of liked Palatino Linotype Italic's little bird (of Character Detail fame on the old MyFonts site).

Nick Shinn's picture

Thanks John, I don't think it's a huge error to omit the .notdef glyph, but I will include it from now on.
Should this be a Unicode character?

Theunis de Jong's picture

Nick,

Per definition it's not a Unicode character :-)
Giving it the name ".notdef" and putting it as first glyph in your font (index #0) should be enough.

Nick Shinn's picture

Per definition it’s not a Unicode character :-)

Then what's it doing in a font?

John Hudson's picture

Nick, the .notdef glyph dates back to Apple's original TT spec: it is to be the first glyph in a font, it is to be named .notdef, and it is to be unencoded. It provides the ultimate fallback -- after font switching or other mechanisms that an application may choose to employ -- to display something when text includes a character that is not supported in the font. The .notdef glyph is not a Unicode characters because it may represent any Unicode character that is not mapped in the font cmap table.

Si wrote: I’m surprised we don’t see more creative shapes – logos, alien heads etc.

I'm glad we don't. I recall some users being confused by the spiral in the Palatino Linotype fonts. It is better that this glyph approximate the evolved conventions: an empty rectangle, a rectangle with diagonal lines corner-to-corner (the form Adobe uses), a rectangle with a question mark in it, a reversed rectangle with a question mark in it.

In traditional Japanese typography, a placeholder sort was used for a similar purpose: to mark a place in the text that would require a character from a different font. This enabled the typesetter to keep working and to leave a visible marker in the text to indicate the place where the other-font character should be inserted. If I recall correctly, the sort had a horizontal rectangle in the upper half, so would print as a black rectangle above a white rectangular space. This form might be appropriate for the .notdef glyph in an East Asian font, although Ken Lunde should comment on this and correct me if I am wrong. It led me to experiment with this form:

Nick Shinn's picture

It provides the ultimate fallback

That doesn't seem to be the case, as there is a variety of glyphs used to represent absent characters, as well as emptiness.
Perhaps it would be better to handle this at a deeper level than fonts?

Alternatively, Unicode could specify a preferred glyph shape for ".notdef", even if it has no code number.
That would at least apply some standard to the situation, and provide direction to font developers, internationally.

Si_Daniels's picture

>That doesn’t seem to be the case,

True, due to app/system fall-back, font-linking etc., but this is the only fallback marker that's within the type designer's control.

cuttlefish's picture

I think it specifies in the Microsoft TT spec (paraphrasing from memory, don't sue me if I'm off by a bit) that the .notdef glyph should be a rectangle, acceptable variants including an X or ? enclosed in the rectangle, but variants beyond that being highly discouraged so as to avoid confusion.

I suppose to make it fit with a particular font the rectangle could be approximately the average glyph height and width of the particular font, and some consideration given to stroke weight too.

lunde's picture

U+3013 (〓) is commonly used as a placeholder glyph in Japanese typography when the desired ideograph is not available.

Dr. Ken Lunde
Senior Computer Scientist, CJKV Type Development
Adobe Systems Incorporated
lunde@adobe.com

lunde's picture

Interestingly, of the 64K glyphs that can be in a CIDFont resource, the only glyph that is required is that for CID+0, which serves as the .notdef. Its function is for rendering as part of a fallback mechanism, and is not encoded. In fact, I would argue that it should not be encoded, which is stronger.

Dr. Ken Lunde
Senior Computer Scientist, CJKV Type Development
Adobe Systems Incorporated
lunde@adobe.com

John Hudson's picture

Nick, as I said, it is the ultimate fallback, if an application doesn't attempt any other kind of fallback or such other methods, e.g. font-switching, fail, the .notdef glyph is supposed to be displayed.

For instance, let's say a piece of text includes a character not supported by the current font cmap table. An application (or system/library/engine) may automatically use the .notdef glyph, or it may go through a series of degrading fallbacks. It may try to find a default font on the system for displaying the character; if it can't find such a font, it may try to find a default fallback font such as Apple use in order to at least identify the Unicode range and script to which the missing character belongs; if the system does not have such a font, the ultimate fallback should still be the .notdef glyph.

Alternatively, Unicode could specify a preferred glyph shape for “.notdef”, even if it has no code number.

This has nothing to do with Unicode. The .notdef glyph is not a Unicode character; its function is defined by the Apple TT spec.
___

Thanks, Ken. I remember now the three-band placeholder. I did try a .notdef glyph based on this with a superimposed question mark, but it was too messy at small sizes.

Nick Shinn's picture

John, wouldn't it make more sense for applications to substitute a standardized ".notdef" glyph from a system font, rather than rely on the vagaries of commercial fonts?

How are we going to get all the foundries to play ball, especially when there is no Unicode guide to suggest a standard glyph shape?
Having the standard in the Microsoft TT spec is not comprehensive enough.

This has nothing to do with Unicode.

By the same token, it's not really a responsibility of font formats
As Unicode is all about character encoding, it may be better positioned to deal with the lack of an encoded character than font formats.

lunde's picture

OpenType (and TrueType) fonts are designed to be fairly self-contained. Depending on external resources, such as the rendering of the ".notdef" glyph goes against this, though some clients may choose to implement their own handling of ".notdef" situations. This is also the reason why the mapping from character codes to glyphs are included in the fonts themselves, in the form of the 'cmap' table.

Dr. Ken Lunde
Senior Computer Scientist, CJKV Type Development
Adobe Systems Incorporated
lunde@adobe.com

Christopher Slye's picture

How are we going to get all the foundries to play ball, especially when there is no Unicode guide to suggest a standard glyph shape?
Having the standard in the Microsoft TT spec is not comprehensive enough.

Nick, it's also in the Adobe CFF specification, so pretty much any font that doesn't have it is buggy.

But I think your point is that it's not required to be a particular mark, or any mark at all. (Maybe there is some suggestion somewhere in Adobe's documentation, I'm not sure.) It would maybe be nice if FontLab and other significant tools had a reasonable default notdef shape built-in.

cerulean's picture

Um... Isn't .notdef Unicode #FFFD?
If not, shouldn't it be?

Ray Larabie's picture

I think all that's needed is some awareness that a blank .notdef could cause problems. A default .notdef box in Fontlab would be a nice touch. Didn't Fontographer generate a default .notdef box?

charles ellertson's picture

Another windmill . . . Where's my donkey.

Nick, We key about 2 manuscripts a year. Well, no, we no longer key, we send out about 2 manuscripts a year to be keyed . . . The other 398 arrive "editorially correct on disk." We set them with one of the few apps that supports OT - Adobe's InDesign. Not one of our customers would accept, say, an Ariel character if the MS called for a glyph not in the font. And not one of the graphic designers will check to see if the typeface they have selected in fact has all the characters called in the MS, (which may, BTW, have substituted characters from another font).

Yes, we do have a little script we run that shows all the Unicode characters called in the MS. But you can miss something; each proofing pass catches about 98 percent of the errors.

So, (1) The appearance of the missing glyphs is important. The starker, the better. And (2) it is important the end user has the right to add the missing glyph -- like the ygrave I encountered today.

I understand ID-CS4 won't let you export to PDF unless all the needed characters are present. But sadly, I flat guarantee you ID-CS2 will.

Nick Shinn's picture

Don't you get a big pink em-square in InD if the character is missing?

John Hudson's picture

Kevin: Isn’t .notdef Unicode #FFFD? If not, shouldn’t it be?

U+FFFD is used to replace an incoming character whose value is unknown or unrepresentable in Unicode, e.g. where you have text in a different encoding for which software can't figure a way to convert to Unicode. So U+FFFD indicates a character not recognised as Unicode, whereas the .notdef glyph indicates a character not supported by the font. U+FFFD appears in text up-stream of display; .notdef appears in text during glyph display.

dezcom's picture

It is one of the fun glyphs so I always include it--sometimes in a humorous vein.

ChrisL

charles ellertson's picture

Don’t you get a big pink em-square in InD if the character is missing?

Well, it is pink. I've seen fonts where the size -- width -- is quite small. I'd assumed the "pink" is the width assigned to the .notdef character; I suppose I should check. I've missed a few.

Pink doesn't work when your primary checking/proofing method is to print out the job and have it read (or skimmed) by someone other then the comp who set it. An old idea, but still a good one. BTW, with "editorially correct" files supplied, we don't read against copy.

paragraph's picture

I thought that pink highlight in ID shows font substitution, not missing character?

Florian Hardwig's picture

@Jan (paragraph): Both. In InDesign, missing fonts and missing characters are both marked with a pink highlight that can be toggled in the preferences (Compositon > Substituted Fonts). If the font has a .notdef glyph, missing characters are substituted with that glyph and get highlighted. Otherwise it’s just a highlighted blank space. I reckon the width of that blank space is one en space.
F

Theunis de Jong's picture

Otherwise it’s just a highlighted blank space..

For entire missing fonts, it seems ID checks for a reasonable match -- the same Unicode in its default font (Myriad?), or in a font matching the name but not the style. A missing glyph in an otherwise good font usually displays the notdef glyph.

Pre-CS4 InDesign, it used to be a matter of carefully examining the document (in InDesign), but hip-hip-hooray! CS4 finally marks 'missing glyphs' as an error in its live preflight.

ralf h.'s picture

By the way:
Is there any other app (beside InDesign) that allows to check fonts against certain Unicode texts and output missing characters?
If people need to perform such tests, one can hardly suggest a CC subscription just for that. ;-)

Syndicate content Syndicate content