New MS white paper on the TrueType interpreter

John Hudson's picture

Today, on the OpenType developer discussion list, Microsoft made this announcement:

A new white paper has been posted to the Microsoft Typography website. The paper was written by Greg Hitchcock to help font designers understand the Microsoft implementation of the TrueType interpreter.

Here's Greg's official summary:
With the implementation of ClearType® in Windows XP and later versions, adjustments were needed for the interpretation of some TrueType® instructions in order to allow existing fonts to work with no updates. This white paper is targeted for font designers and describes the instruction changes and how fonts can continue to have their instructions behave as described in the current OpenType® specification.

Note that this white paper is not part of the OpenType specification, and will not be submitted for inclusion in the ISO Open Font Format. It is provided solely to explain how Windows handles hinted fonts under ClearType rendering.

The paper can be found here:
http://www.microsoft.com/typography/cleartype/truetypecleartype.aspx

gregh's picture

Here is a sample image that addresses one of David’s points and one of John’s requests.

The image is similar to the above image (although not shown from VTT.) Instead of using SHPIX, I used the MIRP to shift the left edges. (I also made a version that does the same with the MSIRP instruction.) I’m moving the edge relative to the left side bearing point. The images with MIRP, MSIRP and SHPIX are exactly the same in VTT.

I made an additional change in the hints by setting the advance width to be 3 1/6 pixels wide. This forces a shift when displaying in sub-pixel positioned ClearType mode. In the above image you can see the pattern repeats every six glyphs.

David> One correction that may help the reader here, is “the left edge of the contour was shifted to the right by four pixel units”, should read, I believe, “the left edge of the contour was shifted to the right by 4/64ths of a pixel.”

I agree that is better wording.

David> And then, there is the interesting question of why any baby yellow or baby blue is showing up at all as those pixels are entirely not within by the contour being rendered.

This has to do with filtering, and is described in detail in our papers on ClearType. The primary purpose of the ClearType filter is to remove the perception of color. This, in theory, is potentially a tunable parameter. But the font hinter cannot rely on what tuning parameter the user has chosen. Additional factors like the foreground and background colors, gamma correct, and of course the display system architecture are going to impact what is emitted by each display sub-pixel. So, big picture, when adjusting the edges, you need to proof the result closely, and verify the visual results (which I know you already do.)

twardoch's picture

Excellent thread, which I only read now. A small anecdote to "sum it up": I just visited Luc(as) de Groot in his studio yesterday. We both worked in FontLab Studio, side by side. We both had the OpenType panel open and I looked at how the feature definition source code was typeset.

Luc(as)' was set in good old Monotype-made TT-hinted Courier New, rendered in GDI ClearType running on Windows 7.

Mine was set in Luc(as)-made CT-hinted Consolas, rendered in Carbon ATSUI running on Mac OS X 10.5.

I thought it was funny.

Rob O. Font's picture

That is so Funny Adam! Mostly because you speak as if font rendering was what it was when this topic was initiated. Pooooooof! All changed! W3CU later old meaning! ;)

Cheers!

twardoch's picture

David,

I think my implicit point was: I have no clue :) When I worked on Windows 98, I always hated the 1-pixel stems of Verdana and I changed the gasp table to make it switch to 2 pixels at smaller ppems. When Windows 2000 came along, I hated the default antialiasing that used 21 shades of gray even more, because straight stems were black and round elements were almost white. When Windows XP came along, I hated ClearType because every morning I felt I had a hangover because my vision was blurry and I was seeing color fringes. I complained to Greg and Kevin, and they told me that my visual acuity or color sensitivity must be poor or something (one of them, I don't remember), so when I visited Redmond some time in 2005, they tested me (with funny little color stones that I had to order in gradients), and I scored the second-best result of all tests they've done. Which apparently meant that what was supposed to be poor was actually very good.

When Acrobat LCD rendering was introduced, I hated it.

When I switched to Mac OS X, I loved the text rendering, and still do. The no-hint-fuzzy approached works for me by far the best. I feel that reading on a Mac screen is like reading letterpress books printed in the 1960s. Windows is like offset on glossy paper.

And I still don't have a clue why. Completely regardless of INSTCTRL and MIRP. Do you know why?

Adam

Rob O. Font's picture

Adam> Do you know why?

I'll guess: 9% color sensitivity, 12% the experience of a black font specification converted to black and grays, 68% that you are not a child or distant viewer in need of fonts 30% larger than print, 10% being interested in a high quality web experience, and I'll leave the other 1% to Greg.;)

Adam>The no-hint-fuzzy approached works for me by far the best
Assuming you are talking about text sizes, it's "no hints" when anti or "all hints" when aliased in the Mac approach, according to the user's specification in appearances.

This may not be the bottom line, but the combined decisions of Apple and MS, have put world wide web text font development in a difficult position with few good clear options.

Your decision, Adam, to switch is perhaps the simplest. Now that you have, you can more easily appreciate how much better both "traditional zooming" (iPhone), and "web zooming" (Safari) environments have to offer without all the fuss.

Cheers!

blank's picture

This may not be the bottom line, but the combined decisions of Apple and MS, have put world wide web text font development in a difficult position with few good clear options.

Financial considerations aside, is just designing fonts for Windows users not a clearly good option? Won’t a font designed and hinted to look good on Windows still look good on macs and iphones just as Verdana, Georgia, and the Cleartype fonts do?

twardoch's picture

David,

I actually like large screen fonts. Back on Windows, I used Opera because at that time, it was the only browser that offered real zoom (not just enlarging the type while keeping the column width, which is ridiculous). Thank God other browser makers realized it, though I still don't understand why Safari and Firefox don't have a normal zoom slider or dropdown list in the status bar (which is a standard UI element in Word, PowerPoint, QuarkXPress, InDesign etc.). My typical reading size for the web on a 24" screen is this:
http://www.twardoch.com/tmp/typophile_webbodytype.png
(This is in fact exactly the way I'm reading Typophile. For some reason Typophile won't let me insert an image here.)

Even back on Windows, I went through a lot of trouble to fine-tune my screen appearance. When I had a 15" 133 dpi notebook as early as in 2001, I suffered from the inability to effectively set the Windows screen DPI to 133. (The "standard" setting is 96 and the "large fonts" mode is 120). In both 120 and 133, many dialog boxes of poorly written applications are reformatted, some UI controls such as "OK" and "Cancel" are invisible (pushed out of the boundaries of the dialog box). So effectively, that setting was unworkable but I came up with another trick: I took a condensed font (first Nina, then ITC Officina in the TrueType version hinted by ParaType) and reduced its UPM size so that at standard point sizes it would appear larger. And I set it as my standard UI font. That worked a bit better.

In fact, I've been using this trick for some time: I would take Nina (which is kind of a condensed Verdana) and set its UPM to a smaller number so that at the same ppem size, the *widths* of my modified Nina and the normal Verdana would roughly match. I would get the same number of characters per line with my customized Nina as I would with the regular Verdana, but the effective body size would be larger in the Y direction. So, while using a font that was "condensed in the X direction", de facto I was using a font that was "extended in the Y direction". Which proved to be very effective.

I remember that the day on which I realized the simple fact, that by reducing the UPM size you can enlarge the body of any font at a given ppem size, I was a happy person :)

Best,
Adam

twardoch's picture

James,

more fonts look "good" on Mac than on Windows, in my experience. The difference between very well hinted fonts and poorly hinted fonts on Windows is paramount: the poorly hinted fonts really look bad. On Mac, the very well hinted fonts may to some look not as good than on Windows (though not for me), but the poorly hinted fonts look on the Mac much better than on Windows. A TrueType font with no hinting at all will look fine on a Mac and catastrophically on Windows.

A.

Rob O. Font's picture

>Won’t a font designed and hinted to look good on Windows still look good on macs and iphones just as Verdana, Georgia, and the Cleartype fonts do?

What Adam said. And, on iphones I have not seen the Cleartype collection, but I have remade a couple of them, along with Lucida Grande and a raft of experimental FB fonts, (for my own experimental evaluation, and for no other purpose), with name tables to intercept verdana a georgia web font calls on the Mac, in Safari, and no, the CT collection does not look nearly as good as Vergeorgia.

Adam, you work a lot harder than Greg says windows users will, just to get their type right for reading. As far as I've been informed, they generally don't want to change even size. This is what a web font developer must plan on, vs. tinkering with the upm of a condensed font and chancing users and web developers to use it at 2x the user's normal size preference. But I like it if you're happy, as ever because what you're doing reminds me of an old double-play combination from the last century. ;)

Cheers!

dezcom's picture

I preferred Grote to Mazzeroski, to Stuart :-)

Richard Fink's picture

This may not be the bottom line, but the combined decisions of Apple and MS, have put world wide web text font development in a difficult position with few good clear options.

As my grandfather, Sherlock Fink, used to say, "Once you have eliminated the unacceptable, whatever is left, however objectionable, is what you have to work with."

Good clear options - what would be the fun in that?

twardoch's picture

The nice thing about CSS webfonts is that you (as a font developer, at least) can tweak them at ends where CSS can only work deal with difficulties. For example, there is the CSS font-size-adjust property which tries to deal with the different face sizes relative to the body size (i.e. at nominally same point/ppem sizes). But with webfonts, the font developer can adjust this easily in the font -- and that even in a very undestructive way, by just changing the UPM size. You can even serve two different fonts depending on the platform of the client (Mac OS X or Windows, for example). The Mac font can be completely unhinted (i.e. smaller in file size) and can have a UPM adjusted so that the same web page will render visually similarly in both Mac OS X and Windows.

I believe that the issue of body sizes will become quite interesting with webfonts. The old principles of type design for print were: make the caps height the same across various typefaces. But this may be a principle worth revisiting. In a way, in print, the vertical size was more of a set limit, since the page had a fixed height. But on the web, the horizontal direction is more of a fixed value, or at least oscilates around fixed constraints (because there is an average line length which is useful for reading), but the vertical space is much more flexible. The user can scroll endlessly. So much more freedom lies in the glyph height and in the linespacing. Choosing good defaults is crucial here.

For example, the one thing that I actually don't like about the ClearType project is that the body size of the typefaces has been set so small. Especially when switching from Verdana and Georgia (which were very opulent, design with "caption size" optical proportions) to Calibri or Constantia, everything suddenly looks much too small.

The "this is too small" perception is actually something that sometimes gets through from users to font developers. Up until Mac OS X 10.3, the AAT version of Zapfino that shipped with the system was set on a 1000 UPM with the normal caps and the lowercase being quite tiny -- to "make space" for the extreme ascenders and descenders of the swashy forms. But apparently, many users complained that Zapfino was "too small for its point size", so with Mac OS X 10.4, Apple made a radical cut and... changed the UPM of the font from 1000 to 400. I.e. effectively enlarged the face size of the font by 250% relatively to the body size!

To make it clear: there are two methods of enlarging the face size at a given point size: you can either enlarge the face size (i.e. scale up all the glyphs) or you can reduce the body size (i.e. set the UPM lower). This will have exactly the same effect, but actually the latter method is often more efficient because you don't have to deal with rounding errors, changing kerning values or lost hinting. This is kind of like when you're trying to have a party or concert that feels like a "full house". You either invite more guests, or you rent a smaller room.

So, apart from hinting issues: type designers, choose your body size (and the built-in linespacing values) wisely for screen purposes, because in some environments it's not so trivial for the end-user to "fix" it.

A.

John Hudson's picture

Adam: For example, the one thing that I actually don't like about the ClearType project is that the body size of the typefaces has been set so small. Especially when switching from Verdana and Georgia (which were very opulent, design with "caption size" optical proportions) to Calibri or Constantia, everything suddenly looks much too small.

Georgia and Verdana were made in ways that pretty much preclude extending their character sets to supporting more diacritic-heavy languages like Vietnamese, at least if one wanted to retain backwards compatible metrics. Even though the initial release of the ClearType fonts had similarly basic character support, we were aware that these fonts were almost certainly going to be extended in future, and so we needed to ensure that there was reasonable vertical space to work with in the body height and default linespacing. Even so, we ran into difficulties during the Win7 Cambria extension work, and I'd argue that seriously multilingual fonts probably should be even smaller on the body (at least if notions of body size are to remain a basis for type scaling and linespacing; only a couple of days ago I was suggesting to some folk at MS that what we really need are language-specific vertical metrics settings).

blank's picture

…only a couple of days ago I was suggesting to some folk at MS that what we really need are language-specific vertical metrics settings).

Ooh, that is a good idea.

Jongseong's picture

I also would like to see language-specific vertical metrics settings. Currently, I'm thinking that if I ever finish a hangul font, I might produce a separate Latin-only version of the font that includes just the Latin glyphs with modifications to make it suitable for setting Latin-only texts, including a reduction of the vertical space, since hangul generally takes up more vertical space than the Latin letters.

Rob O. Font's picture

>CSS font-size-adjust property which tries to deal with the different face sizes relative to the body size

...imagine how long the batteries would last in your remote if each broadcaster controlled their own volume. Then imagine if it was left up to each broadcaster to figure out the difference between their volume, and no standard... and then imagine if it were poorly enough documented to spread really poor documentation. Does that deal?

Adam, I have no idea what tinkering with the upm does for anything at all in user-space. School me with examples if you so desire. All I know is that optical sizing had a place in the print world and as Kevlar said in Mexico City, he's concluded from studies that if typographers used to do something, there is a pretty good chance there was a good reason for it.

>(and the built-in linespacing values)

all gone. what was it for, Adam?

> I'd argue that seriously multilingual fonts probably should be even smaller on the body

lol, go for it! I'll be on the other side of the fulcrum.

The issues here is what users are used to, and the w3c's impossible philosophy towards, and definitions of, the most important parameter of type to any user.

Cheers!

twardoch's picture

John,

A LINE table, somewhat similar in structure to the BASE table, would certainly be useful.

Adam

Rob O. Font's picture

not as good as a size table though.

Cheers!

John Hudson's picture

A size table provides information about relationships between different fonts, so I think careful thought should be given to whether script/language specific metrics might belong in a size table or in a separate table within the individual fonts. I'm inclined to think the latter, and that there isn't an obvious benefit to putting font-specific data in a table that describes relationships of multiple fonts.

Rob O. Font's picture

>I'm inclined to think the latter,

Of course you are! Having such a table for both inter and intra-font size issues is RIDICULOUS, what was I thinking.

Cheers!

twardoch's picture

David,

I think since he nature of the support of these two kinds of relationship in the software is quote different, I'd say this should go into two tables. I do agree that a SIZE table along the lines of what you talked about would be useful.

John Hudson's picture

I do agree that a SIZE table along the lines of what you talked about would be useful.

Yes, although it still means putting family-level data within each individual font, which strikes me as less than ideal. What happens if the data is contradictory, which is always a risk when redundancy is involved.

Data about what fonts to use in what situation -- whether it is size, rendering engine, etc. -- seems to me to belong at a wrapper level, independent of the individual fonts. The XML composite font format standard seems to me an ideal place in which to include such data, with different size variants of a single typeface forming a composite font and the individual component fonts being selected according to situation.

We have a good opportunity to push this in the standard, and I reckon there is more chance of having something like size implemented as part of a new layout mechanism than trying to encourage developers to extend their existing font support to new tables.

Theunis de Jong's picture

John, a family-targeted data file is a fantastic idea!

One can imagine a some quite useful information to be stored in there. For example, it would be a good opportunity to remove those two lousy bits "Italic" and "Bold" from the specs, and replaced with something more substantive (a weight number?).

Rob O. Font's picture

John>What happens if the data is contradictory, which is always a risk when redundancy is involved...

The first skewer-words are always the best-est: What a font-specific model of meta data will do is create contradictory and redundant data.;)

I've been to the XML mountain, and IT'S GreAt! But where we were, (when I started the size and epar table discussions), is a far distance to universal XML c.f.f. compliance.

So as great as another new format might be, I thought then, that we required a transition period that could either occur in public cooperatively, or in private competitively. I interpolated to the best solutions I could for a public, cooperative effort. This would have described, if not en-tabled, many of the subsequent problems users have encountered without such meta-data, in their current transitions from "expert-system" to "public-system" publishing.

What you're saying about the ClearType Collection above, is that you optically sized that series for internationalization.
Let me know when you've finished with the XMLCFF upgrade, in the mean time, literally, don't worry, I'll handle it.

John>Yes, although it still means putting family-level data within each individual font, which strikes me as less than ideal.

Arguing for the future is great, but dealing with users is nourishing. My apologies in advance if you don't know what I'm talking about... again. ;) But John, this is not much different from sitting before a 144 dpi monitor, for years, reading black on yellow and telling me there is no windows rendering problem.

Cheers!

John Hudson's picture

[Note: CFF = Compact Font Format (PS flavour OT); CFS = Composite Font Standard.]

Size information, i.e. information that tells software which 'cut' of a typeface to use in a particular nominal size range -- leaving aside, for now, the related or parallel situations of particular ppem size range, particular rendering engines, etc., and all the other situations that might be capturable --, is something that needs to be interpretable by software. Sure, one can put human-readable info in a font that gives advice on how and where to use that font, but until software automatically selects the correct font for particular circumstances from within a family, there is no real solution. I don't see putting information in a font and hoping that software somewhere will do something other than ignore it as a ‘transitional solution’. Your approach does indeed entail a valuable exercise in describing ‘the subsequent problems users have encountered without such meta-data’, but that process does not require or even presume a specific technical implementation: it's worth doing in any case.

The existence of the GPOS 'size' feature and a significant number of fonts using this feature (made by the same company whom one might expect to be first in line to support such a feature in their applications), for more than a decade, combined with the near-complete failure of support for that feature is strong evidence that simply sticking this information in a font and hoping for the best is not viable. So long as size selection is seen as an optional extra to existing font support, there is little impetus to software developers to do anything to support it.

And please note that I am not saying that in-font meta data will create contradictory data; I am saying that an architecture that makes contradictory data a possibility requires software makers to come up with protocols to handle such an eventuality, and this is a disincentive to supporting such an approach. You may not like it, but looking for data redundancies is one of the first things software makers do when they examine a technology, and one of the things they strive to avoid. [The OpenType format already suffers from multiple redundancies, in large part due to the bolting on of the CFF table. Redundancies were also one of the objections to EOT, which replicated individual font data at the wrapper level. CFS is being designed specifically to avoid redundancies.]

If, on the other hand, size selection is an integral part of a new composite font mechanism -- one that MS, Adobe and Apple have all demonstrated a strong desire to support first through their individual implementations of proprietorial composite font formats (this is already shipping software, not some future intangible) and second through their active engagement in developing a common, open standard --, then it becomes something that gets supported through fresh implementation of that new mechanism, rather than something requiring changes to existing font support.

I'm not ‘arguing for the future’. I bring this up now because now is when the standardisation process for CFS is taking place, and if size selection is something we're interested in seeing in the standard then now is when we need to have this discussion and engage with this process. And, David, in case I'm not clear enough: I would value your input and influence in that process, because I don't think I can alone make size selection a priority.

Rob O. Font's picture

John, I'll keep my eyes peeled for your thread.

But I'm sure you know this first: discrete digital outline fonts are what we say they are, and as such they can't help but be contradictory; If one of us says Roman, and another Medium, and a third says it's Book, and a fourth makes all four, that's what? If one of us says we made a 4-style type family for text and display, screen and print, is that not contradictory? And if a single WOFF lands on a user's desktop via @fontface, and it describes that it has nine "brothers" and 59 "cousins", is that redundant?

To wrap up my input on Greg's paper:

The WP's Conclusion: Significant effort has been taken to make ClearType perform optimally for Microsoft Windows.

I think this is the case as one can see, but what about Microsoft Windows Users? There are a lot of corporations and users that don't work well on Windows and I think more effort is required.

Continued support from font hinters is necessary in order that fonts will continue to be the most readable and legible on many types of displays and many types of rendering environments.

Is it forthcoming fast enough? Frothcoming perhaps... and speaking of which, isn't a quality hinting supply likely to be killed by @ff as practiced today, with simple-to-steal derivatives being served?

This paper has described the techniques used to make existing, unmodified fonts work correctly with ClearType,

I'd need a fairly broad definition of 'unmodified' and 'correctly'. Mine is to allow experts to apply their skills to broaden typographic availability with improved quality over the passage of time for Windows users.

...but more importantly it provides the information to developers of new fonts on how to assure the consistent behavior of the TrueType interpreter for fonts by using the INSTCTRL instruction.

ahem.

And... it would have less contradictory if the paper just said; all size-independent x hints are ignored, and all y-size-independent hints are strictly required.

I agree with John, it would have been less redundant for MS, and all of us developers, if they had just adopted the T1 format in 1989.

Cheers!

Syndicate content Syndicate content