10[1].1.1.76.1518

Experience with OpenType Font Production Sivan Toledo∗and Zvika Rosenberg† 22nd January 2003 1 Introduction This article describes the production of a large number of Hebrew OpenType fonts. More specif- ically, we describe the production of OpenType fonts with TrueType glyph descriptions and with advanced typographic layout features. The project was conducted by MasterFont Studio Rosenberg, a large Hebrew digital font foundry in Tel-Aviv, with assistance from Sivan Toledo from the School of Computer Science in Tel-Aviv University. Hebrew is a right-to-left script that is used in one of two ways: with or without diacritics. Most Hebrew texts are printed almost without any dia- critics, but childern’s books, poetry, and bibles are printed with diacritics. Even texts that are printed without diacritics use them occasionally to disam- biguate pronounciation and meaning. The diacrit- ics in children’s books and poetry are vowels and consonant modifies, which are called nikud in He- brew (meaning “to add points”). Bibles also use a third kind, cantillation marks, which are not treated in this article (but see [4, 5] for a thorough treat- ment). Hebrew diacritics are quite hard to support in fonts, since there are 15 of them and 27 letters, with up to 3 diacritics per letter. Even though not all the combinations are grammatically possible, there are too many combinations to support them conventiently using precomposed glyphs. In ad- dition, diacritics must be positioned relative to the base letter visually, rather than mathematically. For example, the below-baseline diacritics must be set below the visual axis of the letter [14], not below its mathematical center. This means that ligatures, pair kerning, and simple mathematical centering alone are insufficient for correct placement of dia- critics. In addition to diacritics, Hebrew fonts can also benefit from pair kerning, although until re- cently, pair kerning was not particularly common ∗School of Computer Science, Tel-Aviv University, Tel-Aviv 69978, Israel. Email: stoledo@tau.ac.il. †MasterFont Studio Rosenberg, 159 Yigal Alon Street, Tel- Aviv 67443, Israel. Web address: www.masterfont.co.il. in Hebrew fonts. For further information about the Hebrew script, see [1, 13, 14]. The initial objective of the project was to convert older fonts into the new OpenType format. This was motivated by the concurrent development of Adobe InDesign 2.0 ME (middle eastern), the first high-end page-layout program to support Hebrew OpenType features. Microsoft’s Office XP also sup- ports these features, so InDesign was not the only motivator. But it quickly became evident that the same tools that would be necessary to convert the old fonts to the new format could also be used to streamline the entire font production process at MasterFont. Therefore, the project had two objec- tives: to convert the old fonts to the new format, and to streamline the production process. The rest of the paper is organized as follows. The next section provides background on font formats. Section 3 describes how diacritic positioning in- formation is represented in MasterFont’s old fonts. Section 4 describes the objectives of the project, and Section 5 describes the software tools that we used to achieve these objectives. The overall pro- duction process is described in Section 6, and our conclusions from this development and production experience is described in Section 7. 2 Fonts Formats This section describes the OpenType font format and other relevant font formats. Unfortunately, the term OpenType is somewhat vague, so this section also establishes a more precise nomeclature. Type 1, TrueType, and Early Extensions The first widely-accepted device-indepenent font format was the PostScript Type 1 format [9]. Adobe started selling retail Type 1 fonts in 1986, and pub- lished the specification in the late 1980’s, after Bit- stream figured out how to produce Type 1 fonts (prior to that the specification was proprietary and 1 only Adobe could produce Type 1 fonts). Type 1 fonts consist of two or three main parts: scal- able hinted glyph descriptions, metric information, which include glyph dimensions, advance widths, and pair kerning, and possibly bitmaps for screen previewing. Today most systems can rasterize the scalable glyph descriptions so the bitmaps are usu- ally no longer necessary; Adobe distributes freely the Adobe Type Manager which is a Type 1 raster- izer for older Windows and Macintosh computers without a built-in rasterizer. Type 1 fonts are rela- tively easy to design and produce, and today there are tens of thousands of commercial Type 1 fonts. The next widely-accepted device-indepenent font format was the TrueType format [6, 2], which was invented by Apple in the late 1980’s, as part of a collaboration between Apple and Microsoft, and became the native scalable font format for both Ap- ple Macintosh computers and Windows computers. In a TrueType font, all the data associated with the font, such as glyph descriptions, metric informa- tion, administrative informaion (font name, for ex- ample), and possibly bitmaps, are placed in a single file, which is organized as a collection of tables. For example, one table contains glyph description, another character-to-glyph mappings, yet another table contains kerning pairs, and so on. The kinds of data in a TrueType font are almost identical to the kinds of data in a Type 1 font, but they are organized differently and sometimes represented differently. Most importantly, glyph descriptions and hints for low-resolution rasteriza- tion are represented differently. In a Type 1 font, glyphs are represented using cubic-spline outlines with declarative hints. In a TrueType font, glyphs are represented using quadratic-spline outlines and a low-level programmable hinting language. Type 1 outlines are easier to design and hint, and most designers use cubic splines when designing fonts. Well-hinted TrueType fonts are hard to produce, but they can achieve pixel-perfect rasterization at low resolutions, something that Type 1 fonts can- not achieve. Virtually all font editors can convert one type to the other. the conversion of TrueType outlines to Type 1 outlines is lossless, but the other direction only produces an approximation. How- ever, the approximation of a Type 1 outline by a TrueType outline can be made arbitrarily accurate and font editors produce approximations that are visually indistinguishable from the original. Hint- ing information is usually also converted, but not as well. Both Adobe and Apple introduced enhance- ments to these font formats during the 1990s, but none became widely used. Adobe introduced the Multiple Master font technology [10, 11], which al- lows users to interpolate between fonts in a family. This enhancement transforms the font family from a discrete set to a continuous spectrum. For ex- ample, there aren’t just medium and bold fonts, for example, but a continuous weight axis. Apple intro- duced GX fonts [6] (now called AAT fonts), which are TrueType fonts with additional tables that al- low continuous interpolation, like Multiple Master fonts, and which enable high-end typographic fea- tures. In particualr, GX fonts can support multiple glyphs for a single character, such as swash, small caps, or old-style figures, they can reorder glyphs, and so on. Neither format became widely accepted by font vendors, probably because the fonts were too hard to produce. No major font foundry be- sides Adobe produced Multiple Master fonts, and only a few GX fonts were produced by Bitstream and Linotype. OpenType OpenType is a new font format [3] that Microsoft and Adobe specified collaboratively. The format was first used, under the name TrueType Open, in the Arabic version of Windows 95, which was intro- duced in 1996. Adobe later joined the specification effort and the name was changed to OpenType. OpenType adds three capabilities to the True- Type format: advanced layout features, support for Type 1 glyph descriptions, and digital signatures. The advanced layout features are supported by five new tables: , that define glyph properties and glyph sets, , which defines glyph substitution rules, , which speficies positioning informa- tion, , which provides justification information, and , which provides baseline-adjustment in- formation. These features were designed to support layout of text in complex scripts such as Arabic, He- brew, Indic, and east Asian scripts (Hebrew is by no means the most complex), and to support high-end typographic features such swash letters and deco- rative ligatures, old-syle and proportional figures, and small capitals. The support for Type 1 glyph descriptions is de- signed to allow lossless conversion of Type 1 fonts to a TrueType-like table-based file format. Such fonts use two new tables,  and , to store a losslessly-compressed Type 1 font. Technically, the format of the  table corresponds to that of PostScript Type 2 fonts [12], also known as the compact font format, but the essense of the for- mat is a lossless representation of Type 1 glyph 2 descriptions (outlines and hints). The need to sup- port two kinds of glyph descriptions and hints con- stained the design of the layout-related tables, in the sense that positioning information in the  ta- ble could not be hinted using the normal TrueType or hinting language; a different hinting mechanism is used, which supports both Type 1 and TrueType glyphs. The support for digitally signing fonts, which uses the  table, is meant to allow users and font- using software to verify the authenticity of fonts. For example, future operating systems might re- quire that certain system fonts are digitally signed by the operating-system’s vendor, thereby prevent- ing modified versions of these fonts from being used instead. For independent font vendors, there is currently little benefit in digitally signing fonts, since users currently do not have means to verify these signatures, and since fonts rarely pose secu- rity or stability problems to systems. OpenType fonts with TrueType outlines can be used by any TrueType-capable software, since ex- cept for extra tables (which are allowed by the True- Type spefication), the fonts are also valid TrueType fonts. Software such as rasterizers, layout engines, and printer drivers can simply ignore the extra ta- bles. OpenType fonts with Type 1 outlines require special support in rasterizers and printer drivers, support beyond that required to support Type 1 and TrueType fonts. In principle any TrueType-capable layout engine can use any OpenType font, since the metric and layout information are compatible (an exception is pair-kerning information, which can appear in either the ‘kern’ table or the os table or both in a font with TrueType glyphs, but only in the  table in a font with Type 1 glyphs). To emphasize the fact that OpenType fonts with True- Type glyphs can be used by any TrueType-capable software, whereas special support is required for fonts with Type 1 fonts, files containing the first kind use the ttf or ttc extension, whereas files containing the second kind use the otf extension. So what is an OpenType font? According to the Microsoft/Adobe specification, any font that con- forms to the specification, including both TrueType and Type 1 glyphs, with or without advanced lay- out tables and digital signatures. This is somewhat confusing, especially since the file icons in Win- dows do not correspond to the glyph descriptions (Windows 2000 and XP use the ‘O’ icon for otf files and for ttf or ttc files if the fonts have a  table, and the ‘TT’ icon for all other ttf files). We will use the acronyms  for OpenType fonts with Type 1 glyphs, - for fonts with TrueType glyphs and with advanced layout tables, and  for fonts with TrueType glyphs but no advanced layout tables. Microsoft’s main interests in the OpenType for- mat lies in supporting complex scripts in Windows, with initial emphasis on Arabic and later on Indic scripts, and in tighter operating-system/font inte- gration using digital signatures. Adobe’s main in- terests in the format lies in exploiting TrueType’s advantages, mainly the support for non-latin char- acter encodings and the convenient single-file for- mat, and in providing its fonts and applications easier access to so-called expert glyphs, such as old-style figures and small capitals [8]. Windows and Macintosh Font Files Differences in the way font files are stored on dif- ferent platforms cause a few additional difficulties during font production. On Windows and Unix/Linux systems, a True- Type or OpenType font is stored in a single file. A Type 1 font is represented by up to four files: a pfb or pfa file that stores the glyph descriptions and a character-to-glyph encoding, an afm and/or a pfm files that store metric information, as well as the encoding, but no glyphs, and an inf file that stores administrative infomation about the font. On Win- dows system, only the pfb and pfm files are used. On Macintosh systems, there are multiple ways to store fonts. The Macintosh operating system supports two byte streams per file. One is called the data fork and the other the resource fork, which was meant to store application resources such as localized strings, icons, and so on. The format of the resource fork allows multiple resources to be stored in a single file. Before MacOS X (that is, before version 10), fonts were always stored in the resource fork of files whose data fork was empty. Fonts and font families are represented by several kinds of resources, such as the  resource that describes an entire family (regular, bold, italic, etc.), the  for storing TrueType fonts, the  and  for storing bitmap fonts, and so on. The fonts of a family are usually stored in a single file called a suitecase, whose resource fork contains multiple resources. The Macintosh also associates a file type (separate from the extension) and a creator code with each file. Font files need to have a specific type to be recognized as such by the Macintosh’s operating system. Storing fonts in multiple resources within a single file, and storing the font data in the resource fork rather than the data fork creates problems when 3 fonts are moved between platforms. To address this issue, Apple introduced new ways to store fonts in MacOS X. First, suitecases can now be placed in the data fork, using a format called adfontfile (after its extension). This format is essentially a traditional Mac font file, but in the data rather than the resouce fork. Second, MacOS X also allows Windows-style packaging of TrueType and  files in a single file in which the font data is stored in the data fork with no header or trailer. 3 Diacritic-Positioning Infor- mation in Older Fonts Masterfont’s existing fonts used three mechanisms for positioning diacritics. These mechanisms were developed over a long period of time—each addi- tional mechanism was designed to correct deficien- cies in previous ones. The first mechanism uses a number of precom- posed glyphs. These are used for dagesh, a dot that appears inside letters, and which must be carefully positioned to prevent overstriking the letter itself. Such combinations are also used for shin/sin dots, for diacritics attached to a final letter (only two combinations are used in modern Hebrew), and a few other cases. There are Unicode code points for these glyphs, as well as code points in the MacHe- brew 8-bit encoding, and they are used by both Macintosh and Windows systems. On the Macintosh, precomposed glyphs are also used to place diacritics on particularly difficult let- ter: resh, dalet, and qof. The first two are wide letters that have a single vertical stem at the right, under which below-baseline diacritics should be placed. Incorrect placement under these letters looks awful. The qof is the only non-final letter with a descender, so special attention is required to prevent the below-baseline diacritics from colliding with the descender. These glyphs do not have Uni- code code points, but they are used in all the Mac’s Hebrew fonts, including the fonts bundled by Ap- ple with the operating system. These glyphs appear to have been added by Apple quite early on. They do fix the most serious placement cases, but they do not fix all the placement problems. Further- more, this glyph set appears to have been designed by somebody without much expertise in Hebrew, since some of the precomposed glyphs can never appear in a text (e.g., dalet with a hataf-patah). Since the precomposed glyphs are insufficient for correctly positioning all the combinations, Master- Font developed a few years ago a positioning aid for the Macintosh. This software, called Mapik-Ve- Day, is a Macintosh system extension that selects a diacritic glyph according to the base letter. Fonts designed to work with this software have three sets of diacritics: one for narrow letters, one for wide letters, and one for medium-width letter. The dia- critics in all sets usually have the same shapes, and all have zero advanced widths, but each set has dif- ferent sidebearings. The three qamats glyphs, for example, all have the same shape and zero width, and all print to the right of the insertion point (so they appear below the preceding letter in a right- to-left text run), but the amount of right shifting is different. The software essentially divides the let- ters into three categories, each of which has its own set of diacritics. These two mechanisms, the precomposed glyphs and the three sets of diacritics of different widths, essentially mimic hot-led diacritics. These two mechanisms allow fairly good control over positioning, provided the font is designed for this system in mind. In particular, special attention must be given to the left sidebearings of base letters, to ensure that the diacritics from the appropriate category are well placed under them. To allow more precise control over diacritic po- sitioning, MasterFont began to use explicit posi- tioning information in the fonts. These are stored in the kerning table of the font, even though they are not real kerning pairs, since the insersion point should not move after the positioning adjustment. For example, a kerning pair alef-qamats with value 35 means that the qamats glyph must be shifted relative to the alef glyph by 35 font units. The in- sertion point remains exactly after (to the left of) the alef, as if no adjustment took place. This in- formation is not used by the Macintosh operating system, but a some Adobe applications can use it. MasterFont currently has 86 fonts with this level of diacritic positioing, which is essentially the set of fonts designed for book work. In addition, there are many more display fonts with pair kerning in- formation but without elaborate diacritic support. 4 Objectives Now that the setting has been described, we can enumerate the objectives of this project in more detail. First and foremost, we wanted to convert the di- acritic positioning information in the existing fonts to OpenType advanced-layout tables. We wanted 4 to do this without specifying any additional po- sitioning information. This restriction was moti- vated by several reasons: • Efficiency; there was no point in doing ex- tra design work on fonts that already produce perfecly-positioned texts. For example, there is no point in specifying positioning informa- tion for a combination represented by a care- fully designed-precomposed glyph. • Correctness; by not specifying new position- ing information, the chances of introducing new positioning bugs are minizized. • Continued support for older applications; we wanted to be able to produce additional new fonts that would be able to work not only in OpenType applications, but also in older ap- plications. By building new fonts according to the old specification and automati