One of my longer term goals for the open source Source Han Sans project has been to eventually register a Pan-CJK IVD (Ideographic Variation Database) collection that would allow the regional variants to display and be preserved in “plain text” environments, and I think that I may have achieved a breakthrough the other day.
CJK Unified Ideographs is a very deep and fascinating subject, and there are people who sometimes— and anecdotally—claim that it is a bottomless pit (or infinite tunnel, hence the quote in this article’s title). The latter may appear to be accurate, especially when one considers what is happening on that front, such as Extension F with its 7,473 new characters slated for Unicode Version 10.0 in June of 2017, and with work on Extension G commencing.
One of the fringe benefits of moving offices—especially when one has accumulated nearly 25 years of font-related material and it is thus not a pain-free exercise—is discovering historical documents, some of which turn out to be true gems. Our team is preparing to move from the Adobe East Tower to the West one, and part of the process is figuring which material to keep, and which to put into File 13. Anyway, I had been recently looking for a particular presentation that I prepared many years ago, and was fortunate enough to come across it while sifting through my accumulated materials.
It is with great sadness that I write that Unicode Version 9.0, whose beta was authorized yesterday, on the last day of UTC #146, will include no additional CJK Unified Ideographs. The next opportunity for additional CJK Unified Ideographs is therefore Unicode Version 10.0, which is slated for a June 2017 release, and is expected to include 21 Urgently Needed Characters (UNCs) that are appended to the URO (Unified Repertoire & Ordering), along with Extension F (see IRG N2156 for the latest version) that currently includes 7,473 characters.
Interestingly, and as long as Extension F’s block remains stable, there are only 3,088 code points remaining in Plane 2 (SIP), specifically U+2EBF0 through U+2F7FF, along with 1,502 code points at the end of Plane 2, immediately following CJK Compatibility Ideographs Supplement, specifically U+2FA20 through U+2FFFD.
The image above is an excerpt of a PDF that shows what Unicode Version 10.0 is likely to include in terms of ideographs. If you click on the image, you will get the actual PDF. Of course, the yellow stuff is tentative and subject to change.
Updated on 2016-06-26 to reflect the additional UNC appended to the URO at U+9FEA, along with a net decrease of 12 characters in Extension F, reducing it to 7,473 characters.
In late 2015, I collaborated with Daisuke MIURA to submit a proposal (L2/15-328) to the UTC (Unicode Technical Committee) to encode the characters for four tally mark systems. The proposal was discussed during UTC #146, and the result was that the five ideographic tally mark characters were accepted. Good news.
The Script Ad Hoc Committee originally recommended in their report for UTC #146 (see page 9 of L2/15-037) that IDEOGRAPHIC TALLY DIGIT TWO not be encoded, because they felt that it could be unified with U+1D36E (COUNTING ROD TENS DIGIT SIX), but concerns over typographic consistency led to it being accepted as a separate character.
In December of 2015, Unicode launched their Adopt a Character campaign, whose goal is to raise funding for the purpose of encoding a large number of remaining scripts, along with encoding additional characters for scripts that are already encoded. In other words, to help Unicode do its important work. The Unicode Consortium has 501(c)(3) tax status, meaning that donations are tax-deductible in the US, and if your company supports matching grants, you can leverage that to significantly increase the effective donation.
By default, the AFDKO makeotf tool includes Macintosh (platformID=1, encodingID=0, languageID=0) ‘name‘ table strings, and if specified in the “FontMenuNameDB” or “features” files, localized Macintosh ‘name’ table strings will also be included. The next release of AFDKO will include “-omitMacNames” as a new command-line option for makeotf whose purpose is to exclude Macintosh ‘name’ table strings, other than any that are explicitly specified in the “features” file.
IUC39 (The 39th Internationalization & Unicode Conference) took place in Santa Clara earlier this week, and Adobe was once again proud to be a Gold Sponsor. It was another outstanding and successful conference, and as usual, one of the greatest benefits of the conference—besides the many excellent presentations—was the opportunity for face-to-face exchanges with Unicode leaders, experts, and enthusiasts.
(The introductory graphic illustrates how the character 剣 (U+5263) is displayed using the fonts that are introduced in this article. The code point for this character maps to a glyph that displays as “63” in the FDArray Test 257 font, which is the hexadecimal equivalent of the decimal index of the FDArray element to which its glyph is assigned, which is 99. Likewise, the code point for this character maps to a glyph that displays as “52” in the FDArray Test 65535 font, which is the hexadecimal equivalent of the decimal index of the FDArray element to which its glyph is assigned, which is 82.)
I have built several CID-keyed OpenType/CFF fonts that are specifically designed to test various limits, by exercising various implementation limits, such as the number of glyphs (65,535 is the architectural limit), the number of FDArray elements (256 is the architectural limit), and the number of mappings in the ‘cmap‘ table (when the surrogates and non-characters are factored out, Unicode has 1,111,998 possible mappings in its 17 planes). I have sometimes made these fonts available, such as in this May of 2012 article that explains how such fonts can be built.
Anyway, I spent pretty much all day yesterday—except for a somewhat longer than usual lunch break that was actually used to watch The Martian (2015) with my wife—preparing a pair of open source CID-keyed OpenType/CFF fonts that exercise these limits but to different degrees, and I also managed to prepare and release the project on GitHub as FDArray Test.
The Unicode Consortium is planning to once again propose the encoding of the well-attested ideograph whose reading is biáng. Previous attempts at encoding this ideograph have failed due to the lack of sufficient evidence, such as appearing in a dictionary or other printed source. This time, however, there is sufficient evidence, and the simplified form of this ideograph will also be included in the proposal. Both forms, along with their U-Source references UTC-00791 and UTC-01312, are depicted below:
Historically, there have been two methods of supporting vertical writing in CID-keyed OpenType/CFF fonts, in terms of specifying the ‘vert‘ (Vertical Alternates) GSUB feature. One method involved using a vertical CMap resource, which was supplied to the AFDKO makeotf tool as an argument to its no-longer-supported “-cv” command-line option, that was used to synthesize the ‘vert’ GSUB feature. The other method, which is the preferred one, involves defining a ‘vert’ GSUB feature in the “features” file that is supplied to the AFDKO makeotf tool. In this brief article, I will explain why the first method is no longer supported, but more importantly, why the second method is preferred.
I am scheduled to present at IUC39 (The 39th Internationalization & Unicode Conference) in late October, and the title of my presentation is Pan-CJK Font Development Techniques, Tips, Tricks & Pitfalls. While the related presentations that I delivered at IUC38 last November focused on actual Source Han Sans and Noto Sans CJK development details, this presentation will be more general, and will instead focus more on techniques and best practices when developing large multilingual fonts, drawing on the experience of developing and deploying those two joined-at-the-hip typeface families when necessary.
I am currently dealing with properly categorizing the various tidbits of the presentation as Techniques, Tips, Tricks, or Pitfalls. I decided to combine Tips and Tricks into the single category Tips & Tricks, because they’re roughly the same, but mainly because I found an excellent image that conveys the meaning of tricks. ☺
Anyway, I still have a lot of work left to do on this presentation, but at least I have another two months to complete it.
As I may have mentioned in past articles, the benefits of this conference go beyond the scheduled presentations, and much of the value is the golden opportunity for face-to-face interaction with developers who are involved in the development of Unicode, or who are working with Unicode on a daily basis.
For those who are planning to attend IUC39, I look forward to meeting you there. 🍷
To follow up on my June 2011 article about managing XUID arrays in CIDFont resources, which still conveys accurate information, it has come to our attention that the integer values for the second and subsequent XUID array elements should not exceed seven digits, meaning that 9999999 is the largest integer value that should be used. Integer values that exceed seven digits can result in some implementations treating the XUID arrays of different fonts within the same printing job the same, which affects font caching, and which can result in the wrong font being used to render some characters. This printing issue may happen even if the glyphs display correctly in the PDF file on screen.
Another solution is to simply omit the XUID array from the CIDFont resource header, which effectively disables font caching. For modern printers, font caching has little or no benefit.
Lastly, for those font developers who still include a UIDBase value in their CIDFont resource headers, it can be safely removed. In fact, I strongly recommend that it be removed.
(Uni-chan image designed by Mary Jenkins)
IRG44 (ISO/IEC JTC1/SC2/WG2/IRG Meeting #44), which was originally scheduled to take place from 2015-06-15 through 2015-06-19 in Seoul, Republic of Korea and was canceled due to MERS, will instead take place during the first part of next week in Beijing, People’s Republic of China, from 2015-08-24 through 2015-08-26.
Besides the obvious work on Extensions F1 and F2, other items of interest are the three UNC (Urgently Needed Character) proposals that will be discussed, from the UTC (two characters in IRG N2068), Japan (five characters in IRG N2078), and Macao SAR (36 characters in IRG N2071). Of particular interest is IRG N2071, because Section C.2 of IRG Principles and Procedures (IRG N2016) states that UNC submissions should not include more than 30 characters.
2015-08-21 Update: The revised version of Macao SAR’s IRG N2017 (N2071R) includes only 23 characters, meaning that it is now within the terms set forth in Section C.2 of IRG Principles and Procedures.
I have personal interest in IRG N2074, which provides preliminary details about Hong Kong SAR’s forthcoming HKCS (Hong Kong Character Set) 2015 standard, which is intended to replace Hong Kong SCS-2008. One reason for my interest is that I plan to support HKCS 2015 in the Source Han Sans Version 2.000 glyph set.
Although I cannot attend IRG44, a colleague and friend who works in our Beijing office will be attending as my proxy.
While I won’t repeat here any of the exciting details in Typekit’s recent announcement for East Asia web font support (简体中文, 繁體中文, 日本語, 한국어) that employs dynamic kits, I’d like to seize this opportunity to demonstrate some of the default behavior that this new development exposes in various browsers.
Due to an inadvertent error on my part, the glyphs for the vertical-only kana were incorrect in Source Han Sans Version 1.002 (and, by extension, in Version 1.003 because there were no glyph changes). Many thanks to the person who identified and reported this issue, and I’d like to convey my sincere apologies to those who were affected by it.
[This article was written by Masataka HATTORI (服部正貴) and translated into English by yours truly.]
Source Han Code JP（日本語メニューネーム：源ノ角ゴシック Code JP）は、自分がほしくて個人的にはじめたオープンソースプロジェクトでした。Source Han Sans（源ノ角ゴシック）と Source Code Pro をフォールバックするエディタで使うと、漢字・仮名とくらべ英数字が小さくなってしまい全体的に読みにくいと感じていました。そんなとき、友人のプログラマーから、日本語も使えてコーディングにも適したフォントはないか？と相談されて、これは自分で作ってしまえと考えました。
オリジナルの Source Code Pro は、600 ユニット字幅を採用した欧文専用のモノスペースフォントで、まぎらわしいアルファベットや数字をディスプレイで判別しやすくするために、文字のデザインが工夫されています。それを、Source Han Sans JP（源ノ角ゴシック JP）の日本語と合わせてもフィットするようにサイズやウエイトを調整しました。文字幅は 660 ユニットあたりがちょうど良いと思いました。もともと読みやすさの観点から半角欧文はすこしコンデンスすぎると感じていたので、思い切って 2/3（667 ユニット）字幅を採用することにしました。一般的な半角（500 ユニット字幅）の等幅フォントにくらべ、全角文字との正確なインデントには向きませんが、読みやすさを確保しつつ、使い方次第で様々な表現ができると思いました。Source Han Code JP は、オリジナルの Source Han Sans JP と同じ７ウェイトのファミリーですが、ウェイトを切り替えても文字列の長さは変わりません。
結果的に、日本語を含むプログラミングやマークアップなどソースコードの表示や編集に使用できる Adobe Source シリーズの派生フォントとして、Adobe Fonts GitHub サイトから公開することになりました。