NordVPN Promotion

Home / Blogs

How Do You Turn a Typesetting Language Into an Identifier System? (Not Easily)

Protect your privacy:  Get NordVPN  [ Deal: 73% off 2-year plans + 3 extra months ]
10 facts about NordVPN that aren't commonly known
  • Meshnet Feature for Personal Encrypted Networks: NordVPN offers a unique feature called Meshnet, which allows users to connect their devices directly and securely over the internet. This means you can create your own private, encrypted network for activities like gaming, file sharing, or remote access to your home devices from anywhere in the world.
  • RAM-Only Servers for Enhanced Security: Unlike many VPN providers, NordVPN uses RAM-only (diskless) servers. Since these servers run entirely on volatile memory, all data is wiped with every reboot. This ensures that no user data is stored long-term, significantly reducing the risk of data breaches and enhancing overall security.
  • Servers in a Former Military Bunker: Some of NordVPN's servers are housed in a former military bunker located deep underground. This unique location provides an extra layer of physical security against natural disasters and unauthorized access, ensuring that the servers are protected in all circumstances.
  • NordLynx Protocol with Double NAT Technology: NordVPN developed its own VPN protocol called NordLynx, built around the ultra-fast WireGuard protocol. What sets NordLynx apart is its implementation of a double Network Address Translation (NAT) system, which enhances user privacy without sacrificing speed. This innovative approach solves the potential privacy issues inherent in the standard WireGuard protocol.
  • Dark Web Monitor Feature: NordVPN includes a feature known as Dark Web Monitor. This tool actively scans dark web sites and forums for credentials associated with your email address. If it detects that your information has been compromised or appears in any data breaches, it promptly alerts you so you can take necessary actions to protect your accounts.

Unicode’s goal, which it meets quite well, is that whatever text you want to represent in whatever language, dead or alive, Unicode can represent the characters or symbols it uses. Any computer with a set of Unicode typefaces and suitable layout software can display that text. In effect, Unicode is primarily a typesetting language.

Over in the domain name system, we also use Unicode to represent non-ASCII identifiers. That turns out to be a problem because an identifier needs a unique form, something that doesn’t matter for typesetting.

For a name in the DNS, and for most other kinds of identifiers, if a user sees an identifier in use somewhere, she needs to be able to type or otherwise enter that identifier so that what she typed produces the same bits as the stored identifier. In some cases (see mailboxes, below) the rule is slightly relaxed so that given two strings, the computer can decide whether they identify the same thing.

Unicode is full of homoglyphs, characters or groups of characters that look the same but have different internal forms. We (mostly meaning the Unicode consortium, IETF, and ICANN) have come up with three ways to minimize the homoglyph problem and try to limit Unicode internationalized domain names (IDNs) so that two IDNs that look the same actually are the same.

Homoglyphs are nothing new. Those of us old enough to remember manual typewriters remember that they often had only the digits 2 through 9, and we used lowercase letter l and uppercase O for 1 and 0. It didn’t matter because the meaning was obvious from context. But when used as identifiers in the DNS, there’s no context, and a name like “operator: is not the same as “0perator: or “0perat0r”.

Normalization

In some cases Unicode offers multiple ways to write the exact same character, such as á which can be written as two glyphs, “a” followed by “combining acute accent”, or as a single precomposed glyph “a with acute accent”. Unicode defines several normalization forms, one of which consists of characters that are as composed as possible, known as Normalization Form C (NFC.) The IETF’s Internationalized Domain Names for Applications (IDNA) requires that all IDNs be in NFC, and that input Unicode be converted to NFC before being used as a domain name. This only handles composition where the two forms appear exactly the same, not forms where the forms look similar but not identical.

Scripts

A related but different issue is different scripts. Unicode defines a script as a set of characters used to write one or more languages. Familiar scripts include Latin (used to write most European languages), Cyrillic, and Greek, and Arabic. Different scripts often have characters that look the same, e.g. the Latin letter “o”, Cyrillic “o”, and Greek omicron.

Most domain registries have a list of scripts in which they will accept registrations, and each registered name usually has to be in a single script. In some cases, names are restricted to a single language in a single script (e.g., French or Portuguese which use different accents), or a mixture of compatible scripts, notably Japanese names which allow Katakana, Hiragana, Han (Kanji ideographs), and Latin. This largely deters homograph attacks at the registration level other than some arcane examples where people have constructed what looks like English names entirely from homographs in Cyrillic or Greek.

All ICANN contracted registries are supposed to file their tables of permitted characters for each language in an IANA repository, and many have.

Registry script rules are generally only enforced for the name directly registered, and not for anything below it, so you can see names like <mixture>.something.com.

Language generation rules and bundling

The last level of confusion is among characters that don’t necessarily look the same but in some sense mean the same thing. Examples include traditional and simplified Chinese characters, and in some European languages, vowels with and without accents. In script tables, one character can be listed as a variant of another, an registries have rules about them. Some forbid registration of names that differ only in characters that are variants, while others “bundle” names so that a registrant can get some or all variants of a name.

Variants have their limits; they can’t express character sequences of different length such as the German ö and ß which are usually equivalent to “oe” or “ss”, but they avoid a lot of problems particularly in Chinese and Japanese.

So who cares?

The reason I went through all this is twofold. One is related to the DNS: there are good reasons that the characters in Unicode DNS labels are limited, and you can’t use, to revisit a recent argument, emoji. If you want to use emoji in text messages or other contexts that are like typesetting, that’s fine. But they make dreadful identifiers since there are lots and lots of emoji that look almost the same, frequently deliberately so. For most emoji that look like people, you can add modifier glyphs for any of five skin tones, and male or female gender. You can make several emoji display as a super-emoji group, say man and heart and woman as 💑 which looks cute but is a challenge to type since it’s a sequence of six glyphs that have to be entered in the right order: woman, combine, heart, alternate-version, combine, man.

If the emoji for slightly frowning face 🙁 and slightly frowning face with open mouth 😦 look nearly identical, it makes no difference in a text message, but it makes them terrible identifiers. Imagine you registered one, built a website around it, and then a competitor registered the other. How can you explain to your customers which is the real one?

To avoid this problem, in principle people could create an emoji script table that groups together similar-looking emoji as variants, and otherwise limits the allowable emoji to ones that look different enough that people could reliably recognize them if they saw them in an ad on the side of a bus. But nobody will. It’s not worth anyone’s time since emoji DNS names are at most a gimmick.

The other reason is that DNS labels are not the only place on the Internet where we have text identifiers. Two other familiar ones are the path in URLs, the part after the domain name, and the mailbox in an e-mail address. Mailboxes, in particular, are a challenge, since only the system hosting the mailbox knows the meaning of an address and although every mail system does some kind of fuzzy match, the fuzz varies a lot. For ASCII mailboxes, everyone does upper/lower case folding, some ignore dots, some trim off suffixes after hyphens or plus signs, some do other things, but it entirely depends on the mail system. Systems with Unicode addresses will do similar things, but it’s a lot harder since the details of case folding are highly language specific (even among languages written in Latin characters), and there are a lot of things that might be considered to be like dots or hyphens.

While the DNS character rules can be a useful guide to designing rules for other applications, it’s unlikely they can be applied directly (e.g., DNS names never ignore dots, mailboxes sometimes do.) We still have a lot to learn about what’s a usable identifier in what contexts.

By John Levine, Author, Consultant & Speaker

Filed Under

Comments

Comment Title:

  Notify me of follow-up comments

We encourage you to post comments and engage in discussions that advance this post through relevant opinion, anecdotes, links and data. If you see a comment that you believe is irrelevant or inappropriate, you can report it using the link at the end of each comment. Views expressed in the comments do not represent those of CircleID. For more information on our comment policy, see Codes of Conduct.

CircleID Newsletter The Weekly Wrap

More and more professionals are choosing to publish critical posts on CircleID from all corners of the Internet industry. If you find it hard to keep up daily, consider subscribing to our weekly digest. We will provide you a convenient summary report once a week sent directly to your inbox. It's a quick and easy read.

Related

Topics

DNS

Sponsored byDNIB.com

Domain Names

Sponsored byVerisign

Brand Protection

Sponsored byCSC

IPv4 Markets

Sponsored byIPv4.Global

Threat Intelligence

Sponsored byWhoisXML API

New TLDs

Sponsored byRadix

Cybersecurity

Sponsored byVerisign

NordVPN Promotion