How long is a string? Trick question.

10 min readโ€ขbyRohit Roy
Jul 10

How many characters is this: ๐Ÿป

One. A single bear.

How many bytes does that one character take up, compared to the letter A?

Not the same. The bear takes 4 bytes. A takes 1. Same idea of "one character," very different storage cost. Here is why, and why that gap has corrupted databases and crashed iPhones.

Code points

Unicode assigns a unique ID number to every character that exists: every letter, digit, symbol, and emoji, in every language. That ID number is called a code point.

A is code point 65. ๐Ÿป is code point 128,059.

A byte can only hold a number from 0 to 255. 65 fits. 128,059 does not. So large code points get split across multiple bytes, chained together.

But how? If everyone invents their own way of splitting numbers into bytes, no two programs can exchange text reliably. You need a standard. A rulebook that says: here is exactly how to pack a code point into bytes, and here is exactly how to unpack bytes back into code points.

That rulebook is UTF-8. It is the most widely used text encoding on the internet. Almost every web page, every API response, every database connection uses it.

UTF-8 works on a simple principle: each byte has 8 bits. UTF-8 reserves some of those bits as a header and leaves the rest to carry the actual code point number. The header pattern tells a reader whether this byte is the start of a new character or a continuation of one already in progress.

Here is the actual bit layout, by code point range:

Code point rangeDecimal rangeBytesBit pattern
U+0000 to U+007F0 to 12710xxxxxxx
U+0080 to U+07FF128 to 20472110xxxxx 10xxxxxx
U+0800 to U+FFFF2048 to 6553531110xxxx 10xxxxxx 10xxxxxx
U+10000 to U+10FFFF65536 to 1114111411110xxx 10xxxxxx 10xxxxxx 10xxxxxx

Every x is a free bit, used to carry the actual code point number. Every continuation byte always starts with 10. That fixed pattern is what makes continuation bytes recognizable, no matter what data they carry. A reader never has to guess whether a byte is a continuation. It can just check the first two bits.

Take A, code point 65. In binary, 65 is 1000001, which is 7 bits, small enough to fill the x slots of the 1-byte pattern directly: 01000001. That is the full byte. Nothing else needed.

Now take ๐Ÿป, code point 128,059, or 1F43B in hexadecimal. This number needs 21 bits to represent, so it falls in the 4-byte row of the table. Written out in 21 bits, it is:

000 011111 010000 111011

Those three groups slot directly into the three sets of x positions in the 4-byte pattern:

byte 1: 11110  000      -> 11110000
byte 2: 10     011111   -> 10011111
byte 3: 10     010000   -> 10010000
byte 4: 10     111011   -> 10111011

In hexadecimal, that is F0 9F 90 BB, four separate bytes, each carrying a fragment of one code point number. For example, that is what you would see if you opened a file containing ๐Ÿป in a hex editor.

Run this in Go:

s := "๐Ÿป"
fmt.Println(len(s))                    // 4 bytes
fmt.Println(utf8.RuneCountInString(s)) // 1 code point
fmt.Printf("%U\n", []rune(s))          // [U+1F43B]
for _, b := range []byte(s) {
    fmt.Printf("%x ", b)               // f0 9f 90 bb
}

len counts bytes. utf8.RuneCountInString decodes UTF-8 and counts code points. Iterating over []byte(s) shows each byte individually. Same string, three different views.

A rune is Go's name for a single decoded code point. When you convert a string to []rune, Go walks the bytes, decodes the UTF-8 sequences, and gives you back each code point as a single value. Other languages have the same idea under different names.

Byte slicing versus code point slicing

Take Hello, ๐Ÿป๐Ÿป๐Ÿป๐Ÿป.

Hello, is 7 bytes (7 plain characters, 1 byte each). Each bear is 4 bytes. Total: 7 + 16 = 23 bytes, but only 11 characters.

Now say some code grabs "the first 9 bytes." It gets all 7 bytes of Hello, , then only 2 of the first bear's 4 bytes. Those 2 leftover bytes do not form a valid character. Most renderers show this as a broken glyph, a black diamond with a question mark, the standard "I could not decode this" symbol.

s := "Hello, ๐Ÿป๐Ÿป๐Ÿป๐Ÿป"
fmt.Println(string(s[:9]))  // Hello, ๏ฟฝ
// 9 bytes cuts through the first bear. 2 bytes left over. Not valid UTF-8.

Now say the same code grabs "the first 10 code points" instead. It correctly reads Hello, as 7 characters, then the next 3 as full bears, using all 12 of their bytes properly, and stops there. Result: Hello, ๐Ÿป๐Ÿป๐Ÿป. Valid. Nothing broken. Just one bear short of the original.

runes := []rune(s)
fmt.Println(string(runes[:10]))  // Hello, ๐Ÿป๐Ÿป๐Ÿป
// 10 code points, all valid. 3 full bears instead of 4.

Real case: MySQL and the emoji that would not save

MySQL has a character setting literally named utf8. It sounds like full Unicode support. But it only stores 3 bytes per character. That means it can only store code points from the Basic Multilingual Plane (U+0000 to U+FFFF), the first 65,536 characters of Unicode. Emoji live outside that plane, in the range above U+10000, where every character needs 4 bytes. MySQL's utf8 cannot store them.

This naming trap has broken emoji storage for years, and the biggest real-world example is WordPress. In 2015, WordPress shipped an automatic migration converting every site from utf8 to utf8mb4 (the actual 4-byte setting). Before that migration, saving a post with an emoji would silently corrupt the content or replace the emoji with question marks. The WordPress core team documented the migration here.

The migration only ran on sites that met specific server requirements. Sites that missed it stayed broken. WordPress runs a huge share of all websites, so a naming decision from MySQL in 2002 required a coordinated upgrade across millions of independently-run sites more than a decade later.

MySQL eventually deprecated the name. A later release started referring to the old setting as utf8mb3 in its documentation and system output, specifically to distinguish it from real UTF-8.

When one code point is not one character

๐Ÿ‘จโ€๐Ÿ‘ฉโ€๐Ÿ‘งโ€๐Ÿ‘ฆ looks like one emoji. Is it one code point, like A and ๐Ÿป?

No. It is 7 separate code points: man, woman, girl, boy, and three invisible characters between them.

You can see the individual code points for yourself:

s := "๐Ÿ‘จโ€๐Ÿ‘ฉโ€๐Ÿ‘งโ€๐Ÿ‘ฆ"
fmt.Println(len(s))                    // 25 bytes
fmt.Println(utf8.RuneCountInString(s)) // 7 code points
for _, r := range s {
    fmt.Printf("%U %c\n", r, r)
}

Output:

U+1F468 ๐Ÿ‘จ
U+200D   (invisible joiner)
U+1F469 ๐Ÿ‘ฉ
U+200D   (invisible joiner)
U+1F467 ๐Ÿ‘ง
U+200D   (invisible joiner)
U+1F466 ๐Ÿ‘ฆ

Unicode uses a composable approach here. It defines individual building blocks (man, woman, child, skin tones) plus a special character called the zero-width joiner, or ZWJ. The ZWJ has no visible shape. Its only job is an instruction to the renderer: combine the character before me and the character after me into one glyph.

This is not real-time compositing. The font file contains a table called GSUB (glyph substitution) that maps code point sequences to pre-designed glyph IDs. When the shaping engine (Core Text on Apple, HarfBuzz on Android, Uniscribe on Windows) sees a ZWJ sequence, it queries the GSUB table: does a glyph exist for this exact combination? If yes, it substitutes the entire sequence with that one glyph. The font is essentially a database of if-then rules: if you see A + ZWJ + B, render glyph #1042 instead.

Some examples:

FormulaCode Point SequenceResult Glyph
Woman + ZWJ + MicroscopeU+1F469 + U+200D + U+1F52C๐Ÿ‘ฉโ€๐Ÿ”ฌ (Woman Scientist)
Cat + ZWJ + Black SquareU+1F408 + U+200D + U+2B1B๐Ÿˆโ€โฌ› (Black Cat)
Black Flag + ZWJ + SkullU+1F3F4 + U+200D + U+2620๐Ÿดโ€โ˜ ๏ธ (Pirate Flag)
Eye + ZWJ + Speech BubbleU+1F441 + U+200D + U+1F5E8๐Ÿ‘๏ธโ€๐Ÿ—จ๏ธ (Eye in Speech Bubble)

If no matching glyph exists, the engine falls back to rendering each code point individually. That is why some platforms show the family emoji as separate people side by side. Their font does not have that specific ZWJ combination.

So we're using one or more code points that render as a single visual character. The technical name for this is grapheme cluster.

Why family emoji changed

Family emoji before and after the iOS 17.4 redesign

The ZWJ approach works well for a fixed set of family combinations. But it falls apart when you add skin tones. Each person can have one of 5 skin tone modifiers. For a family of 4, that is 5 ร— 5 ร— 5 ร— 5 = 625 possible combinations. For all possible family sizes and compositions, the number grows past 7,000. Every combination would need its own entry in the font's lookup table.

That is why Unicode changed course. The old family set could not fairly cover the range of real-world family structures and skin tones without exploding into thousands of variants, so the standard moved toward a smaller set that vendors could implement consistently.

Unicode approved new family emoji designs in Emoji 15.1, and Apple shipped them in iOS 17.4. The old family sequences were not removed from the standard, but vendors are free to render them in their own style. Apple's version uses gender-neutral gray silhouettes so the sequences do not imply a fixed skin tone or family structure.

Real case: the text sequence that crashed the iPhone

In 2018, a short Unicode sequence could crash an iPhone when the system tried to render it.

Like emoji, complex scripts such as Telugu use multiple code points to produce a single visual character. Telugu lives in the Basic Multilingual Plane, specifically U+0C00 to U+0C7F, and its characters are encoded as 3-byte sequences in UTF-8. In Telugu, every consonant has an inherent vowel sound. To write two consonants without a vowel between them, you insert a virama (U+0C4D) which suppresses the vowel and helps form a combined shape called a conjunct ligature.

One widely reported crash trigger was five code points: เฐœ (U+0C1C, Telugu letter Ja), เฑ (U+0C4D, virama), เฐž (U+0C1E, Telugu letter Nya), ZWNJ (U+200C, zero width non-joiner), เฐพ (U+0C3E, vowel sign Aa). The code points formed valid Unicode text. The bug was in Apple's handling of shaping and rendering for that valid grapheme cluster.

The sequence pushed Core Text into an edge case. The virama tries to form a conjunct. The ZWNJ tells the renderer not to join the characters. Apple's text engine did not handle that combination safely.

Apple later classified the problem as a Core Text memory-corruption bug and fixed it with better input validation. Apple did not publish the exact low-level failure.

The blast radius went beyond one app. When displayed in system UI, the sequence could affect components such as SpringBoard, the process that runs the iPhone home screen. Once this became public, people started sending it to strangers deliberately, just to crash their phones. Similar crash-triggering sequences were later reported in Telugu, Devanagari, and Bengali scripts.

MySQL's bug was a byte counting problem. Apple's bug was a text shaping and rendering problem on a valid grapheme cluster, one level higher.

The takeaway

Every time code touches text, it operates at one of three levels: bytes, code points, or grapheme clusters. Shaping and rendering turn grapheme clusters into visible glyphs. Each layer answers "how long is this string" differently. Most bugs come from assuming these answers are always the same number.

For plain English text, they usually are, which is exactly why this class of bug survives testing and only shows up once real users, real languages, and real emoji hit production.

Before you slice, count, or truncate text, ask which of the three you actually mean.