Styled Unicode Breaks Character Counters
A 76-character bio, styled once, no longer has one useful length. const s = " ๐๐๐ฌ๐ญ๐ก๐๐ญ๐ข๐ " ; [... s ]. length // 9 code points (and 9 graphemes here) s . length // 18 UTF-16 units โ what String.length counts new TextEncoder (). encode ( s ). length // 36 UTF-8 bytes โ what storage counts Three defensible answers, factor of four on one word. I found this while auditing a counter besideโฆ
Styled Unicode breaks character counters by changing the number of code points, UTF-16 units, and UTF-8 bytes. A 76-character bio with one style has the same code points as a plain text, but different UTF-16 and UTF-8 counts. A styled bio can be four times longer in storage but the counter should report code points, not bytes. When a destination only publishes a character limit, the UI should show both bytes and character counts.
JavaScript strings truncate at UTF-16 boundaries, which can cause issues with styled Unicode characters. The solution is to truncate by code points or grapheme clusters instead of UTF-16 indices.
Written by urgent.news from Dev.to's reporting โ not their text. Machine-written โ may contain errors; check the original before relying on it.
