Back to the blog

I ported a correct width table and it fixed nothing

Grapheme widths were wrong in the browser client, so I lifted the tables from an implementation that gets it right. Measured it. No change at all, for a reason that took me longer to see than it should have.

Gaurav Gosain

Emoji were breaking layout in the browser terminal. A family emoji would claim the wrong number of columns, everything after it on the line would shift, and box drawing further along would stop lining up.

The obvious fix was sitting right there. ghostty has a VT implementation that gets this right, compiled to wasm, with width tables I could extract. Take its answers, use them instead of mine, done.

I built the table offline, wired it in, and measured against a corpus of hard cases.

It changed nothing. Not "small improvement", not "fixed some cases". The output was identical.

Why a table cannot work

Here is the thing I had wrong, and you can check it yourself:

clustercodepointstable sumcells used
๐Ÿ‘จโ€๐Ÿ‘ฉโ€๐Ÿ‘งโ€๐Ÿ‘ฆU+1F468 U+200D U+1F469 U+200D U+1F467 U+200D U+1F46682
U+002011
ๆ—ฅU+65E522
ๆœฌU+672C22
่ชžU+8A9E22
U+002011
รฉฬU+00E9 U+030111
7 clusters1711
The table says 17 columns. The terminal uses 11. Highlighted rows are where adding up per-codepoint answers gives the wrong result, because a cluster occupies one advance however many codepoints it contains.

A per-codepoint table answers "how wide is this codepoint". Correctly, in ghostty's case. But the terminal is not asking that question. It is asking "how far does the cursor advance", and the answer depends on how codepoints combine, not on what each of them is worth alone.

A family emoji is four people joined by zero-width joiners. Seven codepoints, several of which are individually wide. Add up the table's answers and you get eight columns. The terminal advances two, because it is one grapheme cluster and a cluster gets one advance.

No table of per-codepoint values can produce that. The information is not in the codepoints, it is in the boundaries between them, which is UAX 29 segmentation and a completely different algorithm. I had spent a day importing correct answers to a question nobody was asking.

What the corpus said

The corpus is 45 cases measured against real ghostty-vt with mode 2027 clustering enabled, so the expected values come from an implementation known to be right rather than from my reading of a spec.

With the official grapheme addon in place, 39 of 45 agree. The remaining six diverge in documented, degenerate cases, and they are asserted as expected values rather than treated as outstanding bugs. That is a deliberate choice: pretending they are bugs means either carrying a patch forever or having a permanently red test.

The real fix was already available and was not a table at all. Use a segmenter that implements UAX 29, and let the cluster boundaries fall out of it. Everything else follows.

What I kept from the exercise

The width table went in the bin, but the wasm build did not. It is now an oracle.

That distinction turned out to be the useful part. As a runtime dependency, ghostty-vt was a large coupling for a benefit I had not verified. As a test oracle, it is the thing that tells me whether my output is right, costs nothing at runtime, and gets to be as heavy as it likes because it only runs in CI.

I would not have found that framing if the optimisation had worked. It only came up because I had to ask what the wasm build was still good for after the reason I built it evaporated.

What parity means once pixels are involved

The corpus asserts integers, and integers can be compared with equality: a cluster advances two columns or it does not. The renderer's own tests do not get that luxury, and deciding what "matches" should mean for pixels took longer than wiring up the oracle did.

Pixel parity is a tolerance, not equality. Four golden scenarios pass under a budget of 2 percent differing pixels against the reference, and the emoji scenario under 5 percent, because glyph rasterisation is allowed to differ in its antialiasing without anything being wrong.

The 24-entry torture corpus needed a different comparison entirely. Those rows are mostly background, and that is exactly what makes a per-pixel comparison useless on them: a subpixel shift in glyph position swings the differing-pixel fraction wildly while nothing is actually wrong, so the number measures jitter rather than correctness. Those rows are compared by ink coverage per cell instead. Does the right cell contain the right amount of glyph, not are the exact pixels identical.

Every threshold in that list is a claim about the size of defect I am prepared not to notice. Choosing them felt like bookkeeping at the time. It was not, and the same choice made carelessly in another suite later cost me three wrong closures on one bug, because a tolerance wider than a defect does not make the defect unlikely to be seen. It makes it invisible.

What the benchmarks refuse to claim

The renderer's benchmarks run under headless chromium, where WebGL executes on SwiftShader and rasterises on the CPU. That is a different machine from the one any user has, so the repo states plainly which columns transfer to real hardware: the CPU timings, the draw-call counts and the allocation figures. Nothing else, and no frame rate is claimed anywhere, because a frame rate measured on a software rasteriser describes the rasteriser.

Writing down what a measurement cannot tell you costs a paragraph in a README. I have since been on the other end of skipping that paragraph, with three rounds of benchmarks vouching for a broken code path, so it no longer reads as pedantry to me.

The bit I would do differently

I should have run the cheap experiment first. Take one family emoji, ask the current implementation how wide it thinks it is, ask ghostty, compare. Fifteen minutes, and if both said eight, the table was never the problem and the bug lived somewhere else entirely.

I skipped that because the fix felt obviously right. Correct data replacing incorrect data is such a clean shape that I did not stop to ask whether the data was the thing that was wrong.

The day was not a total loss. Nobody on this project will spend another day importing width tables, and there is now a corpus and an oracle that would catch anyone who tried.