Fast unicode (UTF-8) validation with autovectorization

kept by lunarchivist aug 30

A branchless, autovectorized UTF-8 validation algorithm using a 4-byte lookbehind window achieves 10-80 GB/s, far outperforming branchy and DFA validators.

Fast and Hard Code
LLMs diminish the need to learn language specifics, letting developers choose languages like Rust and Zig for speed and size, and tackle complex low‑level projects without deep expertise.
also on Daniel Lemire, #performance, #developer-tools
Symmetries in boolean scans
The article shows that by exploiting symmetries of boolean scans, only three primitives (∧, <, ≠) need fast implementations to derive all others efficiently.
also on #performance, #programming-languages, #open-source, #developer-tools
Fable one-shotted a Rust rewrite of the TerminalTextEffects Python library in 11M tokens. Startup time went from 87ms to 2ms and rendering s
Fable rewrote the TerminalTextEffects Python library in Rust using 11M tokens, reducing startup time from 87ms to 2ms and boosting rendering speed 9.6x with zero dependencies in a 3MB executable.
also on #performance, #programming-languages, #open-source, #developer-tools