When str.lower() is a security vulnerability in Python – Seth Larson

International standards primarily support ASCII characters, yet the global community utilizes far more than just Latin alphabet characters. Consequently, a mapping from Unicode to ASCII is necessary for domain name usage. This mapping process, known as NamePrep, is defined in RFC 3491 as a subset of StringPrep and plays a critical role in Internationalizing Domain Names in Applications (IDNA), formerly referred to as "IDNA 2003".

StringPrep itself is outlined in RFC 3454. IDNA 2003 has since been superseded by IDNA 2008, which is supported by the idna package available on the Python Package Index, while IDNA 2003 is supported through the idna codec (str.encode("idna")). The stringprep module in Python's standard library implements StringPrep.

StringPrep outlines the "case folding" step (essentially, how to lowercase/uppercase a Unicode codepoint) in Section 3.2, thereby facilitating case-insensitive string comparisons. This step involves mapping all characters through tables B.2 and B.3. Table B.2 corresponds to Python's str.lower() function, which applies Unicode case-folding rules to all characters.

Table B.3 contains specific exceptions. The Python code implementing this, assuming Table B.3 is accurately captured, is as follows: It might seem innocuous at first glance... but the title of this article reveals the underlying issue. The str.lower() function call within this stringprep implementation constitutes a security vulnerability!

Why? Because Python interpreters ship with varying Unicode data. The interpreter's Unicode version can be determined by accessing unicodedata.unidata_version. Furthermore, Python includes a database of Unicode 3.2.0 data (unicodedata.ucd_3_2_0) specifically tailored for the StringPrep and IDNA algorithms. This database is crucial as the B.2 and B.3 tables in RFC 3454 essentially encode Unicode 3.2.0 case-folding rules into a table.

Therefore, to maintain consistent operation, we must utilize Unicode 3.2.0 case-folding rules rather than newer Unicode case-folding rules. This discrepancy in implementation versus specification constitutes the vulnerability. The remediation involved creating new exceptions so that str.lower() would behave as if it were using Unicode 3.2.0, but only in the context of this specific stringprep function.

By mapping each Unicode codepoint and documenting instances where the behavior of str.lower() differs when compared to Unicode 3.2.0, we have aligned IDNA 2003 with the specification. Reporting of this vulnerability is credited to Bitshift, co-development of the remediation to Stan Ulbrych, and thorough review by Marc-Andre Lemburg and Petr Viktorin. For further information, see CVE-2026-17084.