{
  "id": 3647854,
  "title": "UTF-8 vs. cp932: Why Some Bugs Only Show Up on Japanese Windows",
  "url": "https://urgent.news/2026/08/27/utf-8-vs-cp932-why-some-bugs-only-show-up-on-japanese-windows",
  "topic": "tech",
  "section": "Tech",
  "published": "2026-08-27T02:36:50.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/susumun/utf-8-vs-cp932-why-some-bugs-only-show-up-on-japanese-windows-11i4"
  },
  "original_language": "en",
  "account": "When software doesn't behave as expected on Windows with Japanese settings, the issue often stems from character encoding problems. At its core, character encoding maps symbols to strings of bytes for storage or transfer. Different encodings can produce varying byte sequences for the same symbol. For example, the Japanese character \"あ\" turns into either b'\\xe3\\x81\\x82' in UTF-8 or b'\\x82\\xa0' in cp932, which Windows uses for Japanese locales.\n\nIf a program encodes a string using one encoding and later decodes it using another, garbled text or errors appear. UTF-8 supports nearly all characters, including emojis, while cp932, specific to Japanese text, does not include most emojis. This discrepancy is where bugs commonly occur.\n\nThe source explains that some applications only crash on Windows due to encoding mismatches. Python, when capturing a subprocess's output via a pipe without explicit encoding, assumes the platform's default encoding (UTF-8 on macOS and Linux, cp932 on Windows). Thus, a string containing emojis works fine on macOS but crashes on Windows, reproducing only when the code runs on a Windows machine.\n\nIn a specific incident, a build script named build_app.py ran a version-checking script named bump_version.py. When this script printed an emoji, the subprocess captured its output using UTF-8 on macOS and cp932 on Windows, causing a UnicodeEncodeError and crashing the build process. To prevent such issues, tests were created to catch encoding misalignments before code reaches deployment. These tests include a static check for characters that can't be encoded in cp932 and a behavioral check that simulates the Windows environment's encoding to ensure the script runs correctly under those conditions.",
  "summary": "\"It works fine on my Mac, but it crashes the moment we hand it to Windows\" is a report a lot of developers eventually hear. More often than not, the culprit is a character encoding mismatch. This post walks through the basics of text encoding, then looks at a real build-pipeline crash this project ran into, and the test written to make sure it never happens again. What an Encoding Actually Is…",
  "key_points": [
    "Character encoding maps symbols to byte sequences for storage or transfer.",
    "UTF-8 supports nearly all characters, including emojis; cp932 does not.",
    "Python assumes platform default encoding (UTF-8 on macOS/Linux, cp932 on Windows)."
  ],
  "editors_take": "Character encoding mismatches between UTF-8 and cp932, used on Japanese Windows systems, can cause software bugs that only appear on those systems, leading developers to implement additional tests to catch such issues.",
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}