Urgent.News

What's breaking now, across thousands of outlets.

Tech

Python parallel debugging comes to HPC thanks to TotalView upgrade

TotalView has supported C/C++ and Fortran. With the latest quarterly release, it supports Python as well

Python parallel debugging comes to HPC thanks to TotalView upgrade

Supercomputing heavyweight TotalView has teamed up with Python, the data-science favorite now making waves in HPC. At SC26, Perforce Software showcased how TotalView's parallel debugging tool now supports Python for tackling large, distributed applications within the high-performance computing realm. Bill Burns, Perforce's vice president of software engineering, noted Python's growing popularity in HPC.

TotalView, a debugger specializing in dissecting parallel applications, primarily serves the HPC community where scientific applications run on multiple servers in parallel. While TotalView has supported C/C++ and Fortran, the latest update now embraces Python. Debugging a thousand cores in C++ was already a challenging task, with TotalView simplifying it.

However, Python's ease of learning and use has made it a popular choice in HPC, thanks to libraries like mpi4py that facilitate parallel programming. As Python's multinode scientific simulations and distributed AI training gain traction, debugging becomes more complex with numerous interpreters running in parallel. To address this, TotalView uses Microsoft's debugpy engine to create a hierarchical tree server, aggregating hundreds of stack traces and highlighting those outside of expected bounds.

Currently, developers can choose between debugging the C/C++ side while viewing the Python stack or debugging Python alone. Future releases aim to merge both modes into a single workflow and support other Python concurrency frameworks like Dask and PyTorch's torchrun.

Written by urgent.news from The Register's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.

Other reports of this story

Every outlet that reported this, and when →

Read the original at theregister.com →

More in Tech

Handle Empty Vector Search Results: Fresh Lexical and Web Recovery

The page says duplicate_detection_gap : to handle empty vector search results, the help center should fall back instead of accepting a record with no evidence.

  • System implements fallback methods for empty vector search results
  • Semantic search finds near-duplicates, rejects low-evaluation candidates
  • Keyword retrieval prioritizes recent information, web search as last resort

Fast Disks, Slow WAL Sync: Finding Linux Writeback Throttling

As described in the previous post , the primary now uses a high-I/O Ultra disk. Reads and writes took fractions of a millisecond, yet PostgreSQL WAL synchronization averaged 25-30 ms. Removing synchronous-standby waits didn't help: synchronous_commit=local still requires local WAL durability.

  • PostgreSQL WAL synchronization takes 25-30 milliseconds on ultra high-I/O disk
  • Linux writeback throttling causes delay despite disabling synchronous standby waits
  • HorizonDB avoids Linux throttling issue, achieving higher throughput

More from Tuesday 17 November →