Urgent.News

What's breaking now, across thousands of outlets.

Tech

Meta TLX, a Native GPU Compiler Extension Supporting Advanced Hardware Capabilities

Meta has introduced Triton Low-Level Extensions (TLX), a native GPU compiler extension designed to bridge the gap between high-level GPU programming abstractions and the low-level execution control required to fully exploit modern GPU architectures. TLX aims to give developers explicit control over GPU execution while preserving Triton's programming model, allowing them to retain the advantages of Python, autotuning, and blocked tensor programming while gaining direct access to the execution structures that increasingly determine performance on modern GPUs.

Meta has deployed TLX in production for several months across three major workloads, achieving significant performance improvements in GEM model training, In-Kernel Broadcast Optimization, and distributed grouped GEMM for LLM pretraining.

Brief written by urgent.news from InfoQ's own syndicated text. Machine-written — may contain errors; check the original before relying on it.

Read the original at infoq.com →

More in Tech

Running untrusted pull request code from agents and contributors

Originally published on sandscale.dev . A pull request is a request to run someone else's code on your machine. CI has always worked this way: check out the branch, install dependencies, run the…

  • Pull requests run untrusted code, posing risks like code escaping or data exfiltration
  • Sandscale's sandbox creates a separate kernel and network environment for each PR
  • After tests, sandbox should be stopped and discarded to prevent lingering malicious code

How do you tune speculative decoding in a production serving stack?

Tune speculative decoding against your own traffic, not a benchmark. Choose a method that fits your workload, set draft length from measured acceptance rate, and re-test at production concurrency…

  • Base speculative decoding method on your own traffic data, not benchmarks.
  • Choose n-gram or suffix decoding for modest speedup; use EAGLE or MTP for best latency reduction.
  • Adjust draft length based on acceptance rate; test multiple values and monitor per-request metrics.

More from Friday 20 November →