Netflix Reworks Conductor for 420 Million Monthly Workflow Executions and 10X Larger Workflows
Netflix has reworked its Conductor workflow orchestration engine to handle larger workloads, increasing supported workflow size from about 2,500 to 30,000 tasks and reducing p99 workflow evaluation latency by about 40%. Conductor 4.0 separates workflow metadata from task data, moves evaluation to asynchronous processing, and introduces dynamic worker allocation and concurrency controls. By Leela…
Netflix has revised its Conductor workflow orchestration engine to handle a surge in monthly workflow executions and larger workflow sizes. The updated version now manages approximately 420 million workflows each month, up from around 200,000 workflows. Conductor's capacity has expanded from supporting tasks totaling about 2,500 to handling up to 30,000 tasks per workflow. The redesign has decreased p99 workflow evaluation latency by about 40%.
Written by urgent.news from InfoQ's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
