Urgent.News

What's breaking now, across thousands of outlets.

More in Tech

Airflow and Dagster Can't Backfill ML Experiments Right

TL;DR — Airflow and Dagster both partition work along a single axis: time. ML workflows need backfills along two axes at once — time window and code/config version — and neither tool's partition model…

  • Airflow and Dagster partition work along a single time axis.
  • ML workflows need backfills along two axes: data time window and code/config version.
  • Current orchestrators can't track code/config version as explicit dimension.

Most advice is right for the person giving it

  • Steve Jobs and Mark Cuban gave different, yet valid, advice to graduates
  • Analyze the reason behind advice to determine its applicability to your situation
  • Test advice in small experiments before committing to significant changes

More from Wednesday 21 October →