{
  "id": 4008419,
  "title": "XGBoost Explained: From Gradient Boosting to Weighted Quantile Sketch",
  "url": "https://urgent.news/2026/08/28/xgboost-explained-from-gradient-boosting-to-weighted-quantile-sketch",
  "topic": "tech",
  "section": "Tech",
  "published": "2026-08-28T18:01:00.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/arham_ahmed_63699c0d1def9/xgboost-explained-from-gradient-boosting-to-weighted-quantile-sketch-145d"
  },
  "original_language": "en",
  "account": "XGBoost is a powerful tree-based algorithm that performs calculations significantly faster than traditional Gradient Boosting. It has gained popularity through its success in Kaggle competitions and its ability to handle sparse data and null values efficiently.\n\nTo understand XGBoost, it's essential to grasp the concept of boosting. In boosting, the mistakes of the first model are identified and fed to a subsequent model to correct them. This process is repeated, with each new model attempting to correct the errors of the previous one.\n\nXGBoost improves upon the standard Gradient Boosting by optimizing an objective function that minimizes the difference between predicted and actual values. However, the objective function in XGBoost is non-optimizable in Euclidean space due to the varying decision tree structures. To address this, XGBoost uses the 2nd-order Taylor series expansion, which allows for optimization.\n\nThe algorithm estimates the optimal weights for each leaf node by solving the modified objective function. The split finding process involves calculating the gain for each potential split and selecting the one with the highest gain. XGBoost uses a greedy approach to evaluate split candidates, which can be computationally expensive for large datasets.\n\nTo address this, XGBoost employs an approximate split finding technique known as the weighted quantile sketch. This method represents feature values and their corresponding Hessians as a dataset, and then defines a weighted rank function to determine the fraction of total weight below a certain feature value. By setting an approximation factor (ε), XGBoost generates a manageable number of candidate split points. A smaller ε results in a finer approximation, allowing for more candidate split points.\n\nMoreover, XGBoost handles missing values by assigning a default direction to each branch. During training, the algorithm determines the default direction that yields the best split score when encountering missing values during prediction. This approach ensures that missing values are appropriately handled without negatively impacting the model's performance.",
  "summary": "XGBoost Introduction: XGBoost XGBoost is a tree-based algorithm. It is a variant of Gradient Boosting which is one of the highest-scaled versions of it. It performs calculations 10x faster than Gradient Boosting. Its success is also witnessed in Kaggle's competitions wherein out of 29, 17 solutions used XGBoost. Its capability also shines in handling sparse data and null values. Before…",
  "key_points": [
    "XGBoost is a faster tree-based algorithm than traditional Gradient Boosting.",
    "It optimizes an objective function using 2nd-order Taylor series expansion."
  ],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}