{
  "id": 18598427,
  "title": "Hierarchical Temporal Transformer for Cancer Grade Prediction and Cross-Cancer Transfer Learning from Pathology Reports",
  "url": "https://urgent.news/2026/11/06/hierarchical-temporal-transformer-for-cancer-grade-prediction-and",
  "topic": "ai",
  "section": "AI",
  "published": "2026-11-06T00:00:00.000Z",
  "source": {
    "name": "bioRxiv",
    "slug": "biorxiv",
    "url": "https://www.biorxiv.org/content/10.64898/2026.11.05.764680v1?rss=1"
  },
  "original_language": "en",
  "account": "The Hierarchical Temporal Transformer (HTT) is a language model designed for predicting cancer grades from pathology reports. Unlike traditional models that process one document at a time, HTT reads the sequence of medical records while considering the temporal aspect of the data. This approach allows for a more accurate analysis of the relationship between tumour characteristics and patient outcomes.\n\nHTT was developed by adapting BiomedBERT, a pre-trained language model, to better handle medical terminology. The model reads the sequence of reports using a cancer-type-conditioned temporal transformer, which enables it to capture the timing of medical visits and their impact on the progression of the disease.\n\nIn their study, the researchers evaluated HTT against various benchmarks to assess its performance. On a synthetic benchmark with labels stated in the text, HTT achieved an area under the receiver operating characteristic curve (AUROC) of 0.931 ± 0.012, surpassing a baseline model that used only the final report by 0.018 (95% confidence interval -0.009 to +0.050). In another benchmark that required extracting information from tumour sizes and visit intervals, HTT-IA (a version of HTT that incorporates a time code focusing on intervals) achieved an AUROC of 0.843, outperforming models that disregarded time entirely.\n\nThe study also tested HTT's ability to transfer knowledge across different types of cancer. When presented with unseen tumour types, HTT achieved an AUROC of 0.925 ± 0.007, matching its performance on data from the same distribution as the training set. This suggests that HTT can effectively generalize its knowledge to new cancer types, making it a valuable tool for cross-cancer applications.\n\nHowever, the researchers found that visit timing is only utilized when the data clearly indicate its importance. The time code used in HTT operates on a clinical scale, meaning that it is sensitive enough to capture meaningful temporal patterns without being overly granular. The study also showed that ceilings, order-only ablations, and re-dating probes can quickly evaluate the performance of models like HTT, providing a cost-effective way to assess their effectiveness.\n\nIn conclusion, the Hierarchical Temporal Transformer demonstrates the potential of continuous-time sequence models in cancer grade prediction and cross-cancer transfer learning. By leveraging the power of pre-trained encoders and incorporating temporal information, HTT achieves impressive results even on unseen data and across different cancer types. These findings highlight the importance of considering both the order of medical records and the timing of medical visits when analyzing pathology reports, paving the way for more accurate and informative diagnostic tools in the field of oncology.",
  "summary": "Background: Language models for cancer reports usually read one document at a time and are tested on tumour types seen in training. Continuous-time sequence models address the first gap, but whether they use visit timing, not just order, is seldom tested. Methods: We developed the Hierarchical Temporal Transformer (HTT), which encodes reports with adapted BiomedBERT and reads the sequence with a…",
  "key_points": [],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}