Urgent.News

What's breaking now, across hundreds of outlets.

Tech

How I Automated 10M+ Network KPI Records a Day with Python and Apache Airflow

Originally published on pmpdilshan.github.io . Every hour, a mobile network generates performance counters for every cell: traffic, drops, handovers, throughput, congestion. At Mobitel that adds up to more than 10 million KPI records a day. For a long time, turning that into reports meant exporting files, merging them in Excel and rebuilding the same charts every morning. So I built pipelines in…

Every hour, mobile networks generate performance counters for each cell, including traffic, drops, handovers, throughput and congestion. This results in over 10 million KPI records daily for Mobitel alone. Traditionally, turning these raw data points into reports involved manual export, merging and charting, leading to a massive 40% reduction in effort after implementing automated pipelines using Python and Apache Airflow.

The pipeline begins with extraction, pulling counter exports from network management systems on a schedule. Next, validation checks file arrival, row counts and data completeness. Transformation then standardizes names, calculates KPIs and aggregates results by cell, cluster and region. The final step is loading the data into a queryable database, followed by refreshing dashboards and setting alerts for any issues.

Critical lessons from this automation journey emphasize the importance of validation, making tasks safe to rerun, defining KPIs centrally and maintaining thorough logging for debugging. By automating the laborious reporting process, Pathum Dilshan, an RF & Network Planning Engineer at Mobitel and MSc Data Science & AI student, has freed up valuable time for optimization work. He encourages others to prioritize building the automated reporting system itself rather than solely focusing on the reports themselves.

Rewritten by urgent.news; may contain errors — check the original.

Read the original at dev.to →

Report a problem with this story

More in Tech

Same Character, 25 Clips: How ULTRA Was Made

Originally published on the Character Lock blog . ULTRA is a 3-minute AI brand film (3:13) with one bear dad holding across more than 25 clips.

  • ULTRA is a 3-minute AI brand film with 25 clips of a bear dad.
  • Character Lock method ensures consistent character appearance across 25 clips.
  • Film achieved LUFS master of minus 14, demonstrating Character Lock method effectiveness.

Change a screenshot to the required aspect ratio in the tab before you commit the hero

The docs call for a 16:9 hero, or the OG template is 1200×630. The file in the PR is a 4:3 laptop screenshot. The layout letterboxes it, or a later object-fit: cover chops off the toolbar you meant to…

  • Hero image must be 16:9 aspect ratio, 1200×630 pixels
  • Layout team treats aspect ratio as initial check, not CSS compensation
  • Use in-tab ratio tool to adjust image before committing

A Japanese NLI checker for the space between generation and trust (sealed 0.961, zero-shot baselines 0.79–0.81)

TL;DR — We released ja-nli-v1, a small Japanese natural language inference model: given a premise and a hypothesis, it returns probabilities for entailment / neutral / contradiction.

  • ja-nli-v1 Japanese NLI model achieves 0.9612 accuracy on sealed test
  • Outperforms zero-shot baselines with 0.79–0.81 accuracy
  • Model trained on synthetic data from public-domain government documents

More from Saturday 21 November →