Urgent.News

What's breaking now, across thousands of outlets.

Tech

A scraper isn’t a monitor: detecting new Telegram posts with Apify and n8n

My Actor run succeeded. It returned five recent Telegram posts, each with an ID, timestamp, and permalink. Then I asked the question the automation actually needed to answer: Which posts are new? The honest answer was: the current run cannot know by itself. Nothing had failed. The Actor had done its job. It had retrieved what was visible now. But “new” is not a property contained in one snapshot;…

The actor retrieved five recent Telegram posts, each with an ID, timestamp, and permalink. However, it could not determine which of these posts were truly new without prior knowledge. The actor retrieved visible content at a given snapshot, but "newness" requires a comparison between the current snapshot and a previous one. To achieve this, the tutorial introduces a state layer that provides a stable identity for every record, partitions the state by Telegram channel, incorporates a first-run policy, prevents duplicates across executions, warns when the polling window may be too small, and handles failures gracefully when state is missing or malformed.

This design is applicable to various workflows that need to identify changes, such as price trackers, job monitors, lead feeds, and inventory checks. The code and examples for this article are available in a companion package. The workflow stops at notification candidates, leaving the determination of actual changes to the downstream n8n workflow.

Retrieval and change detection are separate tasks. The actor only returns visible messages, while the n8n workflow compares these messages with durable state to identify new items.

Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.

Read the original at dev.to →

More in Tech

I'm Building an Open-Source MMORPG with Rust & Python — Join the Project

I'm Building an Open-Source MMORPG in Rust and Python — Looking for Contributors Hi everyone! 👋 I'm currently working on The Last Signal , an open-source post-apocalyptic MMORPG project.

  • The Last Signal is an open-source post-apocalyptic MMORPG being developed
  • Project utilizes Rust and Python programming languages
  • Focus on client-server communication and cryptographic integration

More from Tuesday 1 September →