Urgent.News

What's breaking now, across thousands of outlets.

Tech

How to Scrape Tweets from X (Twitter) Without Paying for Empty or Fake Results

Disclosure: I built the X Twitter Scraper used in this guide and earn from its usage on Apify. If you have ever paid for a batch of scraped tweets and opened the file to find half of it empty, duplicated or filled with sample rows, this guide is for you. I will show how to collect tweets from X (Twitter) by keyword, profile, link or replies, how to keep the numbers exact, and how to check that…

This guide teaches how to scrape tweets from X (Twitter) without paying for empty or fake results. It explains common issues with scraped tweet data and how the X Twitter Scraper on Apify handles them. The steps are:

1. Open the X Twitter Scraper Actor in Apify Console. Sign in or create an account.

2. Choose sources for the run, such as search terms, profiles, tweet links, or followers. Enter a search term like "customer support" (from:zendesk OR from:intercom) as an example.

3. Set limits and filters. Specify the maximum number of rows per input item, the total maximum rows for the whole run, date range, minimum engagement metrics like likes or retweets, language preferences, and whether to exclude retweets, replies, or media. Skipped items do not incur charges.

4. Run the scraper and check the SUMMARY record in Apify Console. It lists rows delivered, skipped, duplicates, and the reason each input stopped (like limit reached or provider error). Ensure the delivered row count matches expectations.

5. Export the results as JSON, CSV, or Excel files. Each tweet row contains text, date, language, engagement metrics, media info, quoted and retweeted tweets, and author details. IDs are strings to prevent rounding issues.

The cost is $0.25 per 1,000 tweets scraped from searches, profiles, and replies; $2 per 1,000 tweets fetched by link; and $0.25 per 1,000 followers or following. There are no start fees or per-run charges. A free Apify plan allows up to 100 results per run (10 tweets fetched by link).

The guide also provides code examples in curl and Python for running the scraper via the Apify API, and mentions integrating it with n8n for daily tweets monitoring.

Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.

Read the original at dev.to →

More in Tech

CloudFront VPC Origin returns 504? Your origin's security group is lying to you

I built a clean private-ingress edge on AWS: CloudFront out front for TLS, WAF, and Shield; a private Network Load Balancer as the origin; no public load balancer anywhere.

  • CloudFront VPC Origin returns 504 Gateway Timeout error
  • Security group of Network Load Balancer limits inbound traffic
  • Allow traffic from CloudFront's public origin-facing IP ranges

Building an Offline-First POS That Stays Consistent With an Online Store

We just shipped a Windows point of sale for StoreFlux (a headless commerce backend) that keeps working with no internet connection and never drifts out of sync with the online store it's attached to.

  • Offline-first POS syncs with online store without internet.
  • Local SQLCipher database records all register actions.
  • Server processes unique client-generated IDs safely.

More from Monday 2 November →