Turning a list of company websites into a contact list with a plain HTTP crawl
Anyone who has done outbound sales or recruiting knows this task. You have a list of company websites and you need the contact info off each one. Open the Contact page, copy the email, check LinkedIn, move to the next site. A 50 domain list eats most of an afternoon done by hand. The approach Give it a start URL and it crawls within that same domain, up to a depth you choose, using Crawlee's…
This tool converts a list of company website URLs into a contact list by crawling the websites within the same domain. Using Crawlee's CheerioCrawler, it scans each page for email addresses, phone numbers, and social media profile links from major platforms like LinkedIn, X, Facebook, Instagram, and GitHub. The tool returns a record for each contact found on a page, including the URL, domain, and the discovered contact details.
It does not utilize a headless browser, meaning it only extracts information that is publicly visible in the raw HTML response to a simple HTTP request. This keeps the process fast and inexpensive, but limits its ability to gather data from sites that require JavaScript to render their content.
The tool has specific limitations when dealing with sites that heavily rely on JavaScript rendering. It may not extract all the intended information from such websites. However, for static or server-rendered company websites, which are typically the case for small and mid-sized businesses, the approach works effectively.
Pricing is based on the number of pages where contact information is actually found. A domain with no publicly available contact details costs nothing to scan, while a domain with contact information on multiple pages would cost similarly to manually extracting information from all those pages but with increased speed. The tool is available at github.com/timmKal01/website-lead-extractor and can also be used through a hosted version on Apify (apify.com/m_ctim/website-lead-extractor).
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
