How to check domain expiry dates, DNS records and SSL certificates in bulk

Managing a large number of domains can be a daunting task. Renewals can be missed, SSL certificates may expire on Sundays, and clients may inquire about their DMARC policies. These issues often require checking information from three different sources: WHOIS lookups, DNS tools, and browser padlocks. This can be time-consuming, particularly when checking fifty domains at once.

To simplify this process, a small Apify Actor called Domain WHOIS, DNS and SSL Expiry Lookup (RDAP) can be utilized. This tool allows the input of domains or full URLs and generates a single row with comprehensive data for each domain. The data includes registration details from RDAP, DNS records such as A, AAAA, MX, NS, TXT, CAA, as well as SPF and DMARC policies.

Additionally, it provides SSL certificate information from a TLS handshake on port 443, including the issuer, subject, validity period, days remaining, chain trust status, and the certificate's coverage.

To use this Actor, you can follow these steps. First, open the Actor from the Apify Store and sign in to the Apify Console, which is available with a free account. Input your domains or URLs, one per line, in the Input tab. You can also paste a list separated by various delimiters or a list copied from a spreadsheet. Ensure that all three checks - WHOIS (RDAP), DNS records, and SSL certificate - are switched on, or switch them off if you do not require them.

Once you click Start, the Actor will process all the domains and provide a dataset as a CSV, JSON, or Excel file. For regular monitoring of domain renewals, schedule the Actor to run weekly with your domain list, and then filter the results based on daysUntilExpiry or sslDaysUntilExpiry.

If you prefer to call the Actor from code, you can use the curl endpoint for short lists or the Python client for larger lists. The curl command initiates a run and returns the results once it finishes, requiring your Apify token as an environment variable. In Python, you can use the apify-client package to call the Actor. This method is beneficial when dealing with thousands of domains, as it waits for the run to complete, regardless of how long it takes.

The pricing for this Actor is pay-per-event, at $0.0015 per domain looked up, which amounts to $1.50 for every 1,000 domains. This cost covers registration, DNS, and SSL checks. There is no additional start fee or platform usage charges. A domain is charged when at least one source provides information about it. Invalid inputs and domains that fail all sources are free error rows. You can also set a maximum charge for any run in Apify.

Please note that this tool does not collect registrant data, such as owner names, emails, phone numbers, or addresses. In some cases, like .de domains, RDAP service may not be available, and the registration fields will be empty while the DNS and SSL checks still function. Registries can rate-limit RDAP requests, so the Actor maintains about one request every 0.7 seconds per registry to avoid hitting these limits.

For lists containing various domain extensions, the Actor runs more efficiently. The SSL check is performed on port 443 of the provided host, and sites without HTTPS will show sslStatus: error. Some registries do not publish creation dates, causing domainAgeDays to be empty, and the Actor does not attempt to guess these values.

The maximum number of domains that can be processed in a single run is 20,000. The Actor does not scrape websites, ensuring that your domains remain secure during this process.