{
  "id": 18038249,
  "title": "Measuring AI Crawler Visits and AI Referrals on a Law Firm Site",
  "url": "https://urgent.news/2026/11/03/measuring-ai-crawler-visits-and-ai-referrals-on-a-law-firm-site",
  "topic": "ai",
  "section": "AI",
  "published": "2026-11-03T23:29:24.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/sbdmarketing/measuring-ai-crawler-visits-and-ai-referrals-on-a-law-firm-site-o38"
  },
  "original_language": "en",
  "account": "Measuring AI crawler visits and AI referrals on a law firm's website is a valuable exercise, though imperfect. Two main things to track: AI crawlers fetching the site and people clicking through from AI answers. Neither method is perfect, but together they offer a more precise picture than guessing. Setting this up involves server-side and analytics-side components, with specific considerations for law firm sites.\n\nTo begin, identify the various AI crawlers. Big AI providers each run multiple crawlers, each with different purposes. For criminal defense firms, the focus should be on the search and user-initiated agents. These bots indicate whether the site's pages are being cited or are the source of a real person's query. Confirming these requests comes from the correct provider can be done using IP address lists provided by the AI companies.\n\nAccess logs from the server or CDN are necessary, as JavaScript analytics won't detect crawlers. Many hosts allow you to download these logs, and CDNs can export or query them. A simple first pass involves searching the logs for specific crawler user agents. This reveals a count per agent, indicating how frequently each crawler interacts with the site.\n\nNext, examine which URLs each agent requests. For a law firm site, this could involve checking whether the search agents reach specific pages like charge descriptions, attorney bios, and location information. If they don't, investigate robots.txt settings, CDN bot configurations, and whether those pages are properly linked. Regularly review these logs monthly, noting trends rather than focusing on a single month's data.\n\nCapturing AI referrals in analytics involves looking for specific UTM parameters added by AI tools. For example, ChatGPT appends utm_source=chatgpt.com to outbound links, making those sessions easy to spot in analytics tools like GA4. Other assistants may show up as referrals from their respective domains. However, expect undercounting, as some traffic may lack a referrer, landing in the Direct category.\n\nTie the AI channel back to calls and form submissions. A defense firm's primary concern is calls and consultation requests, not just sessions. Ensure conversion events for phone clicks, form submissions, and call tracking numbers are accurately reported against the AI channel, just as they are for organic search and ads. If a new client mentions they found the firm through AI, record this in the report, even if the session data doesn't capture it.\n\nIt's important to note that Google's AI features, such as AI Overviews and AI Mode, are harder to differentiate. Clicks from AI Overviews and AI Mode appear as ordinary organic traffic in analytics. Google has begun rolling out a Search Console report for these features, but it's not yet available in all countries. A firm may also be named in an assistant's answer without a link click at all. Therefore, periodic manual checks of what assistants say about the firm are necessary to fill in the gaps.\n\nPrivacy considerations are crucial for law firm sites. Access logs include IP addresses and requested URLs, which could hint at a visitor's legal troubles. Therefore, keep raw logs only as long as necessary, limit access to authorized personnel, and avoid sending them to unapproved tools. Lawyers should decide on the retention policy. Technical notes on AI search and legal marketing technology can be found in specialized libraries.\n\nDevelopers may have questions about blocking training crawlers to save bandwidth. However, this is a firm decision, not a technical one. Blocking GPTBot or ClaudeBot does not affect search agents, which use separate user agents. Some AI referrals may show up as Direct due to apps opening links without passing a referrer. While utm_source parameters added by tools like ChatGPT help, not all assistants include one. Review logs monthly, and after significant changes like site migrations or modifications to robots.txt.",
  "summary": "If you build or maintain websites for criminal defense firms, someone at the firm will eventually ask whether ChatGPT or Perplexity is sending them clients. It is a fair question, and the honest answer is usually \"partly, and we can measure some of it.\" Two separate things are worth tracking: AI crawlers fetching the site, and people clicking through to the site from an AI answer. Neither…",
  "key_points": [
    "AI crawlers fetch law firm site, indicating page citations and user queries",
    "Identify AI crawlers via IP address lists and server logs, excluding JavaScript analytics",
    "Capture AI referrals using UTM parameters, tracking calls and form submissions"
  ],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}