Urgent.News

What's breaking now, across thousands of outlets.

Tech

How to fix 'Blocked due to access forbidden (403)' in Google Search Console (it's usually your WAF or bot rules)

Automated technical SEO checks that catch indexing blocks and deploy regressions — seotto.com Search Console's Pages report has a reason that looks simple: Blocked due to access forbidden (403) . Googlebot requested the URL and got HTTP 403 Forbidden, so the page can't be indexed. The confusing part is that the page usually loads perfectly in your browser. The 403 is aimed at Googlebot…

To resolve Googlebot-specific access blocked (403) errors in Google Search Console, follow these steps. First, confirm the issue persists by selecting a few URLs from the report and performing URL Inspection followed by a live URL test. If the live test displays a 403, the problem is current. Next, identify which layer returned the 403 response.

This can be done by analyzing the headers and body of the response, which often reveal identifiable patterns such as CDN or WAF blocks, server-level 403 pages, or custom app error templates. Keep in mind that Googlebot requests are protected and return different headers compared to typical requests.

Common causes of the 403 error include bot protection or WAF rules, rate limiting, geo-blocking, server-level rules, and file permissions. WAF or CDN rules are the most frequent culprits; they often become active after tightening security settings or adding new rules. To address this, verify that verified Googlebot requests are passing through by examining your WAF or CDN logs for requests from Google's IP ranges.

Additionally, if your site implements rate limits or geo-blocking, ensure that Googlebot's IP addresses are exempt from these restrictions. For server-level rules, inspect your server configuration (like .htaccess or Nginx files) for any user agent patterns that might be blocking Googlebot. If only specific files or directories return 403, check file permissions or missing index files.

After making the necessary adjustments, verify the fix by running URL Inspection's live test on the affected URLs to ensure they now return a 200 status. Monitor your access or CDN logs over the following day to ensure verified Googlebot requests are being granted access, and validate the fix through the Search Console. Remember to check for a possible increase in 403 responses in the Crawl Stats report as an indicator of any ongoing issues.

Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.

Read the original at dev.to →

More in Tech

Private Image Originals and Thumbnails: Retention, Resizing, and Signed Downloads

Short answer: use private object storage for originals and generated thumbnails, run resizing in your Node.js worker or an image service, and return short-lived signed download links.

  • Separate private originals and generated thumbnails in distinct prefixes (origins/, thumbs/)
  • Use unique keys for variants (originals/{tenantId}/{imageId}/source.jpg) and record them in DB
  • Generate signed GET URLs for downloads, log request ID, image ID, variant, attempt, and time

How to fix 'too many missed start times' in Kubernetes CronJobs (and why your job silently stopped running)

Reliable HTTP cron and delayed-job scheduling with automatic retries and full run logs — delay.run You check on a Kubernetes CronJob that should run every minute and find that it hasn't created a Job…

  • CronJobs stop running after 100 missed start times in default Kubernetes
  • Missed start times can occur due to suspension, controller issues, clock skew, or frequent schedules
  • Set startingDeadlineSeconds to allow runs within a specified late time window

Should every URL end with a trailing slash? Let's argue about it

Automated technical SEO checks that catch indexing blocks and deploy regressions — seotto.com Few URL questions generate as much quiet frustration as this one.

  • Trailing slash debate centers on URL format preference
  • Supporters argue alignment with static hosting platforms
  • Opponents prefer URLs without trailing slashes for ease of use

My analytics undercounted my audience by an order of magnitude

Last week my own funnel report told me something comfortable and wrong: 123 human visitors in 30 days . At that number, the only rational conclusion is "I have no audience", and the rational move…

  • Analytics undercounted audience by an order of magnitude
  • Bot visits and unclassified visits omitted from human count
  • New metrics report denominator, change date, and known-unknown bucket

x402 Seller Checklist: Verify Your Endpoint Before Buyers Pay

Originally published on lumierepaycheck.org . Written with help from AI and checked against our own data. Before AI agents pay your x402 endpoint, many of them check it first.

  • Verify endpoint grade using Lumière PayCheck
  • List payout wallet in well-known/paycheck.json
  • Complete paid test to achieve A grade

More from Wednesday 18 November →