Openai.com is currently rated at 20/100, placing it in the 'invisible' band on our Pulse board. The site faces significant visibility challenges, primarily due to access blocks and missing content maps. Automated requests to the homepage are blocked, returning an HTTP 403 error, which affects AI crawlers similarly. The absence of a /llms.txt file further complicates content discovery.
What the Probe Saw
Our probe indicates that openai.com is largely invisible, with a score of 20/100. The homepage returned an HTTP 403 error, suggesting that the site either blocks automated requests or is experiencing server errors. This means AI crawlers, like GPTBot and ClaudeBot, face similar access issues. Although there is no blanket Disallow for major AI crawlers, the lack of a /llms.txt file means engines lack a curated map to the site's best content, leading them to guess. Furthermore, the probe couldn't detect any JSON-LD schema or assess the site's answer-first structure due to these access issues.
Implications for Business Sites
For a business site, being invisible to AI crawlers can significantly impact online visibility and search engine rankings. The HTTP 403 error on openai.com suggests a need to review server settings and ensure that automated requests are not inadvertently blocked. Implementing a /llms.txt file can guide AI crawlers to key content, improving discoverability. Additionally, ensuring that JSON-LD schema is present can enhance how search engines interpret and display site information.
To avoid similar pitfalls, businesses should regularly audit their sites using tools like our free checker . Understanding how AI crawlers interact with your site is crucial, and our AI crawler playbook offers valuable insights. Monitoring site performance through our Monitor can help identify and rectify issues promptly.
No comments yet