How to Fix “Crawl Failed” in ChatGPT Ads for Shopify

If ChatGPT Ads reports “Crawl Failed” for a Shopify store, the problem usually is not that Shopify is invisible to OpenAI. It means the crawler used for ad review could not reach, read, or validate the exact landing page you submitted. Start by checking the final URL, its HTTP response, robots.txt, storefront passwords, redirects, bot-protection apps, and any CDN or firewall in front of the store. For product ads, also check whether product images are accessible and whether the Shopify product feed has finished syncing. Follow these checks to fix a ChatGPT Ads crawl failed Shopify error before resubmitting the ad.

The crawler that matters most for ad review is OAI-AdsBot. OAI-SearchBot has a separate role in search discovery and product-image access. Treating those crawlers, the Shopify app connection, the product feed, and the OpenAI Pixel as one system makes troubleshooting harder. This guide separates them without weakening your store’s security or damaging its existing SEO rules.

What “Crawl Failed” Means in ChatGPT Ads

OpenAI visits landing pages submitted for ChatGPT Ads to validate their safety and understand when an ad may be relevant. Its advertiser crawler guidance identifies OAI-AdsBot as required for landing-page validation and review. OAI-SearchBot is recommended for public-web discovery, and it also needs access to product image URLs used in feeds.

A page loading normally in your browser does not prove that OAI-AdsBot can access it. Your browser may have cookies, a selected country, a completed age check, or a trusted session. An automated crawler can receive a different response: a 403, a challenge page, a redirect loop, a login screen, or a rate-limit response.

Identify Which Part Is Failing

“Crawl failed,” “products missing,” and “conversions missing” are different problems. Use the visible symptom to choose the right investigation.

Do not edit robots.txt to fix a Pixel problem. Do not reinstall the Pixel to fix a 403 returned to OAI-AdsBot. This separation is the fastest way to avoid unrelated changes.

Read the HTTP Response Before Changing Anything

Test the exact URL submitted with the ad, including its protocol, hostname, path, and query parameters. Use the store’s primary custom domain rather than a myshopify.com address or an old domain that redirects elsewhere. Use Screpy’s guide to HTTP status codes and crawl errors to interpret the result.

Follow the complete redirect chain and record the final status:

One redirect from http to https, or from a secondary hostname to the primary domain, is not automatically a problem. A chain that changes by country, returns to an earlier URL, or ends on a selector page is. Check the final destination from a clean session without relying on stored market or consent cookies.

Check OAI-AdsBot in Shopify Robots.txt

Open https://yourstore.com/robots.txt on the same primary domain used in ChatGPT Ads. Search for OAI-AdsBot, OAI-SearchBot, and broad user-agent groups that might cover them. Pay attention to rules affecting the submitted product, collection, landing-page, or image path.

Shopify says its default robots file works for most stores. A store often gets into trouble after a theme customization, an old workaround, or an app changes the rendered rules. Shopify’s robots.txt documentation also warns that an incorrect customization can cause a large loss of search traffic. Review how to use robots.txt without hurting SEO before changing crawler rules.

OpenAI provides this basic allow example:

User-agent: OAI-AdsBot
Allow: /

User-agent: OAI-SearchBot
Allow: /

Do not replace Shopify’s complete robots file with those four lines. If you need a templates/robots.txt.liquid file, preserve Shopify’s generated groups and add only the reviewed custom rules:

{% for group in robots.default_groups %}
  {{- group.user_agent -}}
  {% for rule in group.rules %}
    {{- rule -}}
  {% endfor %}
  {%- if group.sitemap != blank -%}
    {{ group.sitemap }}
  {%- endif -%}
{% endfor %}

User-agent: OAI-AdsBot
Allow: /

User-agent: OAI-SearchBot
Allow: /

Shopify’s developer documentation recommends outputting the Liquid robots.default_groups because Shopify updates its default SEO rules over time. A dedicated user-agent group can also affect which rules a crawler applies, so review the rendered result instead of copying the example blindly. If only one landing-page path needs to be opened, a narrower rule may be more appropriate than a site-wide allowance.

After saving, reload the public /robots.txt URL. Confirm that the final rendered file contains Shopify’s normal groups, sitemap entry, and the intended OpenAI rules.

Remove Storefront Barriers That Stop Automated Crawlers

Robots permission does not guarantee access. OAI-AdsBot still has to pass every layer between the public internet and the landing page. Check for:

• A storefront password or unpublished store
• Customer-login requirements
• Age-verification interstitials
• CAPTCHA or JavaScript challenges
• Country-blocking or IP-blocking apps
• Cookie or consent flows that hide the page until interaction
• Geo or language apps that redirect the first request
• A market URL that is inactive or unavailable from the crawler’s location

Shopify’s own Markets system is designed to exclude known search crawlers from automatic customer redirection. That does not guarantee that every third-party geolocation or localization app recognizes OAI-AdsBot. Review app behavior separately. Shopify explains the distinction in its localization and automatic redirection documentation.

Do not permanently disable age checks, consent requirements, or security controls simply to make an ad pass review. Create the narrowest legitimate exception your legal, security, and storefront requirements allow.

Check CDN, Firewall, and Bot-Protection Rules

This step matters when the store uses an external proxy, CDN, firewall, or custom security layer. Standard Shopify-hosted storefronts might not have a merchant-controlled WAF in front of them, so do not manufacture one if your architecture does not include it.

Search security and traffic logs around the exact failure time. Filter for the landing-page URL and look for:

• 403 Forbidden decisions
• 429 Too Many Requests
• Managed bot rules
• Browser-integrity or JavaScript challenges
• Country or ASN restrictions
• Request bursts caused by a large ad upload

OpenAI publishes current crawler network ranges in adsbot.json and searchbot.json. Prefer those maintained lists and verified-bot programs over IP addresses copied from one log entry. OpenAI also states that OAI-AdsBot is a verified and allowlisted bot in Cloudflare, although a custom firewall rule can still deny it. Stores connected through Cloudflare can use Screpy AI & Bot Traffic to review verified crawler activity, response outcomes, and non-2xx paths; the report shows access evidence, not ad approval.

If bulk uploads coincide with 429 responses, test a smaller batch after the access rule is fixed. Rate limits should be adjusted narrowly; removing all request controls is not a durable solution.

Test Product Images and Feed URLs Separately

Landing-page validation can succeed while a product feed still fails. OAI-SearchBot needs to fetch product image URLs used by the feed. Open several affected image URLs in a clean session and inspect their responses independently from the product page. A Screpy Website Audit can help find broken product and image URLs, unexpected response codes, canonical conflicts, and repeated crawl failures across the store.

Check whether images:

• Return 200 without a signed or short-lived URL
• Redirect to another hostname with stricter bot rules
• Require cookies or a referrer
• Return 403 from an image CDN
• Use an inactive product or market-specific path

Also verify that the affected Shopify products are active, published to the relevant sales context, and available. A technically crawlable page does not make an inactive or unavailable product appear in an advertising feed.

Verify the ChatGPT Ads Shopify Connection

If products are absent rather than marked with a landing-page crawl error, follow the official ChatGPT Ads for Shopify setup guide. OpenAI recommends confirming that:

1. The Shopify app is connected to the intended Ads Manager account.
2. The product feed has completed its synchronization.
3. Products are active and available.
4. OpenAI Pixel for Shopify is connected.
5. Ads Manager has been refreshed after the sync.
6. The setup is being managed in Chrome or Safari.

The Pixel sends eligible commerce events for measurement. It does not grant OAI-AdsBot access to a blocked landing page. Likewise, reconnecting the feed will not remove a storefront password or firewall denial.

Retest in a Controlled Order

After making a change, verify one layer at a time:

1. Confirm the submitted URL and primary domain.
2. Check its response and final redirect destination.
3. Inspect the live robots.txt rules for OAI-AdsBot.
4. Remove unintended authentication, challenge, or geo barriers.
5. Review firewall events for 403 and 429 responses.
6. Test product image URLs for OAI-SearchBot access.
7. Confirm feed completion, product availability, and Pixel connection.
8. Resubmit one affected ad or a small batch.

OpenAI advises resubmitting or re-uploading affected ads if their status does not update after access is fixed. Its documentation also says advertisers should not depend on support manually bypassing crawler validation.

Use Screpy to Verify the Public Technical Path

The checks above are easier to repeat when they are part of a site-wide crawl rather than a one-page inspection. Screpy’s guide to how an SEO crawler finds technical website issues explains how crawl data exposes patterns across URLs, redirects, metadata, links, and images before another ad submission.

This is supporting evidence, not an Ads Manager approval test. ScrepyBot does not impersonate OAI-AdsBot, and a successful Screpy crawl cannot prove that OpenAI’s advertising crawler received the same treatment. Compare Screpy’s technical results with your firewall logs and the status shown in ChatGPT Ads.

After the ad issue is resolved, Screpy AI Visibility can monitor selected prompts for brand mentions, citations, sources, and competitor share of voice. That organic AI visibility workflow is separate from paid ChatGPT Ads delivery.

What Not to Do

• Do not replace Shopify’s full robots output with a short copied file.
• Do not permanently switch off every firewall or bot-protection rule.
• Do not assume that an accessible sitemap proves the landing page is accessible.
• Do not treat GPTBot permission as a substitute for OAI-AdsBot.
• Do not edit tracking code to solve a crawler access error.
• Do not repeatedly upload a large catalog while 429 responses continue.
• Do not assume that every Shopify merchant reporting a problem has the same cause.

When to Contact OpenAI Ads Support

If the exact landing page returns usable content, the rendered robots file permits OAI-AdsBot, and no access layer records a denial, contact the support address listed in OpenAI’s Shopify setup documentation. Include the Ads Manager account, exact URL, failure timestamp with time zone, response codes, redirect chain, relevant firewall event, and an affected product or ad example.

That evidence is more useful than stating that the store works in a browser. It lets support compare the failed crawler request with the public behavior you verified.

Frequently Asked Questions

Does Shopify block OAI-AdsBot by default?

Shopify says its default robots file works for most stores. Check the live output because theme customizations, old workarounds, apps, or an external security layer can change what OAI-AdsBot receives.

Which OpenAI crawler is required for ChatGPT Ads?

OAI-AdsBot is required for ChatGPT Ads landing-page validation and review. OpenAI recommends also allowing OAI-SearchBot, particularly when product image URLs must be fetched.

Why does my Shopify page open normally but fail in ChatGPT Ads?

Your browser may hold cookies or have already passed a password, age, consent, language, or country step. The crawler can receive a 403, challenge page, redirect, or login response instead.

Do I need to allow both OAI-AdsBot and OAI-SearchBot?

Prioritize OAI-AdsBot for ad readiness. OAI-SearchBot is recommended for public discovery and should be able to access product image URLs used in feeds.

How long does ChatGPT Ads take to retry a Shopify landing page?

OpenAI does not publish a universal retry time in its crawler guidance. After fixing access, resubmit or re-upload the affected ad if its status does not update, and use a small batch while validating the result.

Fix the Access Path Before Resubmitting

To resolve a ChatGPT Ads crawl failed Shopify error, work from the submitted URL outward: verify the final response, preserve Shopify’s robots defaults, allow OAI-AdsBot, remove unintended crawler barriers, and test product images and feed status separately. Then resubmit one affected ad and confirm the result before scaling the upload.

The goal is not to make the store open to every automated request. It is to give the required OpenAI crawler controlled access to valid public landing pages while keeping the rest of the store’s SEO and security behavior intact.