Product.ai's whole promise is to be the source AI shopping answers can trust, and its llms.txt tells AI systems which pages to read first. But on November 4 a plain request to the homepage, the Truth Graph and even the sitemap got Cloudflare's "Attention Required" block page; only robots.txt and llms.txt came through. SimplyCodes.com behaved the same. Assistants that fetch a page live when a shopper asks a question, and search engines that are challenged, may be getting the same wall, so the Category Bibles and research cannot be quoted or cited.
First action
Open Cloudflare security events for the last 30 days and filter by known AI and search crawlers.
How to measure it
Tracked shopping questions where product.ai is cited
- Now
- –
- Target
- 25%
- By
- 90 days
Steps
Read the security log
Which crawlers and assistants were challenged or blocked in the last 30 days.
Engineering Lead 1 day
Write the allow rules
Verified search crawlers and documented AI assistant agents on public paths only.
Engineering Lead 1 day
Keep protection where it matters
Challenges stay on logins, labs, console and API paths.
Engineering Lead 1 day
Apply to SimplyCodes
Same review for simplycodes.com merchant pages.
Engineering Lead 1 day
Resubmit the sitemap
Search Console and Bing Webmaster Tools.
Growth Lead 1 day
Set up a citation tracker
50 shopping questions from the Category Bibles, asked weekly across assistants.
Growth Lead 3 days
Report
Citations and referral visits from assistants, monthly.
Growth Lead monthly
Why it works
Review the Cloudflare security settings for product.ai and simplycodes.com (bot fight mode, managed challenge rules, AI crawler controls). Allow verified search crawlers and the AI assistants' documented user agents on public content (/, /truth-graph/, /research/, /sitemap.xml), keep challenges on login, checkout-like and API paths, and check Cloudflare's logs to see which assistants were being blocked. Then test citations weekly: ask the main assistants the questions each Category Bible answers and record whether product.ai is cited.
The research and guides only pay off if AI answers and search results can reach them; this is a settings change, not a project, and it unlocks every content idea below.
Ready to use
ChecklistCrawler access check
[ ] Security events reviewed for AI and search agents [ ] Allow rule for verified search crawlers [ ] Allow rule for documented AI assistant agents on public paths [ ] /sitemap.xml returns 200 to a plain request [ ] Challenges kept on labs, console, API [ ] simplycodes.com reviewed [ ] Weekly citation tracker running
Risks and how to handle them
More scraping by bad bots
Allow only verified agents and keep rate limits.
Assistants still rank others first
Fix access first, then content (ideas 2, 10).
Evidence
- Site checkproduct.ai/robots.txt says "User-agent: * Allow: /". But on November 4, 2026 a plain HTTP request to the homepage, /truth-graph/ and /sitemap.xml got Cloudflare's "Attention Required" block page (HTTP 403), while robots.txt and llms.txt loaded. simplycodes.com returned the same block page. Google's PageSpeed test, which runs a full browser, loaded the homepage.Source
- Your websiteproduct.ai/llms.txt describes Product.ai as "the verification layer for commerce" giving "ground-truth product guidance for shoppers, merchants, and AI agents", tells AI systems "If context is limited, prioritize Axioms, Truth Graph, and Research pages first", and lists the Truth Graph, Manifesto, Research Hub, For Shoppers, For Agents, For Merchants and Press pages.Source
- Google PageSpeedPhone performance 93/100 (desktop 97); real Chrome visitors pass Core Web Vitals (75th-percentile LCP 1.94s, INP 99ms). The homepage has no meta description, no social share image and no favicon. Its largest element is the line "THE VERIFICATION LAYER FOR COMMERCE".Source
- Your websiteThe two llms.txt files disagree: product.ai's says Product.ai is "Parent company of SimplyCodes" and that the Truth Graph "Currently covers Smartphones, Running Shoes, and Skincare"; axioms.ai's says Axioms.ai is the parent and counts "61,604 statements across 220 product categories". product.ai's lists "Product.ai is also known as Demand.io, Product AI, and ProductAI" and links LinkedIn, X, YouTube, TikTok, Instagram, Facebook and GitHub profiles under "askproductai".Source