pgdog.dev/sitemap.xml does not return a sitemap. It returns the homepage as an ordinary page, and so does every address that does not exist (/about, or any made-up path), instead of a 404. robots.txt has no Sitemap line either. Search engines therefore get no list of the 23 blog posts, and they index the copies: a site: search shows /sitemap.xml as a page titled with the funding banner ("We raised a $5.5M seed round"), a blog post under that same banner text, /about as a separate page, and the homepage under its old headline, "Postgres scales with PgDog". The docs site already publishes a proper 104-URL sitemap, and the marketing site is fast (real visitors pass Core Web Vitals), so crawl hygiene is the one gap.
First action
Open pgdog.dev/sitemap.xml and pgdog.dev/any-made-up-path, confirm both show the homepage, and find the hosting rule that falls back to index.html.
How to measure it
Indexed posts and organic blog clicks
- Now
- –
- Target
- All 23 posts indexed; +50% organic clicks to /blog
- By
- 60 days
Steps
Find the fallback
Locate the static-host or Cloudflare rule that serves the homepage for every unknown path.
Founding Software Engineer 1 hour
Generate the sitemap
A build step that writes /sitemap.xml with the homepage, /enterprise, /blog and all posts, with lastmod from the RSS dates.
Founding Software Engineer 2 hours
Update robots.txt
Add Sitemap lines for pgdog.dev and docs.pgdog.dev.
Founding Software Engineer 15 minutes
Serve real 404s
Status 404 with links to docs, blog and Enterprise; 301 /about and /enterprise.html to their canonical pages.
Founding Software Engineer 2 hours
Fix titles
Each post gets its own title, description and canonical; the funding banner stays out of the title.
Founding Software Engineer 2 hours
Submit and recrawl
Submit both sitemaps in Search Console and Bing Webmaster Tools; request recrawl of the homepage and the three comparison posts.
Founder & CEO 1 hour
Watch coverage
Weekly check of indexed pages, soft-404 reports and clicks to the comparison posts.
Founder & CEO weekly for 8 weeks
Why it works
Generate /sitemap.xml at build time, listing the homepage, /enterprise, /blog and every post, with lastmod dates taken from the RSS feed. Add Sitemap lines for pgdog.dev and docs.pgdog.dev to robots.txt. Make unknown paths return a real 404 page with links to the docs, the blog and Enterprise, and redirect /about and /enterprise.html to their canonical pages. Give every post its own title tag, meta description and canonical, so the funding banner can never become a page title. Then submit both sitemaps in Google Search Console and Bing Webmaster Tools and ask for the homepage to be recrawled, so search results show "Scale Postgres n times".
A sitemap is how search engines learn which pages exist and when they change. Soft 404s spread duplicate copies of the homepage and waste crawl on addresses that do not exist. For a three-person team with no marketing hire, the comparison posts (vs. PgBouncer, Citus and RDS Proxy) ranking for the searches engineers make before switching is the cheapest acquisition channel there is, and it costs a day of work.
Ready to use
Checklistpgdog.dev crawl fix list
[ ] /sitemap.xml returns XML (urlset) listing /, /enterprise, /blog and all 23 posts with lastmod [ ] robots.txt: Sitemap: https://pgdog.dev/sitemap.xml and Sitemap: https://docs.pgdog.dev/sitemap.xml [ ] Unknown paths return HTTP 404 with links to docs, blog, Enterprise [ ] /about and /enterprise.html 301 to canonical pages [ ] Every post: unique <title>, meta description, rel=canonical [ ] Funding banner not used as a title [ ] Both sitemaps submitted to Search Console and Bing Webmaster Tools [ ] Homepage recrawl requested so results show "Scale Postgres n times"
Risks and how to handle them
A real 404 breaks old links that relied on the fallback
Before switching, list paths that got traffic in the last 90 days and redirect any that still matter.
Duplicates take time to drop out
Use the removal and URL inspection tools for /sitemap.xml and /about.
Evidence
- Your websiteCrawl check (fetched twice, November 10, 2026): pgdog.dev/sitemap.xml returns the homepage as an HTML page, not an XML sitemap; made-up addresses such as /this-page-does-not-exist-xyz, and /about, also return the full homepage instead of a 404; the site links to both /enterprise and /enterprise.html; robots.txt contains only "User-agent: *" and "Allow: /", with no Sitemap line. The docs site, by contrast, publishes a 104-URL sitemap.Source
- Search resultsA site:pgdog.dev web search (November 10, 2026) lists pgdog.dev/sitemap.xml as an indexed page titled with the funding banner, "We raised a $5.5M seed round. Read the post→"; shows the post "Why we built yet another Postgres connection pooler" under the same banner text; lists /about, a copy of the homepage, as its own page; and still shows the homepage under the old headline "Postgres scales with PgDog".Source
- Your websiteBlog (checked November 10, 2026): 23 posts from March 2025 to November 2026, including "PgDog vs. PgBouncer", "PgDog vs. Citus", "PgDog vs. RDS Proxy", "Sharding a real Rails app", "Sharding pgvector" and "Multi-tenant Postgres can be easy"; no newsletter signup; a valid RSS feed exists at /feed.xml but the blog page does not link to it; a sampled post ends with a link to the installation docs and no enterprise or support offer.Source
- Google PageSpeedPageSpeed (November 10, 2026): mobile performance 99 and desktop 100; real visitors pass Core Web Vitals (75th percentile LCP 755 ms, INP 48 ms, CLS 0); accessibility 93, with two issues (no main landmark; ARIA roles missing required children).Source