uta.edu serves its XML sitemap at /sitemap-xml, 16,038 URLs covering every program, news release and admissions page. But robots.txt contains "Disallow: /sitemap-xml" and has no "Sitemap:" line, and the conventional /sitemap.xml returns 404. Any crawler that obeys robots.txt is told not to open the map, and crawlers that look for a sitemap in robots.txt or at /sitemap.xml never find it. That includes Bing (which powers several AI assistants) and the AI search crawlers that students now ask "best nursing program in DFW" or "UTA transfer from Dallas College".
First action
Remove the Disallow line and add the Sitemap line to robots.txt today.
How to measure it
Sitemap URLs indexed in Google and Bing
- Now
- –
- Target
- +25%
- By
- 60 days
Steps
Edit robots.txt
Remove "Disallow: /sitemap-xml"; add the Sitemap line.
Web Services Manager, University Communications 30 minutes
Add the usual address
Redirect /sitemap.xml to /sitemap-xml.
Web Services Manager, University Communications 1 hour
Clean the sitemap
Exclude /testzone/ and other non-public sections.
Web Services Manager, University Communications 2 hours
Check Google
Search Console Sitemaps report: fetched, no errors.
SEO and Analytics Specialist 1 day
Submit to Bing
Bing Webmaster Tools, plus IndexNow for new pages.
SEO and Analytics Specialist 1 day
Split the map
Separate sitemaps for programs, news and admissions so each can be monitored.
Web Services Manager, University Communications 1 week
Monitor
Indexed counts by section monthly.
SEO and Analytics Specialist monthly
Why it works
Delete the "Disallow: /sitemap-xml" line, add "Sitemap: https://www.uta.edu/sitemap-xml" to robots.txt, and redirect /sitemap.xml to it. Take the 51 /testzone/ pages out of the sitemap (robots.txt blocks them anyway). Then check the Sitemaps report in Google Search Console and submit the sitemap in Bing Webmaster Tools.
A readable sitemap helps search engines and AI crawlers find new program pages, news and deadlines quickly, at almost no cost. For a university that adds programs and publishes news daily, freshness in search is enrolment.
Ready to use
Copyrobots.txt change
# remove this line: Disallow: /sitemap-xml # add at the end: Sitemap: https://www.uta.edu/sitemap-xml # and redirect: /sitemap.xml -> /sitemap-xml (301)
Risks and how to handle them
The rule was added on purpose (crawl load)
Keep the crawl manageable with a split sitemap and accurate lastmod dates instead of blocking it.
Evidence
- Your websiterobots.txt (checked October 20, 2026) has 15 lines, no "Sitemap:" line, and the rule "Disallow: /sitemap-xml". /sitemap-xml is exactly where the site serves its XML sitemap; /sitemap.xml and /sitemap_index.xml return 404.Source
- Your websiteThe sitemap at /sitemap-xml lists 16,038 URLs (academics 7,287, news 4,313, administration 1,171, admissions 196). 8,750 entries declare a Spanish (hreflang="es") alternate, and in 8,718 of them the "Spanish" URL is the same English page, including the homepage. It also lists 51 /testzone/ pages, a folder robots.txt disallows.Source
- Your websitePrograms: the homepage says "more than 180 degree programs" and the September 17, 2026 enrollment release says "more than 190", but the sitemap has 153 program pages under /academics/programs/. The program finder's page HTML says "No Majors Found" (the list loads by script). Program pages have no meta description and no structured data, and some URLs are misspelled ("adult-gerentology", "psychiatric-mental-health-nurse-practitioner-msm").Source