#**************************************************************************** # robots.txt # : Robots, spiders, and search engines use this file to determine which # content they should *not* crawl while indexing your website. # : This system is called "The Standard Robots Exclusion." #**************************************************************************** User-agent: * Disallow: /cgi-bin/ User-agent: Googlebot Allow: / User-agent: NinjaBot Allow: / User-agent: Mediapartners-Google Allow: / User-agent: Googlebot-Image Allow: / User-agent: Adsbot-Google Allow: / User-agent: Googlebot-Mobile Allow: / Sitemap: https://www.houstonreptileshows.com/sitemap.xml Sitemap: https://www.houstonreptileshows.com/sitemap_images.xml Sitemap: https://www.houstonreptileshows.com/sitemap_video.xml