# /robots.txt for info.webcrawler.com User-agent: * Disallow: /cgi/ Disallow: /antigua/jolly-beach/ Disallow: /antigua/longbay/ Disallow: /antigua/siboney/ Disallow: /antigua/stjamesclub/ Disallow: /antigua/the-suites-jolly-beach/ Disallow: /*.taf$ Disallow: /jamaica/*.html$ Disallow: /turks-caicos/*/default.html$ Disallow: /turks-caicos/*/rooms.html$ Disallow: /turks-caicos/*/more.html$ Disallow: /turks-caicos/*/map.html$