User-Agent: * Crawl-delay: 30 Disallow: /*/filter$ Disallow: /admin Disallow: /checkout Disallow: /cart