User-Agent: * Crawl-Delay: 20 Disallow: /admin/ Disallow: /conf/ Disallow: /cross-network/ Disallow: /languages/ Disallow: /js/ Disallow: /functions/ Disallow: /logo/ Disallow: /logs/ Disallow: /other/ Disallow: /resumes/ Disallow: /job_bulk_post/ Disallow: /cron_jobs.php Disallow: /cgi-bin/ User-agent: Googlebot Allow: Sitemap: https://jdhuntr.com/sitemap.xml