About our crawler

User agent: Mozilla/5.0 (compatible; MailHunterPro/2.0; +https://mailhunterpro.app/bot)

When a MailHunterPro user runs a search, our crawler visits the public pages returned by the search engine, plus up to three linked contact, staff or about pages on the same site. It reads only publicly available HTML. It does not log in, submit forms, run scripts or follow links off the site.

How it behaves

• It checks robots.txt before every page and obeys Disallow rules for MailHunterPro or *.
• For each search result it reads that page plus at most three linked pages on the same site, a few at a time, with a 9-second timeout per page.
• It skips files that aren’t web pages, such as PDFs and images.

Block it

Add this to your site’s robots.txt:

User-agent: MailHunterPro
Disallow: /

To remove a single address from results, use our opt-out page. Questions: rockrevstermusic@gmail.com.