M
Mojeek
Mojeek publishes a JSON file of MojeekBot IP prefixes. The bot respects robots.txt and limits crawl rate to one page per second. Verify with rDNS: legitimate MojeekBot IPs resolve to *.mojeek.com hostnames.
Mojeek publishes a JSON file of MojeekBot IP prefixes. The bot respects robots.txt and limits crawl rate to one page per second. Verify with rDNS: legitimate MojeekBot IPs resolve to *.mojeek.com hostnames.