how to block MarkMonitor,muso.com and entura.co.uk from indexing my site

[andalusy]

hi
this Anti-Piracy sites Sent hundreds of emails every day and the send to my data-center too and i have to delete files or i will lose my server
does any one know ips of spider of this sites or any why to block them ?

 

[z00ms.com]

Edit your robots.txt

 

[gonzalus]

Edit your robots.txt file, copy-paste these codes:
User-agent: Google
Disallow:
User-agent: *
Disallow: /
It will allow only Google bot to crawl your site. You can add Yahoo and Bing if needed

 

[andalusy]

I am still suffering from this problem

 

[Tango]

Do you know any of the IP's trawlers use?

 

[Drinkordie]

Simple editing of robots.txt is not a good solution here, remember with who he deals with If you're using WP this plugin should help you solve the problem - https://ast.wordpress.org/plugins/blackhole-bad-bots/ - if not, you can copy his solution and customize it to work with your script.

Add your own virtual Blackhole trap for bad bots. The Blackhole plugin includes a hidden link to your pages. You then add a line to your robots.txt file that forbids bots from following the hidden link. Bots that ignore or disobey your robots rules will crawl the link and fall into the honeypot trap. Once trapped, bad bots are denied further access to your WordPress-powered website.

I call it the "one-strike" rule: bots have one chance to obey your site's robots.txt rule. Failure to comply results in immediate banishment. The best part is that the Blackhole only affects bad bots: human users never see the hidden link, and good bots obey the robots rules in the first place. Win-win!