Google Penalizes Lots of Large Websites
So when Google introduced the "Farmer Update" which was about 3 months ago now their was a lot of reaction that while it helped with duplicate content sites that copied stuff with RSS feeds etc it didn't really remove auto generated content that sites like FreshWap, MechoDDL and loads of other DDL sites and DLE sites use.
It was announced unofficially though staff working on the project that they were working on auto generated content but it wasn't in that major update but was due soon. Well that update is now live and a lot of sites have being affected.
Basically what loads of sites used to do was save the keywords that people searched for on their site. You could click on them and it would show a page usually with links to cash sites. What most scripts by default or at least should do is send out a 404 header meaning the page doesn't exist. This tells the search engines it's an error page and their's no need to index the content.
By removing the 404 header and sending out a 200 header your telling Google and other search engines the page or what ever you were looking for was found and so Google will index it.
This allows you to have not just hundreds but MILLIONS of pages to be indexed and can bring huge traffic. The reason it brings in such good traffic is your being indexed for things that don't exist and therefore have no competition for those keywords.
So Google has finally copped onto these soft 404 pages as they are known and is now penalizing sites who have them. You can check if a site is penalized by using tools such as Google Penalty Checker.
Examples of big sites affected
Site: HQ-Uploads.com
Google: Search Google
ScreenShot: http://i.imgur.com/ysgs3.png
Site: FreshWap.net (note it now redirects to .com)
Google: Search Google
ScreenShot: http://i.imgur.com/MglNy.png
Site: Heroturko.org (note it now redirects to .com)
Google: Search Google
ScreenShot: http://i.imgur.com/5O8bS.png
Site: downtr.net (note it now redirects to .org)
Google: Search Google
ScreenShot: http://i.imgur.com/sk7TG.png
Site: KickAssTorrents.com Now kat.ph
Google: Search Google
ScreenShot: http://i.imgur.com/HFna4.png
Site: MechoDDL.com
Google: Search Google
ScreenShot: http://i.imgur.com/78h4Q.png
Site: TorrentDownloads.net
Google: Search Google
ScreenShot: http://i.imgur.com/CkkTY.png
Site: LatestDown.com
Google: Search Google
ScreenShot: http://i.imgur.com/aJn9l.png
Site: PassionDownload.com
Google: Search Google
ScreenShot: http://i.imgur.com/X9pco.png
Site: FreshDL.com
Google: Search Google
ScreenShot: http://i.imgur.com/hNeQB.png
Site: TorrendDownloads.net
Google: Search Google
ScreenShot: http://i.imgur.com/hNeQB.png
The list goes on and on and on. It's pretty much every major indexing site and I expect more to follow over the next few weeks. At the moment it's mostly major sites that are being affected with smaller sites slowly being affected.
So what can you do and what does this mean for the future of not only warez but of Search in general?
Well for starters it's the begining of the end of most DDL sites. Most DDL Sites get traffic to their sites with this method. PhazeDDL will be slightly less affected but it will be affected. This of course affects smaller sites who rely heavily on DDL sites for traffic.
This isn't just affecting warez or download sites but these were in general the sites that used the technique the most.
I've about 20 small under the radar sites that currently use this technique and have lots of different tests running in an attempt to find out how Google detects this. Out of about 20 sites two have being got a penalty in the last 24 hours so I'm still researching and it will be some time.
If you have real websites who don't use this technique you will probably find your organic traffic increase ever so slightly due to the removal of others. It will also be better for people as they search for items in Google as lots of insignificant pages won't be returned.
The peanalty received by most sites is -30 which from my research reduces organic traffic to about 30% of what it used to be but what's critical is most of these pages which used to bring traffic are removed so the drop in traffic is far greater to as low as 10%.
The bad news is their is no quick way to reverse this. Sites will have to modify their code to re-add the 404 header and the method of allowing searches to be indexed. Once you have this done you can fill out a reconsideration form in Google Webmaster Area. You pretty much have to beg Google for forgiveness, say your sorry, that you've fixed your site and you'll never do it again. If your lucky your get the Penalty removed but your traffic will never return to what it once was.
FreshWap and Heroturko have tried a different approach and are now redirecting to the .com instead of the .net and .org respectively.
I don't think this will work in the long run as the penalty will simply transfer to the new domain if they don't fix the issue.
Sending a Reconsideration Email to Google to get the Penalty Removed
First your strongly advised to read the Google Webmaster Guidelines
http://www.google.com/support/webmasters/bin/answer.py?hl=en&answer=35769
If you want the easiest thing is to just watch this video
http://www.youtube.com/watch?v=G959-2RlPyA&feature=player_embedded#at=18
Once you have had a look through them you can send them an email
http://www.google.com/support/webmasters/bin/answer.py?answer=35843
Update Concerting Forums
This doesn't really affect forums so if you have one or are saying the big forums are unaffected it's because this relates mostly to DLE (DataLife Engine) and DDL (Direct DownLoad) sites or other sites who use the recent searches techniques. I don't know of any forums who use this technique nor has anyone replied with a forum yet that uses these techniques so obviously this doesn't affect or concern them besides the fact they could potentially see more traffic.
Last edited:
