Featured Posts

NostalgiaNostalgia A few years ago, I added a shortcut to this blog on my web browser favourites bar. I accidentally clicked it tonight and found out it still exists. Therein started a bizzare existential adventure whereby...

Read more

A Premiere At Last! (My Toils Relearning Video Production)A Premiere At Last! (My Toils Relearning Video Production) In the last few months I've been asked/put myself forward to film and live stream some events. During my rather weirdly-spent youth, I was producing videos for my school of its plays and other staged...

Read more

Why I Keep Banging On About Google - An UpdateWhy I Keep Banging On About Google - An Update Today is the 90th day since one of my sites was penalised by Google. Wanna know what that looks like? Take a look at this! One of the more frustrating things about this situation - and don't...

Read more

Taking the downs with the ups...Taking the downs with the ups... It's been a while, and a blog is just for moments like this. Tempting though it is, a blog shouldn't be just for one's successes. If you don't include the failures you're painting a false picture. So...

Read more

Free Speech is Dead (or it will be when I've bombed it)Free Speech is Dead (or it will be when I've bombed... Before I start - I want to admit I have a hard time putting my exact feelings on this subject into writing. Not because I'm not feeling 'free' enough to do so, but because I'm a computer networks graduate...

Read more

Woohoo Post & Google Penalty Help (now I’m seemingly out again)

Posted by Pat | Posted in 'Work' | Posted on 13-06-2011

0

There’s nothing more frustrating as a webmaster when your site inexplicably disappears from Google’s search results in a sudden and devastating way. As you might have had the misfortune to wade through, I’ve had problems this year with one site and tried over several months to figure out where the site was going wrong.

I have to eat humble pie on this one. Advice from Google’s own mouth (Well, mostly Matt Cutts) turned out correct.

My biggest suspicion is that the site had been penalised because of bad links. It hadn’t. I was also suspicious the problems were caused by someone copying my site and sending duff traffic to it. It wasn’t.

My problems were the result of an algorithmic change on or around the 12-14th January 2011 – tripping my site into the wrong side of a filter. The change was seemingly was related to ‘SafeSearch’, the search results ‘adult content’ filter created by Matt Cutts himself. I can’t help think he must have slightly revelled in having caught me if he took an interest in my tweets and forum threads over the last 5 months.

I say ‘caught me’, I haven’t really done anything wrong at all. The niche of the site has adult overtones and I was catering for those visitors. In fact there are several sites ranking the exact same terms which must have a higher ‘trust’ level as they do exactly the same practices and weren’t penalised.

No hard feelings on the part of Matt by the way – I like him and what he does as the Google Webmaster PR/Spin merchant. He also does engage as much as he can and I’m grateful for that – I never really felt he was ‘against’ my problem, and the more I listened to the repeated advice from Google, the more it sunk in.

So, I’m going to reveal something completely innocuous and unlikely sounding which paved my way out of the penalty.

Recovery to traffic levels near to January 2010's

Firstly let me tell you what didn’t work – but what also didn’t seem to harm the site further:

  • Removing tag cloud from blog – making 1000 pages ‘404 Not Found’ but with intention to eliminate duplicate content
  • Truncating the blog posts on the front page of the blog so all the words of a post didn’t appear in several places
  • Adding ‘noindex, follow’ meta tags to duplicated areas of the Blog (Page 1, 2 etc, ‘archive’, seperate category pages)
  • Removing ‘doorway’ page for link to affiliate site
  • Removing any ‘old’ files which were no longer linked to internally but indexed with Google
  • Reducing keyword density on the front page / Titles / Meta Tags
  • Cleansing all my data (part of which was going through 1500 external links on the site individually to check them)
  • ‘Nofollowing’ all external links
  • Implementing ‘Canonical’ meta tags everywhere
  • Adding new features and content (which generated around 2000 new pages)
  • Moving the site to a dedicated VPS / IP Address
  • Chasing problematic backlinks and removing any files they linked to which I believed caused the issue
  • Badgering the Webmaster Spam team with Reconsideration Requests

I can’t say that none of those didn’t help – obviously when the site bounces back any of the above could have given the site some brownie points – but in my experience, having done all the above (and likely more I can’t remember) there wasn’t even a flicker of a recovery. In fact, the traffic was still on a slow decline from Google right up until I figured it out. International Panda on 11th April seemed to dampen my efforts a small amount too, so my partial recovery is probably where I would have found myself if the original penalty had never happened. In fact the level of the recovery is almost the same as the last bounce back from a penalty in January 2010 – and as you can see on the graph, the site grew well throughout the year, so I’m crossing my fingers for a similar pattern.

A few keywords (especially ‘single word’ ones) haven’t returned to their former glory, but I think that could be the Pandarising effect, or as a result of my deoptimization.

I’ve also lost (probably permanently) my ranks for anything adult related. This is on purpose, because of the way I got out of the penalty, and it really was this simple:

  • I removed anything with the word ‘adult’  in it, or any phrases in the niche which might sound ‘adulty’ – e.g. ‘<niche> chat’ – within internal anchor text, alt tags, page filenames, titles, descriptions, everything.

I know. That’s it – one line of simplicity. To put this in perspective and why as obvious as it sounds, I didn’t do it until very late on is that:

  • There are other sites which still rank fine for the same (or very similar) niche, which still contain the ‘adult’ word and phrases in them
  • The penalised site had run these ads (via an internal doorway page) with the same anchor text (or similar) for the last decade.
  • When the site last emerged from a penalty (Jan ’10) it still had these exact same ads on it

How do I know that did it?

Google Web Preview.

Look for the magnifying glass next to your site’s pages in the search results. If you’re searching with everything ‘default’ – that is, Safesearch is set to ‘Moderate’ and your magnifying glass is missing, you have a problem. Google thinks the content on that page (or site) has adult content which might not be safe for the Web Preview to show.

This actually isn’t the case for the penalised site. There were adult (text) links as a means of steering traffic looking for that elsewhere, whilst maintaining a completely mainstream site and content – that is, there were no adult pictures which someone would find objectionable if they appeared in the web preview within the search results. Matt Cutts does talk about people seeing adult stuff in pages with otherwise mainstream kw terms as ‘accidental porn’ or similar, and that it’s not liked – but in the niche I’m in, it is well known as a mainstream and adult niche… so there is some ambiguity and my running of those ads is almost expected and not a problem with users (never received a single complaint). So in a way, I’m battling an algorithm that basically doesn’t understand the niche, or it believes it to be more mainstream than not.

I was able to replicate the fix to the problem by using another site as a test bed. The site suffered the same issue of the magnifying glass not being present. I removed any ad with ‘adulty’ anchor text. Literally the next time they were crawled, the magnifying glass came back. I could tell that I’d got it back as a result of doing this because any of the pages still without magnifying glasses had an old cache which still contained the ad on it, and over time each one regained their magnifying glass as the new ad-less version was indexed.

Whilst that site didn’t seem to be in a penalty, it got so little traffic I couldn’t test whether the reappearance of the magnifying glass was having much of an effect on traffic. But anything was worth a try so I removed every single ad on the problem site except the front page. (It was a sitewide side bar link)

Two reasons for doing this – first to see if I could replicate the return of the magnifying glass, and second because I didn’t want to earn $0 from the site and couldn’t forsee a way back into placing ads, without being penalised, so I kept one so my residual traffic would at least earn me a little and not all was lost.

This takes us up to the end of my previously updated and ridiculously long blog post, by the way.

Over the next two weeks,  the internal pages without ads slowly got cached and their magnifying glasses returned.  Within days, I noticed more long-tail traffic appearing to those pages, including some fresh, previously unobtained KW terms from the new content I had created. I think that’s cracked it, gromit!

I bit the bullet and removed the last ad – so the site was completely unmonetized. Sure enough, magnifying glass came back and traffic to the front page came back within a few days.

Amazing! But not great. So, basically I can’t run advertising on there any more – or at least the same kind of advertising for which I have over time negotiated a fantastic rate with the merchant (and hence very retiscent to try something much less productive like Adsense).

Coupled with the removal of ‘adulty’ terms (by the way, the word ‘adult’ is the most ‘adult’ they got!) removing any ranking for those terms, the traffic I got back was less, and obviously I’m left in a quandary on how to proceed and monetize the traffic again. I’m working on that.

In conclusion

I’m writing this because regardless of the consequences I feel I should complete the story after a lot of crap on here and hope it can help someone, somewhere in a similar position.

Do I think the penalty was fair? On one hand, not really – Google doesn’t ‘get’ the niche – the affiliate link does add value to the site in that sense because users turn up looking for the adult side of things. On the other hand, yes the penalty was fair – I ranked for terms which essentially I sent people away from the site to see. I don’t do that any more, so the site feels much more squeaky clean, SEO wise and otherwise. Adding in all the changes I did for the better, the penalty was a good kick up the arse for that site and a salient lesson in what Google’s up to.

Would it have helped if the communication within Webmaster Tools was better than even their most recent revamp? Yes. Telling me my problems were down to something adult being detected would have helped me from day 1.  Someone less well intentioned though could probably use the information to manipulate things too much. It must be a fine line for Google to be nice to honest webmasters or give spammers a field day. I understand that, regardless of all the frustrations.

I’ll shut up now, I promise.  My girlfriend considers Matt Cutts as the 3rd person in our relationship and I have become just a little obsessed about Google’s little fluctuations and changes, reading everything there is to read, every day. I think I’m going to slow down on that a bit now… until the next time!