I feel like a post like this needs to begin with a disclaimer.
I suppose in the ideal world disclaimers should not be necessary but there has been plenty of conversations recently about skill shortages in SEO and the tendency for too many account managers to consider themselves experts just because the keep up to date with the echo chamber of opinion and without testing anything for themselves.
I’ve some sympathy for those concerns and so here’s my disclaimer; this post is purely speculation. Sure, it’s based on fact and history but the extrapolation is purely a ‘what-if’ scenario. Hopefully it’s food for thought.
Social Hilltop background
This post is inspired by some of the live coverage from Pubcon. Pubcon is a drinking binge in Vegas in which people are able to make the business case to attend because SEO experts present, on stage, their thoughts and opinions. I’m not jealous that I’ve never been able to make said business case, you understand, it’s just that… well… I should get to go.
SERoundtable has some good, live blog, coverage.
Brian Ussery: Twitter and Facebook are still essentially blocked from Googlebot…
Barry Schwartz:: Lots of people talking about social in terms of SEO and some SEOs are getting ahead of themselves. Google can only see stuff on the open web, not stuff hidden to GogoleBot
I agree with much of that. Modern SEO is not just social SEO. Modern SEO is multi-signal SEO and social is one of the signal collections. In fact, in a very useful interview between Eric Enge and Duane Forrester of Bing, Bing’s Sr Product Manager confirmed that social signals are the second most important ranking factor for them, ahead of links, but behind behavioural factors.
The question, therefore, is Googlebot really blocked from Twitter?
I find it hard to argue that it is. After all, Google has indexed over a trillion pages from Twitter.
Most of those pages, the important pages, are people’s “homepages” or profile pages.
It may be the case that Google does not always get its hands on the individual tweets as fast as they would like. Without the firehose Google may not be able to tell directly from Twitter, in real-time, which pages are popular tweet targets. When Google lost the firehose from Twitter, a few months ago, the world lost real-time search from Google. Despite the growth of Google+ and the brand new launch of Google+ Pages we are yet to see RTS return.
So, can Google cope without direct access to the Twitter firehose? Yes. I’m going to suggest that Google could expand the Hilltop algorithm to the Social Hilltop algorithm and monitor the real-time web of influencers and determine the social authorities.
The Hilltop algorithm is an absolute must know for any SEO account manager or freelancer. It has its own wikipedia page. It was when Google first started talking about authority.
The Hilltop algorithm was created by Krishna Bharat and George Mihaila. It was bought by Google in 2003. That’s right: this was a search algorithm good enough that Google needed to buy it.
The concept is simple. Start with a carefully selected list of “expert sites”. Hand picked, if needs be. Expert sites are those without affiliation, which link to many sites and which have significance of their own.
Sites that many “expert sites” linked to are considered to be “authority sites” by the Hilltop algorithm.
In some ways this is the very first social media search algorithm. Hilltop uses influencers to work out which sites should be ranking for any given keyword.
Krishna Bharat goes on to be employed by Google, be fundamental in producing Google News and LocalRank. Google News, to this day, cares about the influence of your news organisation.
Social Hilltop algorithm
Let’s imagine the Social Hilltop algorithm is used to let Google cope without direct Twitter firehose access. It first needs to find the social equivalent of expert sites.
In the social world expert sites would be those that –without affiliation- surface pages which are popular among the social community. For this blog I’m going to suggest Tweetmeme and Topsy. Both monitor popular URLs as they’re shared on Twitter. They are without affiliation, neither block Google – in fact, both have
Allow: / in their robots.txt and both link to popular pages.
As a result we can find our social authority pages. Google’s crawls will find it simple to identify which pages are considered popular by both Tweetmeme and Topsy. Should Google find this a helpful measurement of engagement, authority or influence – in the way they find the traditional Hilltop algorithm helpful – then these discoveries become an easily inclusion into the legion of ranking factors.
We may, of course, also conclude that Google still has a pretty good idea of what is popular on Twitter by being able to examine that social network directly. Either way; it seems very likely that Google is all too aware of what’s popular and what’s not among today’s blog posts or discoveries.