Attribution: Thomas Shahan
A couple of weeks ago we looked at negative SEO and what you can do to protect your site. One of the techniques we mentioned was hacking. Competitors, upon gaining access to a site, may alter the content or add malware to pages in the hope that Google will delist or penalize a site. Hackers may also simply attempt to use a site to spread their malware without any particular intentions regarding SEO.
We gave a number of suggestions for dealing such an intrusion, but often, after having received a warning from Google, it can be difficult to determine exactly what the Googlebot crawler is seeing. Hackers are adept at making a site appear perfectly normal to those who go directly to a page, while serving malware or undesirable keywords and hidden links to search engine crawlers and those who arrive at a site from a search engine. What you see when you visit a site is not necessarily what Googlebot is seeing.
Fortunately, Google make available a very handy utility that allows webmasters to see exactly what the Googlebot is seeing when it crawls a site. “Fetch as Googlebot” is part of the Google Webmaster Tools, and can be found in the left-hand menu of the main Webmaster Tools dashboard under Health. To use “Fetch as Googlebot”, simply submit a link, and Googlebot will attempt to recrawl the page. If it is successful, it will return information that can be very helpful for diagnosing problems with the page. If you don’t see any problems, it’s possible to directly submit the crawled page to the index.
Among the information that “Fetch as Googlebot returns is:
- The HTTP response that the server returns to Googlebot,
- The data and time of the request,
- The page’s HTML code,
- The first 100KB of visible text on the page.
Using this information webmasters can examine what Googlebot is seeing, and determine the causes of any malware warnings and undesirable content that might be being served. The tool is also quite a handy utility to have in the SEO arsenal. Allowing a bot’s eye view of site content makes on-page optimization easier. Check out the Fetch as Google support page for more information.
Let us know in the comments below whether you find “Fetch as Googlebot” to be a useful tool.
Related articles
