Are XML Sitemaps Files a Welcoming Door in for Scrapers? (Search Engine Roundtable)
Are XML Sitemaps Files a Welcoming Door in for Scrapers? (Search Engine Roundtable)
An excellent WebmasterWorld thread asks if the new Sitemaps Auto-Discovery supported by all four major search engines is not just an easy way for search engines to find and index your content. The thread asks if this enables scrapers to easily find and scrap your most important content? There is no doubt in my mind that having an XML feed helps scrapers do their work. That is part of the …