Google says it will continue to allow other websites to opt out of web crawling after its agreement with the FTC expires
- Google's agreement to allow sites to opt out of its web crawling expires December 27.
- In a letter to the FTC, Google promised to uphold the agreement moving forward, which was originally put in place in 2012 after an antitrust investigation.
- It's a win for sites like Yelp that claim Google uses content that doesn't belong to it on search results pages.
Google said Tuesday that it will maintain an agreement to let sites opt-out of web crawling by the search giant, even though the agreement expires on December 27.
In a letter to the FTC, Google said, "We believe that these policies provide additional flexibility for developers and websites, and we will continue them as policies after the commitments expire."
Google originally made the legally binding agreement in 2012 following an antitrust investigation against the company. The agreement was part of its settlement with the US government.
Websites like Yelp have complained over the years that Google unfairly "scrapes" content from their websites and monetizes that content on search results pages. The agreement lets sites block Google from scraping their content for use on search results pages.
But Yelp accused Google in September of violating the agreement. In a letter to the FTC, Yelp said it found thousands of cases where Yelp images appeared in search results.