![Reddit stock photo Reddit stock photo](https://i0.wp.com/www.androidauthority.com/wp-content/uploads/2021/09/Reddit-stock-photo.jpg?w=640&ssl=1)
- Google is currently the only search engine that’s able to crawl Reddit and display recent search results from the platform.
- This further cements the company’s position as the dominant search engine and makes it harder for people using alternative platforms to access a good amount of user-generated content.
- This has once again sparked concerns about Google’s monopolistic practices.
Well, in a nutshell, it means searching for Reddit content using “site:reddit.com” will no longer yield results on these alternative platforms (via 404 Media). The reporting outlet found that DuckDuckGo is turning up a handful of Reddit search results for now, but there’s no further data on where the links go. Older Reddit links, from prior to last week, that is, still come up, but because alternative search engines are unable to crawl Reddit anymore, Google is your best bet for finding the most recent Reddit content.
So, why does this matter? The fact is that Google has, for some time now, been accused of monopolistic practices. By securing exclusive access to crawl Reddit, the tech giant has further consolidated its position as the dominant search engine. Additionally, given that the quality of search results on Google hasn’t been impressive in the recent past, the company’s move makes it harder for people to migrate to other platforms for a better search experience without compromising access to a good amount of current user-generated content.
404 Media connected with Colin Hayhurst, CEO of search engine Mojeek, who noted that his platform was blocked from crawling Reddit starting in early June. While Hayhurst has attempted to contact Reddit to resolve this issue, he has, unfortunately, yet to hear back.
In the past, Reddit has been vocal about how companies are scraping its platform to train their AI models. Over time, the company began introducing stringent measures to stop AI firms from being able to exploit its content without authorization. In June, the company posted, “In the next few weeks, we’ll be updating our robots.txt instructions to be as clear as possible: if you are using an automated agent to access Reddit, you need to abide by our terms and policies, and you need to talk to us.”
Reddit seems to have gone ahead with this and updated its robot.txt instructions, and the timeline aligns with the recent restrictions that alternative search engines have observed. We’ll have to wait and watch to see how much of an impact this has on smaller search engines and users who want a different search experience than what Google can offer.
You might like
Discover more from TechPros: Innovate, Learn & Connect
Subscribe to get the latest posts sent to your email.