News Score: Score the News, Sort the News, Rewrite the Headlines

AI crawlers haven't learned to play nice with websites

SourceHut, an open source git-hosting service, says web crawlers for AI companies are slowing down services through their excessive demands for data. "SourceHut continues to face disruptions due to aggressive LLM crawlers," the biz reported Monday on its status page. "We are continuously working to deploy mitigations. We have deployed a number of mitigations which are keeping the problem contained for now. However, some of our mitigations may impact end-users." SourceHut said it had deployed Nep...

Read more at theregister.com

© News Score  score the news, sort the news, rewrite the headlines