|0°C
Next edition scheduled
Your Personal Daily Intelligence
Edition 2026-08-31

TECHNOLOGY

Git Kernel Org Experiences Heavy Load from AI Crawlers, Deploys Challenge to Block Traffic

Git.kernel.org receives approximately six million daily requests, most of which are automated crawlers that render git commit history as HTML.

By Hacker News · 54d ago · Source: Hacker News

Full article

Git.kernel.org receives approximately six million daily requests, most of which are automated crawlers that render git commit history as HTML. This process consumes significant CPU resources, with 14 of 90 total cores dedicated to rendering commits, representing about 20 % of the system’s capacity. Anubis proof‑of‑work challenge blocks roughly two‑thirds of the requests; the remaining requests succeed after solving a difficulty‑based math puzzle. Legitimate user traffic is estimated at two % of total requests, while the majority are scrapers seeking the full commit history for training large language models. The project has disabled certain features and limited anonymous access to reduce expensive URL exposure and computational load.

Source transparency

Publisher
Hacker News
Reliability
high
Published
8/31/2026, 10:00:23 AM
Retrieved
8/31/2026, 10:00:23 AM
Relevance
80%
Confidence
85%
Read original at Hacker News

Botwin's Morning Wire publishes the full source article for reading convenience. Please visit the publisher for the original presentation and any updates.