ShieldFont and the challenges of web scraping
Although ShieldFont pages remain easy for people to read, using the font on a published website can affect how digital tools interpret the content. Search engines, screen readers, copy-and-paste utilities, and translation software may struggle with modified HTML, potentially making the page less accessible and useful to visitors.
credit:
ShieldFont
ShieldFont is not a completely foolproof anti-scraping solution. AI scraping systems can still interpret human-readable pages by rendering the site in a browser and using optical character recognition (OCR) to analyze the resulting image.
However, browser rendering and OCR require considerably more processing than extracting text directly from a page’s raw HTML. Most web scrapers currently collect the source code of billions of pages as plain text rather than simulating a browser and reproducing the full rendering process. Estimates from third-party scraping services indicate that pre-rendering can cost five to 13 times more than standard HTML scraping, creating substantial time and infrastructure expenses for large-scale data collection.
That increase in cost is central to ShieldFont’s purpose. Its creators aim to discourage or slow indiscriminate web scraping, particularly the mass collection of online content for AI training. “Our primary underlying objective is to enforce the fundamental principles of AI ethics: Creators should have a meaningful say in whether their work is used to train AI systems,” they wrote. “If consent is not respected, the technical design may make unauthorized acquisition of that work less useful and more costly. …Discoverable does not mean consent to AI training.”
The creators also hope other developers will build on the same principle: content can be presented one way for human readers and another way for automated machines. As more anti-scraping techniques appear across the web, AI crawlers may need increasingly sophisticated methods to bypass them, making unauthorized data collection more difficult and expensive.
Source: arstechnica.com


