Wikipedia urges AI companies to use paid API and stop scraping | TechCrunch

Wikipedia on Monday presented a simple plan to ensure the website remains supported in the AI ​​era despite declining traffic.

In a blog post, the Wikimedia Foundation, the organization that runs the popular online encyclopedia, called on AI developers to use the content “responsibly” by ensuring that contributions are properly attributed and that the content is accessible through the paid product, the Wikimedia Enterprise platform.

The opt-in, paid product allows companies to use Wikipedia’s content widely without “severely burdening Wikipedia’s servers,” according to the Wikimedia Foundation blog message explains. Additionally, the paid nature of the product allows AI companies to support the organization’s nonprofit mission.

While the post doesn’t go so far as to threaten punishment or any form of legal action for using the material through scraping, Wikipedia recently noted that AI bots had been scraping its website while trying to appear human. After updating its bot detection systems, the organization discovered that the unusually high traffic in May and June came from AI bots trying to ‘evade detection’. Meanwhile, it said the number of ‘human page views’ had fallen by 8% year-on-year.

Now Wikipedia is laying out its guidelines for AI developers and providers, saying that generative AI developers must provide attribution to give credit to the human contributors whose content they use to create their output.

“To ensure that people can trust information shared on the internet, platforms must make it clear where the information comes from and increase opportunities to access and engage with those sources,” the message reads. “With fewer visits to Wikipedia, fewer volunteers can expand and enrich the content, and fewer individual donors can support this work.”

Earlier this year the organization brought her AI strategy for editorsthat said it would use AI to help editors with workflows around tedious tasks, automating translations and other tools that help editors, not replace them.