How I (and Stockwits) Feel About Scrapers and Machines in an AI World

Good afternoon from Coronado…

Before I dive in, yesterday I shared Roy Rubin’s post on ‘Intentless Commerce’. It got a lot of great conversations going in my email from smart people in AI and E-commerce. Roy reached out and shared a longer interview/video he just did on the e-commerce and intentless commerce. A must listen if you are in the e-commerce space and wondering about the future as ai and ai agents explode…

Onward…

The future might be AI-driven, but it shouldn’t be built on scraped content…

As Stocktwits co-founder and CEO, I have long been at a serious disadvantage in this digital era because I don’t fully grasp the power of machines and technology. I am a lover not a geek!

I love machines, but they are mischievous when left to people like Sam Altman, Satya, Elon, Google, Elon, Zuck and Wall Street etc etc…

Elon is on all sides of this because he is Elon. He told us ‘only he’ could fix the bot problems at Twitter and bots have increased tenfold…so what chance does an old ‘lover’ like me have against the machines in an AI era?

Short answer…not much.

Long answer, you must constantly adapt and spend to defend the castle and grow.

Luckily, I have some friends in high up places around the internet, and about a month ago I pinged Matthew Prince, co-founder and CEO of Cloudlare ( $NET ( ▲ 0.37% )  ) to chat about the future of AI. Specifically, how Stocktwits should be thinking about the machines and behemoths running the machines like Open AI, Anthropic and Gemini as they scrape our website to hoover our content and data into their LLM’s and spit out to their customers with little or no attribution, let alone monetary consideration.

He told me about a new product they were launching – which is live today – that would combat some of this. They announced the product yesterday, on what they call ‘content independence day’.

Cloudflare just became the first major Internet infrastructure provider to automatically block AI crawlers from accessing websites without permission or compensation. 👏 

Welcome to Content Independence Day! No AI crawl without compensation.

— Matthew Prince 🌥 (@eastdakota)
1:02 PM • Jul 1, 2025

I am not a Chat GPT hater. Ellen and the kids use Gemini and Chat GPT all day. The AI/LLM magic trick bundling endless machines with code and all the world’s content with giddy venture capital is wonderful, but it will lay waste to the creator boom at hand if we don’t set up some rules of play.

Here is my quote as a customer and partner of this with Cloudflare…

“At Stocktwits, original content and real-time conversation are at the heart of our community. As AI transforms how information is discovered and used, it’s essential that innovation doesn’t come at the expense of the creators and platforms driving it. We’re proud to support Cloudflare’s efforts to ensure a more transparent, fair, and sustainable Internet.”

I am not so naive to think Stocktwits can beat the machines so we give a little and battle a lot. I am ok with the machines having a lot of our content. I want people to be able to ask Chat GPT ‘What is the BEST site for stock market sentiment?’ Stocktwits comes back number one. BUT, too many obvious questions that should include Stocktwits recommendations or links or data do not even list us.

That is the battle. It is a big one and a complicated one and one that will be never ending because you cannot tire the machines.