AI: 'When it Rains it Pours', some less noticed AI trends. ARD #90.

Michael Parekh AI: Reset to Zero

1 day ago

11 MIN READ

Today’s theme: ‘when it rains it pours’ times in the AI Tech Wave — three disparate realities around the current bull market worth understanding. AI coders getting too productive for their own systems. AI forward-deployed engineers (FDEs) flooding enterprise customers. And big tech’s booming stock-compensation taxes — rivaling the AI infrastructure spend itself. Three Takes today, each with my Take.

(1) Coders ‘Too Productive’ With AI Coding — Causing Computing Overloads

The Information had it — “OpenAI coders’ AI coding use overwhelms internal systems” — the unintended consequences of Codex’s comeback: OpenAI’s own coders finding their productivity up 2x-3x or more, and instead of bringing updates into internal and external systems once or twice a week, it’s now multiples of that — overloading systems, interrupting internal users and external customers, and creating a whole new set of issues to program around. And for mainstream users: when your favorite product or app feels a little slower, it may not be AI compute limits on your plan — it may be the company’s AI-augmented coders uploading multiples of the usual updates. The AI agents arrive in the enterprise frame is in AI-RTZ #494.

MP Take: As more companies beyond big tech and the AI companies embrace AI tools that make their coders and other employees more productive, expect similar traffic surges in internal systems — especially as firms begin to automate much of software development. Such tools have already caused a surge of traffic to GitHub, which has led to outages and other issues at Microsoft recently. These incidents highlight a broader trend in the offing.

(2) AI Forward-Deployed Engineers (FDEs) Flood Enterprise Customers

The Information ran the piece — “Why Forward Deployed Engineers Are the Rage.” FDEs became popular via Palantir on the defense side, and now everyone across Silicon Valley has them — with OpenAI and Anthropic creating multi-billion-dollar partnerships with consulting companies and private equity to flood the enterprise zone at their customers: helping them figure out, implement, and maintain all these new AI technologies at scale. The people doing it are often better paid than other engineers at the same companies. I covered AI’s FDEs going from engineers to entities in ARD #68.

MP Take: Expect this FDE trend to be a multi-year phenomenon. FDEs are a new buzzy term for an old IT phenomenon prevalent for decades. New enterprise technologies require a whole host of computer software and services that by itself amounts to trillions in additional spend by businesses — on top of the investments in the ongoing compute hardware and software. AI FDE services are the AI-dressed-up version of a multi-decade enterprise software-and-services reality. India’s IT industry will have a role to play too here at scale before too long. Contrary to investor concerns of AI impact on India’s global tech services business.

(3) Big Tech’s Booming Stock-Compensation Taxes in a Bull Market

The Information had the finance read — “Alphabet’s fine print reveals hidden cost of AI talent war” — of Google’s $80B+ equity raise (about what SpaceX expects from its mega IPO), some $30-40 billion is headed to stock-based-compensation taxes — the arcane reality where tech companies pay the taxes on employees’ vested shares, big cash inflows to the IRS. Across all of big tech the numbers approach $200 billion — not far from what each of these companies spends per year on AI data-center capex. The big tech AI talent run and options frame is in AI-RTZ #769.

MP Take: Given the prevalence of stock-based compensation in the tech industry, expect this trend to continue beyond this year as well — especially as the secular bull market in AI runs parallel with the financial bull market in the same.

Then there’s the larger issue of big tech stock-based compensation (SBC) and its accounting by Wall Street analysts — especially when it comes to non-GAAP-based valuation methods that need to be watched in vigorous bull markets. Very different from my day running Goldman’s Internet Research in the nineties. But that’s a topic for another day.

For now, it’s notable that big tech has to finance not just tens of billions of AI data centers and power, but also stock-compensation taxes that rival the AI investments.

Gadget AI — Google’s Gemma SLMs on 16GB (or 8GB) of Local RAM: the New Local-AI Inference Thing

Small Language Models (SLMs) that work with 16GB of local RAM or less — even 8GB — are becoming the new local-AI-computing-for-inference-tokens thing.

Nvidia’s DGX Spark chip, announced with Microsoft this week, is the hardware side of the puzzle on Windows devices. Apple, with iOS running on Apple Silicon, is the other side of the device spectrum. Now we need SLM models optimized for local AI inference — and VentureBeat had the model side: “Google’s new open-source Gemma 4 12B analyzes audio + video and runs entirely locally on a typical 16GB enterprise laptop.” We’re talking hundreds of millions of laptop units, then smartphones — a force that could dent the loads on AI data centers and the cloud: meterless computing instead of a-la-carte pricing. Apple’s iPhone chips powering the new value MacBook Neo with 8GB RAM is in AI-RTZ #1017.

MP Take: More efficient AI chips that run on local devices with small amounts of RAM, plus more capable SLMs doing local inference, are a key step for the AI Tech Wave ahead. Good to see Google, Microsoft, Nvidia, Apple and others focusing on this important end of the spectrum.

Questions

Q1 — What is MP’s favorite local SLM AI application?

Apple Intelligence is likely the closest — Apple runs a lot of small, discrete models working with personal information locally on Apple chips, with the AI-inspired Siri with Google Gemini expected at WWDC next Monday. The same is true for Google Android phones running local apps like Gemini Nano — very efficient, with data privacy in mind. But broader, mainstream SLM applications are still on the horizon — either from Google, Apple and the other tech companies, or from AI startups and app developers.

Q2 — What is MP’s most-wished-for local SLM AI application?

An app that processes my screenshots — across iPhone and Android phones.

I take screenshots of everything around me, every day — and have for years. Over half of my almost 100,000 photos are screenshots: articles, papers, everything that catches my attention. All waiting for a time when SLMs — or LLMs for that matter — can securely use them as a continuous data input on my interests and activities, personal and work.

There is a gusher of personal attention information in these screenshots — waiting to be harvested and used as productivity fuel. Waiting for a customizable local app and service to do its automagic. I think we’ll see more of this in the next year or two.

Source Reading — For the Full Context

For the full context, see the canonical sources:

Shorts Clips from today

Clip 1 — AI Coding: Too Productive?

Watch on YouTube Shorts

OpenAI’s own coders found their productivity jumping 2x-3x or more with AI coding — and instead of bringing updates into internal and external systems once or twice a week, it’s now multiples of that. Interruptions to customers and internal users, overloaded systems, and a whole new set of issues to program around.

MP Take: This is going to be true of every company, large, medium and small, in the next two or three years as they use AI to update their systems. Such tools have already caused a surge of traffic to GitHub, leading to outages at Microsoft. A broader trend in the offing.

Clip 2 — Big Tech’s $200B Stock Tax

Watch on YouTube Shorts

Of Google’s $80B+ equity raise, some $30-40 billion is headed to stock-based-compensation taxes — the arcane reality where tech companies pay the taxes on employees’ vested shares. Across all of big tech, the numbers approach $200 billion — not far from what each of these companies spends per year on AI data centers.

MP Take: Expect this trend to continue beyond this year, as the secular bull market in AI runs parallel with the financial bull market in the same. Big tech has to finance not just tens of billions of AI data centers and power, but stock-compensation taxes that rival the AI investments.

Clip 3 — AI Updates Overload Systems

Watch on YouTube Shorts

Why this matters for mainstream users: when your favorite product or app feels a little slower, it’s not just that there may not be enough AI compute on your current plan. It could be that the company’s coders — newly augmented by AI — are uploading multiples of the usual updates and slowing things down.

MP Take: This phenomenon has not yet been thought through in its implications. As companies large and small use AI to update their systems over the next two or three years, this becomes everyone’s reality — not just OpenAI’s.

Clip 4 — FDEs: AI’s Hot New Job

Watch on YouTube Shorts

Forward-deployed engineers — FDEs — became popular via Palantir on the defense side, and now everyone across Silicon Valley has them. OpenAI and Anthropic have created multi-billion-dollar partnerships with consulting companies and private equity to flood the enterprise zone at their customers. The people doing it are often better paid than other engineers at the same companies.

MP Take: This is basically the AI-buzzy version of something that’s been true in IT services for decades — a multi-trillion-dollar global industry. The FDE trend will be a multi-year phenomenon, then it’ll normalize into forward-deployed AI software and services.

AI Ramblings Daily on AI-RTZ is here to think through AI and reset. Together.

Today’s AI-RTZ #1107 — Meta Tries Mixed AI Services for Consumers and Businesses — on Meta’s AI Agent offerings for consumers and businesses, with its MSL properties, Muse Spark AI and Hatch AI agents. If you’re one of the 3.5 billion people on WhatsApp, Facebook or Instagram, you’ll be seeing a lot of these things in front of you.

Tomorrow — ARD 91 on AI-RTZ #1108. And Friday, of course.

Thanks for joining us, AI Curious Folk. Stay tuned.

(NOTE: The discussions here are for information purposes only, and not meant as investment advice at any time. Thanks for joining us here.)