The AI Spending Blind Spot
Organizations are pouring money into AI tools while junior engineers rack up token bills nobody expected. The real problem isn't adoption anymore — it's efficiency.
Listen to under 2-min AI recap •TL;DR ↓
The AI Efficiency Gap Nobody's Talking About
The past few months, the internet has been absolutely buzzing about AI. Mostly the same topics recycled across every blog and newsletter — Claude as the centerpiece of every discussion, security leaks making headlines, accidental production database deletions raising eyebrows about questionable integration practices. I took a deliberate pause from writing about the same stuff half the internet was already covering. Content market seems oversaturated, yet a lot is happening too at the same time.
But while I stepped back from the content treadmill, I kept observing. And what I'm seeing now feels worth writing about.
Big Spends, Unknown Returns
Many organizations still consider themselves in the "AI adoption phase." Fair enough. But here's what's actually happening: spending is climbing fast, yet only some are tracking what they're getting back.
The investment into AI tools, tokens, and integrations keeps growing — often justified with "it's nice to have" or "it's fun" or "everyone else is doing it." Some teams do measure returns, and a few genuinely extract value from the exercise. The most common measurement unit? Hours saved per person. Which, let's be honest, is often a rough guess far from reality. But at least it's something.
From there you can translate hours saved into theoretical cash value — people multiplied by hours saved multiplied by cost per hour. A neat spreadsheet that may or may not reflect what's actually happening.
The Organizational Tug-of-War
There's a fascinating tension playing out across the industry right now.
Business leaders are pushing hard for AI adoption. Stakeholders want "smart things" integrated everywhere. Meanwhile, security teams are pushing back — and rightfully so — about data residency, about internal or confidential information leaving the building. Yes, most company data already lives in the cloud, but that's a fundamentally different situation from having data processed by an external intelligence layer. Different risk profile entirely.
The good news: the industry seems to be reaching a more thoughtful stage. Organizations are starting to draft safe AI usage guidelines, investing in AI literacy, rolling out actual staff training. That's progress.
Where the Cost-Value Gap Gets Obvious
Here's something I've personally observed that I think illustrates the core problem beautifully: junior engineers often generate the highest token usage bills.
This isn't a knock on junior engineers. It's a symptom of a gap in understanding how AI actually works — what tokens are, how they're calculated, and how context size directly impacts cost.
AI token usage is driven by your request, yes. But the "I didn't ask for much" excuse for sky-high bills usually hides what's really going on: unscoped prompts, wide-open context windows, throwing many or massive files into context, no clear expectations for the response format, or simply researching across multiple sources. The user's input might be small. The generated response might look reasonable. But what happens in between — the context processing, the back-and-forth, the reasoning chains — can burn through hundreds of thousands or millions of tokens.
That's the cost-value gap in action. High spend, uncertain return, no visibility into why.
Efficiency Over Adoption
My call at this point is simple: stop obsessing over adoption and start focusing on efficiency.
Train people. Not just on how to use AI tools, but on how AI works under the hood — enough to make informed decisions about when and how to use it. Help them understand that a well-scoped prompt with clear constraints will outperform a vague request thrown at a massive context window every single time. And it'll cost a fraction of the price.
The organizations that will get the most out of AI aren't the ones spending the most. They're the ones finding ways to achieve results with less waste.
Tips for Using AI More Efficiently
- Scope your context. Only include files and information the AI actually needs. Smaller context = fewer tokens = lower cost.
- Be specific in your ask. Tell the AI exactly what you want, in what format, and what to skip. Vague prompts are expensive prompts.
- Set response constraints. Ask for concise answers. "Answer in under 200 words" is a legitimate instruction.
- Understand what tokens are. Know that everything — your input, the context, and the output — costs tokens. Awareness alone changes behavior.
- Don't use AI for everything. Sometimes a search engine, a colleague, or five minutes of reading documentation is faster, cheaper, and more accurate.
- Track your usage. If you can't measure it, you can't improve it. Check your token consumption regularly.
- Learn to prompt iteratively. Start narrow, then expand. Don't dump everything upfront hoping the AI figures it out.
Are you tracking what AI is actually costing you — and whether you're getting that value back?
TL;DR
- AI spending is climbing fast across organizations, but almost nobody is rigorously measuring what they're getting back beyond rough guesses at "hours saved."
- Junior engineers often rack up the highest token bills — not because they're careless, but because they lack understanding of how context windows, token costs, and prompt scoping actually work.
- The real organizational tension is between business leaders demanding AI everywhere and security teams rightly flagging the risks of external intelligence layers processing internal data.
- Training people on how AI works under the hood — not just how to click buttons — is the single highest-leverage investment most organizations aren't making.
- Well-scoped prompts with clear constraints will outperform vague requests thrown at massive context windows every time, at a fraction of the cost.
Stop measuring AI success by adoption rates and start measuring it by cost-per-outcome — efficiency and value is the game that matters now.
Geddy
Senior Web Engineer / Lead
Engineering leadership • AI innovation • Product thinking. 20+ years of web engineering, from independent contractor to engineering leader. Passionate about developer experience and product engineering.