How AI is changing Software Engineering: A Conversation with Gergely Orosz, @pragmaticengineer
Gergely Orosz, author of The Pragmatic Engineer, discusses the bizarre trend of 'token maxing' in Big Tech, the evolving role of software engineers in the AI era, and why companies are heavily investing in internal AI infrastructure despite uncertain productivity gains.
The Rise of “Token Maxing” in Big Tech
A strange cultural phenomenon known as “token maxing” has emerged within large tech companies like Meta, Microsoft, and Salesforce. It stems from these companies measuring developers’ usage of internal AI tools, often through public leaderboards or spend tracking. At Salesforce, for example, there’s a tool to see how many dollars colleagues have spent on AI tokens, with some teams having a target minimum spend of around $175 per month.
This measurement, coupled with industry-wide job insecurity, has led engineers to feel pressured to increase their token count to avoid being perceived as low performers. This pressure results in counterproductive behaviors:
- Artificial Usage: Engineers run autonomous agents to “build junk” or ask AI assistants to summarize documentation (even if the AI does a poor job) simply to generate tokens and stay out of the bottom percentile.
- Weaponized Metrics: While token count is just one of many data points in performance reviews, it can be “weaponized.” A low performer with a low token count is seen as “not even trying,” while a high performer with a high token count is seen as an innovator.
This trend is reminiscent of earlier flawed metrics like “lines of code,” but it’s now being driven by top tech companies. It’s a response to an initial push from leadership who, seeing the success of AI-native companies like Anthropic, wanted to force adoption among skeptical, experienced engineers. The situation at Coinbase was an extreme example, where the CEO mandated AI tool usage under the threat of termination.
Is AI Actually Making Engineers More Productive?
While individual productivity is certainly increasing, team-level productivity is more of a question mark. It’s proven difficult to retrofit AI into established workflows. An interesting perspective is that the most significant productivity gain may not be for engineers themselves, but for their non-technical collaborators. By enabling them with coding agents, they no longer have to wait for an engineer, effectively creating “serverless developers” and unlocking organizational productivity.
Mastering these AI tools requires a new mindset:
- Continuous Learning: There is no manual for AI tools. It takes a long time to get good, and workflows are constantly changing.
- Practice Over Theory: Unlike traditional computer science, understanding the underlying theory of models doesn’t necessarily make you a better user. Hands-on experience is key.
- Open-Mindedness: Success requires a low-ego approach, being “open to leaving your priors behind” and experimenting.
The Changing Role of the Software Engineer
AI is accelerating a trend that was already underway: the expansion of the software engineer’s role. The role has already absorbed responsibilities from dedicated tester and DevOps teams. Now, it’s beginning to incorporate product skills, giving rise to the “product engineer.”
Expectations for seniority and business awareness are increasing even for early-career engineers. As a result, teams are shrinking. A VP of Engineering at John Deere noted their “two-pizza teams” are becoming “one-pizza teams,” a direct result of these new tools and expanded roles.
The idea that engineers are becoming “engineering managers for AI agents” is a flawed analogy. The role is more akin to a Tech Lead or someone operating a “mech suit” (an analogy from DHH). You orchestrate tasks and can do more, faster, without the people-related challenges of management—the drama, conflicts, and slow feedback loops. With agents, the feedback loop is immediate.
Big Tech’s Massive Investment in Internal AI Infrastructure
Many large tech companies like Uber, Airbnb, and Meta are investing heavily in building bespoke internal AI infrastructure, even if it hasn’t yet translated to a dramatic increase in external product features. They are building their own:
- Custom background coding agents integrated into their monorepos.
- Internal “MCP” (Model-Compute-Provider) gateways.
- AI-powered on-call and code review tooling.
There are three primary reasons for this internal build-out:
- Low-Risk Learning: It’s a way for the company to gain hands-on AI expertise without shipping risky, half-baked features to users.
- Superior Results: Custom solutions tailored to a company’s massive, proprietary codebase can often outperform generic, off-the-shelf tools that lack full context.
- Easy Funding: Internally, any project with “AI” in its name gets funded. A request for headcount for “developer platform” might be denied, but a request for an “agent experience” team gets approved.
Shopify is a prime example of a company willing to pay the price of early adoption. They gained access to GitHub Copilot a year before its public release, trading significant expense and the “churn” of dealing with an early-stage product for a multi-month head start on their competition. For them, it’s a strategic investment in innovation and a crucial tool for recruiting and retaining top talent.