How Forward Deployed Engineering is done at Cognition — Jia Wu
Cognition's deployed engineering team focuses on measuring customer outcomes, not just token usage, achieving significant productivity gains with their AI agent, Devin. This involves understanding deep business problems, strategically applying Devin's capabilities, and feeding insights back to product development, fundamentally reshaping the role of a deployed engineer.
Cognition’s approach to deployed engineering for its AI agent, Devin, centers on delivering measurable outcomes rather than simply optimizing token usage. Jia Wu, a deployed engineering lead, emphasizes that the true value of Devin is demonstrated by significant improvements in customer workflows and productivity, measured by a real delta before and after activation. Initially, Devin faced skepticism as an early 2024 product (“SweepBench 13%”), but through focused deployed engineering, it has evolved into a robust solution for enterprise software development.
The Evolution of Devin and Its Impact
Cognition, with roots including the Windsurf acquisition, offers Devin through CLI, IDE, and primarily the Devin Cloud agent. Internally, Cognition itself has seen a “step function increase” in engineering leverage, shipping an “order of magnitude more good quality robust PRs” by using its own agent. Externally, customer adoption shows “parabolic growth,” indicating a profound shift in how companies integrate AI agents into their operations.
The Role of the Forward-Deployed Engineer
The success of Devin hinges on Cognition’s forward-deployed engineers (FDEs). Their primary objective is to maximize the overlap between Cognition’s product capabilities and a customer’s specific problems. This involves:
- Deep Problem Understanding: FDEs dedicate significant time (e.g., 4-5 hours of calls daily) to understand the customer’s strategic initiatives and “problem space.” They recognize that while coding is “mostly solved” (20% of the software development problem), the real challenges lie in testing, reviewing, deploying, and maintaining software, especially in complex legacy environments.
- Strategic Application of Devin: Instead of “token maxing” (wasting compute by deploying agents without direction), FDEs map Devin’s capabilities to high-leverage customer problems. This involves identifying areas like backlog items, remediations, delayed code, unwritten tests, or automatic alert triage.
- Feedback Loop for Product Development: FDEs don’t just solve customer problems; they also feed insights back to Cognition’s product team. This “highest fidelity evaluation set” from the field helps de-risk the product roadmap, ensuring that common enterprise challenges encountered during deployment are translated into new features or improvements, making subsequent deployments even better.
The role of a deployed engineer is evolving beyond pure technical skills. As “the pure engineering part trends toward zero,” the job increasingly demands strong business and people skills. FDEs at Cognition possess a broad skill set (people, business, process, customer, technology) with “deep spikes” in specific areas, often hiring from backgrounds like product management or founders, recognizing that technical expertise is harder to teach on the job than business acumen. Their mantras include “relentless curiosity for why” and being “relentlessly tied in to the customer,” as customer success is the company’s “lifeblood.”
Measuring Real Outcomes and Quantifiable Impact
Cognition has shifted its focus from maximizing token usage—an outdated KPI from when everything was subsidized—to measuring tangible outcomes in the delivery space. The goal is to make an “organization 10x faster,” not just individual engineers, as single-point tools often fail to achieve this broader impact.
Proof points from anonymized and public case studies demonstrate Devin’s effectiveness:
- Engineering Capacity: Internally, Cognition achieved the equivalent of a 150% headcount increase on a project over three months.
- Delivery Timeline Reduction: Customers experience an “82% reduction across like delivery” timelines when Devin is fully activated, meaning tasks move from inception to market significantly faster.
- Increased PR Output: Engineers using Devin can deliver “almost double the amount of PRs” compared to using single-point tools alone.
- Case Studies: Examples include Nubank (ETL migration by 50 engineers completed in 1/3 timeline with Devin autonomously), a large Latin American bank (tax identification system migration with half the effort, tackling legacy code like COBOL/JCL), and Built (order of magnitude more PR acceptance, weekly output of over 10 engineers).
Core Values: Correctness and Customer Success
Cognition’s core values extend beyond internal operations, permeating its external engagements. The company is committed to “correctness” in engineering practices and “customer success at all costs.” This includes a willingness to “ship the hard thing and leave nothing on the table,” even deploying an engineer to live near a customer for 10 months to ensure their success. The philosophy is that “everybody is go-to-market” because ultimately, every action contributes to making the customer successful and positioning Cognition as the obvious choice for enterprise partnerships.