Our Smaller Model Beat Opus on Finance Tasks - Alexander Shevchenko, Ramp
Alexander Shevchenko of Ramp Labs explains why an application company keeps running applied research rather than waiting for the next frontier model, and walks through three projects in production: a post-trained Qwen 3.5 specialist for financial question answering at Haiku-level latency, KV cache cartridges that prepare agents with far more context ahead of time, and a 10,000-agent swarm that pen-tested Ramp and found seven high-severity vulnerabilities using cheaper open-weight models.
Why an application company keeps doing research
Ramp is a company where the bitter lesson plays out constantly. LLM use cases grew 123% over the last year, and although token spend on the top features keeps rising, unit economics improve because each new frontier model is cheaper, faster, and better, so the same accuracy costs less. The obvious question is why not simply wait for the next model. The answer starts with 2023, when agents needed strict guardrails, could not backtrack, had no thinking tokens, broke on synonyms in prompts, and needed context cut to the bone. Ramp shipped agents like Ramp Tour Guide anyway because financial professionals needed them immediately. By 2026, models handle MCP access and broad context well, but there is still a layer below the default experience where better products are built, and reaching it takes more effort.
A specialist model for Ramp Sheets
Ramp Sheets launched in November as one of the earliest spreadsheet agents, before the frontier labs shipped their own. Users who dropped many spreadsheets in and asked for a DCF or a financial scenario hit a clear friction point: Opus 4.6 spent a long time planning and researching every sheet to get its bearings, and its thinking tokens were slow and expensive. The team kept frontier models as orchestrators and handed financial question answering to a much faster specialist. Working with Prime Intellect, they post-trained a 35B-parameter Qwen 3.5 model that exceeded Opus 4.6 on exact-task accuracy while serving at Haiku-level latency. It has been in production as part of the Ramp Labs effort.
KV cache cartridges
Cartridges compress up to 20 to 30 times more context into the KV cache offline, so an agent can be prepared for work it knows is coming. Month-end close is the motivating case: a financial professional has to reconcile the ledger, aging reports, and supporting documents on the first of the month. Instead of having the agent retrieve and reason over all of it at that moment, a cartridge can be built overnight, so the agent opens with the full picture already loaded. Because the self-play procedure generalizes well, cartridges are also much lighter to produce than post-training runs. The caveat is that the labs may ship this as an API feature under continuous learning. Ramp is comfortable with that, since it is building up internal know-how on applying these techniques and could use a shipped version from day one.
Agentic swarms for pen testing
The third production project is a map-reduce swarm of 10,000 parallel agents that pen-tested all of Ramp. Infrastructure for the run was handled by Ramp Inspect, the internal cloud agent. The agents mapped the attack surface and filed findings, then the reduce step deduplicated the tickets with LLMs and attempted adversarial reproduction to confirm which vulnerabilities were real and important. The result was seven high-severity vulnerabilities, all patched. One was a Lua script injection that could be delivered over email. No single component of the stack was unsafe; the bug only appeared when the swarm reasoned end to end across several components. The process runs unattended, and its main cost is tokens, including the tokens spent on passes that fail.
Cheaper models in swarms
A finding from the same work is that open-weight models up to five times cheaper than frontier reasoning models can be used in these swarms. Because the work is distributed across many agents, the detection rate does not fall by a factor of five. Redundancy recovers most of the difference, so the savings can be reinvested in more coverage while still getting nearly the same detection rate.
Saving time and money
Across the projects, the recurring goal is to cut latency, which is time, and token cost, which is money. That matches Ramp’s own slogan of saving customers time and money. Two products reflect this research. Token spend management connects to a company’s API keys and shows usage by key, by user, by accounting category, and even by tax credits that the spend qualifies for. It also builds a semantic taxonomy of LLM usage, so companies can see what the models are doing rather than only who is using them. Ramp Router adds routing strategies developed with Ramp Labs, including smart flex offloading of work, to improve both latency and cost per token, with more strategies planned.
Other research directions
The Ramp Labs blog covers several threads beyond these projects. One post describes transferring post-training gains between models: a LoRA post-train on Qwen 3.5 can be projected onto newer Qwen versions or even different model families, which amortizes the post-training cost over time. Other work includes self-improving monitoring, where LLMs create Datadog monitors for production services and refine them as incidents arrive without a human in the loop, and semantic tagging, a map-reduce over user sessions that discovers open-ended taxonomies of LLM usage and reconciles them against spend.