Small Models, Big Results: Training a Finance Agent for Under $500 — Charles Dickens, Snorkel AI
Snorkel AI and UC Berkeley's Sky Computing Lab show that a 4B Qwen3 model, trained with reinforcement learning for under $500 using the rLLM framework, beats its 235B sibling on expert-validated financial question answering built from SEC 10-K filings. The talk covers the FinQA dataset construction and verification, common failure modes of frontier models, transfer to multi-table and general tool-use tasks, and ablations showing that simple data and binary rewards outperform more elaborate setups, since the real bottleneck is disciplined tool use rather than reasoning depth.
Charles Dickens, a research scientist at Snorkel AI, argues that for enterprise AI workloads, reliability and specialization can outweigh raw parameter scale. The talk covers a partnership with UC Berkeley’s Sky Computing Lab in which a 4B Qwen3 model, trained with reinforcement learning, outperformed its 235B sibling on a simulated financial question-answering environment, reaching roughly 60% pass@1 against 51%.
Why specialists over generalists
When financial institutions ask how well an agent reasons, they usually mean something narrower: can it operate inside a stack of complex schemas, legacy APIs, and technical debt, can it stay reliable across long workflows without compounding errors, and can its behavior be audited? Dickens frames this as a choice between a large generalist and a small specialist, and argues that specialized workflows do not need polymaths. His analogy is that you would not call Terence Tao for a tax audit. You would call the specialist who knows the forms, the tools, and the rules.
Building FinQA from 10-K filings
The dataset is built from SEC 10-K annual reports, which analysts use to identify risks and inform investment decisions. The pipeline pulls filings from EDGAR, uses Qwen3 30B to extract roughly 6,900 SQL tables, and generates one question-answer pair per table. Generation is guided by a question taxonomy developed with financial experts and by metadata capturing lineage, table names, and columns.
Verification happens in three layers. Programmatic consistency checks confirm that referenced tables and columns actually exist in the metadata. Independent agents perform automated review. Human experts then manually check that questions and answers are realistic and correct. The answers are single, verifiable final answers. The split is roughly 4,000 train, 500 validation, and 290 held-out benchmark examples, arranged so that no company appears in more than one split.
Failure modes of frontier models
Even frontier models showed recurring gaps:
- Schema hallucination: the model assumes tables or columns exist, possibly an artifact of pretraining data.
- Context flooding: the model issues broad queries such as SELECT STAR, which can overwhelm its own context limits.
- Poor recovery: the model repeats the same failed strategy instead of using error messages to adapt.
Dickens notes that similar patterns appeared in insurance underwriting work presented at CAIS.
Training with rLLM
The team used the open-source rLLM framework from the Sky Computing Lab. Dickens highlights that it works with any agent framework with near-zero code changes through a decorator pattern, offers a CLI-first workflow, and includes built-in RL algorithms such as GRPO, REINFORCE, and RLOO. It also supports multiple training backends, and he mentions gains beyond finance, including math reasoning.
The agent runs in a ReAct loop with tools for interacting with the roughly 7,000 generated tables. The reward is binary correctness, judged by GPT-5 nano using reference-based evaluation. The base model is Qwen3 4B, optimized with GRPO and a binary reward. Training ran about 1,000 concurrent environments on eight H100s, costing roughly $420 in compute and $40 in judge API calls, for a total under $500.
Results
The base model’s accuracy more than doubled, and the 4B model outperformed the 235B variant on the 290 held-out expert-curated samples, at roughly 60% versus 51%.
To test transfer to harder problems, the team built a multi-table variant with 2 to 5 tables requiring sequential decisions and planning, split into roughly 1,000 train, 120 validation, and 80 benchmark examples. The tool-use discipline learned on single-table data transferred directly, with no explicit multi-table training needed. On the BFCL benchmark for general tool calling, overall accuracy slightly improved, with minor gains on multi-turn and memory tasks. Specialization did not erode broader tool-use competence.
Why simple data and simple rewards won
Ablations compared training on single-table FinQA alone, a mixture of single and multi-table data, and a curriculum moving from single to multi-table. The simplest set produced the most lift. Dickens concludes that the bottleneck was never reasoning depth but tool-use reliability. Once the fundamentals were mastered, the model could compose those skills.
The same logic applied to rewards. The team tried more informative signals, including intermediate-step rewards for table access and query completeness, and a fine-grained weighted rubric developed with experts. A single binary reward gave the most lift.
Blueprint for enterprise agents and evaluation
The study suggests small models can be trained with RL successfully when the data represents the target tasks and outcomes, yielding better deployment economics for specialized domains. Similar results have appeared in healthcare, law, and insurance.
For evaluating agents, Dickens proposes three axes. Environment complexity asks how realistic and dynamic the environments are. Autonomy horizon asks whether agents are tested over the horizon lengths people actually use, whether they make safe decisions on their own, and whether they improve their judgment as copilots over time. Output complexity asks whether evaluations capture the wide range of outputs in daily work. He considers this axis underexplored, with most benchmarks still focused on text artifacts.
He closes by promoting Snorkel’s Open Benchmarks Grants, a multi-million dollar commitment to open benchmarks like this one, which has already supported projects such as Agent Last Exam and JudgmentBench. Snorkel is also hiring across research and engineering roles.