The Future of Frontier Model Architectures with Walter Goodwin, Founder & CEO of Fractile

Fractile is building chips for very fast inference on the largest models, a bet on speed it has held since founding in summer 2022. Walter Goodwin frames the thesis around two shifts: foundation models trained on the internet that generalize broadly, and the need to pour more compute into models at test time. Because every deployment pays the inference cost again, making deployment fast is where the value sits.

The chip landscape and why full-stack teams differ

The accelerator landscape looks more varied than it is. Hyperscaler chips such as Google’s TPU, Meta’s MTIA, Microsoft’s Maia, and OpenAI’s custom effort are typically built with a small set of ASIC design partners, with Broadcom the largest. Across these platforms the same ingredients recur: HBM memory, tensor cores for matrix multiplication, and advanced packaging from TSMC. Few teams own the whole stack, from architecture and front-end design (which still mostly means writing code that specifies the chip’s logic) through physical design, process technology, and foundry relationships. The handoff from architect to back-end partner is where, in Goodwin’s words, you end up at the mercy of how that partner behaves. What ultimately ships to TSMC is a full layout file (GDSII) that specifies where every metal layer and transistor goes, tied to a specific process node.

Fractile keeps this loop in house with front-end designers, physical design, backend implementation, and advanced packaging. At roughly 150 people the team is deliberately lean in each area, but the closed loop lets it adjust quickly. Goodwin argues that in AI chips you have to place your bets correctly, which requires both skill and luck, and then strike fast. The team also runs ahead on workloads, examining where a model architecture should change to align with Fractile’s bets, and then advocates those changes to customers and partners.

Memory bandwidth as the central technical bet

Fractile’s early architecture was SRAM-based, similar to Groq or Cerebras. SRAM sits on the same silicon as the logic, giving extremely high bandwidth and thousands of tokens per second. The concern that emerged in late 2023 and through 2024 was scalability. Model parameters grow, but so does context length, and SRAM offers very little capacity. The company therefore moved toward a platform that pairs the bandwidth of fast inference chips with the capacity and cost of DRAM, working with memory vendors and logic foundry partners to reach far higher bandwidth from DRAM. That platform is slated to ramp in the second half of next year.

Goodwin’s argument for why this matters is that a snappier chatbot is the faster-horses version of fast inference. The more consequential use is running multi-trillion-parameter models at many thousands of tokens per second for long-running agents. Those workloads need long-context attention, which current fast chips cannot handle, so they fall back to GPUs. At data center scale, the economics of inference collapse to cost per gigabyte of memory, which is why cheap DRAM is central to the plan.

Workload bets and bandwidth scaling laws

Some properties of frontier workloads are stable even as the models churn. New LLMs want orders of magnitude more memory bandwidth, and autoregressive, low-batch generation remains common. Attention mechanisms and sparsity levels, however, shift every few weeks in the open-source frontier, so the target keeps moving.

With roughly 25 times more bandwidth per chip than an HBM-based chip, Fractile can explore scaling laws for bandwidth, not just FLOPs. Mixture-of-experts models would ideally become far sparser, moving from roughly 1 in 6 active experts toward 1 in 128 or 1 in 256 for the same intelligence. On HBM-based GPUs that sparsity becomes bandwidth-bound and runs at low FLOP utilization. Some attention variants are less bandwidth hungry but more FLOP hungry for a given capability level. Goodwin points out that FLOPs have scaled roughly a millionfold over twenty years while memory bandwidth has grown about 40x. Chips that scale bandwidth let you trade FLOPs for capability, which he sees as a multiplier on global throughput for these models.

Compressing the design cycle and the payoff window

There is a deliberate tension here. Foundry cycle times run 3 to 5 months even for hot lots, and a chip needs a 3 to 5 year amortization window to make financial sense, with a useful lifespan beyond three years. The goal is therefore not to ship a new chip every few weeks. The physical world imposes constraints on data center power, installation latency, and financing. What can be compressed is the gap between observing a workload shift and realizing that bet in volume. The value comes from having more shots on goal, so you are always ready to take a flagship platform and ramp it over a 12 to 18 month volume window.

Goodwin likens this to what frontier labs do with models: if you can structurally carve out a 3 to 6 month advantage, you win the deployments. Nvidia systems combine six to nine custom chips, so for a company building one chip today, becoming productive enough to run a rolling frontier of aligned bets is the way to build a comparable machine.

Using AI to accelerate chip design

Asked how soon an architect’s intent could reach a usable GDSII file end to end, Goodwin reports that a top semiconductor CEO said ten years. His heuristic is to take such an estimate, divide it by four, and subtract a bit more. He expects end-to-end prototyping within a few years. Conventional place-and-route tools still run for days, and as in AI research, the bottleneck is shifting from thinking to computationally expensive steps. Amdahl’s law applies: whatever cannot be parallelized becomes the limit, and fab latencies remain fixed.

Sign-off will stay with Cadence and Synopsys, whose DRC and LVS checks confirm compliance with foundry rules, and that work remains very valuable. In the meantime, fuzzy placement algorithms and crude approximations of simulation-heavy steps such as finite element analysis and thermal estimation could get most of the way to a final design much faster. As thinking gets cheaper, trials become the bottleneck, so accelerating trials matters as much as accelerating intelligence. Goodwin argues that thinking hard before each costly experiment becomes almost a moral obligation.

Market structure and frontier risk

Goodwin expects supply diversity to persist because compute is existential for these players. He notes a common joke that first-party chips mainly exist to lower what companies pay Nvidia, and says there is some truth to it because many of those chips are architecturally similar and do not enable capabilities others cannot. Real differentiation comes when a chip enables something fundamentally new, and deploying at the frontier increasingly requires running models orders of magnitude faster.

Frontier labs face an asymmetric risk. A lab that goes all in on proprietary silicon could be overtaken if a rival discovers a computational breakthrough that only runs on a different platform, and the first lab could lose in the nine months before it deploys enough of its own chips. That is why frontier players need to deploy compatible platforms, and why Goodwin considers deeply differentiated chip bets dangerous for anyone at the frontier. He sees a lasting place for third-party frontier chip players in the premium end of the market, where speed matters above everything else.

• Tags:
• Mixture-of-Experts
• DRAM
• AI Chips
• AI Inference
• SRAM
• HBM
• Memory Bandwidth
• Full-Stack Chip Design
• Chip Design Automation
• Semiconductor Market Structure