Solving AI Video: How Fal.ai is making AI Video Generation Fatser & Easier
Fal co-founder Burkay Gur and head of engineering Batuhan Taskaya discuss their journey building a high-performance generative media cloud. They cover their strategic pivot to media models, core optimization principles born from early GPU scarcity, and the development of a customer-obsessed culture to navigate the fast-paced AI model landscape.
The journey of Fal.ai began not with generative media, but with foundational machine learning infrastructure challenges at companies like Coinbase. The initial idea was to build ML pipelines for fraud detection and other enterprise use cases. However, the landscape shifted dramatically with the advent of models like DALL-E and ChatGPT, prompting a pivot towards the emerging world of generative AI.
The Genesis of Speed: From Scarcity to a Core Tenet
The company’s deep focus on performance was born from necessity. In the early days, as the team explored running Stable Diffusion 1.5, they were struck by its slowness, with generation times often exceeding 10-15 seconds. This technical curiosity was compounded by a severe “GPU crunch.” The team had a total quota of just eight GPUs, forcing them to maximize the efficiency of every single unit. This constraint became a catalyst for an obsessive culture around optimization, which started as a survival mechanism and evolved into the company’s core differentiator.
A Principled Approach to Optimization
With a background in compilers and the Python interpreter, head of engineering Batuhan Taskaya applied first-principles performance engineering to ML models. The process involved:
- Profiling and Benchmarking: Calculating the theoretical maximum FLOPs achievable by the GPU and comparing it to the actual performance of a workload to identify the largest gaps.
- Identifying Bottlenecks: Discovering patterns where simple calculations were blocked, preventing full utilization of the GPU.
- Systematic Optimization: Implementing solutions like sharding workloads, more efficient threading, and grouping common patterns in the execution layer.
- Custom Kernels: Developing highly efficient, custom kernels specifically for the targeted diffusion model workloads. This work was designed as a pluggable system, allowing the same optimizations to be applied to new models as they emerged.
A critical piece of the infrastructure puzzle was data access. To operate in a multi-cloud environment and overcome slow cold starts (which could be up to 5 seconds with standard Kubernetes orchestration), Fal built its own distributed file system. This system features multi-layered caching—at the data center, node, and in-memory levels—which drastically shaved off model loading times compared to fetching from object storage like S3 for every request.
Strategic Focus on Generative Media
While the AI world rushed to host LLMs following the release of Llama 2, Fal made a deliberate decision to focus exclusively on generative media (images, video, audio). The team recognized that media models presented a unique set of challenges and opportunities, distinct from language models. They saw an emerging market with complex, chainable workflows (e.g., image-to-image refinement, upscaling, background removal) that were fundamentally different from the more generalizable, prompt-based interactions with LLMs.
This conviction led them to coin and champion the term “generative media cloud,” positioning themselves as the torchbearers for this specific domain. The subsequent announcement of models like OpenAI’s Sora validated their bet, proving that media generation was evolving into its own major market.
Navigating the Competitive Model Landscape
The video model space is characterized by “fierce competition” where leadership changes on a weekly basis. Unlike the image space where models are converging on quality while retaining unique differentiators, video is still in a “leapfrogging” phase with significant, non-marginal improvements happening with each new release.
“I remember when Sora came out even in our team people were like oh my god OpenAI is like so far ahead that no one’s going to be able to catch up and then like Luma released their model, Runway released their model, Kling released their model… You can never like say oh this is the model and then this is not going to have any competition for like even a month right like even like two weeks is… we are operating at like weeks as I said.”
Fal has positioned itself as a neutral, two-sided marketplace. They provide developers with access to the best-performing models while offering model creators a distribution channel to a large user base. This flywheel effect is amplified by Fal’s infrastructure expertise, helping model labs optimize their workloads for launch.
The Importance of Fine-Tuning and Complex Workflows
A key learning has been the importance of fine-tuning in the media space, which is estimated to be “1,000 times more” prevalent than in the language space. For tasks requiring high consistency—such as virtual try-ons, character generation, or product photography—fine-tuning models like LoRAs remains the most effective method, even as models with better prompt controllability emerge. These bespoke workflows require more than just an inference engine; they need a platform that can efficiently chain multiple models and processes together.
Building a Customer-Obsessed Culture
Fal’s culture is deeply engineering-driven but equally obsessed with business outcomes and customer success. The team was composed of 28 engineers before the first non-engineer was hired, yet the founders leaned heavily into founder-led sales early on. They believe that to win, you must “win the market,” which requires being as good at go-to-market as at engineering.
This customer-centricity is built into their operations. The company is a top user of Slack Connect, maintaining shared channels with nearly all customers. This practice embeds Fal’s engineers directly into the customer’s workflow, allowing them to act as partners and advocates, ensuring that the infrastructure is built to serve real-world needs.