Building Planetary-Scale Data Systems with Venice • Felix GV & Olimpiu Pop • GOTO 2026
Félix GV, an architect of LinkedIn's Venice database, discusses its unbundled, planetary-scale architecture. He covers how components like Kafka and RocksDB form independent distributed systems, details their rigorous chaos engineering practices, explains CAP theorem trade-offs in multi-region deployments, and explores the experimental integration of DuckDB for SQL-based analytics.
Félix GV, who worked on LinkedIn’s Venice data system for a decade, describes it as a distributed database built on an “unbundled” pattern. In this architecture, each core component is itself a complete, standalone distributed system.
The Unbundled Architecture of Venice
The Venice system is composed of several key, independent distributed systems:
- Write-Ahead Log (Pub/Sub): Utilizes Apache Kafka as its commit and replication log, handling the write path and data replication between replicas.
- Server Fleet: Thousands of servers per region, each running a Java process that manages a local RocksDB database instance. These servers handle requests and can perform computations directly.
- Control Plane: A separate distributed system using Apache Zookeeper and Apache Helix to manage metadata, partition placement, and overall cluster orchestration.
- Client Ecosystem: Clients are also distributed systems. Venice offers multiple ways to access data:
- A standard client-server model where requests are sent over the network.
- An “eager cache” pattern where the client application embeds a RocksDB instance and pulls data directly, effectively becoming another follower replica. This provides extremely low latency at the cost of higher local resource (RAM/SSD) usage.
Multi-Region Reliability and Chaos Engineering
To ensure planetary-scale reliability, the system was designed for multi-region deployment. This design was not just theoretical; it was rigorously and frequently tested using chaos engineering principles inspired by Netflix’s “Chaos Monkey.”
Several times a week, during peak morning traffic, the team conducted “load tests.” These tests involved failing traffic out of one data center and concentrating it into the remaining ones. The goal was to ensure a single data center could handle not only a 50% increase in its normal load (the expected scenario if one of three data centers fails) but an even greater load to test for future growth. This proactive testing regularly uncovered bottlenecks, ensuring that the reliability mechanisms were always functional and ready for real-world failures. The core principle is that reliability mechanisms you don’t regularly test are mechanisms you don’t actually have.
Data Flow and Consistency Trade-offs
Venice is primarily a derived data system, meaning it hosts machine-generated data rather than user-generated primary data. This design leads to a different set of trade-offs compared to traditional databases like PostgreSQL.
- Asynchronous Write Path: Data ingestion is asynchronous, coming from either batch jobs (e.g., daily or hourly) or streaming processors (e.g., Kafka streams), or a hybrid of both.
- High Throughput vs. Strong Consistency: This asynchronous model allows for extremely high ingestion throughput. The trade-off is the sacrifice of strong consistency, specifically linearizability (often referred to as “read-your-own-writes”). When a write is acknowledged, it is durable in the write-ahead log (Kafka), but it is not yet indexed or readable in the database servers. This is a fundamental design choice made at the “drawing board level” to optimize for availability and throughput, which are critical for derived data use cases.
A Practical Look at the CAP Theorem
The discussion provides a clear, practical explanation of the CAP theorem, which states that a distributed system can only provide two of the following three guarantees: Consistency, Availability, and Partition Tolerance.
Since network partitions are a reality of distributed systems, you must always design for Partition Tolerance (P). Therefore, the only real choice is between:
- CP (Consistency and Partition Tolerance): The system remains consistent even during a network partition, which often means sacrificing availability (e.g., refusing writes or reads).
- AP (Availability and Partition Tolerance): The system remains available during a network partition, which means sacrificing consistency.
Venice is an AP system by design. It’s almost as if it operates in a state of continuous high latency between the write path and the read path, so strong consistency is never guaranteed. This same trade-off applies to multi-region primary data systems. For example, if a shopping cart is updated in a US data center, there will be a delay before that update is asynchronously replicated to a European data center. During this replication lag, the system is inconsistent across regions but remains highly available.
Integrating DuckDB for SQL Capabilities
While RocksDB serves as the core key-value storage engine, the team experimented with integrating DuckDB, an in-memory analytical SQL engine. This was a hackathon project to test the flexibility of Venice’s abstractions and explore new capabilities.
The motivations for this integration included:
- Data Exploration and Debugging: Providing users with a way to run ad-hoc SQL queries on their data without the high-inertia workflow of running an ETL job to move data into an offline system like Spark. This allows for quick checks on data shape, cardinality, and values.
- Leveraging the DuckDB Ecosystem: Gaining access to DuckDB’s rich ecosystem of extensions for advanced analytics, including vector math capabilities. While Venice had some basic vector math functions since 2019, DuckDB offers a more modern and extensive set of tools “for free.”
While not yet used in production, the integration proved successful and opens up possibilities for adding powerful, on-the-fly analytical capabilities directly within the Venice ecosystem.