One Operator, Many Drones: Inside Skydio's Autonomy Stack — Suchet Bargoti, Skydio
Suchet Bargoti, Skydio's Director of Inspection and Mapping, demos a fleet of docked drones orchestrated through an agentic layer, then explains how full-stack autonomy, maps as world models, edge-cloud splitting, VLM-driven tracking and the limits of end-to-end learning combine to make drones function as infrastructure.
Suchet Bargoti, Skydio’s Director of Inspection and Mapping, opens by setting aside slides and flying the talk live. From a laptop on conference Wi-Fi he launches a docked drone over San Mateo, sends a second one to a power line incident in Colorado, has a third track a car autonomously, and then recalls the whole fleet to dock. The demo is the argument: drones should work as always-available infrastructure, not as hobbyist gear or tools that someone has to carry to the field.
From hobby toy to infrastructure
Bargoti traces the field’s arc over roughly fifteen years. Early drones were hobbyist toys that owners tinkered with and learned to fly through hours of practice. About a decade ago they became tools that industries carried in trucks and deployed on demand. The next era, which Skydio is building toward, is drones as infrastructure: physical embodied agents that are available anywhere at any time and can launch, execute their tasks, and return without a person at the controls. Skydio now has thousands of docks deployed with power utilities, public safety agencies and construction companies, and about 16 million people live within two miles of one.
Real-world deployments
Two examples anchor the claim. A client on the northeast coast running routine patrols next to a power station spotted a utility pole burning from the inside, a fire risk that would have been hard to catch without sending crews out at considerable expense. In San Francisco, working closely with SFPD, a drone followed a person who had stolen a car and was swapping its license plate. The thief never knew they were being tracked, so police could position themselves safely and intervene at the right moment instead of running a dangerous high-speed chase through the city. Bargoti also points to reliability work across Alaska’s cold, Texas’s heat, and day, night, rain and sun, with a target of reliability to many nines, validated through simulation and testing.
Why one pilot per drone does not scale
Bargoti is candid about the limits of the current model. Skilled manual piloting takes hours to learn, and first-person-view flying takes longer, so every drone needs a trained operator beside it. As 911 calls and alerts multiply, staffing grows linearly and breaks down. The product direction is therefore to move from a first-person view of each flight to a strategic, multi-agent view in which an operator commands the whole fleet toward an objective, such as searching an area for a missing person or watching for a certain vehicle and holding position, without managing the flight itself. He imagines a near-future interface where a chat-style alert reports an incident and the autonomous system dispatches a drone, with safety determined by the system rather than a certified pilot.
Full-stack autonomy and the data flywheel
Skydio’s advantage, in his framing, is controlling the hardware, software, cloud and user interface together. This lets the team design the vision system, planning and tracking around real operating conditions: high altitude, clouds, high speed, rain, and urban canyons where GPS is unreliable. Every flight generates logs that feed a learning flywheel. Data is sanitized to remove private information, and customers are kept informed about what they share, much as Street View handles imagery. The logged data, including moments when the drone did something unexpected, flows back into learning and reinforcement learning pipelines for retraining, evaluation and redeployment.
Splitting intelligence between edge and cloud
Autonomy is divided between the drone and the cloud. Immediate control runs on the edge device, while heavier work such as longer-horizon planning, GPU-based inference and reasoning over the full map runs in the cloud. Because cloud agents depend on what they see, video quality over constrained links matters. The team invests in encoding video into smaller sizes and decoding it cleanly, so high-quality feeds reach the servers even in low-bandwidth areas. The trade-off between latency, bandwidth and capability is a constant design concern.
Maps as world models
Bargoti uses the term world model in the context of maps, comparing the approach to Waymo. A prior map supports local perception and global planning, so a drone can choose an optimal route rather than navigating building by building. Prior building data is merged with vector layers such as power lines and roads, with different behaviors for different zones. Maps go stale, though, so the fleet itself maintains them. In his example, a construction site missing from the map shows up as a mismatch between rendered map points and live video. Drones observe such changes, the data syncs back through the map pipeline, and in the next iteration every drone plans against the updated map.
Tracking through occlusion
A second model class handles object tracking, including when a target disappears behind buildings. The system keeps an implicit representation of the object, so it can predict where the target will re-emerge and move to follow it. Five years ago this relied on manually engineered motion logic. Now reinforcement learning and learned end-to-end approaches can handle many edge cases without hand-coding them. Vision-only tracking must hold up in rain, snow, day and night. Lightweight tracking runs on the device, while heavier VLM-based reasoning runs in the cloud at roughly 7 to 10 hertz with one to two seconds of latency. That is too slow for direct control but adequate for broad decisions about where to move.
Semantic reasoning and agentic find and follow
For infrastructure customers, the system performs semantic reasoning about what is in the scene, such as identifying utility poles when an instruction says to inspect a particular line. The team builds primitives and tools that an agent can call to understand the scene and act on it. In the demo, a VLM given the instruction ’look for a white Jeep’ uses a drone API to command a trajectory, runs detection to find the vehicle, and then invokes tracking and follow tools. No rules for that behavior were hand-coded. The agent reasons over drone state and the tools available to it.
End-to-end learning and its limits
Bargoti acknowledges the appeal of feeding raw sensor data into a single network that outputs actuation, pointing or flight paths, and says the team is testing reinforcement learning toward that goal. For physical systems, however, end-to-end approaches make observability and reliability guarantees hard to achieve. His answer is selective decomposition: identify which parts of the pipeline can move to learned world-model representations and which still need explicit engineering. Search and rescue illustrates the tension. Instructions like ’look for trees, then check underneath, switch to thermal, then try elsewhere’ quickly become tangled if-statements and branching code. The aim is to let agents call high-level tools instead.
Beyond quadcopters
The team is extending the approach to a smaller quadcopter form factor and is exploring fixed-wing platforms. The goal is basic APIs that cloud agents can tap into, so drones can be launched and recovered from anywhere and high-level reasoning can be built quickly on the cloud. He closes by recruiting for full-stack roles spanning hardware, software, autonomy, front end, back end and wireless networking, framing the work as making the technology in everyone’s phone fly.