Why Robotics Still Isn't Solved - But Could Be Soon | YC Paper Club

The session opens with Francois Chaubard reflecting on the persistent “next year, robotics is solved” narrative over the last decade. Despite impressive demonstrations from systems like AlphaGo, Mujoco, ALOHA, Diffusion Policy, and VALAs, fully autonomous, general-purpose robots are still elusive. He identifies four core roadblocks hindering scalable, real-world robotics deployment:

1. The Sim-to-Real Gap: Current video-based world models fail to accurately respect real-world physics when deployed, leading to unrealistic outcomes (e.g., a simulated car not crashing into a grocery store). This gap is even more pronounced when conditioning transition functions (ST to ST+1) on actions, which demands an immense amount of data.
2. Action Representation: Learning efficient action spaces remains a significant open problem. Fast learning is hampered by the lack of effective representations for complex action spaces, an issue likened to the historical challenge of optimizing giant Q matrices with linear approximations.
3. The Sensorimotor Problem: Robots severely lack the rich, ubiquitous haptic and proprioceptive sensing capabilities of humans. While humans can infer complex object properties (normal/tangent force, moisture, temperature, vibration, friction coefficient) and build detailed world models purely through touch (e.g., finding items in a backpack), robots typically only possess minimal fingertip force-torque sensors and wrist cameras, profoundly limiting their understanding and interaction with the physical world.
4. Embodiment Drift: The physical degradation of robot hardware over time (e.g., dust, corrosion in actuators, battery decay) leads to a divergence between programmed actions and actual physical outputs. This “drift” renders previously collected teleoperation data stale, necessitating costly and frequent retraining of entire VLA systems.

Addressing these challenges is crucial for robotics to move beyond impressive, yet limited, demonstrations.

---

Marcel Torne introduces MEM (Multi-Scale Embodied Memory), addressing the critical limitation of most current Vision-Language-Action (VLA) policies—their lack of memory. Policies like RT1 or Gato operate without context, leading to repetitive errors or inability to complete long-horizon tasks (e.g., endlessly washing dishes, burning food). The challenges of incorporating memory are effectiveness (distribution shifts, data scarcity) and efficiency (resource-intensive training and inference).

MEM proposes a two-tiered architectural and memory decomposition:

• High-level policy: A VLM that predicts the “what” (next symbolic step).
• Low-level policy: A VLA that executes the “how” (robot actions).

Memory is similarly split:

• Short-context visual memory: Dense visual frames for dexterous manipulation, fed to the low-level policy. It uses a novel ViT-based encoder with temporal attention layers, compressing sequences by dropping tokens for faster inference. This enables tasks like preparing grilled cheese (knowing when it’s done), unloading groceries (remembering items), and cleaning windows (avoiding endless repetition).
• Long-context language representation: A compressed textual “memory scratchpad” for the high-level policy, tracking past events. Being text-based, it’s efficient, less prone to distribution shifts, and physically accurate as the policy can modify its memory. This supports complex, multi-minute tasks like full recipe preparation.

Crucially, MEM facilitates in-context adaptation, allowing robots to learn from and correct mistakes in real-time (e.g., re-grasping a dropped chopstick, switching sides to open a fridge), a capability largely absent in memory-less policies. The textual nature of long-term memory currently relies on human-annotated Supervised Fine-Tuning (SFT) data, but future research aims to use reinforcement learning to automatically discover relevant information to store.

---

Milan Ganai’s work focuses on Self-Supervised Bootstrapping of Action-Predictive Embodied Reasoning, a method to improve VLA action prediction by teaching robots “what’s worth reasoning about.” Drawing an analogy to Chain of Thought for Large Language Models (LLMs), embodied reasoning aims to provide rich, human-readable logical steps, which is particularly valuable in data-scarce robotics and for debugging.

The core challenge is a “chicken-and-egg problem”: there’s no oracle source for reasoning data or models, and reasoning can be overly verbose or irrelevant. The approach, R&B Encore (Refine and Bootstrap Embodiment-Specific Chain of Thought Reasoning), treats reasoning as an unobserved latent variable, leveraging variational inference.

R&B Encore employs two main components in a self-improving pre-training cycle:

1. Reasoning Proposer: An annotator model suggests diverse reasoning types (e.g., planning steps, visible objects, gripper positions) for a given demonstration.
2. Reasoning Validator: Scores these proposed reasoning traces based on three criteria: concision (not verbose), non-triviality (encourages generalization), and most importantly, action predictiveness (ensuring the reasoning is grounded and useful for generating actions for that specific embodiment).

This process generates synthetic, action-aligned reasoning datasets, leading to more robust VLA policies. R&B Encore has shown effectiveness across various embodiments:

• Manipulation: Found that ‘move’ and ‘gripper position’ reasoning are useful, while ‘perceptual reasoning’ often contains distracting objects and is less effective.
• Hardware Manipulation: Improved out-of-distribution performance for novel objects and cluttered scenes.
• Legged Locomotion: Emphasized the importance of reasoning about structural affordances and movements over terrains or counterfactuals.
• Self-driving: Refined human annotations, removing “hallucinated experiences” and reducing L2 path errors and collision rates, confirming meta-action, visible objects, and perceptual reasoning as beneficial.

The key takeaway is that selective reasoning is more effective than exhaustive reasoning; valid reasoning is not always useful. R&B Encore demonstrates that self-supervised bootstrapping can effectively address the data and model scarcity in embodied reasoning, and its approach generalizes across different robotic systems and VLA scales. The proposed “action forcing” mechanism allows for reasoning during training to embed priors, while dropping it during inference to mitigate latency concerns in real-time applications like autonomous vehicles.

---

Tyler Lum presents SimToolReal, a method for achieving zero-shot dexterous tool manipulation by training policies entirely in simulation. Traditional teleoperation struggles with highly dexterous tasks due to embodiment mismatch and limited force feedback, while prior Reinforcement Learning (RL) methods typically train separate policies for each skill.

SimToolReal overcomes this by:

• Massively Parallel Simulation: Utilizing GPU-accelerated simulation to run tens of thousands of robots in parallel, generating “decades of interaction data in a few days” at 1000x real-time speed. This scales data collection with compute, not human effort.
• Goal-Reaching Formulation: Unifying diverse manipulation as a goal-reaching problem. The policy doesn’t receive task labels but aims to move objects from their current pose to a desired goal pose.
• Training: A goal-conditioned policy is trained on procedurally generated primitive objects (cylinders, cuboids) and random goals within simulation, incorporating crucial system identification and domain randomization.
• Zero-Shot Inference: At inference, a human video demonstration provides the sequence of desired object goal poses (extracted using foundation pose and SAM). The frozen RL policy then tracks these goals at 60Hz. The human video acts as a “task prompt” for the policy, not for training or fine-tuning, allowing new tasks to be performed in minutes.

The policy, an LSTM taking proprioception, object pose, grasp bounding box, and goal pose as input, controls a 22-DOF hand and 7-DOF arm. It achieves zero-shot generalization across 12 unseen tools and tasks, demonstrating high dexterity (e.g., screwdriver spinning). Evaluation confirms that improving performance on generic sim goal-reaching tasks directly translates to success with real-world tools and human-demonstrated trajectories. SimToolReal significantly outperforms fixed-grasp baselines (highlighting the importance of in-hand reorientation) and kinematic retargeting methods (which ignore contact forces).

Failure analysis reveals that pose tracking failures (over 50%) are the dominant issue, especially for smaller or occluded objects. Interestingly, the policy exhibits strong recovery behaviors (e.g., immediately re-grasping a dropped hammer), which emerged as an accidental byproduct of applying random forces to objects during simulation training.

A follow-up, Play to Perfect, extends this concept to precise contact-rich assembly by leveraging a pre-training/fine-tuning paradigm: learning a shared dextrous prior through task-agnostic play (similar to SimToolReal) and then fine-tuning on sparse-reward assembly tasks for zero-shot real-world deployment.

---

Niko West from Rerun.io, a company providing a unified data layer for physical AI, champions a new model for successful robotics companies: “robotics application companies” or “neo-integrators.” These companies focus on solving a full business problem end-to-end, excelling in operations, support, minimizing custom hardware, and starting with teleoperation to validate the business case before scaling with autonomy. He predicts this model will be the “new SaaS” for physical work.

West outlines a three-part pattern for starting such a company:

1. Solve a single customer problem with teleoperation and off-the-shelf hardware: The “physical world is brutal,” demanding rapid learning from real-world failures. Teleoperation proves commercial viability and offers a path to data collection for later model training. Learnings can include critical production metrics, failure tolerance, onboarding new designs, hardware limitations, and potential V2 products.
2. Set up the basics for learning:
   • Early Model Fine-tuning: Fine-tune a simple model (e.g., a Pi model) on minimal teleop data to establish a baseline. This early integration of learning influences data collection and operations.
   • Performance Evaluation: Maintain a replica of the customer’s environment for testing. Develop repeatable, business-specific metrics for success (e.g., sharp plane edges, symmetry). Start with manual evaluation to build intuition, then automate. Track metadata and failure classifications rigorously.
   • Effective Data Collection: Continuously train, evaluate, and debug data. Real-world robotics often requires adjusting data collection instructions based on model performance, emphasizing iterative rather than upfront data collection. Physical data (multimodal, multivariate, episodic, 3D) requires specialized tools like Rerun.io, as traditional databases are ill-suited.
3. “Hill Climb” (Iterate and Scale): Once a viable service is established, the focus shifts to continuously improving robot intelligence (scaling data, advanced algorithms, new sensors) while simultaneously excelling in non-modeling areas like sales, manufacturing, logistics, and customer support. These operational aspects, often overlooked by pure model-focused companies, form a significant competitive moat.

West observes that successful teams, whether frontier labs with massive compute or lean startups, prioritize fast iteration, pragmatic systems, minimal moving parts, and full-stack debugging. He encourages founding robotics application companies, citing enormous markets, improving base models, strong business moats in the physical world, lower capital requirements than general AI, and the need for great AI and engineering talent. Historically, the high cost and immaturity of basic robotic components prevented this model, but current advancements make it more feasible.

---

Bill Jiao and Guanming Wang from General Instinct introduce their work on making World Action Models (WAMs) practical for real-world robotics. WAMs, exemplified by Nvidia’s Dream Zero, are Vision-Language Models (VLMs) that simultaneously predict future frames and kinematics, leveraging current visual observations to imagine future states and guide actions. While WAMs demonstrate strong performance, their primary drawback is immense computational cost, often requiring prohibitively expensive hardware (e.g., two GB200s) for real-time operation.

General Instinct’s solution focuses on optimizing WAM inference for speed and efficiency:

• Problem Identification: The slowness stems from the Diffusion Transformer (DIT) running numerous times (e.g., 32x for 16-frame prediction) due to Classifier-Free Guidance (CFG) during the autoregressive flow matching inference process (typically 50-100 steps).
• Architectural Shifts:
  • Image One: Instead of predicting full future video chunks, predict only the future end state (a single frame at t+N).
  • Fast One: Further optimizes by assuming the world representation is implicitly learned within the DIT’s hidden state, eliminating the need for any video decoder. The hidden state directly conditions the action head.
• General Instinct’s Infra Optimizations:
  • Distillation: Applied to both the VLA encoder-decoder and the DIT to reduce model size.
  • Split Transformers: Divides the DIT into a video transformer and an action transformer. The action transformer uses cross-attention to learn world representations from the video transformer’s hidden state, bypassing explicit future frame decoding.
  • Flow Matching Distillation: Drastically reduces autoregressive sampling steps from 50-100 to 1-2, yielding a ~50x speedup without performance drop.
  • Modality Exploration: Investigates alternative modalities (e.g., masks, optical flow) beyond raw pixels to retain rich world representation more efficiently. Heatmap visualizations guide action prediction.

These optimizations enable WAMs to run at 500 milliseconds per chunk (16 actions) on a much more economical Jetson Orin, a significant step towards scalable, real-time physical AI. The core belief is that WAMs offer “physics-informed” action generation by explicitly learning future kinematics, which VLAs typically lack by only reacting to current observations. The business model focuses on providing this crucial inference infrastructure rather than building end-to-end robotics applications, ensuring these powerful models can be deployed on the edge.

• Tags:
• Self-Supervised Learning
• Robotics
• Reinforcement Learning
• Vision-Language-Action Models
• VLA
• Machine Learning
• Real-Time AI
• Tool Use
• RL
• Diffusion Models
• Teleoperation
• Sim-to-Real
• Embodiment Drift
• Sensorimotor Problem
• Action Representation
• Multi-Scale Memory
• In-Context Adaptation
• Embodied Reasoning
• Dexterous Manipulation
• Robotics Companies
• World Action Models
• WAMs