Tokenless

Microsoft research

What AI gets wrong and what failure teaches us

What AI gets wrong and what failure teaches us

Jennifer Neville discusses her research into "surprising failures" in AI systems, where models, particularly LLMs, struggle with complexity in realistic user workflows. Her team focuses on practical evaluation for multi-turn conversations and agentic knowledge work, emphasizing the need for robust AI that understands context and user intent over time. She offers insights into bridging the gap between user expectations and AI capabilities, advocating for detailed user feedback and outlining the future of human-AI collaboration.

Your LLM Judge Is a Confident Liar: Building Better Verifiers — Browserbase

Your LLM Judge Is a Confident Liar: Building Better Verifiers — Browserbase

Browserbase and Microsoft Research unveil their Universal Verifier for computer-use and web agents, addressing the critical flaw of inaccurate LLM-as-judge benchmarks that yield "confident liars." They detail its four guiding principles, demonstrating how it drastically cuts false positives and provides high agreement with human evaluations, leading to better training data. An auto-research experiment highlights the synergy between AI and human expertise in developing robust evaluation systems.

MagenticLite is here: A full-stack agentic experience powered by Small Models

MagenticLite is here: A full-stack agentic experience powered by Small Models

Microsoft Research introduces MagenticLite, an agentic framework powered by two new small, open-weight models: Magentic Orchestrator for planning and coding, and Fara-1.5 for browser automation. The talk details the novel synthetic data generation techniques and training strategies used to achieve state-of-the-art performance in small models, enabling them to compete with much larger ones.

Will machines ever be intelligent?

Will machines ever be intelligent?

Doug Burger, Nicolò Fusi, and Subutai Ahmad explore the intelligence of AI, contrasting transformer-based LLMs with the human brain's distributed, continuously learning architecture. They delve into differences in efficiency, representation, and sensory-motor grounding, debating what intelligence truly means and how future AI might bridge the gap.