KDD '25 AI Reasoning Day keynote: Improving AI Reasoning through Intent, Interaction, and Inspection

AI models can be improved by focusing on three core pillars: capturing richer user intent, enabling collaborative interaction, and performing systematic inspection. A fourth emerging pillar, memory, promises to turn reactive tools into evolving collaborators.

1. Capturing Richer User Intent

Standard natural language is often insufficient for specifying complex, structured tasks. Richer forms of intent, such as input-output examples and temporal context, can guide models to produce more accurate and relevant results.

Input-Output Examples: The Power of Demonstrations

Input-output examples are a powerful form of specification because they are verifiable. This concept was pioneered in features like Microsoft Excel’s Flash Fill, which synthesizes a program to automate repetitive string transformations from just one or two examples.

The core ideas behind this symbolic, program synthesis approach are:

• Domain-Specific Language (DSL): Restricting the search space to a DSL (e.g., with substring and concatenate operators) instead of a general-purpose language like Python makes the problem tractable.
• Goal-Directed Search: Instead of enumerating all possible programs, a goal-directed search backpropagates constraints from the input-output examples through the structure of the DSL grammar, pruning the search space.
• Intelligent Ranking: Multiple consistent programs are ranked based on features like program size and uniformity of outputs when applied to other data in the spreadsheet.

While powerful, Large Language Models (LLMs) often struggle with this type of task. They generate code that is approximately correct but fails on edge cases because they lack a deep understanding of execution semantics. To overcome this, one can trick an LLM into performing a more robust search:

1. Ask the LLM to generate the first statement of a program.
2. Execute that statement on all provided input examples.
3. Show the intermediate results back to the LLM as context.
4. Ask the LLM for the next statement.

By repeating this process and even exploring multiple potential statements at each step, the LLM is guided to build a program that is correct by construction, effectively simulating a breadth-first search. This technique leverages the key advantage of examples: they can be executed and validated at each step.

Another key idea is using distinguishing inputs. When multiple synthesized programs are consistent with the user’s examples, the system can find an input where the top-ranked programs produce different outputs. It then presents this input to the user, asking them to clarify their intent for this specific edge case, thus helping the user provide the most informative examples.

Temporal Context: Learning from Recent Actions

User intent is also hidden in their recent actions. By observing a user’s repetitive edits across a codebase, a system can infer the underlying transformation they wish to perform and offer to automate it. This is the principle behind IntelliCode Suggestions in Visual Studio.

The key to its success was a user experience that did not require users to stop their workflow, copy-paste examples, and explicitly invoke a tool. Instead, the system works in the background, observing keystrokes to find patterns of repetitive edits and proactively offering suggestions. This approach respects the user’s flow and addresses the “late awareness problem,” where a user might not think to use a tool until they are already deep into a manual, repetitive task.

2. Collaborative Interaction

LLMs are often trained to provide an answer to any query, which leads to hallucinations when they lack sufficient information. A better paradigm is to allow the model to interact, ask for clarification, or even admit uncertainty.

The “Investigate and Respond” Pattern

Instead of directly answering a question, a model can be prompted to follow a more robust, agentic pattern:

1. First, determine if it has enough information to answer.
2. If not, it should investigate further using available tools and context.
3. If user input is required for the investigation, it should ask the user a specific, clarifying question.

This pattern was applied to Copilot for debugging, transforming its success rate from 25% to 88%. The initial version would often guess at a fix, while the agentic version would first perform root cause analysis, mimicking a proper debugging workflow. This not only solved the problem more reliably but also taught users better debugging practices.

Evaluating Conversation Quality

To systematically improve interactions, we first need to measure them. Drawing from the philosopher Paul Grice’s maxims for effective communication, we can define rubrics for what constitutes a good AI-human conversation:

• Quantity: Is the information concise?
• Quality: Is the information truthful and evidence-based?
• Relevance: Does it stay on track with the user’s goals?
• Manner: Is it clear and unambiguous?

These high-level principles can be used to generate domain-specific rubrics for tasks like bug fixing or data analysis. To evaluate agents at scale, one can build simulated user proxies. These proxies are equipped with ground-truth information (e.g., the correct code for a data analysis task) that allows them to automatically answer an agent’s clarifying questions. By simulating different user personas, these proxies can effectively measure and amplify the performance gaps between different agent designs.

Abstaining with Placeholders

An alternative to hallucination or asking questions is for the model to return a partial response, explicitly marking its uncertainty with a placeholder. This can be achieved by modifying the training loss function to be more lenient on producing a special placeholder token (e.g., ?) instead of a wrong token, while a regularizer prevents the model from overusing it.

This idea can be extended to creative domains. For instance, an image generation model can produce an initial image and also expose key parameters from its internal prompt as user-adjustable controls, complete with suggested alternatives. This transforms the interaction from a single-shot generation into a collaborative exploration of the creative space.

3. Systematic Inspection

Instead of treating model outputs as final, they can be subjected to automated, programmatic inspection. The structured feedback from this inspection can then be used to guide the model through iterative refinement.

For example, when extracting a table from a PDF, an LLM’s initial output can be flawed. A symbolic checker can be run on this table to find errors:

• Does all data in a column match a consistent type or pattern?
• Can all content in the table be traced back to the source text (to detect hallucinations)?
• Is all source text represented in the table (to detect omissions)?

The list of detected errors is then provided as structured feedback to the LLM, which can then retry and produce a more accurate table. This code-based, systematic feedback loop provides a significant (e.g., 15%) lift in quality compared to simply asking another LLM to “review” the output.

This principle also applies to improving an agent’s reasoning process. By analyzing the model’s action trajectories, one can perform root-cause analysis on failures, abstract these into general “insights” (e.g., “the model failed to account for X”), and feed these insights back into the prompt to make the agent’s behavior more robust in the future.

Conclusion: The Future is Memory

By combining richer intent, collaborative interaction, and systematic inspection, we can significantly improve the quality and reliability of AI reasoning. The next frontier is memory. Without it, every interaction is like a “first date,” where the user must re-teach their preferences and correct the same mistakes. Memory will allow AI systems to learn from past interactions, turning them from reactive tools into true collaborators that evolve with the user.

• Tags:
• Prompt Engineering
• LLMs
• Program Synthesis
• Reasoning
• Model Evaluation
• Human-AI Interaction