Shaping Model Behavior in GPT-5.1— the OpenAI Podcast Ep. 11

The Evolution to Reasoning Models in GPT-5.1

A primary goal for GPT-5.1 was to address user feedback on GPT-5 and, most significantly, to establish “reasoning models” as the default for all users. This marks a fundamental shift where every model within the chat experience, including for free-tier users, now has the capability to “decide to think.” This process, akin to a Chain of Thought or Daniel Kahneman’s “System 2” thinking, allows the model to deliberate on complex prompts, refine its answers, and utilize tools before responding. This enhancement provides a notable boost in overall intelligence, leading to significant improvements in instruction following and performance on a wide range of tasks, even those not immediately perceived as requiring deep reasoning.

Key improvements in GPT-5.1 were driven by community feedback that GPT-5 could feel “less warm” or have “weaker intuition.” This was traced to several factors:

• Context Window: The previous model’s context window didn’t retain enough information, making it feel “cold” when it forgot important details from earlier in the conversation.
• Auto-Switcher Jarringness: The previous system automatically switched between chat and reasoning models, which had different response styles. This could be jarring, for instance, by providing a clinical-sounding answer to a sensitive, emotional topic after the switch.
• Instruction Following: GPT-5.1 is substantially better at adhering to custom instructions, giving users more reliable control over the model’s behavior and quirks.

Deconstructing “Model Personality” and EQ

The concept of “model personality” is an overloaded term. On one hand, it refers to a specific product feature that allows users to control the response style and tone—adjusting for traits like conciseness, length, or emoji use. However, for most users, personality encompasses the entire experience of interacting with ChatGPT. This broader definition includes everything from the app’s UI and latency to the underlying “harness” of the system, such as the context window’s effectiveness and how different modalities like voice and text integrate. Users perceive this as a single, omni-experience.

Improving the model’s Emotional Intelligence (EQ) is an open-ended research challenge. It’s not just about the model’s tone but its ability to listen, remember, and understand user intent. Progress is made through:

• Technical Improvements: Enhancing the context window and implementing features like Memory are crucial for demonstrating high EQ.
• User Signals Research: This involves training reward models on production data to better interpret subtle user signals and intent during Reinforcement Learning (RL).
• Customization: Allowing users to select a response style that resonates with them is another facet of improving the model’s perceived EQ.

The Art of Steerability, Safety, and Bias

A core principle guiding model development is to “maximize user freedom while minimizing harm.” This creates a complex balancing act. The post-training process is an art form, carefully weighing trade-offs to avoid breaking functionality while refining behavior.

Part of the art here is figuring out how to pull out these quirks of the model that can come across as personality without breaking steerability, which is what users ultimately want. That’s the freedom component.

Early models were often overly cautious and prone to refusal. The technology has since evolved toward safe completions. If a request touches a safety boundary, the model now attempts to fulfill the user’s request earnestly without generating harmful content, rather than issuing a blanket refusal. This allows for more nuanced and helpful responses in sensitive contexts, such as a lawyer proofing a legal document versus someone writing a malicious email.

Furthermore, GPT-5.1 has a wider expressive and creative range. While the default may not feel dramatically different, users can push the model to adopt more elevated or, conversely, much simpler styles of communication. The model is also being trained to better handle subjective domains by expressing uncertainty and anchoring its responses in objective truths where they exist.

Personalization Through Memory and Customization

The ultimate goal is to move toward a world where the model is highly steerable and can adapt to each of the 800 million weekly active users, as a single personality cannot serve everyone. Features like Memory are a key step in this direction. Memory allows the model to record and recall key facts from conversations, which prevents users from having to repeat themselves and grounds the model’s responses in the user’s specific context.

While the model will become better at inferring a user’s context and expertise—reducing the need for explicit prompt engineering—the user will always remain in control. Features are designed to be transparent, allowing users to view, edit, or delete memories to steer the model’s behavior.

To get the most out of the models, users are advised to continuously pressure-test them with hard questions on subjects they know well, as capabilities are constantly improving. It’s also effective to ask the model itself for help in crafting a better prompt to unlock its full potential.

• Tags:
• Prompt Engineering
• Reinforcement Learning
• Reasoning Models
• GPT-5.1
• Steerability
• Model Personality
• User Feedback
• Model Alignment