Every AI Founder Should Be Asking These Questions
Jordan Fisher, co-founder of Standard AI and now at Anthropic, poses critical questions for startup founders facing the imminent arrival of AGI. He explores challenges from software commoditization and building trust in automated teams to finding durable moats and the ethical responsibility of building world-changing technology.
In a world hurtling towards Artificial General Intelligence (AGI), the familiar playbooks for building a startup are becoming obsolete. The future is more uncertain than ever, and the most critical task for founders is not to have answers, but to ask the right, fundamental questions about strategy, product, and the very nature of building a company. While conventional wisdom advises planning for the next model release in six months, a more radical approach is needed: planning for a world with AGI in the next two to three years.
Rethinking Strategy in a Post-AGI World
The core challenge for founders is navigating the paradox of focus: the need to concentrate on a niche while simultaneously considering how AI will transform everything from hiring and fundraising to go-to-market strategy.
- The AGI Planning Horizon: Instead of planning 6 months ahead, founders must operate with a 2-3 year mindset, assuming AGI’s arrival. This doesn’t mean creating a rigid plan but actively thinking through how your company, product, and team will function in a world where superintelligence is a reality.
- AI-Native vs. Retrofitting: A key strategic question is whether new, AI-native products built from the ground up will triumph over incumbent products that retrofit AI features. Existing players have the advantage of distribution, but AI-native companies might create entirely new, more effective operational patterns and product experiences. The winner may vary by vertical, and founders must develop a clear, validated hypothesis for their market.
- The Evolving “Buy Side”: The impact of AI isn’t one-sided. Enterprises will also be armed with powerful AI agents, accelerating their procurement cycles and technical capabilities. They may even opt to build solutions in-house with a small team and advanced AI tools, bypassing traditional SaaS providers. Startups can no longer assume a slow-moving enterprise market.
The Future of Software, Products, and Interfaces
The very concept of software is being questioned. As AI’s ability to generate code advances, the way we build and interact with applications could fundamentally change.
- The Commoditization of Software: Will it still make sense to be a SaaS provider when an enterprise can prompt an AI to build a custom internal tool? We may see a future where consumers and businesses generate personalized, on-demand applications, rendering app stores obsolete. The counter-argument is that AI will also enable dedicated teams to raise the quality bar to exceptional new heights, creating a gap that on-demand generation cannot close.
- On-Demand Code Generation: A potential pattern is for applications to generate new code and functionality on the fly in response to a specific user’s needs. This offers incredible personalization but introduces massive challenges around trust and security. You must deeply trust an AI to modify backend systems and databases without human oversight.
- Multimodal, Contextual Interfaces: The future of UI is not just generative but multimodal and contextual. The best interface should meet the user where they are, seamlessly weaving together text, images, audio, and video, and accepting input via touch, voice, or whatever is most convenient for the user’s current environment.
Trust, Security, and the New Social Contract
As teams become smaller and agents take on more responsibility, trust emerges as the most critical currency. The old mechanisms for ensuring corporate accountability are eroding.
- Trust in the Agent: Beyond trusting the underlying AI model’s alignment, can a user trust the agent built by a startup? An agent from an ad-based company may be aligned with the company’s goal to sell products, not the user’s best interests. This is especially concerning when agents handle both personal and professional data, raising questions about data segregation and whose behalf the agent is truly acting on.
- The Erosion of Human Guardrails: We traditionally trust companies partly because they are composed of a diversity of people. If a CEO makes an unethical decision, there’s a chance an employee will act as a whistleblower. In a semi-automated company, a single bad actor could deploy a harmful change with no one else aware.
- A New Model for Trust: AI-Powered Auditing: To solve this, we may need new guardrails. One powerful idea is the concept of a neutral, AI-powered audit. A company could commit to its public mission statement and allow an AI to inspect its operations—from code to internal communications—to verify compliance. To protect IP, such an auditor could be designed to have “no memory,” deleting all its findings if no malfeasance is discovered.
Finding a Durable Advantage
In a world where a competitor can prompt an advanced AI to replicate your entire business, finding a defensible moat is paramount.
- Data Defensibility: While generalist LLMs have devalued many custom datasets, a moat may still exist in industries with deep, tacit knowledge that has never been digitized on the public internet. Fields like advanced materials science or semiconductor manufacturing (e.g., TSMC) possess proprietary knowledge that frontier models do not have, creating a real, defensible advantage.
- Solving Truly Hard Problems: The most durable advantage will come from solving hard problems that remain difficult even with AGI, particularly those that interface with the physical world. Robotics is lagging behind pure software intelligence. Therefore, focusing on areas like infrastructure, energy, manufacturing, and logistics can create a moat that is difficult to erode with prompts alone.
- The Intelligence Ceiling: Founders should consider if their vertical has an “intelligence ceiling”—a point where models are “good enough” for the task at hand (e.g., generating a marketing image). Once a task hits this ceiling, the service becomes rapidly commoditized, as switching to the next, more powerful model provides no additional value.
The Founder’s Final Responsibility
This unique moment in history may be the last opportunity for founders to shape the future in a meaningful way. The pursuit of profit, while understandable, should not overshadow the profound societal implications of the technology being built. The YC slogan “build something people want” should be interpreted more deeply: build something society needs. Build products that are not just delightful for a moment but are also trustworthy, secure, and beneficial to humanity’s long-term mental and societal health.