The 2025 AI Engineering Report — Barr Yaron, Amplify

Based on a survey of 500 respondents, the 2025 State of AI Engineering report reveals key trends in how professionals are building with and thinking about AI. The community is broad and technical, with a notable trend of seasoned software engineers (10+ years of experience) being newcomers to AI (3 years or less). While titles vary, the work of “AI Engineer” is becoming more defined and is a rapidly growing area of interest.

LLM Adoption and Use Cases

More than half of respondents are using LLMs for both internal and external applications. The landscape is heavily influenced by OpenAI, whose models account for three of the top five (and half of the top ten) models used in customer-facing products.

The primary use cases are code generation, code intelligence, and content/writing assistance. A key takeaway is the heterogeneity of application; 94% of those using LLMs employ them for at least two use cases, and 82% for three or more, spanning internal and external needs.

System Customization: RAG and Fine-Tuning

Retrieval-Augmented Generation (RAG) is the most popular method for customizing LLM systems, with 70% of respondents reporting its use.

Surprisingly, fine-tuning is being performed much more widely than anticipated, particularly among researchers and research engineers.

• Methods: There’s a strong preference for parameter-efficient fine-tuning (PEFT), with 40% of those fine-tuning using LoRA or QLoRA.
• Approaches: Good old supervised fine-tuning remains the most popular core training approach, though many hybrid methods were also reported.

The Development Lifecycle: Rapid Iteration and Prompt Management

The field is characterized by a rapid pace of change. Over 50% of engineers update their models at least monthly, with 17% doing so weekly. Prompts are iterated on even more frequently:

• 70% of respondents update prompts at least monthly.
• 10% update prompts on a daily basis.

Despite this constant iteration, prompt management is a significant blind spot. A full 31% of respondents have no formal system for managing their prompts.

The Multimodal Production Gap

There is a significant disparity in the adoption rates across different data modalities. Text-based models see widespread use, while image, video, and audio lag considerably, a phenomenon termed the “multimodal production gap.”

However, looking at future intent reveals a strong interest in audio. Among professionals not currently using audio models, 37% plan to adopt them, the highest intent-to-adopt ratio of any modality. This suggests an “audio wave” may be coming.

The State of AI Agents

For the survey, an AI agent was defined as “a system where an LLM controls the core decision-making or workflow.” While 80% of respondents feel LLMs are working well at their jobs, less than 20% say the same for agents.

Despite low current production use, the interest is extremely high. The majority of engineers plan to use agents, and fewer than one in ten have no plans to ever use them. For agents already in production, a majority have write access, often with a human-in-the-loop, and some are capable of taking action independently.

Monitoring, Evaluation, and Storage

• Monitoring: Most engineers use multiple methods, with 60% employing standard observability and over 50% relying on offline evaluation.
• Evaluation: For assessing model and system quality, a combination of user data collection and benchmarks is common, but human review remains the most popular method.
• Storage: 65% of respondents use a dedicated vector database, suggesting that their specialized capabilities provide significant value over general-purpose databases with vector extensions. Of this group, there’s a close split between self-hosting (35%) and using a third-party managed provider (30%).

Community Perspectives and Pain Points

• Biggest Challenge: “Evaluation” was cited as the single most painful aspect of AI engineering today.
• Future Predictions: A majority believe Transformer-based models will still be dominant in 2030, and that open-source and closed-source models will eventually converge in capability.
• Disclosure: Most respondents believe agents should be required to disclose that they are AI and not human.
• AI Companions: When asked to guess what percentage of the US Gen Z population will have AI girlfriends or boyfriends, the mean response was 26%.

• Tags:
• AI Agents
• Prompt Engineering
• LLM
• RAG
• Fine-Tuning
• AI Engineering
• Model Evaluation
• Vector Databases