Vertical AI Agents Are the Opening, and Prompting Plus Evals Are the Skills

I spent time going through the distilled notes from more than 72 hours of AI agent talks at the big spring conferences this year, and I want to give you the real signal rather than the highlight reel most coverage produced. The real signal is narrower and more actionable than a list of exciting possibilities, and it points to one specific play I would start building for a client this week.

The clearest takeaway across all those hours of sessions is this: the only AI opportunity that has not already been absorbed by the major platforms is vertical agents. Not AI in general. Not chatbots. Not general-purpose assistants. Vertical agents built specifically for one industry or one function, running workflows that only someone with deep domain knowledge would know how to design correctly. And the two skills that determine who captures that opportunity are not model selection and not coding ability. They are prompt engineering and evals. Almost nobody who is excited about vertical agents has both of those skills developed well enough to ship something a real business will trust with real operations. That gap is where the opening is.

The investor framing from one of the YC roundtable sessions was blunt about scale. A vertical agent does not just sell software to replace part of the workflow. It can do the work the software operators used to do, which means the market it addresses is the software license plus the payroll of the team the software was helping manage. IBM's CEO study published this year found that 85 percent of chief executives say their staff can use AI, but only 25 percent actually do. That 61-point gap exists almost entirely in vertical use cases, because general-purpose AI does not understand the specific workflow, vocabulary, and quality standard of any particular industry. Every point of that gap is an addressable opportunity, and the window to build there is open right now.

The platforms absorb everything horizontal, but vertical domain knowledge resists commoditization

The first wave of AI startup products that appeared after the large language model era began were almost entirely horizontal: meeting notes, calendar scheduling, email triage, document summarization, basic research assistance. These were genuinely useful when they launched. Within 12 months most of their core functionality was built directly into Google Workspace, Microsoft 365, Notion, and similar incumbents that already had the customer relationships and distribution to ship AI features as part of a bundle customers already paid for. The startups building horizontal tools had to pivot or found themselves competing against a free feature bundled into a tool their customers already used.

That is the absorption pattern. It is not a prediction. It already happened, and it is still happening to anyone building AI tools in horizontal use cases today. When I say horizontal, I mean capability that is useful across every industry without modification: summarize this, draft that, translate this, search that. When a capability is horizontal, the company that wins is the one with the best distribution, and the best distribution in software is almost always the incumbent that already has the installed customer base.

Vertical agents operate on different terrain. What makes a vertical agent valuable is not the underlying AI capability, which anyone can access for a few cents per token through an API. What makes it valuable is the knowledge of how a specific business actually operates: the edge cases that general models do not know about, the domain vocabulary that changes the meaning of a request, the quality standard that distinguishes a useful output from a technically correct but practically useless one. An agent built by someone who has spent real time inside a specific industry encodes all of that knowledge in ways that are genuinely difficult to replicate from the outside, even with access to the same underlying model.

The human capital example that stuck from the talks was an HR agent running personalized conversations with more than a thousand employees, work that used to require a full department. That agent was not generic. It knew the specific tone the organization uses internally, the policies it was bound by, the escalation paths for different situations, and the cadence of follow-up that matched that culture. None of that comes from the model. All of it comes from the operator who understands that organization specifically. And it is exactly the knowledge that is hardest for a platform to absorb, because absorbing it requires understanding each organization individually rather than shipping a generic tool that works adequately for all of them.

The prompt framework that holds up across serious vertical agent work has six parts: the role the agent plays in this specific context, the precise task it is performing in this session, what inputs it can access and in what format, what the output should look like so the reviewing human can act on it efficiently, the constraints it must operate within including what requires human judgment, and the specific capabilities and reminders that encode the domain knowledge that makes it work for this particular business rather than in the generic case. Every part matters, but the last two are where the vertical knowledge actually lives.

A dental practice running this right recovers more than a year of salary from one afternoon of setup

Here is what the play looks like in a specific context, because the theory of vertical agents is easy to follow but the practice is where most people stop making progress.

A dental practice has a specific administrative task that repeats every morning before the first patient arrives. Someone needs to check each scheduled patient's insurance eligibility for the procedures planned that day, identify which cases need pre-authorization before treatment can begin, flag patients whose annual benefits are expiring so the practice can reach them before the year resets, and prepare the recall queue for patients overdue for hygiene appointments. At a practice running a full daily schedule, this work takes two front desk staff members roughly 90 minutes each morning before they can start answering phones and checking patients in.

An agent built around this workflow carries a prompt structured in six parts. The role is a dental front-office coordinator. The task is preparing the eligibility and recall summary for the day's scheduled appointments. The inputs are the day's schedule and the patient records in the practice management system. The output is a ready-to-review document organized by appointment time so the staff can work through it quickly. The constraints include never contacting a patient who has already rebooked, staying within what the practice is legally permitted to communicate, and flagging anything requiring clinical judgment rather than resolving it independently. The reminders encode the specific domain knowledge: which insurance plans the practice sees most frequently, which of those have year-end benefit periods that create urgency, and which procedure codes on the schedule require pre-authorization from specific plans.

Anthropics own guidance on agent prompting from the workshop talks reflects the same approach: start with the smallest possible prompt, something as simple as check the schedule and prepare the eligibility summary, then stress it against unusual inputs and refine from whatever breaks. Resist writing the perfect comprehensive prompt on day one. Build the eval set in parallel with the prompt from the very beginning, so you have something to test against before deployment rather than after.

That prompt, built well by someone who has spent real time inside a dental front office, turns a 90-minute task into a 20-minute review. With two staff members each spending 90 minutes daily on eligibility and recall, and the agent reducing that to about 20 minutes of review per person, the practice recovers roughly 560 minutes of front-desk labor per week. At a front-desk rate of 18 dollars per hour, that is about 168 dollars per week in recovered labor, or roughly 8,700 dollars per year against tool costs that might run a few hundred dollars per month. The math on the investment is not subtle.

But the more important point is not the labor savings. It is quality consistency. A human handling 50 eligibility checks and recall contacts on a busy morning will miss some and rush others. An agent runs the same quality process every time, which reduces billing disputes, improves benefit utilization for patients who would otherwise miss their year-end coverage, and produces a recall queue that is complete rather than the subset a busy front desk managed to get to before the phones started. Those downstream effects compound over time and are worth more than the direct labor savings in isolation.

The eval framework is the only thing that turns a working demo into a trusted product

This is where the vertical agent conversation almost always breaks down in practice. Someone builds a clever prompt, runs it against a handful of test cases, the agent produces impressive outputs, and they declare it ready for real use. Then the agent encounters a case it was not designed for, produces an answer that is confidently wrong, and the business that trusted it spends a week undoing the error. The agent gets turned off. The conclusion is that AI is not reliable enough for real operations. That conclusion is wrong, but it is the predictable outcome of skipping the eval step.

An eval is a test case with a known correct answer. For the dental agent, an eval is a real past appointment record where the outcome is already documented: whether the insurance was eligible, whether pre-authorization was required, whether the patient's benefits were expiring, and what the correct action was. Five to ten real cases like this, selected to represent the range of situations the agent will actually encounter, form the baseline test set. Every time the prompt changes, you run the agent against those cases and check whether every answer is still right. If a change improves three cases and breaks two others, you know it immediately. If it improves all five without breaking any, you can deploy it with confidence.

Companies that have shipped reliable production agents treat their eval sets as their most important intellectual property. Not the prompts, the evals. The prompts are relatively easy to reconstruct from observing the agent's behavior over time. The evals encode the specific quality standard of a particular domain in a particular business, including the edge cases that only someone with deep domain experience would even think to test. Those definitions are built from real operational history and are genuinely hard to replicate from the outside, even with access to the same prompt framework.

The five-step workflow that holds across every serious vertical agent build is: observe how the task is done today in detail, decompose it into its component steps, map how those steps connect and where the edge cases live, prototype a simple agent version starting with the smallest useful scope, and evaluate against the test set and iterate. Starting small and building the eval set in parallel, rather than writing the perfect comprehensive prompt on day one, is what produces agents that improve predictably rather than ones that surprise you with failures once they are in production with real users.

Building the evals in parallel with the prompt is the single habit that most consistently separates reliable vertical agents from impressive demos that break in production. You do not need a thousand labeled examples. Five to ten real past cases with known correct answers, covering the main types of situations the agent will encounter, catch most of the failure modes before they reach a user. Three rounds of running those cases, finding what breaks, adding the domain knowledge to handle it, and re-running the full set produces an agent that a business will actually trust enough to change their daily workflow around. That is the threshold that matters.

The agent engineer role and where the compounding begins

The new role that holds all of this together has been given a name: agent engineer. It is not a software engineer who learned to prompt, and it is not a domain expert who learned to use AI tools. It is the specific combination of prompting skill, traditional engineering capability to connect the agent to real data and systems, product sense about what a business will actually use daily, and enough machine learning literacy to understand what the underlying model can and cannot be trusted to do without supervision. That combination is genuinely rare right now, and the first person in any given vertical who develops it is in an advantaged position that builds quickly because their eval set grows more useful with every real case added to it.

Andrew Ng's specific prediction from the talks is worth taking seriously. He named voice agents and coding assistance as the two most underrated bets for near-term AI agent adoption. Voice agents are particularly relevant for service businesses because they allow customers to interact in the mode they already use, a phone call, without requiring any change in customer behavior. A dental practice that routes initial recall outreach through a voice agent can scale that contact without proportionally scaling front-desk headcount. The call quality is consistent on every contact. The timing is precise. The practice gets data on every call outcome automatically rather than relying on staff logging.

The combination of a vertical agent handling the back-office workflow and a voice agent handling the outreach creates something more powerful than either alone: an operation that runs its administrative and patient communication pipeline at higher quality and lower per-contact cost than a comparable human team, while freeing the staff to focus on work that genuinely requires human judgment and presence. That is the direction 72 hours of talks was pointing toward. Not AI instead of people, but AI running the rule-based repetitive work so that people can focus on the parts where their skill and presence actually matters.

The businesses that apply this seriously now will have production agents running before their competitors have finished reading about the technology. The advantage that creates is not just operational efficiency. It is data about how the agent performs on real cases, which informs prompt improvements, which improves the agent further, which generates more useful eval cases. That compounding is what makes early investment in vertical agents worth more than the immediate labor savings suggest. The window to get there first in any given vertical is open. For most industries, it will not stay open much longer.