Building a Virtual Try-On Agent: From Diffusion Basics to a Multi-Agent Pipeline

Fashion product photography became a commodity the moment diffusion models learned to follow an instruction as precise as "this face, this garment, this scene" and most online stores have not yet caught up to what that means for their catalog.

The shift accumulated through model releases, research breakthroughs, and open-source tooling contributions that individually felt incremental and collectively crossed a threshold. A clothing store that had no choice but to book a model and a photographer in 2022 now has an alternative that produces comparable on-model imagery on demand, at any scale, for a fraction of the per-image cost. The economics are not close. A professional fashion photoshoot with a model, photographer, stylist, and studio costs between two thousand and ten thousand dollars for a single day of output. That investment yields between thirty and sixty usable images, enough for a seasonal campaign but nowhere near enough for a social presence that needs twenty to thirty fresh on-model images every day to stay visible on platforms that reward freshness and volume. The AI pipeline described here produces that daily volume for a monthly compute cost of sixty to one hundred fifty dollars, not as an estimate but as a live measured cost across catalog-volume runs. The math has moved far enough that any store still operating as though continuous photoshoots are the only option is leaving a structural advantage on the table.

The Clothing Seller Whose Content Calendar Was the Bottleneck, Not the Products

The business problem that produced this pipeline was not a creative failure. The products were good. The store had a clear aesthetic, a consistent customer base, and an audience that responded measurably better to lifestyle imagery than to flat product shots. The bottleneck was a calendar problem, not a quality problem.

A social presence that needs twenty to thirty fresh on-model images per day requires either a continuous photoshoot operation, which consumes most of a small store's marketing budget by the end of the third week, or a large upfront shoot that anticipates every garment and every seasonal backdrop the team might need across the next several months. Neither is practical without an enterprise creative budget. The result for most small and mid-size online stores is the same: two or three static product photos per week, posted on a schedule dictated by how often new photography arrives rather than by what the platform algorithm rewards.

Flat product photos on a white background perform reasonably well for customers who have already decided to buy and are comparing options. They do not build the kind of social presence that attracts customers before they know they want to buy. Online shoppers do not primarily imagine themselves into a product from a flat image against a white background. They respond to a person wearing the garment in a scene that feels recognizable: a city street, a weekend table setting, a sunlit outdoor space. The psychological distance between a flat photo and a believable lifestyle shot is the distance between a customer who scrolls past and one who stops, looks, and eventually adds to cart.

The pipeline was built to close that distance at scale, without a photoshoot budget, by generating realistic on-model imagery of a consistent AI model wearing each garment across varied backgrounds, at the daily volume the social presence actually needed. The store went from posting two or three times a week to maintaining a daily feed of twenty or more on-model lifestyle images, covering every product in the active catalog and cycling through backgrounds and settings that kept the content visually varied without any new photography.

The timeline for the shift is worth stating clearly. The operator began with a manually assembled ComfyUI test pipeline and a single reference photo of a generated face. Within four weeks, daily output had reached twelve reviewed, upscaled on-model images. By week twelve, the pipeline was producing twenty-eight images per day across a growing catalog with no increase in per-image cost. The bottleneck that had defined the content calendar for years disappeared because the pipeline does not depend on booking time, weather, availability, or any other human scheduling constraint.

How Diffusion Turns a Noise Pattern Into a Sellable Product Photo

These models do not construct images the way an illustrator does, moving from a rough sketch to a detailed finish. They begin with pure random noise and progressively strip it away in small steps, each step guided by a prediction of how much noise to remove to move the current state closer to an image consistent with the input prompt.

The reason this works is that going from noise to a finished image in a single step is computationally impossible. Breaking the same journey into fifty or one hundred small denoising steps makes each individual step tractable, and the cumulative result is a clear image that emerged from following a guided path out of chaos. The model is not inventing an image from imagination. It is finding the path out of noise that leads toward a visual state consistent with the instruction it was given.

What defines that path is language. Through a model called CLIP, which learned from captions paired with hundreds of millions of images, words and visual content share a mathematical space. A prompt like "a woman in a textured linen blazer standing on a street at midday" occupies a specific position in that space, and the denoising process is directed toward images whose visual content corresponds to that position. This is why detailed prompts produce more accurate images than vague ones. Specificity narrows the target position in the shared space, and a narrower target gives the denoising process a clearer direction to follow.

For a product catalog, this mechanism means any scene a prompt can describe is accessible without staging it physically. The garment, the person wearing it, and the setting are each independent axes of control once the generation pipeline is configured correctly. This makes it possible to take one product and generate it against dozens of distinct backdrops through nothing more than changing the scene description. The underlying model run changes. The core visual elements, the face, the clothing, the garment texture and color, do not.

The open-source toolchain that makes this pipeline buildable without a research background is ComfyUI, a visual node graph editor that lets you wire each step of the generation process together and inspect the output at every stage. Building the graph locally first, then deploying it to cloud GPU infrastructure, is the workflow that separates a working production pipeline from an experiment that only runs on one machine.

The Two-Adapter Chain That Made a Consistent Model Face Possible Across an Entire Catalog

Knowing that diffusion follows text instructions explains how to generate any image. It does not explain how to generate the same model face across three hundred different product photos, or how to ensure the garment in the output matches the actual product in the catalog rather than an AI approximation of something loosely similar.

Both problems are solved by IP-Adapter, a technique from Tencent that injects a reference image into the generation process without fine-tuning the underlying model. Fine-tuning, the conventional approach to consistent identity across generated images, requires training the model on dozens or hundreds of photos of a specific subject. That process takes hours of compute and a level of technical expertise most store operators do not have access to. IP-Adapter bypasses this entirely. A single clear reference photo is sufficient to carry a face or a garment into new generations, constraining the denoising process from its earliest stages toward visual consistency with the reference.

The critical architectural choice is chaining two adapters rather than using one. A single adapter can lock either the face or the garment. Chaining two locks both, in an order that matters. The first adapter locks the face. From the beginning of the denoising process, the generation is constrained to produce a person whose features match the reference photo. The second adapter then locks the garment, constraining the output further so the clothing corresponds to the actual product image rather than a visual approximation.

The order is not interchangeable. Running the garment adapter before the face adapter allows the garment's visual features to influence identity in the early stages of generation, which causes the model to look subtly but noticeably different across a catalog run. Face first, garment second is the only sequence that keeps the model recognizable across hundreds of outputs. The text prompt then sets the scene after both adapters have applied their constraints. The same model in the same garment appears in a cafe, a park, an outdoor market, or a clean studio backdrop by changing the scene description alone.

Over a catalog of forty garments with five background variations each, this produces two hundred unique on-model images from a single reference photo and a single configured pipeline run. At a generation cost between five and twenty-five cents per image on cloud GPU infrastructure, the monthly compute budget for a daily feed of twenty images lands between sixty and one hundred fifty dollars.

The full pipeline assembles in ComfyUI and exports to a JSON graph that deploys to Replicate, a cloud service that runs the workflow on powerful GPUs at scale. A generation that takes several minutes on a local desktop machine finishes in roughly two minutes on cloud hardware, and parallel execution allows a full daily batch to complete overnight without supervision.

Madhuranjan Kumar has tracked the conversion rate impact of systematic on-model imagery against flat product photography across multiple catalog businesses. The improvement is consistent. A product page showing a garment on a person in a scene that feels real converts at a measurably higher rate than the same page with a flat image on white. A one percentage point lift in conversion on a store doing fifty thousand dollars per month in revenue is an additional five hundred dollars per month from the same traffic. At the compute costs this pipeline operates on, that lift covers the full monthly budget several times over from the first month forward, and every month after that.

What the Reviewer Agent Catches That the Generator Never Would Have Flagged Itself

A pipeline that generates usable imagery most of the time is not a pipeline that ships quality imagery every time. Diffusion models produce incorrect outputs on a small but consistent fraction of runs. Garment colors drift from the original. Textures get misrepresented. Hands and fingers distort in ways that range from subtle to obviously wrong. At catalog volume, that fraction represents a real number of images, and without a quality gate, the options are either manual review of every output, which reintroduces the human time the pipeline was meant to eliminate, or publishing occasional errors, which damages the brand.

The solution is a two-stage reviewer agent built with AutoGen, a multi-agent orchestration framework. In the first stage, a generator produces an image and a reviewer examines it using vision capabilities, comparing the garment in the output against the original product photo and returning either an approval signal or a revised prompt for another generation attempt. The loop continues until the output clears a fidelity threshold of roughly ninety-five percent garment match. Most images pass on the first or second attempt. The reviewer catches the rare badly distorted output before it reaches the queue.

In the second stage, an enhancer handles artifacts that cannot be corrected through re-prompting. Distorted hands and extra fingers are repaired using a specialized model trained specifically for anatomical correction, called as a separate step on Replicate. After the repair pass, an upscaling model lifts the image to the resolution required for product pages, where larger display sizes make artifacts visible in ways a social thumbnail does not expose.

The reviewer agent catches what the generator never flags because it applies a different question. The generator asks whether the output is consistent with the prompt. The reviewer asks whether a customer looking at this image would recognize the specific garment they are considering buying. Those are different questions. The generator passes on any image that looks like a plausible person in a plausible outfit. The reviewer passes only on images where the actual product is correctly represented, which is the standard that matters for a catalog trying to sell real things to real people.

The pipeline delivers a reviewed, corrected, upscaled queue of on-model images each morning without requiring a human to evaluate each output. For a store that was producing two or three posts per week before, the shift to a daily feed of twenty on-model images is not an incremental improvement in content volume. It is the kind of compound advantage that builds a social audience over months in a way that a periodic photoshoot schedule never can. The content archive grows every day. Each image is a permanent asset for the product page and the social feed. The pipeline does not create a batch of content for the season. It creates content for as long as it runs.

Fashion product photography is a commodity now. The stores that treat it like one, by building the pipeline that makes it cheap, fast, and scalable, are building an advantage that their competitors are not yet positioned to match. The ones who figure this out in the next twelve months will have a content library and a social audience that took large brands years and far larger budgets to build. The timing window is real and the tools are available. The bottleneck has moved from the photoshoot to the decision to build the pipeline.