We Tested AI UGC Ads Against Human UGC for 90 Days. Here’s What the Data Actually Says.
We did not run this test to prove AI UGC Ads work. We ran it because our creative pipeline was slow and expensive, and we needed to know where synthetic video was genuinely useful and where it was a trap. Ninety days later, the answer is more specific than “it depends” — and it is not the answer most vendor content gives you.
Why we ran the test, and what we were actually trying to learn
Our team spends roughly $180K per quarter on paid social across four categories: skincare, supplements, home goods, and pet. Before the test, every creative batch went through human creators. A typical batch took two to three weeks from brief to live asset, and about one in eight videos survived our kill rules. The question was not whether AI could make a video. The question was whether AI UGC Video could produce a readable signal faster and cheaper than the human pipeline, without collapsing our conversion rate.
We wrote down three hypotheses before launch. First, AI UGC Ads would win on cost per variation. Second, human UGC would win on trust-sensitive categories. Third, the metric that would separate them would not be CTR. Two of the three held. The third one surprised us.
The test design (and the rules we locked before launch)
We ran four cells, each with equal budget for fourteen days:
- AI hook-focused: AI UGC Ads built around a single objection-led hook.
- AI proof-focused: AI UGC Video built around product demonstration and before/after framing, with synthetic-performer disclosure.
- Human hook-focused: creator-shot videos using the same hook brief.
- Human proof-focused: creator-shot demonstration videos.
Before launch, we pre-registered three kill rules: a CPA ceiling per category, a minimum spend before any read, and a hold-rate floor. Pre-registering the rules mattered more than any tool choice. It stopped us from scaling a variant because it looked good on day one, which is the most common way small tests produce confident but wrong conclusions.
Tool note: for the AI cells we used a mix of platforms. We ran the first AI UGC Video drafts through AivideosUGC because it was the fastest way to produce a readable first batch without committing to a long production setup. Once a direction survived the first read, we moved volume production to a heavier platform. That two-stage approach — light tool for discovery, heavier tool for scale — is the part of the workflow we kept after the test.
What the numbers showed, category by category
The results were not uniform, and that is the point. Here is the honest split.
| Category | Where AI UGC Ads won | Where human UGC won | What we concluded |
|---|---|---|---|
| Home goods | Hook rate, cost per variation, speed to first read | Nothing material | AI-first pipeline is defensible here |
| Pet | Hook rate, format coverage | Hold rate in demo formats | Hybrid: AI for hooks, human for demos |
| Skincare | Cost per variation only | Hold rate, purchase intent, comment sentiment | AI as discovery layer; human for scaled asset |
| Supplements | Speed to first read | Purchase intent, trust signals | Human-led, AI for variant coverage |
In home goods and pet, the AI cells produced a usable read in roughly a third of the time, and the winning AI UGC Ads variant held up when we scaled it. In skincare and supplements, the AI cells often won the click but lost the next few seconds and the purchase intent read. That gap was consistent enough across fourteen days that we stopped treating it as noise.
The metric that lied to us
CTR was the most misleading number in the test. Several AI UGC Video variants had a cleaner opening frame, a more legible product shot, and a higher CTR than the human equivalents. If we had optimized on CTR, we would have scaled videos that looked strong and converted weakly. The hold rate told a different story: in skincare and supplements, the human variants held attention longer even when they lost the click.
The metric that held up best across all four categories was cost per incremental purchase at a fixed spend, read after the pre-registered minimum. It is a slower metric and it requires patience, but it was the only one that did not reward a video for being merely legible. If you run this test yourself, lock that metric before you launch, or you will end up scaling the wrong thing.
Where AI UGC Ads genuinely changed our workflow
The biggest gain was not cost per video. It was the number of readable tests per month. Before the test, we could run roughly one creative batch every two to three weeks. During the test, the AI cells let us run a new hook direction every few days without waiting on creator scheduling or shipping. That changed the shape of the creative meeting: we stopped arguing about which concept to produce and started arguing about which result to trust.
The second gain was format coverage. AI UGC Ads made it cheap to produce the same concept in four or five formats — talking head, demo, unboxing-style, text-led — and see which format carried the hook best. That is a job human creators can do, but not at the same speed or cost. For the discovery layer, that coverage is the actual product.
What we would not do again
We would not run AI UGC Ads in a trust-sensitive category without a human control cell. In skincare, the synthetic variants looked fine in isolation and underperformed against a real person once purchase intent was in the read. Without the human control, we would have concluded the opposite.
We would also not skip disclosure. Since June 9, 2026, New York law requires conspicuous disclosure when an ad features an AI-generated synthetic performer, and the FTC has been active on fabricated testimonials and overstated AI claims. We kept synthetic-performer disclosure in the proof-focused AI cells as a default. It did not hurt performance in home goods or pet, and it removed a compliance question we did not want to litigate later.
Our working rule after 90 days
AI UGC Ads are the discovery layer. Human creators are the trust and scale layer. In categories where the conversion mechanism is legibility and format coverage, AI can carry more of the pipeline. In categories where the conversion mechanism is lived experience, AI should find the direction and a human should close it. The brands that get this wrong are the ones that treat AI UGC Video as a universal replacement instead of a faster way to learn which concept deserves a real shoot.