Split Testing Ads Based Upon Impressions With Ad Optimizer Enabled

I'm interested in split testing multiple ads with Google's
Ad
Optimizer enabled.  As you know, Ad optimizer will shows the
better
performing ad more often, so it accrues more impressions,
and
therefore more clicks.

There is a fairly simple statistical equation for
determining a
winning ad based upon # of clicks for a true A/B ad split
test.
However, the quicker form of testing using the Ad Optimizer
is based
upon impressions.

For example, Ad 1 has accrued 821 clicks & 51,567
impressions for a
1.59% CTR.  Ad 2 has accrued 51 clicks & 3,385
impressions for a 1.50%
CTR.  Based upon clicks & impressions, how confident
should I be that
Ad 1 is the long-term better performing ad?

Here's what my Adwords Rep said:

"I can't really give you a specific number number that
would yield a 95% confidence interval because there are so
many
factors
involved. I can say that for making decisions about ad
deletion,
impressions is the metric that makes the most sense sense
(at least as
a
measure of duration). Making decisions based on click
numbers is
somewhat
flawed because, depending upon CTR, the number of clicks
doesn't
necessarily give a solid idea of the the amount of exposure
the ad has
received (essentially, your sample size in a more
traditional
experiment)."

Thanks.

-Josh


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the
Google Groups "SEM 2.0" group.

To post to this group, send email to SEM2googlegroups.com

**The moderators highly recommend you visit http://groups.goo
gle.com/group/SEM2 and log into the forum on the web, to
access the full range of posting, reading, and searching
options. To make your subscription "web-only" (no
email), you can set your preferences while logged into the
forum.**

To unsubscribe from this group, send email to
SEM2-unsubscribegooglegroups.com
-~----------~----~----~----~------~----~------~--~---

You can't.  Your rep is right.  This is why we turn off
auto-
optimization when we're testing ads.  Impressions can get
really out
of whack with the optimizer on, and we've always felt we
don't get a
good, statistically significant result when that's the case
-
especially when the numbers are fairly close, like yours
are.

Also, you'll want to turn off content targeting when
testing, since
ads don't rotate properly in the content network, even with
optimizer
turned off.

My bigger question for you is, which ad converted better? 
Or were you
just looking to drive traffic?

Melissa


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the
Google Groups "SEM 2.0" group.

To post to this group, send email to SEM2googlegroups.com

**The moderators highly recommend you visit http://groups.goo
gle.com/group/SEM2 and log into the forum on the web, to
access the full range of posting, reading, and searching
options. To make your subscription "web-only" (no
email), you can set your preferences while logged into the
forum.**

To unsubscribe from this group, send email to
SEM2-unsubscribegooglegroups.com
-~----------~----~----~----~------~----~------~--~---