Google Experiments Beta

mandeep383 — Sat, 08 Jan 2011 00:54:29 +0000

We have a client who is interested in integrating image ads into their existing Google text ad campaigns. Before we could confidently ad image ads across the entire campaign we first needed to test their performance while running alongside text ads versus the current performance gained from solely using text ads. An image ad takes up the space of up to four separate text ads depending on the placement, meaning the image ad has one possible cost whereas the text ad could have up to four possible costs per click (CPC) depending on its rank within the four said ads. Factors to monitor will be the click through rate (CTR), the CPC, as well as the conversion rate which will be determined by how many sales each side receives. We believed that the image ads would make up the extra needed sales and be equally cost effective if not more so than the text ads alone.

The problem we faced with such an experiment in the past was not being able A/B test the two strategies during the same time period. The next best method was to test the control the first month and then the experimental campaign the next month. From our experience this data would not truly be comparable because of the level of varied performance we receive month to month. To accurately know if an image and text ad campaign would perform stronger than a text ad only campaign we needed to test within the same time period.

Google is slowly integrating a new tool called Google Experiments. This new tool allows for an advertiser to set up a test experiment where they have control over both the control (text ads only) and/or the experiment (image & text ads). Each experiment can be set to run for a specific time period as well as the percentage of traffic you wish to be distributed between the control and the experiment. Once set up we were able to collect data with both the control and the experiment running side by side over a one month period with the traffic split evenly between the two.

The table below shows the results of the experiment:

	Impressions	Clicks	CTR	CPC	Spend	Conversions	Conv. Rate	CPA
Control	563,187	14,305	2.54%	0.13	1,859.64	94	0.66%	19.78
Experiment	549,871	28,483	5.18%	0.11	3,133.16	163	0.57%	19.22

Once all of the data had been collected and we calculated our results we saw that our theory was indeed correct. Our Experiment side comprised of both image and text ads resulted in significantly more clicks (28,483) compared to the Control side (14,305) which resulted in a similar difference in CTR. Over the course of the experiment the CPC paid by the Experiment side dropped to be $ .02 below the Control’s CPC due to its higher CTR granting it a higher quality score. The final question was did the Experiment sides conversion rate stay close enough to the Control sides so that the extra cost was made up for in extra sales? As we can see from the table the Experiment sides conversion rate did drop to .57% from .66%, however it was still enough to garner a lower CPA ($19.22) then the Control side ($19.78).

This left us and our client with the conclusion that adding image ads to their campaigns would indeed prove successful, not only granting them higher revenues with more sales but also lowering their actual CPA.