A Proper Split Test
Do you really think you know how to do a proper ads split test? You may be surprise to find out that you have been using a wrong method…
If you don’t know what split test is…
Split test is a process to test the effectiveness of two or more Ad copies or, in another term, creatives. It is usually measured in term of click through rate (CTR). When we do a split test, we want to find out which Ads are more effective in generating click.
Setting:
First step of split test is to tick off (in another words, stop running) Seach network and Content network. Leave only Google search in Campaign settings.
Generally, the CTRs of an Ad in Google search and Search network are quite different. Comparing CTRs of two Ads with one in Google search and another in Google+Network search are like comparing apple with orange. When we add a new ad, this ad will display instantly for Google Search but not Search Network. Google team will take a while to review the new ad before approving it to be displayed in Search Network. So if we compare the CTRs of an old ad (displayed in both Google Search & Search Network) with a new ad (displayed only in Google search), we will get a distorted picture of comparison.
We can wait until the new ad is displayed in Search Network to do the comparison, but we don’t know the exact date when the new ad started to be displayed in Search Network. So the logical method is to deactivate the old ad from displaying in Search Network for a fair comparison.
Second, you tick of Ad rotation and, therefore, NOT to show better ads more often.
More about split testing technique at Google Groups Adwords Help forum.
Unit of comparison:
CTR, instead of number of clicks, should be the comparison factor. But how many clicks should be sufficient for a fair comparinson of CTR? This is from the thread by jtara from the Google Groups Adwords Help forum.
30 clicks is a good rule of thumb. A more accurate way is to use statistical confidence. There’s a handy statistical confidence calculator for AdWords here;
You enter the # of clicks and CTR for each of the two ads, and it will calculate a confidence level.
This is the explanation on splittester’s methodology.
Comparing CTR:
After a few days (it really depends on the click popularity of your ads and keywords.) we may start comparing your newly create ads and old ads. From the image below, we should choose a date after the new ads were created. We will have a fair comparison of the effectiveness of the ads.
Creation and variations of ads:
You may find more about creating variations of ads for split testing at the blog by Brad Geddes.
