Posts Tagged ‘creative testing’

Three Search Marketing Resolutions for the New Year

By January 9th, 2013

With January underway, most search marketers have already made their personal New Year resolutions. Whether it’s better health, spending more time with friends and family, or helping out in the community, we all have high hopes for 2013. For me, and many other search marketers, the motivation to do better each year defines the strategies I hope to execute on in 2013. It all starts with planning and creativity. After analyzing this past year’s performance and factoring in the objectives that have been set, three additional New Year resolutions made it on my list. The following are what I hope to do more of for my paid search program in 2013 and how I plan to get there.

1. Test More of Everything

TestingMy testing plans tend to go through cycles of high activity and mild dormancy. Building a test, analyzing results, implement the findings, and building the next test takes time and often loses steam when coupled with a busy schedule. This year, I’m going to test more and test often. This will include not just search landing pages, but also ad creative, conversion pages, and display banners. I’m working with my design team to create an optimization calendar that fits their bandwidth and allows me to prioritize tests across our multiple marketing channels. Testing more will be especially be critical this year as I look to grow leads incrementally without expanding budgets.

2. Re-Organize Messy Campaigns

Account TaxonomyThough this resolution is much more time consuming and has more long term than short term implications, I believe that it will pay big dividends during critical seasonal periods of 2013. Many times search campaigns are built out with a structure that makes sense at that time. However, as campaigns mature and outgrow that structure, they no longer perform at optimal levels. Aggressive keyword expansion can lead to less relevant creative; and less aggressive negative keyword expansion can result in unwanted clicks. On top of that, as Google and Bing come out with new features and functionality, the account structure may become dated and less productive.

This year, I’m reviewing the structure of my campaigns and groups. Groups that achieved lower than average click-through rates (CTR) in 2012 will be split out into more granular ones. Since keywords under this new structure will vary less within any one group, I can then generate highly relevant creative and leverage more specific landing pages in hopes of increasing Quality Score, CTR, and conversion rate.

3. Prioritize Creative Optimization

AB Landing Page TestingI’ve talked about this particular resolution in my previous articles, yet it always seems to fall lower in the pecking order of my daily optimization efforts. Especially among small to medium sized businesses where budgets are tight, creative testing and optimization is critical to incremental growth and improvements in overall paid search performance. In fact, all three of my resolutions for 2013 fall into the category of maximizing performance without increasing budgets. Splitting keywords out into more granular groups will require continuous testing to increase relevance and improve creative performance. To capitalize on the largest revenue opportunities, I’ll prioritize my optimization efforts to groups with high traffic volume, but poor performance. This is likely where I’ll get the most return on my time spent optimizing creative.

The key this New Year is building on the momentum from 2012. There are plenty of new publisher features and tools like remarketing and Product Listing Ads out there to try. Regardless, I’ve decided to rely on these three basic optimization strategies—aligning them with my marketing objectives—to improve my paid search program and increase conversions in 2013.

The Search Marketers Guide to Creative Testing and Optimization, Part 4

By September 5th, 2012

From selecting creative elements for testing to reaching statistical significance, this four part blog series reviews basic and advanced tips for conducting a successful creative test. Last week, we discussed how to prioritize tests based on return and the importance of limiting test elements. Today, in the final part of this four part series, we’ll walk through campaign rotation settings and the importance of implementing test tracking and reaching statistical significance.

Check Campaign Rotation Settings

Ensuring even creative rotation is one of the most commonly overlooked steps when implementing a creative test. Google offers three creative rotation settings: optimize for clicks, optimize for conversions and rotate evenly. Optimizing for clicks is the default setting, but does not promote a fair test since creative that provide more clicks, are displayed more often. Opting to rotate evenly results in a fair and more statistically significant test, as each creative receives an even number of impressions. This setting also favors the utilization of key performance indicators (KPI) outside of clicks and conversions, such as conversions per impression or return on ad spend (ROAS), to determine top performing creative.

AdWords Campaign Rotation Settings

 

 

 

 

 

 

 

Keep in mind that Google campaigns set to rotate creative evenly will only do so for 90 days after the last creative was enabled or edited (unless opted-out of this functionality). After this 90-day period, creative will automatically optimize for clicks. This shift in rotation setting occurs regardless of whether or not statistical significance has been reached within the test. Focusing on higher volume groups will help to reach significance within the rotation period. Pausing poor performing creative and generating new creative for testing will reset this 90-day clock.

Implement Tracking Before Each Test

Day-to-day campaign management and optimization has a tendency to overshadow ongoing creative tests. Once a new test has been launched, it is critical to take note and track the where and when of each new creative. Where in the account is the creative being tested? And when was the creative activated?

Marin AB Test Tracking

 

 

 

 

 

 

 

 

With larger scale accounts, locating old creative tests can feel like searching for a needle in a haystack. Taking note of when the account, campaign and group a new creative was added can mean the difference between a successful test and one that never happened.

Reaching statistical significance in a creative test can take days, weeks and sometimes months. Even for high volume groups that reach significance faster, knowing when a new creative was added defines the date range used for reporting and analysis. In addition, low-volume groups that utilize longer date ranges are more susceptible to Google’s 90-day rotation functionality and thus require more attention. Knowing when the 90-day rotation period is nearing its end means tracking when a creative test started.

Reach Statistical Significance and Stop

Online marketers often end creative tests too early or let them run for too long. Determining a winning creative through statistical significance requires patience, and is one of the most difficult steps in a successful creative test. Achieving statistical significance leaves little doubt that a new creative outperforms the other creative in the group.

Prior to calculating significance, a KPI such as conversions per impression, CTR, conversion rate or ROAS should be selected for evaluating the performance of each creative. By definition, creative performance is statistically significant if it is unlikely to have occurred by chance. To calculate this, many creative testing tools utilize a Student’s T-Test with a user-defined confidence level. This test determines the likelihood that the difference between a single creative and the average of all creative in the group, has not occurred by chance. A confidence level between 80% and 99% is standard, but keep in mind that low-volume groups do not support high-confidence levels and are not likely to achieve statistical significance as fast as high-volume groups.

In Conclusion

Search marketers are constantly searching for ways to find and engage their target audience. No optimization strategy is more central to accomplishing this goal than creative testing. Continuously testing to find more relevant and more compelling creative serves to not only increase CTR and Quality Score, but decrease costs and drive more revenue.

When implementing creative tests, search marketers must remain disciplined at every step. New creative must remain focused and relevant. Tests must be prioritized, tracked and statistically significant. Adhering to best practices and avoiding common pitfalls will help ensure that new iterations of creative will incrementally improve account performance. Though search marketers cannot guarantee that all creative tests will be successful, they can guarantee that all creative tests have been set up for success.

 

Download The Search Marketers Guide to Creative Testing and Optimization for additional best practices and two case studies from BoostCTR on how they successfully test and optimize creative.

 

The Search Marketers Guide to Creative Testing and Optimization, Part 3

By August 22nd, 2012

From selecting creative elements for testing to reaching statistical significance, this four part blog series reviews basic and advanced tips for conducting a successful creative test. Last week, we discussed how to prioritize and test keyword tokens and leverage dynamic keyword insertion to increase creative relevancy. Today, in part three of this four part series, we’ll walk through how to prioritize tests based on return and the importance of limiting test elements.

Prioritize Tests Based On Return

As paid search programs grow, it becomes increasingly challenging to implement and manage creative tests across all groups within an account. To optimize creative at scale, prioritize tests to focus on groups with the most potential to shift overall account performance. These groups are characterized by a high share of impressions, clicks or conversions within an account.

Due to limited resources, our fictional retailer, PowPow Sports, decided to only test creative in two of the groups within their account. Group A received 10,000 impressions per week, while group B received 1,000. Each test resulted in equal improvements in performance within its respective group. The table below highlights the improvements in group performance after creative testing and highlights the potential performance of another, untested Other group.

Prioritize Creative Tests Based on Return

 

 

 

 

 

 

This example simplifies a common challenge where groups with little to no volume are prioritized over Other, higher volume groups. Though both groups benefited from a creative test, group A experienced a greater increase in clicks and conversions. Each test took the same amount of time to implement, but one resulted in a greater revenue return on time investment. Prioritizing creative tests for high volume groups has the greatest potential for incremental improvements in overall account performance.

Limit Test Elements

A new creative might be subject to one or many test elements. It can be triggered by a single set or multiple sets of keyword tokens. And it might share impressions with another or many other creative within the group. Without controlling these variables, it becomes difficult to reach statistical significance and to determine what factors contributed towards a successful or unsuccessful creative test.

Limiting the number of elements within a creative test makes it easier to identify why one creative performed better than another. For example, assume that PowPow Sports is testing two new creative. One tests a free shipping offer, creative B, and the other tests several formatting and language elements, creative C. Even with improved performance on the new creative, it would be unclear as to which test element in creative C contributed to its success. Testing each element one at a time will better determine its individual impact on creative performance.

 

Good Test

Tests a single element in Description Line 2

A                                                      B
Shop PowPow Sports OriginalShop PowPow Sports Good

 

 

 

 

 

Bad Test

Tests too many elements across the entire creative

A                                                      C
Shop PowPow Sports OriginalShop PowPow Sports Bad

 

 

 

 

 

To promote an optimal creative testing environment, keep keyword lists concise when building out new campaigns and groups. Groups that contain a small set of highly granular keywords allow the creative within that group to focus on a small set of tokens. Rather than having to test tokens to improve relevancy, creative within these groups can test compelling offers and calls-to-action that drive greater increases in CTR and conversion rate.

The rate at which a creative test reaches statistical significance is associated with the number of creative within the group. Testing a large number of creative requires a large number of impressions. With smaller, low volume groups, this requirement becomes an issue. For a group that receives only 1,000 total monthly impressions, testing ten creative variations might take several months to reach statistical significance.

For larger, high volume groups, reaching statistical significance is less of a concern. However, the opportunity cost of running on underperforming creative must be monitored much more closely. Underperforming creative within these groups accrue a high volume of impressions that are better served on top performing creative, and should be paused once statistical significance is reached.

To Be Continued

Adhering to best practices and avoiding common pitfalls will help ensure that new iterations of creative will incrementally improve account performance. Though search marketers cannot guarantee that all creative tests will be successful, they can guarantee that all creative tests have been set up for success. In part four of this series, we’ll review three additional best practices for creative optimization.

 

Download The Search Marketers Guide to Creative Testing and Optimization for additional best practices and two case studies from BoostCTR on how they successfully test and optimize creative.

 

The Search Marketers Guide to Creative Testing and Optimization, Part 2

By August 15th, 2012

From selecting creative elements for testing to reaching statistical significance, this four part blog series reviews basic and advanced tips for conducting a successful creative test. Last week, we discussed how to select an appropriate test, maintain keyword relevance and limit opportunity costs. Today, in part two of this four part series, we’ll take a look at how to prioritize and test keyword tokens and leverage dynamic keyword insertion to increase creative relevancy.

Test Keyword Tokens

Keyword tokens within a creative will appear in bold whenever they match or closely match a user’s search query. Tokens are the individual terms that make up a keyword. For example, the keyword “mens hiking backpacks” contains three tokens: “mens”, “hiking” and “backpacks”. Improve the relevancy of your creative to your keywords by testing tokens. Discovering the most relevant tokens promotes higher Quality Scores by increasing CTR.

A highly granular group might only contain keyword variations using the three tokens “mens”, “hiking” and “backpacks”. Generating creative that includes these tokens is rather trivial. Conversely, a very general group might support more tokens and additional keyword variations. Consider the search results below, where each creative utilizes a different set of the search query’s tokens. A group might contain keyword variations of “mens hiking backpacks”, “mens rucksacks” and “mens bags”. Though all three keywords share similarities, generating a relevant creative for this group proves to be a difficult task. Opting to include “rucksacks” rather than “backpacks” makes the creative less relevant to consumers searching for backpacks. Not including “mens” makes the creative less relevant to male consumers. Without splitting these keyword variations out into individual groups, deciding which tokens to include will require thorough testing.

Testing Tokens Search Results

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

To prioritize your test variables, pay close attention to the impression share of each token within the group. For instance, assume keywords that contain the token “backpacks” account for 90% of the total group impressions and keywords that contain the token “packs” account for 20% of the total group impressions. Because searchers are more likely to include “backpacks” in their query, generating a creative that includes “backpacks” will likely results in a greater overall CTR than a creative that includes “packs”.

Leverage Dynamic Keyword Insertion

One of the easiest ways to incorporate keyword tokens within creative is through the use of dynamic keyword insertion. Inserting {keyword:default text} into the headline, description line or display URL dynamically populates the creative to include the keyword that triggered the creative. In Google, modifying the keyword insertion parameter controls which tokens in the keyword are capitalized. For both publishers, if the inserted keyword causes the creative to exceed the character limitations, the default text is used instead. For example:

 

Creative:

Dynamic Keyword Insertion Default

 

 

 

 

 

Inside Limit:  Keyword: [red hiking boots]

Dynamic Keyword Insertion Inside Limit

 

 

 

 

 

Outside Limit: Keyword: [mountain hiking boots]

Dynamic Keyword Insertion Outside Limit

 

 

 

 

Using dynamic keyword insertion multiple times in a single creative is a quick and effective way to increase relevancy. However, keep in mind that not all keywords make grammatical sense when inserted into a creative. Take the headline “Shop {KeyWord: Hiking Boots}” for example. If the keyword triggering the creative was “hiking boot”, the headline would read “Shop Hiking Boot”. Even a simple keyword variation such as this can result in an awkward-sounding creative. Granular and organized groups with well-written creative will benefit most from dynamic keyword insertion—resulting in increasing CTRs and Quality Scores.

To Be Continued

Adhering to best practices and avoiding common pitfalls will help ensure that new iterations of creative will incrementally improve account performance. Though search marketers cannot guarantee that all creative tests will be successful, they can guarantee that all creative tests have been set up for success. In part three of this series, we’ll review two additional best practices for creative optimization.

 

Download The Search Marketers Guide to Creative Testing and Optimization for additional best practices and two case studies from BoostCTR on how they successfully test and optimize creative.

 

The Search Marketers Guide to Creative Testing and Optimization, Part 1

By August 7th, 2012

In a search landscape where millions of keywords define intent, generating the most compelling creative can prove to be a daunting task. Understanding your audience and formulating a message might be easy but packaging up that message within the narrow limits of a creative is far from it. For paid search programs, big and small, creative optimization remains the single most impactful strategy for increasing traffic, lowering costs and acquiring more revenue.

From selecting creative elements for testing to reaching statistical significance, this four part blog series will review basic and advanced tips for conducting a successful creative test. Novice and advanced testers will gain a complete understanding of how to generate, analyze and iterate on new creative to maximize performance across paid search programs. Furthermore, they will be equipped with the best practices necessary to make the implementation of a disciplined and statistically significant creative test a reality. Today, in part one of this four part series, we’ll review how to select an appropriate test, maintain keyword relevance and limit opportunity costs.

Select an Appropriate Test

All creative tests begin with a choice, and marketers are subject to a plethora of test elements to choose from. Even at a basic level, creative can be characterized by and tested with functional, emotional or promotional qualities. Functional creative focus on the product or service and provide information such as pricing or features. Emotional creative pull at metaphoric heart strings and attempt to form a connection between the customer and the product or service. Promotional creative highlight discounts and evoke a sense of urgency. The table below lists common elements that helps define how a creative is characterized. Before engaging in creative testing, it is important to identify these elements and understand the benefits each one provides.

Creative Testing Elements

 

 

 

 

 

 

 

Maintain Keyword Relevance

Testing too many different elements can lead to a sacrifice in relevancy. For instance, removing keywords in favor of testing pricing or unique selling propositions may correspond to drops in Quality Score, resulting in increased cost-per-clicks (CPCs) and decreased CTRs. When testing creative, the objective is to generate and test compelling creative while maintaining keyword-to-creative relevancy.

Let us use PowPow Sports, a fictional sporting goods retailer, as an example. A group containing variations of the keyword “mens hiking backpacks” has been set up to test between functional and promotional creative. Creative A describes various hiking backpacks for men, while creative B highlights a free-shipping offer during Father’s Day weekend.

A.

Hiking Backpacks - A

 

 

 

B.

Hiking Backpacks - B

 

 

 

C.

Hiking Backpacks - C

 

 

 

Because creative B only utilizes the phrase “mens hiking backpacks” once, it has become less relevant to the keywords within the group. Successful creative utilize compelling language, while remaining highly relevant to the user’s search query. A more effective creative C incorporates the free-shipping offer, which influences conversion, without sacrificing occurrences of the phrase “mens hiking backpacks”, which influences click.

Limit Opportunity Costs

Opportunity cost is the cost of any activity measured in terms of the value of forgoing the next best alternative. To better understand this, let us assume that our fictional sporting-goods retailer, PowPow Sports, has decided to generate and test three creative: A, B and C. The table below compares the performance of each creative, at statistical significance.

Opportunity Cost Summary

 

 

 

 

Let us assume PowPow Sports decides to pause creative B, and continue testing creative A and C. Assuming that performance remains consistent, the opportunity cost of this activity, as well as testing creative B against creative C, is highlighted in the table below.

Opportunity Cost A-C

 

 

 

 

In continuing to test creative A, even after achieving statistical significance, PowPow Sports has lost out on 100 clicks, five conversions and $375 in revenue. This is because half the available impressions were allocated to the under-performing creative A, rather than all the available impressions being allocated to just creative C.

When testing creative, the adverse effects of opportunity cost are twofold. One, if you do not test at all, you forgo the benefits of running on better performing creative. But marketers also make a common mistake when testing – continuing to test after statistical significance has been reached. This results in the missed opportunity to simply run on the better performing creative. In our example, PowPow Sports should have paused creative A and B, leaving 100% of the impression share to creative C.

To Be Continued

Search marketers are constantly exploring ways to find and engage their target audience. No optimization strategy is more central to accomplishing this goal than creative testing. Continuously testing to find more relevant and more compelling creative serves to not only increase CTR and Quality Score, but decrease costs and drive more revenue.

Adhering to best practices and avoiding common pitfalls will help ensure that new iterations of creative will incrementally improve account performance. Though search marketers cannot guarantee that all creative tests will be successful, they can guarantee that all creative tests have been set up for success. In part two of this series, we’ll review two additional best practices for creative optimization.

 

Download The Search Marketers Guide to Creative Testing and Optimization for additional best practices and two case studies from BoostCTR on how they successfully test and optimize creative.

 

Tester Beware, Google To Tweak Ad Rotation Settings

By May 1st, 2012

Yesterday Google announced a significant tweak to their “rotate evenly” creative rotation setting. Today, campaigns that are set to “rotate evenly” will have their creative rotated evenly for an indefinite amount of time. Starting next week, campaigns using this setting will only rotate creative evenly for 30 days after the last creative was enabled or edited. After the 30-day period, creative will automatically optimize for clicks (one of the three existing ad rotation settings). Keep in mind that the campaign will remain labeled “rotate evenly” even after the rotation period has ended for any or all of the groups within that campaign.

Though this tweak is said to provide users with more relevant ads and help advertisers achieve better performance, it forces search marketers to be more in-tune with their ongoing creative tests. When testing, hone in on creative that deliver statistically significant results for your business, whether that’s clicks or conversions. Focus on high traffic groups to reach significance within the 30-day rotation period. Once significance is reached, pause “losing” creative and test new iterations of the “winning” creative. This resets Google’s 30-day clock and preserves the continuity of your creative test.

Creative testing tools, like Marin’s AB Testing solution, automatically reveal “winning” and “losing” creative once statistical significance has been reached. This logic, coupled with a best practice focus on testing creative in top traffic groups, should make the new 30-day rotation period a reasonable time frame for testing. As a best practice, take note of start dates when engaging in creative testing. Marin users should continue leveraging Dimensions as part of their creative testing workflow. These tags will help alert you when a test is nearing the end of its 30-day rotation period.

Resources

Find us on Facebook