Testing Cart Abandonment

Test­ing has evolved for online mar­keters so much in the last few years that it’s easy to test any­thing and every­thing. It’s also easy to seg­ment your audi­ence to test and tar­get con­tent just for them. It’s mind-boggling to think of the days when we were lim­ited to sin­gle ses­sions and to single-page test­ing. With the lat­est test and tar­get tech­nol­ogy, all that has changed.

Recently, we were work­ing with a client and wanted to intro­duce them to behav­ioral tar­get­ing for test­ing. We had already started the ini­tial rounds of tests to tar­geted paid search traf­fic but felt there was an oppor­tu­nity to move the nee­dle a bit more. There are end­less things that one can do with behav­ioral tar­get­ing. At a high level though, behav­ioral tar­get­ing allows mar­keters to tar­get or seg­ment based on pre-defined behav­iors of their visitors.

Start Small Enough to Show a Difference

For this par­tic­u­lar client, a large retailer, we wanted some­thing rel­a­tively easy to do in terms of exe­cu­tion and design so as to deliver a small taste of what is pos­si­ble. We cre­ated a test that would be tar­geted to those vis­i­tors that added some­thing to their cart but failed to com­plete their pur­chase, and then made a sub­se­quent visit to a cat­e­gory page, prod­uct page, or the home page on the same ses­sion or a later ses­sion. This seg­ment can be described as “aban­doned carts.”

Set Up a Mon­i­tor­ing Cam­paign First

Now before I we explained to the client what we wanted to do we thought it best to get some data to help explain the goal and our desired out­come. To do this we set up a mon­i­tor­ing cam­paign to track these cart aban­don­ers. A mon­i­tor­ing cam­paign is sim­i­lar to a reg­u­lar cam­paign or test but does not deliver con­tent. It is mainly used to iden­tify base­lines so as to visu­al­ize the gains of sequen­tial A/B or mul­ti­vari­ate test­ing.

For this mon­i­tor­ing cam­paign to cap­ture the desired data we had to cre­ate the seg­ment for cart aban­don­ers. To do this, we pro­filed or ‘tagged’ those vis­i­tors that added some­thing to their cart. We then pro­filed or ‘tagged’ those vis­i­tors that checked out or made a pur­chase. Pro­fil­ing is an enor­mous fea­ture that will cer­tainly be blogged about here soon in much greater detail.

Now, to qual­ify for the mon­i­tor­ing cam­paign a vis­i­tor had to be a cart-adder but not a pur­chaser. The vis­i­tor would then have vis­ited a cat­e­gory page, prod­uct page, or the home page. We did not limit these vis­its to sin­gle session.

The Mon­i­tor­ing Cam­paign Results

The results of this mon­i­tor­ing cam­paign were def­i­nitely impact­ful. We found that 30% of vis­i­tors finally made a pur­chase after leav­ing the fun­nel. We then had some inter­est­ing data to help us under­stand who we were focus­ing our test on and what would be our suc­cess met­ric. Our ini­tial goal wasn’t to increase the rev­enue met­rics such as RPV, AOV, or total sales. Our aim was to increase the con­ver­sion rate for the 70% of vis­i­tors that com­pletely failed to check out despite hav­ing added some­thing to their cart and mak­ing a sub­se­quent visit back to one of the ear­lier men­tioned areas of the site.

Now that we had the pro­files or ‘tags’ in place we were all set. We added an mbox to the top of the cen­ter “hero area” to the home page, cat­e­gory page, and the prod­uct pages. This mbox was named the same, which allowed for quick and easy set up in Test & Tar­get. The only thing needed now was a cre­ative ele­ment to remind vis­i­tors that they have some­thing in their cart.

Add Cre­ative

For the cre­ative, we included a hand­ful of dif­fer­ent designs to see if there were any dif­fer­ences in the per­for­mance. Lim­it­ing the test to no cart aban­don­ment mes­sag­ing vs. one piece of cre­ative lim­its your analy­sis quite a bit. If cart aban­don­ment mes­sag­ing is effec­tive but the cre­ative was poor you would get mis­lead­ing results in such a test.

The test or cam­paign was set up as follows:

Recipe A: Default Con­tent (No Cart Aban­don­ment mes­sag­ing)
Recipe B: Aban­don­ment mes­sag­ing (design 1)
Recipe C: Aban­don­ment mes­sag­ing (design 2)
Recipe D: Aban­don­ment mes­sag­ing (design 3)

With this type of test design we get to under­stand a cou­ple of things from the results.

1. Does cart aban­don­ment mes­sag­ing improve the con­ver­sion rates of this seg­ment of traf­fic?
2. Does one cre­ative over another affect things differently?

The results were pretty inter­est­ing. We saw a decent amount of con­ver­sion lift across Recipes B, C, D. This was great to see, as we were able to con­firm that we could pos­i­tively affect this seg­ment of traffic.

Recipe C though did a lot bet­ter then Recipes B and D. The cre­ative was very dif­fer­ent which lead us to believe that cre­ative does impact things as well.

Right now this cus­tomer is run­ning a champion/challenger test by test­ing Recipe C against another new vari­a­tion to see if we can lever­age the gains even more. This time we have 10% of traf­fic going to the con­trol, and 45% of traf­fic going to each of the cart aban­don­ment mes­sag­ing. This way we can quan­tify the impact of the mes­sag­ing while get­ting the biggest bang for our buck.

If you want to see a very sim­plis­tic mock up of the cart aban­don­ment mes­sag­ing using the Test and Tar­get plat­form, I invite you to visit this link.

Once you visit you will see this prod­uct page with four products:

If you actu­ally add one of these nice Mac prod­ucts to your cart and then revisit the prod­uct page you will see the tool in action with this messaging:

If you are a Test and Tar­get user and would like to see how I set up this Cart Aban­don­ment test please email me or make a com­ment on the post. I will then cre­ate a login into my Test and Tar­get account. Here you can not only see how I set up this test but the many more exam­ples I plan on post­ing here.