I am currently working on a website, and we have just started an A/B test for revenue. I can not use exact details, but my question outline is as follows. Please note I am a developer, I am purely posting this query out of interest for myself. It may be a fundamental question which marketers would know by default, so please don't assume I know what I am talking about if it turns out to be simple.
We have a website (for example) which sells cats, you go to the website and can select the number of cats you want (they all look the same, are as cute as each other). They cost £5 each. There is a dropdown on the page which allows you to select a hat for the cat. The default blue hat is free, but a snazzy gold hat costs £3 extra. So if you pick a gold hat, the total costs are £8 per cat.
When you get to the page, four cats are selected by default (including a free blue hat each). So the total starting displayed example price is £20 (4 x cats, 4x free blue hats), you can amend this as you want.
When the user makes an order we are using tracking this revenue using e-commerce tracking in GA.
OK, so we now have created a 'B' page from the above with the gold hats auto-selected (to A/B test). So when the user gets to the page, the default starting displayed example price is £32 (4 x cats, 4x £3 gold hats). The user can again amend this (even switching the dropdown back to free blue hats).
OK, my query is, if we are running this test using the total revenue. Is this fair? As the price for the B is higher. We may make more money from 'B' but how would we know if we have lost more sales because people thought it too expensive (without realising they could opt for free hats initially). Do the lost sales matter? Or will it all work out in the end?
Any recommendations welcome.