What is a confidence threshold of 95,0% in Google Analytics Experiments?

by Jurik   Last Updated May 25, 2016 08:01 AM

In Google Analytics Experiments you can Set a confidence threshold 95.0% (or 99,0% or 99,5%). The documentation says:

Set a confidence threshold: The minimum confidence threshold that must be achieved before Analytics can declare a winner. The higher the threshold, the more confident you can be in the result. A higher threshold can result in Analytics taking longer to declare a winner.

I understand the words but I have no clue what that means. What is a confidence treshold of 95%?

Answers 1

Confidence threshold (or level) is a statistical term which, in a nut shell, denotes the confidence with which you can say that experiment A is better than experiment B.

It denotes the statistical significance with which you can declare the winner. You can read more about it here - http://www.stat.yale.edu/Courses/1997-98/101/meancomp.htm

To reach higher confidence level, you need more data and thus google will need more time to reach 99.0% level than it require to reach 90% confidence level and thus they ask this number so they can declare winner when they achieve this confidence level.

May 25, 2016 09:49 AM

Related Questions

Tracking alternatives withing goal funnels

Updated October 10, 2017 21:04 PM