Wayne Parham Messages: 18758 Registered: January 2001
Illuminati (33rd Degree)
This is why the volume levels are to be held constant in any A/B test. It's also why I don't consider power increases less than 2x to be particularly significant. If you A/B test two things with 10% difference, you'll surely hear it and the louder one has a slight advantage. But if you're not A/B testing, you really have to double power to notice much of a change. A ten percent increase seems like almost nothing at all, barely perceptible unless compared side-by-side.