The Customer Service Survey
The problem with statistics
Mon - April 16, 2007 10:15 AM in
In geometry, there's something called the "left hand rule," which is used to visualize 3D vectors. Unless you use a right-handed coordinate system, in which case you use the right hand rule. It's completely arbitrary, and you find both in computer graphics. Right-handed coordinates are the standard in physics, while left handed (arguably) makes more sense to artists.
Similarly, computer processors have a notion of "endianness". Some systems are big-endian, others are little-endian. The choice is so arbitrary that the name comes from the debate in Gulliver's Travels over whether Lilliputians should crack their eggs from the big end or little end.
Whenever you have two independent groups of people inventing (or using) a complicated tool, they develop different names and conventions for the same things. That can make it difficult to figure out what's really going on. Indeed, it makes it tricky for one group to even discover that the other group exists. (The Wikipedia entry for Right Hand Rule [http://en.wikipedia.org/wiki/Right_hand_rule] doesn't even mention coordinate systems, even though it links to a site that does [http://mathworld.wolfram.com/Right-HandRule.html].)
The problem with statistics is that it's a mathematical tool that every branch of science and engineering uses, and each one uses it differently. The book I learned statistics from is "Statistics for the Behavior Sciences." But it doesn't even mention margin of error, even though that's the most common statistic used to describe opinion polls. And much of behavioral science (psychology) is conducted with surveys! Apparently pollsters and psychologists don't talk much to each other.
Nor is it just psychology that's different. I've been trying to figure out the derivation of Peter's margin of error formulas from last week [http://www.vocalabs.com/resources/blog/C834959743/E20070402150458/index.html]. He has a physics background. Wikipedia is maddeningly inconsistent (no surprise), and even MathWorld [http://mathworld.wolfram.com/] is troublesome-- although more along the lines of "Note that some authors define the term as...."[http://mathworld.wolfram.com/Erf.html].
Statistics are like the water heater in your basement which you install once and don't think about until it breaks. People figure out what formulas they need, and then forget about them. Even statisticians don't get hung up on nomenclature. Which is too bad. The math is hard enough, without having to deal with different--and often conflicting-- definitions for similar things.
One thing you should know is that the formulas in Peter's blog entry are approximations. Instead of 1/sqrt(N), most people would write the margin of error formula as 0.98/sqrt(N). (Our approximation is slightly more conservative.) But if you need that much precision, getting the margin of error right is the least of your worries.
Posted by David Leppik
Whenever you have two independent groups of people inventing (or using) a complicated tool, they develop different names and conventions for the same things. That can make it difficult to figure out what's really going on. Indeed, it makes it tricky for one group to even discover that the other group exists. (The Wikipedia entry for Right Hand Rule [http://en.wikipedia.org/wiki/Right_hand_rule] doesn't even mention coordinate systems, even though it links to a site that does [http://mathworld.wolfram.com/Right-HandRule.html].)
The problem with statistics is that it's a mathematical tool that every branch of science and engineering uses, and each one uses it differently. The book I learned statistics from is "Statistics for the Behavior Sciences." But it doesn't even mention margin of error, even though that's the most common statistic used to describe opinion polls. And much of behavioral science (psychology) is conducted with surveys! Apparently pollsters and psychologists don't talk much to each other.
Nor is it just psychology that's different. I've been trying to figure out the derivation of Peter's margin of error formulas from last week [http://www.vocalabs.com/resources/blog/C834959743/E20070402150458/index.html]. He has a physics background. Wikipedia is maddeningly inconsistent (no surprise), and even MathWorld [http://mathworld.wolfram.com/] is troublesome-- although more along the lines of "Note that some authors define the term as...."[http://mathworld.wolfram.com/Erf.html].
Statistics are like the water heater in your basement which you install once and don't think about until it breaks. People figure out what formulas they need, and then forget about them. Even statisticians don't get hung up on nomenclature. Which is too bad. The math is hard enough, without having to deal with different--and often conflicting-- definitions for similar things.
One thing you should know is that the formulas in Peter's blog entry are approximations. Instead of 1/sqrt(N), most people would write the margin of error formula as 0.98/sqrt(N). (Our approximation is slightly more conservative.) But if you need that much precision, getting the margin of error right is the least of your worries.
Posted by David Leppik
Posted at 10:15 AM | | | | |