Pawel Flajszer Slip-boxNotebooksWorkAbout
written
rests on
1
cited by
3

Outliers mask themselves under ±2σ detection because they inflate the σ

It’s a self-fulfilling prophecy. Standard deviation’s (σ) very definition relies on the mean. Mean is not robust to outliers. The mean explodes with outliers present, therefore, the differences between all values (especially the outliers) and the mean grows. Those distances between mean and each data point then get squared in the process of calculating variance (en route to computing the σ). In result, the standard deviation is much larger and using “the golden rule” of ±2σ does not work - 2 SDs from the mean engulf the very outliers it was designed to catch.

An example: [2,4,6] has SD ≈ 1.63; add a single 100 and SD explodes to 41.6 - a 25× jump from just one outlier.

Rests on

Cited by