Variance for grouped data

For frequency data $(x_i, f_i)$ with $N = \sum f_i$: $\sigma^2 = \frac{1}{N} \sum f_i (x_i - \bar{x})^2 = \frac{\sum f_i x_i^2}{N} - \bar{x}^2.$

For grouped data, take $x_i$ as the midpoint of each class.

When the $x_i$ are large or awkward, two tricks reduce the arithmetic.

Short-cut method

Pick a convenient origin (assumed mean) $a$. Let $d_i = x_i - a$. Then $\sigma^2 = \frac{\sum f_i d_i^2}{N} - \left(\frac{\sum f_i d_i}{N}\right)^2.$

Reason: variance is shift-invariant; subtracting $a$ from every $x_i$ leaves $\sigma^2$ unchanged.

The mean is $\bar{x} = a + \dfrac{\sum f_i d_i}{N}$.

Step-deviation method

When the class widths are equal to some $h$, define $u_i = (x_i - a)/h$. Then $\sigma^2 = h^2 \left[\frac{\sum f_i u_i^2}{N} - \left(\frac{\sum f_i u_i}{N}\right)^2\right].$

This makes the $u_i$ small integers, which keeps the arithmetic clean. The mean is $\bar{x} = a + h \cdot \frac{\sum f_i u_i}{N}.$

Step-by-step procedure

1. Tabulate classes, find midpoints $x_i$.
2. Choose assumed mean $a$ (often a central midpoint).
3. Compute $u_i = (x_i - a)/h$.
4. Compute $f_i u_i$ and $f_i u_i^2$.
5. Apply formulas above.

Worked examples

Example 1. Find variance and SD for:

Midpoints $x_i$: $5, 15, 25, 35, 45$. $N = 40$.

Use $a = 25, h = 10$.

$u_i$: $-2, -1, 0, 1, 2$. $f u$: $-10, -8, 0, 7, 10$. $f u^2$: $20, 8, 0, 7, 20$.

$\sum f u = -1$. $\sum f u^2 = 55$.

$\sigma^2 = h^2 [55/40 - (-1/40)^2] = 100 [1.375 - 0.000625] = 100 \cdot 1.374375 = 137.4375$.

$\sigma = \sqrt{137.4375} \approx 11.72$.

Mean: $\bar{x} = 25 + 10 \cdot (-1/40) = 25 - 0.25 = 24.75$.

Example 2. Find variance for:

$N = 30$. $\sum f x = 40 + 160 + 300 + 200 + 150 = 850$. $\bar{x} = 850/30 \approx 28.33$.

$\sum f x^2 = 400 + 3200 + 9000 + 8000 + 7500 = 28100$.

$\sigma^2 = 28100/30 - (28.33)^2 \approx 936.67 - 802.56 \approx 134.11$.

$\sigma \approx 11.58$.

Example 3. Compute SD for the data with the same midpoints as Example 1 above using direct formula (no shortcut).

We already found $\bar{x} = 24.75$. Squared deviations $(x_i - \bar{x})^2$: $(5 - 24.75)^2 = 390.06$, $(15 - 24.75)^2 = 95.06$, $(25 - 24.75)^2 = 0.06$, $(35 - 24.75)^2 = 105.06$, $(45 - 24.75)^2 = 410.06$.

Multiply by $f_i$: $1950.31, 760.50, 0.94, 735.44, 2050.31$. Sum: $5497.5$. $\sigma^2 = 5497.5/40 = 137.44$. (Matches example 1.)

Example 4. Find variance of:

Midpoints: $5, 15, 25, 35, 45, 55$. $N = 76$. $a = 35, h = 10$, $u_i$: $-3, -2, -1, 0, 1, 2$.

$f u$: $-15, -18, -17, 0, 10, 12$. Sum: $-28$.

$f u^2$: $45, 36, 17, 0, 10, 24$. Sum: $132$.

$\sigma^2 = 100 [132/76 - (-28/76)^2] = 100 [1.7368 - 0.1356] = 160.13$.

$\sigma \approx 12.65$.

Mean: $35 + 10 \cdot (-28/76) = 35 - 3.68 = 31.32$.

Example 5. Without computing, what happens to the SD if every observation is multiplied by $5$?

$\sigma_y = |5| \sigma_x = 5 \sigma_x$.

Try it yourself

1. Find variance and SD of: | $x$ | $2$ | $4$ | $6$ | $8$ | $10$ | |---|---|---|---|---|---| | $f$ | $3$ | $5$ | $7$ | $4$ | $1$ |

2. Find SD of: | Class | $0$–$5$ | $5$–$10$ | $10$–$15$ | $15$–$20$ | |---|---|---|---|---| | $f$ | $4$ | $6$ | $5$ | $3$ |

3. Compute mean and SD using step-deviation method for: | Class | $20$–$30$ | $30$–$40$ | $40$–$50$ | $50$–$60$ | $60$–$70$ | |---|---|---|---|---|---| | $f$ | $5$ | $8$ | $10$ | $4$ | $3$ |

4. The mean and SD of $20$ observations are $10$ and $2$. Each observation is multiplied by $3$. Find the new mean and SD.

5. If the SD of $x_i$ is $\sigma$ and each $x_i$ is replaced by $a + b x_i$, what is the new SD?

6. Mean of $50$ observations is $40$ and SD is $6$. Find $\sum x_i$ and $\sum x_i^2$.

7. SD of the first $100$ natural numbers (Use $\sigma = \sqrt{(n^2 - 1)/12}$).

8. The variance of $5$ numbers $a, b, c, d, e$ is $9$. Find the variance of $a + 2, b + 2, c + 2, d + 2, e + 2$.

9. Find variance of $x_i$: $-2, -1, 0, 1, 2$.

10. The mean of $5$ observations is $4.4$ and variance $8.24$. If $3$ of them are $1, 2, 6$, find the other two.

11. SD of $20$ observations is $5$. If three new observations $0, 4, 7$ are added, find new SD.

12. If $\sum (x_i - 5) = 30$, $\sum (x_i - 5)^2 = 230$, $n = 10$, find $\bar{x}$ and $\sigma$.

Pitfalls / Tricks

• Always use midpoints for grouped data.
• Step-deviation: $u_i$ should be small integers.
• Remember $\sigma_y = |b| \sigma_x$ when $y = a + b x$ , shift contributes nothing.
• Insight. Variance is a quadratic statistic , it cares about squared deviations. This both amplifies outliers and makes it mathematically tractable.