TAOCP 1.2.10 Exercise 12
Let G(z)=\sum_{k\ge0}p_kz^k, \qquad \sum_{k\ge0}p_k=1.
Section 1.2.10: Analysis of an Algorithm
Exercise 12. [HM21] When $G(z)=p_0+p_1z+p_2z^2+\cdots$ represents a probability distribution, the quantities
$$ M_n = \sum_k k^n p_k \qquad\text{and}\qquad m_n = \sum_k (k-M_1)^n p_k $$
are called the "$n$th moment" and "$n$th central moment," respectively. Show that
$$ G(e^t) = 1 + M_1t + M_2t^2/2! + \cdots; $$
then use Arbogast's formula (exercise 1.2.5-21) to show that
$$ \kappa_n = \sum_{k_1,k_2,\ldots,k_n \ge 0} \frac{(-1)^{k_1+k_2+\cdots+k_n-1}n!(k_1+\cdots+k_n-1)!} {k_1!1!^{k_1}k_2!2!^{k_2}\cdots k_n!n!^{k_n}} M_1^{k_1}M_2^{k_2}\cdots M_n^{k_n}, $$
where $k_1+2k_2+\cdots=n$. In particular,
$$ \kappa_1 = M_1,\quad \kappa_2 = M_2-M_1^2,\quad \kappa_3 = M_3-3M_1M_2+2M_1^3, $$
$$ \kappa_4 = M_4 - 4M_1M_3 + 12M_1^2M_2 - 3M_2^2 - 6M_1^4. $$
What are the analogous expressions for $\kappa_n$ in terms of the central moments $m_2,m_3,\ldots$, when $n \ge 2$?
Verified: yes
Solve time: 9m58s
Solution
Let
$$ G(z)=\sum_{k\ge0}p_kz^k, \qquad \sum_{k\ge0}p_k=1. $$
The moments are
$$ M_n=\sum_k k^n p_k. $$
Substituting $z=e^t$,
$$ G(e^t) =\sum_{k\ge0}p_ke^{kt} =\sum_{k\ge0}p_k\sum_{n\ge0}\frac{(kt)^n}{n!}. $$
Interchanging the sums,
$$ G(e^t) =\sum_{n\ge0}\frac{t^n}{n!}\sum_{k\ge0}k^np_k =\sum_{n\ge0}\frac{M_n}{n!}t^n. $$
Since $M_0=\sum_k p_k=1$,
$$ G(e^t) = 1+M_1t+\frac{M_2}{2!}t^2+\frac{M_3}{3!}t^3+\cdots. $$
Define the cumulants by
$$ \log G(e^t) = \sum_{n\ge1}\kappa_n\frac{t^n}{n!}. $$
Let
$$ F(t)=1+\sum_{n\ge1}\frac{M_n}{n!}t^n. $$
Then $\log G(e^t)=\log F(t)$. Applying Arbogast's formula to $\log F(t)$, the coefficient of $t^n$ is
$$ \frac{\kappa_n}{n!} = \sum_{\substack{k_1,\ldots,k_n\ge0\ k_1+2k_2+\cdots+nk_n=n}} \frac{(-1)^{k_1+\cdots+k_n-1}(k_1+\cdots+k_n-1)!} {k_1!1!^{k_1},k_2!2!^{k_2}\cdots k_n!n!^{k_n}} M_1^{k_1}\cdots M_n^{k_n}. $$
Hence
$$ \boxed{ \kappa_n = \sum_{\substack{k_1,\ldots,k_n\ge0\ k_1+2k_2+\cdots+nk_n=n}} \frac{(-1)^{k_1+\cdots+k_n-1} ,n!,(k_1+\cdots+k_n-1)!} {k_1!1!^{k_1},k_2!2!^{k_2}\cdots k_n!n!^{k_n}} M_1^{k_1}\cdots M_n^{k_n} }. $$
For $n=1,2,3,4$ this gives
$$ \kappa_1=M_1, $$
$$ \kappa_2=M_2-M_1^2, $$
$$ \kappa_3=M_3-3M_1M_2+2M_1^3, $$
$$ \kappa_4 = M_4-4M_1M_3+12M_1^2M_2-3M_2^2-6M_1^4. $$
Now let
$$ m_n=\sum_k (k-M_1)^n p_k $$
be the central moments. Write
$$ X=k-M_1. $$
Then
$$ G(e^t) = \sum_k p_k e^{kt} = \sum_k p_k e^{(M_1+X)t} = e^{M_1t}\sum_k p_k e^{Xt}. $$
Since
$$ \sum_k p_kX=0, $$
the expansion of the second factor is
$$ \sum_k p_k e^{Xt} = \sum_k p_k \sum_{n\ge0}\frac{X^n t^n}{n!} = 1+\sum_{n\ge2}\frac{m_n}{n!}t^n. $$
Therefore
$$ G(e^t) = e^{M_1t} \left( 1+\sum_{n\ge2}\frac{m_n}{n!}t^n \right). $$
Taking logarithms,
$$ \log G(e^t) = M_1t + \log!\left( 1+\sum_{n\ge2}\frac{m_n}{n!}t^n \right). $$
Thus $\kappa_1=M_1$, and for $n\ge2$ the cumulants are obtained by applying Arbogast's formula directly to
$$ 1+\sum_{j\ge2}\frac{m_j}{j!}t^j. $$
Since there is no $t^1$-term, we have $k_1=0$. Hence, for $n\ge2$,
$$ \boxed{ \kappa_n = \sum_{\substack{k_2,\ldots,k_n\ge0\ 2k_2+3k_3+\cdots+nk_n=n}} \frac{(-1)^{k_2+\cdots+k_n-1} ,n!,(k_2+\cdots+k_n-1)!} {k_2!2!^{k_2},k_3!3!^{k_3}\cdots k_n!n!^{k_n}} m_2^{k_2}m_3^{k_3}\cdots m_n^{k_n}. } $$
This is the required analogue of the moment formula.
The first few cases are
$$ \kappa_2=m_2, $$
$$ \kappa_3=m_3, $$
$$ \kappa_4=m_4-3m_2^2, $$
$$ \kappa_5=m_5-10m_2m_3, $$
$$ \kappa_6=m_6-15m_2m_4-10m_3^2+30m_2^3. $$
These are exactly the cumulants expressed in terms of the central moments. ∎