1

The answer to this question shows that the product of two Gaussian PDFs is also a Gaussian PDF.

My questions are:

  • Is there a multiplicative identity for this product?
  • More generally what algebraic structure (e.g. semigroup/monoid/ring/field) do Gaussian PDFs belong to?
Community
  • 1
NietzscheanAI
  • 319
  • OK, `algebraic structure' then. Edited accordingly. – NietzscheanAI Jun 09 '16 at 15:53
  • Are you allowed to include delta functions in the structure? This is necessary to have an identity in the structure. – Paul Jun 09 '16 at 15:55
  • Uh...the product of two Gaussian PDFs is a Gaussian PDF in a different dimensional space. This seems pretty bad from the perspective of algebra. – Ian Jun 09 '16 at 15:55
  • Algebra seems to be able to handle such things with matrices fairly well... – NietzscheanAI Jun 09 '16 at 15:57
  • @Paul - if it's a necessity, then yes. Presumably sums, products etc of delta functions are necessary? – NietzscheanAI Jun 09 '16 at 16:03
  • There can't be an identity, because doing any multiplication whatsoever raises the dimension. It might be a semigroup. – Ian Jun 09 '16 at 16:06
  • @Ian, this is not an issue if the 2 Gaussians have the same outcome variable. – Paul Jun 09 '16 at 16:08
  • @Paul I thought we were understanding the variables to be different, so that the multiplication corresponds to independence. I guess if they are the same then you add $\frac{(x-\mu_1)^2}{2\sigma_1^2}$ and $\frac{(x-\mu_2)^2}{2\sigma_2^2}$, which I guess does give another normal PDF...interesting. Still, this issue raises the question of the domain of these PDFs. Is it a common domain or what? – Ian Jun 09 '16 at 16:11
  • @Ian- yes, the domain is common. – NietzscheanAI Jun 09 '16 at 16:14
  • 1
    Oh, in that case, then it's commutative, associative, and the Dirac delta is an identity. There are no inverses, and there's no obvious "secondary" operation here to make it into a ring or anything like that. – Ian Jun 09 '16 at 17:24
  • @Ian - would you care to add that as an answer? – NietzscheanAI Jun 13 '16 at 17:40
  • I'm not certain enough about the lack of a "secondary" operation to post an answer. – Ian Jun 13 '16 at 17:43
  • @Ian - I see. Perhaps convolution could be considered to play the role of the other operation? – NietzscheanAI Jun 13 '16 at 17:45

1 Answers1

0

I will start with the 1-dimensional case. In the question you linked to, they have the identity:

$$ N(\mu_1, \sigma_1^2) \times N(\mu_2, \sigma_2^2) \propto N \left( \frac{\sigma_1^2 \mu_2 + \sigma_2^2 \mu_1}{\sigma_1^2 + \sigma_2^2}, \frac{1}{\frac{1}{\sigma_1^2} + \frac{1}{\sigma_2^2}} \right) $$

where the multiplication is of PDFs, not random variables.

Introduce alternate parameters $\lambda_i = 1/\sigma_i^2$ and $\xi_i = \mu_i / \sigma_i^2$. Note that we can recover the original parameters as $\mu_i = \xi_i / \lambda_i, \sigma^2_i = 1/\lambda_i$. This parameterization is called canonical or information form. Note that:

$$ N \left( \frac{\sigma_1^2 \mu_2 + \sigma_2^2 \mu_1}{\sigma_1^2 + \sigma_2^2}, \frac{1}{\frac{1}{\sigma_1^2} + \frac{1}{\sigma_2^2}} \right) = N \left( \frac{\xi_1 + \xi_2}{\lambda_1 + \lambda_2}, \frac{1}{\lambda_1 + \lambda_2} \right) $$

Hence the new canonical parameters are $\lambda' = \lambda_1 + \lambda_2, \xi' = \xi_1 + \xi_2$. Since $\lambda \in \mathbb{R}_{>0}$ and $\xi \in \mathbb{R}$, I suppose that in the 1D case you want the Cartesian product of those spaces under addition, $\mathbb{R}_{>0} \times \mathbb{R}$. As $\mathbb{R}_{>0}$ does not support an additive identity, I think this is a semigroup.

In multiple dimensions, canonical form is given by $\bf{\Lambda} = \bf{\Sigma}^{-1}, \bf{\xi} = \bf{\Sigma}^{-1}\bf{\mu}$. I think the addition properties still hold, and you would want to restrict $\bf{\Sigma}$ or equivalently $\bf{\Lambda}$ to be (symmetric) positive definite to avoid degeneracy.

Source: Kevin P. Murphy, Machine Learning, 2nd. ed., section 4.3.3.

Update: I have changed the $=$ in the first equation to $\propto$ because there is a proportionality constant: https://www.johndcook.com/blog/2012/10/29/product-of-normal-pdfs/. Murphy and the linked question both overlook this.

Brian Z
  • 1
  • I posted an answer assuming the operation is multiplication. Gaussian PDFs are clearly not closed under addition, but they are closed under convolution. However, I'm not sure at the moment whether or not there are good distributive laws between the two operations (convolution and multiplication), see https://mathoverflow.net/questions/3455/do-convolution-and-multiplication-satisfy-any-nontrivial-algebraic-identities – Brian Z Feb 12 '18 at 18:48