0

we have : a vector $z$ of dimension $n$.

a Matrix $A \in \mathbb{R}^{n \times n}$

a Vektor $d$ of dimension $n$.

and we are looking at

$$S = (z-Ad)^{T}(z-Ad)$$

is it possible to express $S$ as a Sum in terms of $z$ ?

best regards

edit :

we have a factor model with $k$ variables and $f$ factors.

my lecturer told me : $z$ has $\frac{k^2 -k}{2}$ entries and $d$ has $kf - \frac{f^2 -f}{2}$ entries (because $z$ is the vector of a symmetrix matrix, where we dont need the diagonal-entries and $d$ is the error-vector)

now he said we can express S as a sum of $$((k-f)^2 -(k+f))/2$$ summands, which are linear-combinations of $z$. (he mentioned the singular value decomposition of A).

  • What is $A$? I mean is it a special matrix? $$S = z^{T} z-z^{T}Ad-d^{T}A^{T}z+d^{T}A^{T}Ad$$ – Amin Apr 15 '17 at 06:29
  • $A$ comes out from a loading-matrix $L$ which is 'trapezial' matrix -- so i also think about special properties.. but lets assume we dont know something special about $A$ -- its just real and regular. –  Apr 15 '17 at 06:34
  • Thanks for your comment :) The idea is : we should be able to write S as a linear combination of z, because z follows a normal distribution... and maybe we can write S as a sum of independent, squared normally distributed things -- maybe if we decompose the matrix A ? –  Apr 15 '17 at 06:37
  • Here, $A^T A$ is a symmetric matrix; it seems if you want to go further, we need to know more about matrices and vectors. – Amin Apr 15 '17 at 06:37
  • So the complete context is the minimization problem: $$S=(z−Ad)^{T}(z−Ad) \to min$$ should lead us to a generalization of the chi-square-distribution... we know z is normally distributed... so if S is a sum of squared normally distributed elements of z we are done. –  Apr 15 '17 at 06:43
  • Notice that, here based on your product: $Ad$ which has dimension $n \times 1$, then dimension of $z$ is $n \times 1$ and so this is a dot product; hence, your result is not a matrix! – Amin Apr 15 '17 at 06:44
  • hmmm - but do we get S as a sum of squared normally distributed z's ? –  Apr 15 '17 at 06:47
  • Here, you can write: $\mathbf{A} \mathbf{d}=\mathbf{w}$ then, $$(\mathbf{z}-\mathbf{w})^T(\mathbf{z}-\mathbf{w}) = \sum_{k=0}^{n}(z_k- w_k)^2,$$ do you mean this (it is based on the square of the ements of $\mathbf{z}$)? – Amin Apr 15 '17 at 06:49
  • thank you - but is it obvious that $(z_{k} -w_{k})$ is normally distributed? –  Apr 15 '17 at 06:54

1 Answers1

0

Yes, $S$ can be expressed as a sum of square normal variables if $z$ is normally distributed.

First, notice that $z^T A d = d^T A^T z$. To see this, define $v = Ad$ and note that

$$z^T v = \sum_{i=1}^n z_i v_i = v^T z$$

for any two vectors $z$ and $v$ of the same dimension $n$.

Then, using $v$ as defined above, we have

\begin{align} S &= z^T z + v^T v - z^T v - v^T z \\ &= z^T z + v^T v \\ &= \sum_{i=1}^n \left( z_i^2 + v_i^2 \right), \end{align} which is indeed a sum of squared normal variables, since each $z_i$ is a normal variable. Note that each $v_i = \sum_{i=j}^n A_{ij} d_j$ is a deterministic variable. Then $S$ is a generalized chi-square variable.

I think some confusion about this question arises from the phrase "in terms of $z$". Perhaps you meant "in terms of the components of $z$"?

Another point of confusion might be the phrase "as a linear combination of $z$". To show that $S$ is a generalized chi-square variable, we wish to show that it is a linear combination of chi-square variables, which in this case are the squared components of $z$ rather than $z$ itself.

And to address Amin's concern that $S$ is not matrix, I believe the goal of the exercise was to show that $S$ is a single chi-square variable rather than a matrix of such variables, so the given expression for $S$ is indeed meaningful.

Also note that if $z \sim \mathcal{N}(\mu, \sigma^2)$, and $w$ is a deterministic variable, then $z - w$ is also a normal variable, distributed as $\mathcal{N}(\mu-w, \sigma^2)$, since expectation is a linear operator. However, $(z-w)^2$ is not a normal variable. So, $S$ isn't normal, but this doesn't contradict the fact that it's chi-square.

Justin Le
  • 305
  • Hello Justin Le and thanks for your answer. the vector $z$ is build of the non diagonal elements of $B^{1/2} X B^{1/2}$ where B is a covariance matrix, and X follows a normal distribution - therefore $z$ as well. The $d's$ are 'unkown' approximation errors - does it change the thing that S is generalized chi square distributed ? what can we say about the covariance matrix of $z$ ? maybe expressed with respect to $B$ ? best regards –  Apr 15 '17 at 07:56
  • and : the solution of the minimization problem must be $$d = (AA^{T})^{-1} A^{T}z$$... am i right? –  Apr 15 '17 at 13:58
  • added maybe something important in my main post –  Apr 15 '17 at 14:52