A linear complementarity problem(LCP):
$y=Mx+b$
$x_iy_i=0$
$x_i ≥0$, $y_i≥0$
$M=\left[ \begin{array}{ccc}0 & -A^T\\A & 0\end{array}\right]$
$x \in R^n$, $y\in R^n$
Problem: $f(x)=\frac{1}{2}x^TMx+b^Tx$, please calculate the gradient $\nabla f(x)$.
My answer: $\nabla f(x) = x^TM+b^T$, right or not?
Let the $(i,j)^{\text{th}}$ element of matrix $M$ to be: $~M_{ij}=m_{ij}$
$$\begin{align} f&=\frac{1}2 \sum_{i}\sum_{j} x_i m_{ij} x_j+\sum_{i}b_ix_i\\ \\ \partial_k f&=\frac{1}2\sum_{i}\sum_{j} (\partial_k x_i) m_{ij} x_j+\frac{1}2\sum_{i}\sum_{j} x_i m_{ij} (\partial_k x_j)+\sum_{i}b_i(\partial_k x_i)\\ \\ \partial_k f&=\frac{1}2\sum_{i}\sum_{j} (\delta_{ki}) m_{ij} x_j+\frac{1}2\sum_{i}\sum_{j} x_i m_{ij} (\delta_{kj})+\sum_{i}b_i(\delta_{ki})\\ \\ \partial_k f&=\frac{1}2\sum_{j} m_{kj} x_j+\frac{1}2\sum_{i} x_i m_{ik}+b_k\\ \\ \nabla f&=\frac{1}{2}Mx+\frac{1}{2}x^TM+b \end{align}$$
Write them into row-vector form:
$$\begin{align} \nabla f&=\frac{1}{2}x^TM^T+\frac{1}{2}x^TM+b^T\\ \\ \nabla f&=x^T\left(\frac{M+M^T}{2}\right)+b^T \end{align}$$
The differential is: $$ df(x)\cdot h = \frac{1}{2}(x^tMh +h^tMx)+b^th $$ To get the gradient, express this linear form using the scalar product : $$ df(x)\cdot h = \frac{1}{2}\langle x,Mh \rangle +\frac{1}{2}\langle h,Mx\rangle+\langle b,h \rangle = \langle \frac{1}{2}(M + M^t)x+b,h \rangle $$ By consequence the gradient $\nabla f$ is the vector : $$ \nabla f_{|x} = \frac{1}{2}(M + M^t)x+b $$ If $M$ is symmetric this reduces to $$ \nabla f_{|x} = Mx+b $$
Rational: if $f:\mathbb{R}^n\rightarrow \mathbb{R}$, then the differential at $x$, $df(x)$ is a linear form $\in\mathcal{L}(\mathbb{R}^n,\mathbb{R})$. But we know (1) that there exists a vector $v$ such that $df(x)\cdot h = \langle v_x, h \rangle$. This vector $v_x$ is by definition the gradient at $x$, denoted by $\nabla f_{|x}$
(1) Riesz representation theorem
