0

How can I compute $$ \min_{x \in \Delta_n} \frac{1}{2}\lVert Bx\rVert^2 + x^tAy$$ with $x \in \mathbb{R}^n, y \in \mathbb{R}^m, A_{m \times n}$, $B_{n \times n}$ where $\Delta_n$ is the unit simplex $$\Delta_n = \{x \in \mathbb{R}^n_+ | \sum_{i=1}^n x_i = 1\}$$ Are there standard algorithms for computing it?

Royi
  • 8,711
karlabos
  • 1,257
  • 1
    This is known as quadratic programming. The Wikipedia page lists a number of solution methods. –  Apr 12 '15 at 13:01
  • But how do I write the condition $e^tx = 1$ in terms of $Ax \leq b$ in this case? – karlabos Apr 12 '15 at 13:32
  • 2
    The definition given in Wikipedia is slightly incomplete. In quadratic programming one allows both linear inequality constraints, $Ax\le b$, and linear equality constraints, $Cx=d$. So the unit simplex can be defined as $-Ix\le0,e^Tx=1$. –  Apr 12 '15 at 15:18
  • And for the special case when $B$ is diagonal, it can be computed efficiently in $O(n \cdot \log n)$ time. If you are interested, I will write a full answer. – Alex Shtoff Apr 13 '15 at 09:14
  • Wikipedia is not incomplete. One equality can be written as two inequality constraints. – LinAlg Jan 28 '21 at 20:37

1 Answers1

2

You can also solve this by Projected Gradient Descent since the Projection onto the Unit Simplex is known.

The Code:

vX = pinv(mA) * vB;

mAy = mA.' * vY;
mAA = mA.' * mA;


for ii = 1:numIterations
    stepSize = stepSizeBase / sqrt(ii);
    vG = (mAA * vX) + mAy;
    vX = vX - (stepSize * vG);
    vX = ProjectSimplex(vX, simplexRadius, stopThr);
end


objVal = (0.5 * sum((mA * vX) .^ 2)) + (vX.' * mAy);


disp([' ']);
disp(['Projected Gradient Solution Summary']);
disp(['The Optimal Value Is Given By - ', num2str(objVal)]);
disp(['The Optimal Argument Is Given By - [ ', num2str(vX.'), ' ]']);
disp([' ']);

You can even accelerate this using Nesterov / FISTA to get really fast and efficient algorithm.

You can have the Simplex Projection function from my Ball Projection GitHub Repository.

Royi
  • 8,711