Computational costs of different methods of solving systems of linear equations

Question

Homework question: given a regular matrix $A\in M^{n\times n}$ and a linear system of equations $A\vec{x} = \vec{b}$, state the preferred method of solving and its computational costs:

$A$ has no special properties.
$A$ has positive eigenvalues.
$A$ is orthogonal.

Solutions that I've been able to find so far:

LU-Decomposition, $\approx \frac{2n^3}{3}$. (Source)
Cholesky-Decomposition, $\approx \frac{n^3}{3}$. (Source)
Orthogonal Diagonalization, depends on matrix multiplication $\approx n^{2.376}.$ (Source)

Are these answers correct, or are there better methods/are the best methods slover?

Here's a hint about the third case: there's a much easier method, which you can discover if you think about the various equivalent definitions of an orthogonal matrix. — littleO, Mar 01 '18 at 20:39
@littleO Ah! $A^T = A^{-1}$, and the costs should be therefore $n^2$ — Zyx, Mar 01 '18 at 21:27

score 2 · Answer 1 · answered Mar 01 '18 at 18:44

General principal is that solving linear equations is no harder than multiplying matrices, regardless of what you know about A (see Burgisser Claussen "Algebraic Complexity Theory" Chapter 15). So the bound is $O(n^{2+\epsilon})$ for an every changing value of $\epsilon$ (by ever changing I mean it changes every few years, we don't know if $\epsilon$ can become arbitrarily small.) But for example at mimimum using vanilla Strassen you get $O(n^{\log_2 7})$ complexity always.

But to be honest, a lot of that is not quite accurate. Missing from this is the issue of numerical stability. If you solve problems over the reals your floating point precision becomes a problem. So the complexity of "practical" algorithms is highly varied.

Computational costs of different methods of solving systems of linear equations

1 Answers1