37

We can compute a determinant of an $n \times n$ matrix in $O(n^3)$ operations in several ways, for example by LU decomposition. It's also known (see, e.g., Wikipedia) that if we can multiply two $n \times n$ matrices in $M(n)$ steps, then we can compute the determinant in $O(M(n))$ steps as well.

However (and this is the motivating observation here), as in this question, if $\det(A) = 0$, then I can find a vector $\mathbf x$ such that $A \mathbf x = \mathbf 0$, and tell you: "$A$ is a singular matrix. Here is a vector $\mathbf x$ such that $A \mathbf x = \mathbf 0$". I might have done lots of work to find $\mathbf x$, but you can check my work in only $O(n^2)$ steps by computing $A \mathbf x$: faster than you could compute $\det(A)$ without help.

Is it possible, in a similar way, for me to take a matrix $A$ with $\det(A) \ne 0$, and write a proof of this fact which you can also check faster than computing $\det(A)$? (A perfect solution would check the proof in $O(n^2)$ steps; this is best possible, since we need that many steps to even read $A$.)

Observations:

  • A probabilistic argument exists based on Freivalds's algorithm: I give you $A^{-1}$, and leave you to check that $AA^{-1} = I$. As far as we know, this still needs $O(M(n))$ time to do deterministically, but a probabilistic algorithm can take $O(n^2)$ steps to achieve a one-sided error rate of $\frac12$: if $A^{-1}$ is correct, it will always say "yes", and if $A^{-1}$ is wrong, it will say "no" with probability at most $\frac12$. As a result, you can take $O(n^2\log n)$ steps to achieve one-sided error rate of $n^{-k}$ for any $k$.

  • More generally, we could ask for a proof that $\det(A) = x$ for any specific nonzero value of $x$. This was the original question, but there's no hope of solving that for general $x$ if we can't even solve the $\det(A) \ne 0$ case. (After all, a proof that $\det(A)$ has a specific nonzero value $x$ is in particular a proof that $\det(A)$ has some nonzero value.)

Misha Lavrov
  • 142,276
  • 2
    This is a stupid idea, but it might inspire something better. Let the given data be a list of the eigenvectors and their eigenvalues. Then check them probabilistically by taking a random subset $I$ of the eigenvectors and verifying that $A\sum_{i\in I}v_i=\sum_{i\in I}\lambda_iv_i$. This takes $O(n^2)$ time. Repeat this $\log(n)$ times and you can be pretty sure every eigenvalue is correct. – Oscar Cunningham Feb 25 '18 at 22:35
  • 1
    I guess it is not too surprising that a probabilistic algorithm exists. (Similarly, we could use $\log n$ iterations of Freivalds's algorithm to check the LU decomposition in $O(n^2 \log n)$ time to the same standard of "pretty sure".) I would be happier with a deterministic solution, though. – Misha Lavrov Feb 25 '18 at 22:46
  • 2
    Do you even know of a fast way to prove that a matrix is invertible? Otherwise, it seems appropriate to ask that as a question, since you need to be able to do that efficiently to do this efficiently. Though to be honest, that in itself requires observing $n$ dimensions' worth of matrix-vector products (so you can verify the output dimension), but that itself implies matrix multiplication, so I would be surprised if it's possible to do it faster at all. (But I'm not a mathematician...) – user541686 Feb 26 '18 at 08:10
  • @Mehrdad That's a good point - I don't, and that is a good place to start. (Do you think I should edit this question, or ask a separate one?) – Misha Lavrov Feb 26 '18 at 17:28
  • @MishaLavrov: If it were me I would probably just edit this question, but I'm actually not sure what's appropriate here. – user541686 Feb 26 '18 at 20:41
  • Certainly, for some special cases, there are proofs that are checkable in $O(n^2)$ time - for example, if the proof is "the matrix is upper triangular with nonzero diagonal entries". – Daniel Schepler Feb 26 '18 at 23:04
  • In numerical linear algebra algorithms for finding the smallest eigenvalues, without finding all of the eigenvalues, are important, both for dense and sparse matrices. Performing a literature search in that area may provide some interesting results. – Jap88 Apr 07 '18 at 13:07
  • If we were $n$-dimensional beings, then simply by looking at $n$ vectors it would be blindingly obvious whether or not they are collinear. It's interesting to see that this cannot be turned into an easy mathematical verification. – Arnaud Mortier Dec 26 '19 at 10:29
  • Checking that $n$ vectors are eigenvectors takes $O(n^3)$ time. – 1Rock May 06 '20 at 07:01

1 Answers1

1

Just to share another possible alternative, a randomised algorithm expanded from Weidemann provides a method to sample solution from the solution (affine) subspace $Ax =b$ where $A$ is a square matrix and $b$ is in the column space of $A$. So in this case, one can choose $b = 0$ and the algorithm should sample uniformly from the nullspace of $A$ for finite fields. The idea is to run the algorithm a few times for the same input $A$ and let $b= 0$. It will return you some $x$ and if $x$ is not a zero vector for any of those trails, then we know that $A$ is not invertible.

The situation on infinite fields is not so transparent; but it should yield relatively comparable results if you choose a large enough sampling set.

This algorithm claims to take $O(n^2 \log n \log \log n)$ computations in the base field and it requires the base field to be sufficiently large, else, take a field extension. The success probability of this algorithm is quite high so with only finite repetition, I think it is possible to determine whether the matrix is invertible quicker than $O(n^3)$

Edit: Another alternative is to consider the minimal polynomial of the linearly recurrent sequence $\{u^\top A^i b\}$. (A sequence $s_i$ in a $F$ -vsp is linearly reccurent if there exists $n \in \mathbb{N}$ and $a_0,\dots,a_{n-1} \in F$ such that $\sum_{i=0}^{n-1} a_is_{j+i}$ for all $j \in \mathbb{N}$. An example of a linearly recurrent sequence would be the sequence $A^i$ where $A$ is a $n\times n$ matrix, and it is not difficult to see why $\{u^\top A^ib\}$ is also linearly recurrent. The minimal polynomial of a linearly recurrent sequence is just the polynomial $\sum_{i=0}^{n-1} a_ix^i$.)

Now, another result in the paper by Kaltofen and Saunders (which is the paper above) is that one can 'guess' the minimal polynomial of $A$ well enough using the minimal polynomial of the sequence $u^\top A^i b$ by choosing $u$ and $b$ randomly, provided that the base field is sufficiently large. (But in fact, one would always get a factor of the polynomial of $A$).

One may ask why is even is the sequence $u^\top A^i b$ relevant. This is because it is easy to find the minimal polynomial of this sequence of elements in the base field via a dynamic algorithm, the Berlekamp - Massey algorithm. One can just pick randomly sample $u,b$ and find the minimal polynomial of the sequence $u^\top A^i b$.

But now the issue is that one requires a sequence of length $2d$ where $d$ is the degree of the minimal polynomial of $A$, which is unknown and reasonable guess is $d = n$. I am doubting if this method is even sub-$O(n^3)$ since the algorithm relies on Berlekamp Massey.

monikernemo
  • 526
  • 2
  • 9
  • Interesting! I think we want it the other way around: we get a probabilistic proof that $\det(A)\ne0$ by trying to solve $Ax=0$ many times and getting $x=0$ every time. Still, this beats the previous probabilistic argument, because we didn't even need any "advice" about $A$. – Misha Lavrov Sep 21 '20 at 14:51
  • If I recall the details of the paper correctly, we can bypass solving the linear system. Note that the sequence ${u^\top A^i b}$ is linearly recurrent and one can write a minimal polynomial $f$ for this sequence. In fact $f$ divides the minimal polynomial of $A$. Now, by choosing $u$ and $b$ randomly, we can in fact guess the minimal polynomial of $A$ with high probability (again, given that our field is large enough) and in fact, a factor that has zero constant term will suffice. The minimal polynomial can be easily found via the Berlekamp Massey algorithm. – monikernemo Sep 21 '20 at 15:32
  • Right, by "trying to solve" I meant "using the algorithm to randomly sample a solution", I was just using sloppy wording. – Misha Lavrov Sep 21 '20 at 15:43
  • Hmm... after some thinking, the algorithm uses relies heavily on Berlekamp - Massey algorithm and that requires $2n$ vector-matrix and vector-vector multiplication, so I am not quite sure if this still works faster than matrix multiplication. Perhaps it would be only ideal for very sparse matrices. – monikernemo Sep 21 '20 at 16:18