19

Chess is not completely solved yet but what efforts have been undertaken to solve it? Has there been a large project like the HGP or is something like that even worth pursuing? What failed efforts have been there if any? I am speculating that the the Alpha Zero project must have taken some steps towards that end.

EDIT:-Answers ponder on the point that is extremely difficult to solve chess which i have clearly indicated in the first line in the question. What I would like to know that are there or have there been any projects that try to solve chess. Like that Moscow university 7-piece tablebases and any efforts like that. It is clearly an expensive problem to solve but who is doing it and what approaches are being taken? It is often said that there are more chess combinations than atoms in the universe (probably because of that ad) but so are the real numbers but we have a rich theory about that. We also know a great deal about the universe as well but we had to try hard to get to that point. Is chess being treated the same is my question.

shashank shekhar singh
  • 837
  • 5
  • 15

11 Answers11

43

Endgame tablebases can be seen as such an effort, in that we started at few pieces and have increased the number. In 2012, 7 piece tablebases -from Moscow state university were generated, so chess is solved for positions of 7 or fewer pieces (including the kings).

The problem is, the 7 piece tablebases take about 140 TB in storage. 8 piece tablebases would take about 10 petabyte of storage, and a computer with 50 TB of ram (source) to keep hashes in memory to generate them in reasonable time (a few years?). I guess it would be doable these days if someone makes the millions of dollars available. 9 seems way out of reach.

So it will take a while before we get to 32.

shashank shekhar singh
  • 837
  • 5
  • 15
RemcoGerlich
  • 27,974
  • 80
  • 122
19

7-piece endgame tablebases were completed in 2012. 8-piece endgame tablebases is the next logical step. This page says a little about how those are going (context is searching for the longest forced mate in pawnless positions):

Many show interest in what is to expect from [8-piece] endings ... Unfortunately the size of [8-piece tablebases] will be 100 times larger than size of [7-piece] tablebases. To fully compute them, one will need about 10 PB (10000 TB) of disk space and 50 TB of RAM. Only top 10 supercomputers can solve [8-piece] problem in 2014. So don't hold your breath expecting new breakthroughs too soon - the first 1000-move mate is unlikely to be found until 2020 when a part of a TOP100 supercomputer may be allowed to be used for solving this task.

If we get 8-piece tablebases one day (if we get them) that'll be the next step to solving chess, but we don't expect to get them for many years more at least. Even if they are created, they'll take huge amounts of disk space to store.

In principle chess will be solved when we have 32-piece tablebases, but if 8-piece tablebases are this hard to create, we are not going to solve chess in the foreseeable future, AlphaZero or not.

Greg Martin
  • 162
  • 4
Allure
  • 26,534
  • 1
  • 68
  • 146
  • 5
    In 2020 you can rent 10PB of disk and 50TB of RAM (and a few tens of thousands of CPU cores into the bargain) for maybe 5 million dollars per year. – hobbs Jul 09 '20 at 01:28
  • @hobbs Right, but I guess the people who are willing to spend 5 million dollars per year have better applications for it, like COVID-19-related protein folding. – user253751 Jul 11 '20 at 14:04
15

AlphaZero has no intention of solving chess. It's just an engine that plays spectacularly well. The number of positions you'd have to evaluate in order to solve chess is larger than the number of atoms in the universe, so even if you could solve chess (which is nowhere near the capabilities of the current capability of all computers on Earth combined), you wouldn't be able to store the entire solution anywhere

Rewan Demontay
  • 17,514
  • 4
  • 67
  • 113
David
  • 16,275
  • 26
  • 61
  • 12
    Well, you're assuming that you'd have to store every position. You could certainly discard the "worst" moves without storing them, or encode a "short" algorithm that works from a number of similar positions, avoiding the need to store them directly. Also, you can make the case that any chess-playing AI is a storage of the solution to every position it is able to win from, regardless of whether it had any a-priori knowledge of that position or not. – jpaugh Jul 08 '20 at 15:24
  • @jpaugh exactly . A0 does not "store" every position that is why i said that , that particular project is particularly interesting when it comes to solving chess. if we could follow A0 with big investment maybe it can take us very far into uncharted territories – shashank shekhar singh Jul 08 '20 at 16:53
  • @jpaugh in order to "solve" chess, you'd need to have the solution stored somewhere. Otherwise you'd have no way to know whether you solved it or not. – David Jul 08 '20 at 22:17
  • 3
    As I mentioned elsewhere, results like PSAPCE completeness of generalized chess suggests we should not expect to be able to compress a solution into sub-galactic size, even implicitly like in the guts of an engine. This contrasts with NP-complete problems whose solutions may be hard to find, but are small. – usul Jul 08 '20 at 22:47
  • 2
    @David Nim with initial heaps of sizes {1,2,...,100} has a larger space of valid positions than chess, but it's solved. The solution consists of an algorithm that computes a move given a position, and a proof that the move it computes is always optimal. Chess is probably harder than Nim but the size of the position space by itself doesn't tell you much. – benrg Jul 09 '20 at 00:55
  • 2
    @benrg sure, the argument about storage space implicitly assumes that you have to store a significant fraction of positions. However, no algorithm has been found for chess, and given the numerous attempts at finding one, it's likely that an implementable one exists. The chess computer community certainly has given up on finding one, anyway. – toolforger Jul 09 '20 at 06:25
  • @David "in order to "solve" chess, you'd need to have the solution stored somewhere" - yes, you would need to store the solution, not all other non-solution positions, which essentially comes down to only 1 game of chess (the forcing solution). – gented Jul 10 '20 at 12:19
  • 1
    @gented That wouldn't be a solution. For instnace on the first move you have 20 different alternatives, all of them leading to a draw (therefore equivalent), it's not enough to have analyzed one of them. – David Jul 10 '20 at 12:49
  • 2
    You can store it in a black hole:) – postoronnim Jul 10 '20 at 15:16
  • @David That's not what we said (do you even read the comments?). You do analyse all of them, but you only store the one that is winning. – gented Jul 10 '20 at 20:55
  • 1
    @gented Why do you expect any winning move to exist? Chess is more likely to be a draw with perfect play. Even after you only store the winning move (maybe 1.e4), then you have to store the answer to all twenty possible answers by Black, and after you store that you need to store the winning move against all 400+ combinations of moves (just by move two!) – David Jul 12 '20 at 09:23
  • @David "Why do you expect any winning move to exist?" I don't, that's not the point of what we are discussing, please read the comments above again.You don't need to store anything, that's exactly the point: searching a solution does NOT require storing the attempts of the search. If and only if you accidentally find a solution (you may or you may not) then you store only that one chess game leading to that solution (which is 1 .pgn file only): all your previous attempts, however many numerous they are, need NOT be stored anywhere. – gented Jul 14 '20 at 09:50
  • @pented It's a bit rude to assume that I hold the view I hold because of my ignorance or lack of reading comprehension rather than for the possibility that I may be right. The solution must include all answers to the alternative moves by the losing side (or both sides if the game is a draw). Sure you can store everything in a single .pgn file, of course! Only that .pgn file's size in TB will be several orders of magnitude beyond the number of atoms in the observable universe. But sure, it's still a single .pgn file – David Jul 14 '20 at 10:35
  • @gented Typically in mathematics, when we "solve" something, we are expected not only to solve it, but to produce a way to verify that the solution was indeed found, without requiring the reader to solve it themselves. This can be substantially smaller than "everything," but typically larger than "here's the best Chess game playable." – Cort Ammon Apr 07 '21 at 01:06
  • @David "The solution must include all answers to the alternative moves by the losing side. Only that .pgn file's size in TB will be several orders of magnitude beyond the number of atoms in the observable universe" This is false, see all my comments above (you are in fact still not comprehending any of the comments). Other users in the comments have indicated this too (so I am really puzzled on why this discussion still continues). – gented Apr 07 '21 at 08:53
6

It is funny that you bring up mathematics and the universe as things we know quite a bit about. Yet neither of these fields is 'solved'. Moreover, neither ever can be.

Mathematics can never be fully solved due to Goedel's Incompleteness Theorems. There will always be true facts about numbers that we cannot find proofs of.

As with the Universe, we can never know the full 'state' of the universe (eg what every atom and subatomic particle is doing) precisely due to theHeisenberg's Uncertainty Principle. We can never hope to tell what is going on outside the observable universe due to relativity.

So in some sense, they are pretty fair comparisons to chess. We know a lot, but due to universe we live in, we will never know it all.

The best I believe we can hope for in terms of solving chess would be an ultra-weak solution (possibly involving some strategy stealing) that proves a eg a forced draw (or better) for white from the starting position without explicitly solving how.

For an interesting article about endgame table-bases and methods for solving games, check out this. Checkers (a MUCH simpler game) was solved to be a draw over a decade ago

EDIT To clear up some confusion:

As pointed out in the comments, the unsolvability of these three systems are immensely and qualitatively different. The incompleteness theorems derive from first principles and would hold up in ANY universe. The unsolvability of the state of the universe is based on the physical laws of the one we live in. The unsolvability of chess is only due to the fact that the universe we happen to live in is too small and chess is too large for a (strong) solution to be held inside it. The reason they are similar here are pure coincidence, simply a by-product of universe we all happen to inhabit.

DongKy
  • 1,183
  • 7
  • 12
6

Can't answer your main question, but to the point about alphazero: from my perspective, AlphaZero, although it may be a major step in superhuman performance in games, is not even a minor step towards solving chess.

if we could follow A0 with big investment maybe it can take us very far into uncharted territories

I don't think this is true. Its approach, which is to explore the game-tree by picking paths which have high probability of being good, doesn't seem to have much implication to fully solve chess. Although it may be very very good at being very very good at chess, I think this heuristic-based approach is less viable to exactly solving chess than you think.

To be very charitable, maybe... and by maybe I mean this is a pie-in-the-sky thought that just came from my dumb brain... maybe the AlphaZero work could someday lead to a proof that with training, the model converges to a model that plays correctly with probability 1 (and 1 in this case is aka 99.999...%), but even this may not be considered "solving" chess as we still cannot not prove that any model we have trained in real life makes every move correctly.

shashank shekhar singh
  • 837
  • 5
  • 15
Kevin Wang
  • 187
  • 4
  • ".... Its approach, which is to explore the game-tree by picking paths which have high probability of being good" what is wrong with this approach? – shashank shekhar singh Jul 09 '20 at 04:39
  • 1
    @shashankshekharsingh I just don't see how it is related to the prospect of fully solving chess. As I said, there's nothing "wrong" with it -- it's very very good at playing games better than anything we've seen before, but I don't see the connection to the goal at hand. – Kevin Wang Jul 09 '20 at 05:57
  • well lets say there are 10^40 sensible positions , a0 can at least bring that number further down without needing those humongous amounts of storage since it is the best at chess. and then maybe some other approach can be applied. – shashank shekhar singh Jul 09 '20 at 10:00
  • Is a0 even being used to solve chess , say the 8 piece table-bases ? – shashank shekhar singh Jul 09 '20 at 10:08
  • "Is a0 even being used to solve chess , say the 8 piece table-bases?" That's a good question. I don't know the answer to that. – Kevin Wang Jul 09 '20 at 18:56
  • @shashankshekharsingh Solving chess requires proof. High probability isn't the same as solving. – David Liu Jul 15 '20 at 21:47
4

Others have already mentioned tablebases and their development, but something that hasn't been touched on yet is the sheer space complexity of chess.

In order to say that chess is solved, we need a function gameState -> move, and that function needs to be defined for all possible states of the game, or at least all of the ones that are reachable within the rules of the game. There are vastly more of these than there are atoms in the observable universe. So even if we could somehow encode each game state and its corresponding best move into a single atom, there isn't enough matter in the universe to build the server that would run the database.

Endgame tablebases are interesting, but there's nobody who's seriously attempting to solve chess.

MattPutnam
  • 532
  • 3
  • 3
  • 9
    Who said that in order for a function to be defined, all possible inputs and their outputs need to be stored in a database? By this argument, the function f(x, y) = x + y cannot be defined, even for natural numbers, because there are more natural numbers than there are atoms in the universe. – LarsH Jul 09 '20 at 01:49
  • @LarsH that's just what solving means in this context. If you're saying it can be a function that we compute on the fly... then we're right back to game tree search. – MattPutnam Jul 09 '20 at 03:03
  • @MattPutnam "....but there's nobody who's seriously attempting to solve chess." well that is the actual answer I guess. :) – shashank shekhar singh Jul 09 '20 at 04:28
  • 7
    A mathematical proof of whether the starting position is a win, or a forced draw, would be one kind of "solution" of chess without necessarily requiring a brute-force exhaustive tablebase. – nanoman Jul 09 '20 at 09:06
  • 1
    @nanoman yes exactly. i am more interested in that kind of solution rather than engine based brute force attempts. Chess theory has to be developed. – shashank shekhar singh Jul 09 '20 at 10:32
  • @Matt Not necessarily. There are many ways that functions can be computed on the fly, besides tree search. – LarsH Jul 10 '20 at 14:15
2

Shirish Chinchalkar has a paper which calculates an upper bound of ~10^46 for the number of positions in chess (different from moves obviously). Arguments which talk about the number of atoms in the universe are very facetious on account of using position differences (we can only get to a position from another, so why not just store what piece moved instead of the whole new position?) as well as compression. IIRC, in the Syzygy endgame tables, each position can be represented by less than a single bit after taking into consideration compression.

One thing that I personally wonder, and I could be completely wrong, is whether there would be at a certain point a relative small decrease in complexity as n -> 32 as there will be less starting nodes.

It's an amazing question though. I would love to see it (even weakly) solved in my lifetime.

shashank shekhar singh
  • 837
  • 5
  • 15
Colin McDonagh
  • 21
  • 1
  • A quick calculation shows 3x10^23 possible positions, of which some number are unreachable. There are 13 possible states for each square - 6 pieces on each side - pawn, rook, knight, bishop, queen, king plus 1 empty space. There are 64 squares - so 64^13 possible positions. This seems much more reasonable than 10^46, though still a very large number. the fact that the game-tree has 10^120 moves (shannon number) is irrelevant. you only need to keep 'the one winning move' for each of these 10^23 states. – Jim Jul 10 '20 at 14:53
  • the unreachable states would take into account that there are only ever 2 kings on a board with 2048 total positions of them, instead of up to 64 kings. it would also limit to a maximum of 8 pawns each (again down from 64), and have a maximum bounds on the other pieces due to promotion - say, 4 queens, 6 rooks, 6 bishops, and 6 knights. my math is too rusty to calculate this, but i expect orders of magnitudes of possible positions can be discarded. – Jim Jul 10 '20 at 15:19
  • one final note on unreachable states - there has to be a minimum of 32 empty spaces, up to 61. so, again, anything outside this range can be excluded. – Jim Jul 10 '20 at 15:54
  • Hi Jim, it's actually 13^64 (~10^71) according to your reasoning. So using the different pieces of information you've given you could look to reduce it. However, positions also take account of whether castling is possible for either side and if there are any en-passant squares. See here for position vs diagram: http://wismuth.com/chess/statistics-positions.html – Colin McDonagh Jul 10 '20 at 16:40
  • Colin, that's embarassing. like i said, my math is rusty. what is the etiquette for a wrong post? should i delete it or leave it up? i think the follow-on posts are valid. one more interesting limiter - with 8 pawns in row 2, there are only 16 possible states of row 1, 4 on each side with the rook & knight possibly switching places or the knight not being there. – Jim Jul 14 '20 at 15:12
  • @Jim haha don't worry, we all make mistakes. I've looked at all of this in the past hence anything less than 10^46 would arouse my suspicions. If you're too embarrassed you can delete it, but I don't think there's any reason to hide mistakes. Yeah, one of my friends once wrote up a little article on the same! Having a lot of pawns on their starting positions really restricts the possible positions of pieces – Colin McDonagh Jul 14 '20 at 15:21
  • 1
    @Jim I'm currently trying to publish a small paper on pawn positions. I'll post it here if it ever gets published! – Colin McDonagh Jul 14 '20 at 15:23
2

Another line of attack is reducing dimension: Gardner's 5x5 minichess has been weakly solved by Mehdi Mhalla and Frederic Prost in 2013. 3x3 and 3x4 boards have been strongly solved in the 2000s with complete tablebases for each position by Kirill Kryukov, and 9-men tablebases for 4x4 chess have been completed in 2011 by the same author.

Federico Poloni
  • 515
  • 5
  • 15
1

I think the other answers and comment have made it clear that this is a really complex problem. So complex, that no classic computer will ever come close to solving it.

But I have asked myself the same question. Isn't chess a perfect example to use a quantum computer? Although quantum computers are not yet fully ready for use there do exist "programming languages", and there are already many algorithms for a quantum computer described this way. There actually even exist simulators for running these quantum computer "programs" on classic computers.

I wonder how feasible it would be to write an algorithm for solving chess on a quantum computer. Or would that require so many qbits, that we can never expect an actual quantum computer that could run this algorithm? But if it would once exist, shouldn't it be able to answer the question within a second?

Just to clarify, since I mentioned the possibility to simulate these algorithms on classic computers. This simulation of course only makes this very complex problem solved even less performant, so those simulators could only be used for testing of the algorithm, not for really using it.

Claus
  • 11
  • 1
  • there are no actual quantum computers as of now hence those algorithms don't mean much as you still don't get the time advantage of quantum computers. so we have to wait for that – shashank shekhar singh Jul 11 '20 at 04:11
1

Nice question. The wikipedia page Solving chess also report only endgame tables as partial results.

Personally, I believe that tools from combinatorial game theory are essential to solve chess completely (unless by something similar to an exhaustive search; I would argue that the machine learning method is under the hood an exhaustive search [plus some pruning-type techniques]). Don't expect the the machine learning tools to be of help here if you think the solution should build a chess theory. Combinatorial Game Theory (CGT) is the study of combinatorial games. Chess is not a combinatorial game. But, tools from CGT sometimes do a good job of (partially) solving games which are not combinatorial (e.g.: dots and boxes).

See papers [1] and [2] for a taste of what to expect.

By the way, the fact that generalized chess is computationally hard says practically nothing about chess.

Linked resources:
[1] Qingyun Wu, Frank Yu, Michael Landry, A combinatorial game theoretic analysis of chessendgames.
[2] Noam D. Elkies, On numbers and endgames: Combinatorial game theory in chess endgames.

Cyriac Antony
  • 1,371
  • 9
  • 26
-1

I think everyone here has a good and necessary point with the space complexity of chess being larger than the number of atoms in the observable universe. However, I think I can add some additional points onto that.

  1. As others have pointed out, the definition of "solved" is not super clear. If "solved" means "find the optimal move from any possible game state", solving chess is impossible. However, if we take solved to mean that we just want to play the single "perfect" game of chess we may be able to reduce space complexity.

  2. The problem with point one is that chess does not have a single perfect game. A perfect game of chess implies that you are always making the move that leaves you with the highest chance of winning. It also implies that you are accounting for your opponent making optimal moves, as otherwise it would not be a perfect game (and would also break the previous space complexity issues). However, chess is a well balanced game and any game perfectly played would necessarily end in a draw.

  3. There are likely a near-infinite number of optimally played games that end in a draw. While the number of optimal moves from any position is much smaller than the number of possible moves, according to the best chess engines we have today there are many moves that have near-optimal results from the starting position and in the first few moves. Because a significant advantage is needed to win any game of chess, many of these moves would likely be included in a "perfect" or solved game.

  4. Lastly, how would you prove my points 2 and 3 right or wrong? You can't. Again, you run into the space complexity issue. To prove any position is either winnable or unwinnable if both players play optimally, you need to diagram out every single possible move from a given position. As we have shown earlier, this is impossible until we are below a certain piece number.

Richard Boone
  • 17
  • 2
  • 1
    I'm not sure you understand what "solving" a game means. Once you "solve" a game you have a strategy by which you always win (as with checkers, which has been solved), or for some games, always tie (as in tic-tac-toe). It isn't just playing "a single perfect game" – Kevin Jul 09 '20 at 21:13
  • "There are likely a near-infinite number of optimally played games that end in a draw.": It is believed that an optimally played chess game is a draw. proving it would be a huge step toward solving chess. – Bit Chaser Jul 11 '20 at 01:33
  • @KevinWells That definition of "solving" is not available to chess, so it makes the conversation much less interesting. (You can't force a win from every possible game position, and it isn't clear that you could force a win from the starting position, either.) – jpaugh Jul 14 '20 at 22:27
  • @jpaugh What other definition of "solving" are you using then? The whole point of this question is if we are able to move towards that goal, and if so what efforts are being made to do so. The real answer is that solving chess is certainly out of reach currently, and may never be in reach, but there are still efforts to be made to improve our knowledge of solving increasingly complex starting positions – Kevin Jul 15 '20 at 14:27
  • @KevinWells How about, being able to know the outcome of any game, given the starting position, and two opponents who play perfectly? Solving chess is not about winning an individual game, but predicting the outcome of every game well enough to give yourself the best chance of winning. – jpaugh Jul 15 '20 at 14:36
  • @jpaugh Your question is answered by my point four. You can't know the outcome of any game from a given starting position because that would be equivalent in complexity to knowing the outcome of all the possible games that could be played. As has been discussed in the other answers, this is impossible. – Richard Boone Jul 15 '20 at 15:27
  • @jpaugh Solving for every possible game from the starting position is exactly what I'm talking about. I even said that solving a game isn't about playing a single perfect game, which is exactly what you're saying, so I don't understand your criticism. You said that that is a bad definition because it's out of reach for chess, but that doesn't make it a bad definition, it just makes the answer kind of boring – Kevin Jul 15 '20 at 18:44