Good mathematical book on algorithms

Question

I’m a sucker for mathematical elegance and rigour, and now am looking for such literature on algorithms and algorithm analysis. Now, it doesn’t matter much to me what algorithms are covered, but very much how they are presented and treated.¹ I most value a very clear and precise language which defines all used notions in a stringent and abstract manner.

I found that the classic Introduction to Algorithms, by Cormen, Leiserson, Rivest and Stein is pretty neat, but doesn’t handle the mathematics well and is quite informal with its proofs and definitions. Sipser’s Introduction to the Theory of Computation seems better in that regard, but still offers no seamless transition from mathematics to algorithms.

Can anyone recommend something?

¹: The algorithms should at least invole the management of their needed data using classical non-trivial abstract data structures like graphs, arrays, sets, lists, trees and so on – preferably also operating on such data structures. I wouldn’t be too interested if the issue of usage and management of data structures was ignored altogether. I don’t care much about the problems solved with them, though.

This is subjective; define "good". Also, while we don't have a strict policy for list questions, there is a general dislike. Please note also this and this discussion; you might want to improve your question as to avoid the problems explained there. If you are not sure how to improve your question maybe we can help you in [chat]? — Raphael, Jul 27 '15 at 15:28
@Raphael Thanks. I only used the word “good” in the title, in my question I specified what I want. While I intentionally did not get too specific, it should at least be clear that my focus is (as hinted) at mathematical elegance and rigour. I don’t think this answer screams for a list of books, because there shouldn’t be too many books falling into that category, – and even if so, I don’t believe in that “preserving the purity of a strict question–answer structure” thing going on at several stackexchange sites here – but not my call, I guess. I’m not sure if I can improve the question. — k.stm, Jul 27 '15 at 16:52
Also, I don’t think the “reference-request” is appropriate for the question. At least not according to its description: Neither am I looking for papers nor am I interested in a specific and narrow issue. In fact, I am pretty open as to what contents are covered, see my second sentence. Is it okay if I remove the tag again? Maybe I could and should narrow down with what sort of algorithms I would be content? — k.stm, Jul 27 '15 at 16:55
Perhaps you should look into denotational semantics and program verification. — Yuval Filmus, Jul 28 '15 at 05:12
use of the word "good" alas is too subjective on SE but maybe there is a way to rescue this question somehow... this comes to mind: the two books by Flajolet, Sedgewick on analysis of algorithms are intro/ very advanced and are highly mathematical. — vzn, Jul 28 '15 at 18:29
@jrodatus Elements of Programming sounds like the most promising book of all suggestions so far (for all of which I am very thankful by the way). After reading the preface, skipping over the second chapter and listening to half of Stepanov’s talk at Stanford, my impression is that they wrote a very odd book which nevertheless is likely the next place I should be looking at. I’m not sure if like the idea of paying (that) close attention to real-world computer limitations, though. Anyways, you could make one answer out of both your comments, I think. — k.stm, Jul 29 '15 at 10:15
@k.stm, Pardon me if this is not what you're referring to, but if you're referring to the EE380 lectures, that link seems a strange misnomer, as they are exclusively Computer Systems Colloquium seminars of which subject-area is made no mention at all in the TOC. I only found one single lecture explicitly covering the book here http://youtu.be/Ih9gpJga4Vc — , Jul 29 '15 at 16:55

score 7 · Answer 1 · answered Jul 27 '15 at 22:28

7

I think the book you describe has a name. It is in seven volumes, only three and half of which have been published. It is called The Art of Computer Programming, (TAOCP) and is written by Donald Knuth.

It may be though that he will sometimes describe applications. But you can always skip that, and I doubt it makes much of the content. You should not be too disappointed with the maths.

answered Jul 27 '15 at 22:28

babou

19,445
40
76

I feared this answer might come up. I onced looked into it and it didn’t suit me. “You should not be too disappointed with the maths.” – maybe, but Knuth also doesn’t seem to define stuff, but he rather introduces it informally, explaining it. Great to get the idea across, not what I expect of mathematics, though. – k.stm Jul 27 '15 at 22:38
Maybe you could contribute to the field by publishing the mathematically obfuscated version of his work. But if what interests you is the "seamless transition from mathematics to algorithms", a better topic for you could be type theory. The relation between mathematics and algorithms is still a research topic. – babou Jul 27 '15 at 22:50
1

P.S. If you "feared this answer might come up", you should have said so in your question as it is one of the major reference books. Complete and documented questions get better answers. – babou Jul 27 '15 at 23:02
It only came to me after having asked the question, when I was away from my computer. My remark on Knuth’s book shouldn’t be taken in a pejorative way (which is how I think you have taken it for the “publishing the mathematically obfuscated version” comment) – that would be like next to blasphemy, probably. His way just is not what I am looking for either. Maybe there just isn’t anything like what I imagine out there … – k.stm Jul 27 '15 at 23:28
Also, I don’t want to dismiss The Art of Computer Programs just because of my first impression and the observation that several things are left informally defined. The suggestion is very valid (as would also be Sipser’s book had I not already mentioned it), I just hope there will be others as well. – k.stm Jul 27 '15 at 23:35
3

@k.stm Well, forget the obfuscation thing which is a whole topic in itself. The point is, I believe that algorithmics is not (yet?) mathematics. It may be that you can get formalizations, but they are likely to be simple encodings that lose the substance, with little benefit. That depends on the topic. It is probably much less true for automata theory, which does have significant algorithmics. The point is that identifying adequately the abstract mathematical nature of structures is non obvious. It is a matter of understanding, not of formalization, and it takes years. Think of it like physics – babou Jul 27 '15 at 23:51
@k.stm "Knuth also doesn’t seem to define stuff, but he rather introduces it informally, explaining it. Great to get the idea across, not what I expect of mathematics, though." -- definitions don't have to be formal in the sense that they use symbols. In fact, most of my mathematics lectures were less formal than my TCS lectures, because they used lots of natural language in proofs. You need to give specific examples of what you want in the question. – Raphael Jul 30 '15 at 06:54
@Raphael By formal definitions I don’t mean definitions using symbols, I mean precise definitions, introducing very sharply a new word usage, preferably built on already defined notions. I have no problem with informal proofs (they still can be rigorous), but I have a problem with informal definitions. If you don’t define your stuff formally, you can throw rigour out of the window. It must absolutely clear what statements say in order to assess them as true or not. – k.stm Jul 30 '15 at 07:17
@Raphael The Art of Computer Programming, p. 238 defines a linear list as “a sequence of $n ≥ 0$ nodes $X[1]$, …, $X[n]$ whose essential structural properties involve only the relative positions between items as they appear in line.” This is a very informal definition as far as it relies on words like “essential” and “relative” and notions like “structural properties” and “items” which just aren’t that clear. – k.stm Jul 30 '15 at 07:24
@k.stm The definition is "sequence of $n \geq 0$ nodes", $X[i]$ gives notation, and the rest is an explanation. – Raphael Jul 30 '15 at 07:44
@Raphael Okay, then I can’t immediately tell what the definition is and what is not (which is also bad if you demand precision of language). Then take the introduction of doubly linked lists on p. 281, I can’t find any definition at all. As I said Knuth informally introduces stuff. Nothing inherently wrong with that, still not what I want of a mathematical treatment. – k.stm Jul 30 '15 at 08:15
@k.stm Well, there's a trade-off. Check out Coq or Isabelle/HOL for a completely rigorous approach to algorithms; tell me if that was instructive, let alone fun. (Also, beware false assumptions. You seem to equate "mathematical" and "rigor"/"precision". That can easily cause unhelpful expectations; I think you'll find that most of mathematics is not very mathematical, in your sense.) – Raphael Jul 30 '15 at 09:48
Let us continue this discussion in chat. – k.stm Jul 30 '15 at 10:04
@k.stm, Raphael was right, the conversation maybe interesing for other people (like me). I'm glad that you stick to english. – FNH Feb 06 '17 at 07:42

score 7 · Accepted Answer · 2015-07-30T02:18:23.190

7

Hendrik Lenstra wrote in 1992:

Although it is, from a rigorous mathematical point of view, desirable that I define what I mean by an algorithm and its running time, I will not do so. My main excuse is that I do not know these definitions myself. Even worse, I never saw a treatment of the appropriate theory that is precise, elegant, and convenient to work with. It would be a laudable enterprise to fill this apparent gap in the literature.

I do not know whether any progress has been made since then, or whether this is even considered a "gap" by the consensus. But the point remains that at least some eminent mathematicians have been dissatisfied with the mathematical rigour of the derivation of algorithms. So, it may be there exists no book with the OP's desired level of formalism.

The cornucopia of practical perspectives we have due to Knuth, Gries, Stepanov, and others are intended to aid programmers more than mathematics and so inevitably fall short on rigor and long on subjectivity.

Nonetheless, Stepanov's work is widely acclaimed in Silicon Valley as one of the best attempts at a scientific synthesis.

In Elements of Programming, Alexander Stepanov and Paul McJones attempt to lay the abstract algebraic foundations of algorithms. The book begins with admittedly somewhat informal axiomatic definitions of entities, values and their attributes, but in 288 pages progresses deductively via a series of lemmas to the foundations of the Standard Template Library.

The TOC, preface and a sample chapter on Transformations and Their Orbits can be found here, and an introductory lecture here.

Stepanov's more recent and relaxed book, From Mathematics to Generic Programming, is structured more by a roadmap of the history of mathematics, building from Egyptian multiplication to monoids, semigroups, and Lagrange's theorem, eventually developing modern data structures with their iterators and algorithms used in the STL.

edited Jul 30 '15 at 02:18

answered Jul 29 '15 at 18:09

?!? "there currently exists no precise, elegant, and convenient mathematical derivation of the concept of an algorithm..." how about a Turing machine! – vzn Jul 30 '15 at 01:01
1

Lenstra does address Turing machines in the linked paper, but I think the idea is that they don't provide a full algebraic analysis. The quotation is pretty much verbatim, including the "gap" part, if you'd like to read it yourself. I'll edit the post to remove the suggested "consensus", in case it should be considered debatable. – Jul 30 '15 at 01:07
ofc am/ was aware you were quoting Lenstra but you could add quotes or a longer blockquote of his remarkable/ debatable/ questionable assertion – vzn Jul 30 '15 at 02:05
think that Turing machines cannot be derived, to the contrary they form the axiom of a computational system (ala the Church-Turing thesis). it is all of Lenstras analysis and CS in general that is derived from the axiom of the TM existence. – vzn Jul 30 '15 at 02:13
Then you agree that an algebraic treatment of algorithms involves much more than the axiomatic existence of a TM. There may just be a miscommunication here. Thanks for suggesting the edit. Hope it cleared things up. I don't mean to waste your time with an argument. – Jul 30 '15 at 02:40
Excellent answer! Pretty much what I was hoping for, I’m very happy you wrote this. – k.stm Jul 30 '15 at 06:48
I disagree. Given formal syntax and semantics, it is easy to define mathematically rigorous notions of cost of algorithms (at least for additive cost measures). Of course nobody does that, but that does not mean it can not be done. For a hint, see here. – Raphael Jul 30 '15 at 06:52
@Raphael, very interesting link (your answer). The lack of agreement here seems due to scope, purpose and opinion, as should probably be expected from a question that uses the word "good"... which you fairly warned about :) – Jul 30 '15 at 07:32
@Raphael With what do you disagree? In Lenstra’s quote, he is talking about the lack of a precise, elegant and convenient theory – you seem to suggest yourself that formally defining syntax and semantics is neither elegant nor convenient to work with? So with what are you disagreeing? – k.stm Mar 29 '16 at 21:50
@k.stm Wherever did I say that the result would be inelegant or inconvenient? I only said that nobody did it, which a) is an exaggeration and b) does not imply what you say. So yes, I did and do disagree with the gist of the answer which I take to be, "don't bother with formal algorithm analysis". – Raphael Mar 29 '16 at 22:52
@Raphael Okay, thanks for clearing that up. (Though I think you’re wrong about what you think the gist of the answer seems to be: Neither jrodatus nor Lenstra said something in the lines of this. And it wasn’t the philosophy I got out from that answer either.) – k.stm Mar 30 '16 at 06:28
1

@Raphael, if mom says it would be a "laudable enterprise" to clean my room properly, the gist of her speech is "don't bother with cleaning your room." Am I misunderstanding you, or are you misunderstanding me? – Mar 30 '16 at 08:31
The latter, apparently. Anyway, that quote is from 1992? The author was wrong, afaik. Formal models for algorithms and costs existed well before that. (Whether they were/are "elegant and conveniant to work with" is, of course, a matter of taste.) – Raphael Mar 30 '16 at 08:50

Good mathematical book on algorithms

2 Answers2