Why is non-determinism a useful concept?

Question

An automaton is an abstract model of a digital computer. Digital computers are completely deterministic; their state at any time is uniquely predictable from the input and the initial state.

When we are trying to model real systems, why include nondeterminism in Automata theory?

It would possibly help to ask who originally described NTMs and what their purpose/goal was at the time. — usul, Mar 10 '14 at 21:13
Note that the fact that the machine is deterministic does not always mean our code is. Anyone who's done multitasking/multithreading can attest to the fact that the times at which task switching occurs is often unpredictable in any practical terms and we have to design explicit interlocks to make their behavior appear deterministic. (Basically, there are hidden variables in the state.) Communications raises the same issue. I honestly don't know whether NDAs help address these -- I'm a software engineer, not a computer scientist -- but in the real world your premise is overoptimistic. — keshlam, Mar 11 '14 at 03:26
When you talk about multithreading, arguably, you have non-determinism, at least if you consider metal and OS to form the machine. What's funny is that the code itself is deterministic. — Raphael, Mar 12 '14 at 20:37
@ Raphael, @keshlam In other words we can say that "Non-deterministic models are also useful to simulate parallel execution of code" — Grijesh Chauhan, Mar 13 '14 at 14:33
@keshlam I added your point in my answer, @ Tanmoy read updated my answer. — Grijesh Chauhan, Mar 13 '14 at 15:35

score 19 · Accepted Answer · edited May 23 '17 at 12:37

Yes, you are correct computers are deterministic automate. Non-deterministic models are more useful for theoretical purpose, sometime the deterministic solution is not as obvious to the definition(or say problem statement) and so little hard to find solution. Then one approach is that first design a non-deterministic model that may be comparatively easy to design and then try to convert it into a deterministic one. Below, I have tried to demonstrate what I mean with an example. Consider regular expression:

(01)*01(0 + 1)*

Now suppose, if you are asked to draw DFA for the language generated by above RE.

With my knowledge of designing FAs, I know that (1) when a * present in regular expression indicated I need corresponding loop in FA (2) concatenate operations like a.b means something like: (q₀)─a→(q₁)─b→(q₂).

So, at my fist attempt I would draw an NFA like:

fig

Thought this is not a deterministic solution but looks very simple FA that can be easily designed using the given regular expression. My kind-of-analogy to show similarity between the above regular expression and my NFA is as below:

The loop at state q₀ should be for (01)*
01 (after (01)*) gives (q₀)─0→(q₁)─1→(q₂)
(0 + 1)* gives a self loop at state q₂ for label 0, 1

According to my analogy I think the FA I drawn above is comparatively simple to draw from given RE. And luckily in class of finite automata every Non-deterministic model can be converted into an equivalent deterministic one. We have algorithmic method to convert an NFA into DFA. So I can easily convert above NFA into a DFA:

fig-2

Other part is unfortunately this is not always possible to convert a non-deterministic model into deterministic one, for example class for deterministic push down automate is subset of class of deterministic push-down automate "check venn diagram" and you can't always convert an NPDA into a PDA.

Usually when it is not possible to convert a non-deterministic solution into deterministic one then with the help of non-deterministic solution we define deterministic solution in sub-domain (or say partial domain) instead of complete domain. Or we define solution in some other ways (e.g. greedy approach) that of-course may not give you an optimal solution.

Sometimes non-determinism is an effective mechanism for describing some complicated problem/solution precisely and effectively, for an example non-deterministic machines can serve as model of search-and-backtrack algorithm (read: How string process in non-deterministic model using backtrack). Oppositely deterministic models better represents efficient, minimized and less-redundant solutions.

Here I would also like to quote from Wikipedia Use of Nondeterministic algorithm:

In algorithm design, nondeterministic algorithms are often used when the problem solved by the algorithm inherently allows multiple outcomes (or when there is a single outcome with multiple paths by which the outcome may be discovered, each equally preferable). Crucially, every outcome the nondeterministic algorithm produces is valid, regardless of which choices the algorithm makes while running.

A large number of problems can be conceptualized through nondeterministic algorithms, including the most famous unresolved question in computing theory, P vs NP.

As @keshlam also mentioned in his comment: "Nondeterminism" is in practice used to refer to any unpredictability in the outcome of some process. For an example, Concurrent programs exhibit non-deterministic behaviour - two executions of the same program with the same input can produce different results (if concurrency control mechanism are not applied ). Read more about this in "Usefulness Of Non Determinism".

I would also suggest you to read following links:
1. What is the difference between non-determinism and randomness?
2. 9.2.2 Nondeterministic vs. Probabilistic Models: (a). Nondeterministic: I have no idea what nature will do. (b). Probabilistic: I have been observing nature and gathering statistics.
3. Nondeterministic programming

@Grijest: many thanks for such a huge elaboration.Only one confusion:"Oppositely deterministic models better represents efficient, minimized and less-redundant solutions."-But I think deterministic models are less efficient than nondeterministic one.(That is why NP problems are more complex than P. Isn't it?) — Tanmoy Banerjee, Mar 21 '14 at 15:08
@tan Actually using word "efficient" is wrong, And Yes, you are correct that non-deterministic models are more capable than deterministic one. Class of problems covered by deterministic models is subset of Non-deterministic model. — Grijesh Chauhan, Mar 21 '14 at 15:15
so in which context deterministic models are "efficient" than nondeterministic one(As you mentioned)? — Tanmoy Banerjee, Mar 21 '14 at 15:19
@tan Suppose if you wants to perform further operation (e.g. wants to convert FA to RE, or explain proof for pumping lemma, or some other..) then Deterministic model give you better results (so I said efficient). — Grijesh Chauhan, Mar 21 '14 at 15:23
Yes:Existance of two or more leftmost or rightmost derivations for a string w belongs to L. Ambiguity may lead to incorrect parsing of mathematical expressions. — Tanmoy Banerjee, Mar 26 '14 at 10:00
@tan exactly, this is also a case of non-determinism, but ambiguous form preferred for compiler writing, and ambiguity is resolved explicitly read last-para in this answer — Grijesh Chauhan, Mar 27 '14 at 05:17

score 9 · Answer 2 · answered Mar 10 '14 at 16:40

9

It is more the other way around: automata arose first, as mathematical models. And nondeterminism is quite natural, you often have several paths open before you. Instead of some messy way of specifying that all paths must be followed to the end in some order, and perhaps getting bogged down by infinite branches, and... just use nondeterminism.

And while nondeterministic programming languages aren't mainstream, they have an ilustrious history, perhaps starting with Dijkstra's GCL. As machines acrue more and more cores (independent processors), some form of nondeterminism is seeping into all programming.

answered Mar 10 '14 at 16:40

vonbrand

14,004
3
40
50

I think the first part of your answer is factually wrong. Why do you think automata arose first? Both DFAs and NFAs were defined 10+ years after Turing defined TMs. See the discussion on cstheory – Artem Kaznatcheev Mar 10 '14 at 20:19
@ArtemKaznatcheev, the Turing machine model is an automaton, and it certainly predates computers by at least a decade. – vonbrand Mar 10 '14 at 20:21
yes, but when people say automata they don't mean TM but they mean finite state automata and their direct extensions (PDAs, NPDAs, etc). See the question I linked for a history there and you will see that both TMs and the von Neumann architecture were developed independent of what we now call automata theory. – Artem Kaznatcheev Mar 10 '14 at 20:23
4

@ArtemKaznatcheev, DFA/NFA, PDA, LBA, TM are all automata. As are transducers (FA with output, PDA with output). – vonbrand Mar 10 '14 at 20:35
1

The last paragraph is wrong. Prolog predates GCL, and is even still around and fairly mainstream. Prolog of course wasn't designed in a vacuum, building on previous nondeterministic programming languages such as PLANNER. The credit probably goes to Golomb and Baumert "Backtrack Programming" from 1965. – Pseudonym Mar 13 '14 at 01:23
FWIW, I have always understood "automata" to include Turing machines. Maybe this is a language barrier issue? – Patrick87 Mar 14 '14 at 13:04
The first use of non-determinism in programming that I know of is the work of Bob Floyd in 1966 at CMU, later published in JACM The style is outmoded, but it is clearly about non-determinism in programming languages. He does reference Golomb and Baumert. Interestingly, his example is the 8 queen problem, reused in later in presentations of non-deterministic languages such as Prolog. I do concur that I always comsidered TM to be automata. CC @Pseudonym (BTW last parag says "perhaps", so it cannot be wrong). – babou Oct 23 '14 at 11:43

score 7 · Answer 3 · edited May 23 '17 at 12:37

7

NFAs might be used in practice, check out this answer on stackexchange. The reason is that the powerset construction can be simulated on-the-fly, so to speak. In order to simulate an NFA on a deterministic computer, we just keep track of the possible states that the NFA could be in. Typically, this number would be small, and so the simulation would be fast. This is much more practical than running the actual powerset construction: the resulting automaton could be very large, even though in practice most of the sets would rarely be reached.

Nondeterminism is also important for computation complexity, where it is used for defining the class NP. (The class NP also has other, equivalent definitions, for example using witnesses.)

edited May 23 '17 at 12:37

Community

1

answered Mar 11 '14 at 03:04

Yuval Filmus

276,994
27
311
503

understanding your answer but couldn't grasp it properly.could you please elaborate the fact that how powerset construction can be done easily using nondeterminism? – Tanmoy Banerjee Mar 11 '14 at 05:54
"Nondeterminism is also important for computation complexity, where it is used for defining the class NP." -- that supports the importance of non-determinism only if we assume that NP is a useful concept, which it is only if non-determinism is useful. – Raphael Mar 11 '14 at 11:28
@Raphael NP-completeness is an important concept regardless of your stance on non-determinism. – Yuval Filmus Mar 11 '14 at 11:46
2

@Tanmoy If you have nondeterminism you don't need the powerset construction, but unfortunately real computers are deterministic. Nevertheless, it might be easier to simulate an NFA directly rather than converting it to a DFA first. Check out the answer I link to for more details. – Yuval Filmus Mar 11 '14 at 11:48

reinierpost · Answer 4 · 2019-02-28T19:43:38.887

(This is a rewording of some of the other answers but I'll post it anyway:)

You write: An automaton is an abstract model of a digital computer.

I disagree! Automata model how we humans specify computation, not only how computers execute it. Nondeterminism is exactly the difference. Our specifications are often nondeterministic.

For instance, take merge sort. Merge sort is sorting by splitting the items to be sorted into two halves of roughly equal size, sorting each half using merge sort, and merging the sorted results. This completely specifies the idea of merge sort, but it isn't deterministic: it doesn't specify an order in which to sort the halves (for all we care, it may be done concurrently), nor does it specify an exact way to determine the split. Those details will need to be filled in in order to arrive at a deterministic, sequential version of merge sort that can be implemented by a single-threaded computer program, but I would say they are part of a particular way of doing merge sort, not the idea of merge sort itself.

The same thing is true for algorithms in general - e.g. cookbook recipes. Some people define algorithms to be deterministic, in which case this more general and in my opinion more natural notion of 'algorithm' needs a different name.

The idea of working with nondeterministic specifications was formalized by Dijkstra's method of programming, which starts out by specifications that only give pre- and postconditions to be met by the program, and systematically develops a deterministic, imperative program from them. Dijkstra would probably have said: sorting is the problem, the relationship between pre- and postconditions we're trying to establish; merge sort is an approach to doing that, somewhere halfway between the problem specification and a deterministic solution; a particular, deterministic merge sorting algorithm is a concrete deterministic solution. But the same general approach can be used for developing concurrent programs, in which the eventual program is still nondeterministic. Such programs can e.g. be run in distributed computing environments.

score 4 · Answer 5 · answered Mar 11 '14 at 11:37

You state correctly that automata are models, so there are two parts of use non-determinism can have:

Use in modelling real problems.

Not all automata are equally powerful if you remove nondeterminism, e.g. pushdown automata (CFL $\neq$ DCFL). So while we will have to simulate an NPDA in a deterministic fashion in the end, i.e. when we actually implement a parser, we need it as model for some languages.

Furthermore, non-deterministic automata can provide more compact representations of languages. For example, it is well-known that there are NFA whose minimal equivalent DFA are exponentially larger.
Use in theory.

Using non-determinism can simplify proofs, see e.g. converting regular expressions into finite automata.

score 2 · Answer 6 · answered Mar 10 '14 at 16:23

2

You are right, we can NOT build a nondeterministic machine. Therefore, the aim is not using the concept for building better machines. Rather, nondeterminism is a useful concept when trying to understand computation. For instance, we now know that, from a computability perspective, nondeterminism is not something more powerful than determinism, meaning that we can simulate a nondeterministic machine by using a deterministic one. However, from the complexity perspective, nondeterminism allows us, for instance, to reason and try to understand the relation among the difficulty of finding an efficient solution for a problem and the difficulty of verifying a solution (which is the famous P versus NP problem). And so on. Therefore, the main reason for studying nondeterminism is theoretical.

answered Mar 10 '14 at 16:23

Massimo Cafaro

4,247
18
27

Context-Free versus Deterministic Context-Free? – alto Mar 10 '14 at 19:13
@alto What about it ? – babou Mar 10 '14 at 20:22
@babou I was trying to point out that "nondeterminism is not something more powerful than determinism," is a false statement. NPDAs are more powerful than PDAs. – alto Mar 10 '14 at 20:38
1

@alto: no, you are misunderstanding the statement. From a computability perspective, they are fully equivalent, since the class of problems (or languages if you prefer) that you may solve INDEPENDENTLY of how much computational resources are required is the same. And indeed, you CAN simulate a nondeterministic machine with a deterministic one. Again, time and space required DOES NOT MATTERS in the computability context. – Massimo Cafaro Mar 10 '14 at 21:33
Not being a computability expert, I am not sure I am following you. I understood your statement as meaning that non-detrministic whatever can always be simulated by some appropriate deterministic device. I think that @alto understood that as implying that you meant that a non-deterministic device of type X can be simulated by a deterministic one of the same class, which is not true for PDAs. I did not interpret your text as meaning that. Was I correct? – babou Mar 10 '14 at 21:49
The misunderstanding is probably due to the fact that I and alto are referring to DIFFERENT things. I am speaking of non determinism referring to Finite State Automata. In this case, non deterministic FSA are fully equivalent to deterministic FSA. alto instead refers to non deterministic PushDown Automata which are obviously more powerful than deterministic PushDown Automata. In my answer I never refer to nondeterministic PDA, always to nondeterministic FSA. And, when commenting on @alto comment, I did not see that he was referring to PushDown Automata (that was my fault). – Massimo Cafaro Mar 10 '14 at 23:49
Yes, I generally take automata to mean DFA, NFA, PDA, NPDA, etc. In your answer you simply say "machine," so it wasn't clear to me what type of machine you were referring to. I'd also disagree with the statement "the main reason for studying nondeterminism is theoretical." Even if we are restricting the discussion to DFAs and NFAs, as @Yuval Filmus points out in his answer, in many cases it can be much more efficient to simulate the NFA than convert it to a DFA, which is certainly of practical importance. – alto Mar 11 '14 at 13:58
@alto, simulating the NFA rather than converting it is exactly what I said in my answer. Regarding the importance, my bias towards theory rather than practice is motivated by the fact that I am a CS professor, so, I would weight practice as 20% and theory as 80%. Of course, depending on your background, you are free to disagree ;-) – Massimo Cafaro Mar 11 '14 at 14:05
1

@MassimoCafaro Couldn't agree more, in theory. In practice it appears I prefer to quibble about semantics. – alto Mar 11 '14 at 15:40
We can easily build a nondeterministic machine. – reinierpost Oct 23 '14 at 10:51

score 2 · Answer 7 · answered Mar 11 '14 at 02:49

the invention of the Turing Machine was in 1936 by Turing. FSM-like models were introduced by McCulloch and Pitts, two neurophysiologists, as a model for neurobiological activity in 1943. from the Stanford CS history page:

The exciting history of how finite automata became a branch of computer science illustrates its wide range of applications. The first people to consider the concept of a finite-state machine included a team of biologists, psychologists, mathematicians, engineers and some of the first computer scientists. They all shared a common interest: to model the human thought process, whether in the brain or in a computer. Warren McCulloch and Walter Pitts, two neurophysiologists, were the first to present a description of finite automata in 1943. Their paper, entitled, "A Logical Calculus Immanent in Nervous Activity", made significant contributions to the study of neural network theory, theory of automata, the theory of computation and cybernetics. Later, two computer scientists, G.H. Mealy and E.F. Moore, generalized the theory to much more powerful machines in separate papers, published in 1955-56. The finite-state machines, the Mealy machine and the Moore machine, are named in recognition of their work.

not a CS historian, but suspect that the McCulloch-Pitts model did not include nondeterminism and the Mealy-Moore model did, in a natural generalizing/abstraction of the formal/theoretical concept. note that DFAs and NFAs have the same representational power so that if one wishes to model real systems there is a choice of either. one basic difference is that an NFA may be much smaller than an equivalent DFA (so for example there is a natural element of data/information compression). there are also natural aspects/analogs of parallelism in NFA study.

Hey I saw you profile and looks like someone intentionally down-voting your answers (every where you have just two downvotes) ... This answer not wrong, the answer add useful information. +1 — Grijesh Chauhan, Mar 11 '14 at 06:34

Tanmoy Banerjee · Answer 8 · 2014-03-19T17:33:13.787

First of all I would like to say thanks to all the people who answered the question.All the answers are important and add some useful information.But as it is a tricky question to the beginners, and need sufficient time to understanding it well, I would try to summarize what I have gained from all the answers and from some books:

Actually I had a confusion which was about the mechanism of nondeterministic model. I always wondered about the nondeterministic machine as it is a non-mechanical machine which does not exist in real world. I always compared Automata with our now days computers which is completely deterministic in nature.Actually I wasn't properly understanding the nondeterministic model. Now I think I am understanding nonderministic model quite well: A nondeterministic machine is a machine which always follow that path of execution which leads to acceptance of the string(Without Backtracking).But how this can be possible in real life? : It is absolutely impossible for present days computer to be such intelligent to predict the future. So why nondeterminism at all?. Answer of this question is quite tricky.What I concluded about the question is that: Automata Theory did exist when computers did not exist(First Theory then practical). It is purely theoretical subject and concept of nondeterminism came intuitively.The motive of the subject 'Automata Theory' was not to dealing with practical computers. But when practically computer come then using Automata Theory we are able to define practical computers precisely: what are the limitations of present day computers.which algorithmic problem are very complex to computers and so impractical(Here the role of nondererminism is very crucial by which we can distinguish two complexity classes P and NP).What is the solutions for these impractical problems by which it can be executed comparatively faster.etc. This is the usefulness of nondeterminism.

Please correct me if there is anything wrong.

It is wrong to say that a nondeterministic machine is a machine which always follows the path of execution which leads to acceptance of the string. It does not do that! A nondeterministic machine is a machine whose operation allows certain unpremeditated (= nondeterministic) choices to be made during execution. There is nothing unrealistic about such machines, e.g. they can ask the environment to make those choices. These machines are then applied to tasks for which it holds that certain choices will produce an accepting state. — reinierpost, Apr 29 '14 at 12:05
@reinierpost: So you are saying that non-deterministic machines do exist in real life. — Tanmoy Banerjee, Apr 30 '14 at 15:32
Yes. Here is an example: suppose you are driving a car, and you are not making any decisions on the route to follow. For instance, you may be driving around aimlessly, or you may be taking your directions from a human navigator or a navigation device. The car and you are a nondeterministic system to drive places. You move through traffic and keep encountering choices which direction to take. To you and the car, these choices are nondeterministic: you are not deciding which direction to go, but given that decision, you will follow it. — reinierpost, May 01 '14 at 16:16
@reinierpost: Is there any non-deterministic computer exist? My answer is NO. because if it exists then NP problems would have polynomial time complexity. isn't it? — Tanmoy Banerjee, May 02 '14 at 17:30
Whether computers are deterministic or nondeterministic depends on how you look at them. When a computer stops and waits for the user to do something, and its next actions will depend on what the user does, you can say that is a nondeterministic choice. No, this doesn't imply that P = NP. — reinierpost, May 02 '14 at 22:03

Tom · Answer 9 · 2014-08-30T19:57:53.600

0

non-determinism is useful because it helps you understand determinism, but not the other way around. You could say non-determinism is the bigger idea. A deterministic turing machine is a special case of a non-deterministic one. - Nondeterminism can help us understand why, on todays platforms, some problems are hard to pin down. There are a number of computational problems that have no efficient solution on a deterministic computing platform, but we understand that there can be efficient solutions on nondeterministic ones. ... state, encoding, nondeterminism they are all linked http://people.cs.umass.edu/~rsnbrg/teach-eatcs.pdf

In a deterministic Turing machine, the set of rules prescribes at most one action to be performed for any given situation. A non-deterministic Turing machine(NTM), by contrast, may have a set of rules that prescribes more than one action for a given situation. http://en.wikipedia.org/wiki/Non-deterministic_Turing_machine If you can build a software box that can manage state transitions so well that it can handle more than one action you can get performance beyond deterministic machines.

edited Aug 30 '14 at 19:57

answered Aug 30 '14 at 19:48

Tom

1
1

I'm not sure the alleged links to reality are at all helpful. It's quite clear that we can not build a non-deterministic machine (today at least) so it's an entirely theoretic construct. – Raphael Sep 01 '14 at 10:20
We can build a nondeterministic machine by letting the nondeterministic decisions be made by something external to the machine. – reinierpost Oct 23 '14 at 10:53
@ reinierpost, more importantly we can build non-deterministic machines as deterministic ones, without incurring exponential overhead. see Savitch's Theorem. http://en.wikipedia.org/wiki/Savitch's_theorem – Tom Oct 24 '14 at 16:37
@ Raphael, some references to the real world are important. Why does caching work? Because events in the real world, which is ultimately the source of all data, happen to follow a normal distribution. see Temporal locality: http://durablescope.blogspot.co.at/2009/11/caching-how-and-why-it-works.html – Tom Oct 24 '14 at 16:40
@ reinierpost, and that something external is what Turing called an oracle machine. I think you can think if that as data coming out of cache or something like a multi-tape machine or even tapping into to random access memory. – Tom Oct 24 '14 at 16:46

score 0 · Answer 10 · answered Jan 25 '15 at 14:50

Why is non-determinism useful concept?

Determinism has a strong tendency to break symmetries. _{This tendency is even stronger for sequential determinism, but the notion of an acyclic directed graph and a topological order of such a graph allows to ignore the difference between determinism and sequential determinism.} Non-determinism is a superset of determinism, which allows to preserve more symmetries. When designing the solution of a problem, starting with the non-deterministic solution allows to preserve useful symmetries, and which keeps the description of the solution small and compact. The breaking of the symmetries can then be delegated to a later stage during implementation, while converting the non-deterministic solution to a deterministic solution.

Often non-determinism means that the notion of a partial function is replaced by the notion of a relation. In that case, a non-deterministic machine can run both forward and backward in time, while this is not possible in general for a deterministic machine. If we work with total functions for determinism and multivalued total functions for non-determinsim instead, the symmetry is no longer as nice, but it still can be made to work.

Can you give a specific example? I find it hard to see what you mean by "symmetry" here. — Raphael, Jan 26 '15 at 10:29
@Raphael How about reversing (01)01(0 + 1) to (0 + 1)10(10) such that it recognizes the reversed input string, and apply this symmetry to the non-deterministic machine by reversing all arrow and swapping start and end state? I'm not sure whether there are significantly more interesting examples for finite state machines, but I could try to come up with an interesting example for a PDA instead. — Thomas Klimpel, Jan 26 '15 at 11:13
I wrote about my answer to a similar question in a blog post about Reversibility of binary relations, substochastic matrices, and partial functions. — Thomas Klimpel, May 12 '15 at 06:24

Why is non-determinism a useful concept?

10 Answers10

Linked