3

I am trying to understand connection between non determinism of grammar and LL(1) conflicts introduced by it.

As per my understanding non deterministic context free grammar is a context free grammar in which at some point in time during parsing, multiple actions are allowed. Also I believe non determinism occurs when we have multiple productions in grammar with same prefix of production (right hand side) bodies. Also I know we eliminate non determinism by left factoring grammar.

Wikipedia article says:

  1. We do left factoring to eliminate FIRST-FIRST conflicts. ref
  2. Left recursion is a special case of FIRST-FIRST conflict. ref
  3. Substitution is used for removing FIRST-FOLLOW conflict. However that may introduce FIRST-FIRST conflict.ref
  4. Left recursion is eliminated using rule like below ref :
    $A\rightarrow A\alpha | \beta $
    ios converted to
    $A\rightarrow \beta A'$
    $A'\rightarrow \alpha A' | \epsilon$

I have following doubts:

  1. Since both non determinism and FIRST-FIRST conflicts are removed with left factoring and since left recursion is special case of FIRST-FIRST conflict, is non determinism a reason behind FIRST-FIRST conflict and left recursion (and also for FIRST-FOLLOW conflict)? If yes, does non determinism cause anything else and there is nothing else apart from these three (FIRST-FIRST conflict, left recursion and FIRST-FOLLOW conflict) in which non determinism manifests?

  2. Does the solutions (left factoring, substitution and left recursion removal rule) for removing above can also remove non determinism from all grammars?

  3. I know LL grammar should be both non deterministic and unambiguous. Does ambiguity also manifests in some way in LL parsing table or grammar? I read ambiguity can be eliminated by enforcing association and precedence between grammar symbols and this resolves SHIFT-REDUCE conflicts in LR parsers. Is anything like this required for eliminating ambiguity to make grammar a valid LL grammar?

RajS
  • 1,667
  • 4
  • 26
  • 46
  • "Does the solutions (left factoring, substitution and left recursion removal rule) for removing above can also remove non determinism from all grammars?" First, some languages are inherently non-deterministic. Whatever grammar is used to generated one of such languages, it will be non-deterministic grammar. (I am speaking loosely as non-determinism can have in-equivalent definitions.) Second, it is far from clear whether a grammar that generate a deterministic language can be transformed to a deterministic grammar by those three operations. – John L. Jun 09 '19 at 18:59

1 Answers1

3

  1. No algorithm can remove non-determinism (or ambiguity) from every grammar, not even for every grammar for a language which has a deterministic grammar.

  2. Some deterministic grammars -- and even some deterministic context-free languages -- have no $LL$ parser. Such grammars are not necessarily ambiguous. An accurate deterministic parser obviously cannot be generated for an ambiguous grammar, but failure to generate a parser doesn't say anything about ambiguity.

  3. Yacc-style precedence/associativity declarations do not, in general, solve grammar ambiguity. But they do work for a couple of common simple cases (operator precedence, dangling else) allowing for the use of simpler grammars. That's handy in practice, but it's a very limited technique.

rici
  • 12,020
  • 21
  • 38
  • 1
    Thanks for 1st point. Was unaware of it. But are your 1,2,3 points answers to corresponding 1,2,3 points in my question? If yes, I am unable to get how. Specifically, I want confirmation for this: All FIRST-FOLLOW conflict, FIRST-FIRST conflict and left recursion contribute to non determinism. None of them directly contributes to ambiguity. – RajS Jun 16 '19 at 12:52
  • @amir: no, they are just three points. In response to your request for confirmation: No. It's the other way around. Non-deterministic grammars will necessarily result in LL (and LR) conflicts. But, as I've tried to say here several times, the fact that a grammar cannot be parsed by a particular left-to-right algorithm is not evidence for non-determinism -- much less a "contribution", whatever that might mean. – rici Jun 16 '19 at 14:25
  • For long I am thinking about these concepts and trying to understand their interplay. I am putting these below and numbering them (a), (b), (c), (d), (e). Please let me know if I am correct with them. (Super sorry for long & a lot of comments)(Point (e) doubts correctness of point (d)) – RajS Jun 16 '19 at 15:29
  • [..from prev comment] (a) I was also somewhat confused about how should I use words. Yes I know what you said: "Non-deterministic grammars will necessarily result in LL (and LR) conflicts." But I am talking about non-determin"ism" and ambiguous"ness". Earlier I was thinking "non determinism manifests as conflicts". But now I feel its more apt to say "conflicts lead to non determinism", as non determinism is a grammar characteristic which has different types of conflicts as underlying reason. (Dont know whether changing words adds much value.) [to next comment...] – RajS Jun 16 '19 at 15:30
  • [..from prev comment] (b) Also I got confused by your sentence, "the fact that ... evidence for non determinism". Isnt the given grammar non deterministic, if there is no LR parser for given grammar? If yes, isnt this statement contradicts yours? Am I misinterpreting? [to next comment...] – RajS Jun 16 '19 at 15:31
  • [..from prev comment] (c) Also I am trying to map conflicts to non determinism and ambiguousness. Given the fact that both non determinism and FIRST-FIRST conflicts are eliminated by left factoring (though not always), I feel all FIRST-FIRST conflict, FIRST-FOLLOW-conflict and left recursion lead to non determinism. Such grammars may or may not be ambiguous. [to next comment...] – RajS Jun 16 '19 at 15:31
  • [..from prev comment] (d) Also given the fact that enforcing association and/or precedence resolved SR conflicts and eliminates ambiguity (though not always) (as dragon book says), I feel that these conflicts lead to ambiguity (which in turn leads to non determinism, i.e., these conflicts do not "directly" lead to non determinism, though I dont know whether its correct to make such "directly" & "indirectly" distinctions) [to next comment...] – RajS Jun 16 '19 at 15:32
  • [..from prev comment] (e) After thinking more, now I feel whatever I just said in point (d) is wrong, because there are non-LR, non-deterministic, unambiguous grammars. In these grammars SR, RR conflicts dont cause ambiguity, but non determinism. So I got unnecessarily confused in thinking SR, RR conflicts cause ambiguity and hence "indirectly" cause non determinism, miss interpreting what dragon book said: "we can eliminate ambiguity by enforcing association and precedence which also resolve SR, RR conflicts". To summarize, as you said, all conflicts simply leads to non determinism. – RajS Jun 16 '19 at 15:34
  • 1
    @anir: No. Non-determinism leads to parsing conflicts. Parsing conflicts are a symptom, not a cause, but they are not always a symptom of non-determinism, much less ambiguity. And there is no procedure which can reliably "remove conflicts". You are misreading your sources, probably because you insist on your own incorrect mental model. There are some techniques which you might be able to use to debug grammars, but nothing guarantees that they will work with every grammar. – rici Jun 16 '19 at 15:48
  • 1
    It is correct to say that "non-determinism manifests as conflicts". I don't know why you feel so uncomfortable with that statement. A parsing conflict is not a feature of a grammar. Non-determinism is. – rici Jun 16 '19 at 15:55
  • Thanks for all inputs. Now just need clarification for: "A grammar for which there is no LR parser might be deterministic". [tnc...] – RajS Jun 16 '19 at 18:23
  • [...fpc] In this answer, you said "A grammar is deterministic if the LR(k) parser generation algorithm works for some k, and non-deterministic if it doesn't work for any k. But you can't prove non-determinism that way, because you'd have to try all possible values of k and that would literally take you forever". When you say, "a grammar for which there is no LR parser", it means no LR(k) parser for any k. Shouldnt this make the grammar non deterministic? – RajS Jun 16 '19 at 18:24
  • 1
    @anir: you're right; that was a mistake. If a CFG is deterministic then there is some $k$ for which an $LR(k)$ parser exists. The problem is that there is no way to find such a $k$ other than trying all of them. So if we run the LR algorithm with some value of $k$ and find a conflict, we don't know that the grammar is non-deterministic. It might be that we just need a larger $k$. – rici Jun 16 '19 at 19:52
  • 1
    In that sense, the conflict is produced by the parsing table construction algorithm. A conflict produced in the construction of an LR(1) parse table might or might not be present in the construction of an LALR(3) table. (Just as an example.) Non-determinism, on the other hand, is a feature of the grammar (but not a computable feature). – rici Jun 16 '19 at 19:57
  • Sorry for the confusion. – rici Jun 16 '19 at 19:58