The problem states: The probability of a man telling the truth is = $3/4$. He rolls a dice and claims he got a six. What is the probability that he actually got a six?
And my textbook solves this using Bayes' theorem and gets $3/8$ as the answer.
The way I'm looking at it is: The probability that he really got what he's claiming (a six) should be equal to the probability of him telling the truth which = $3/4$. Where am I going wrong here?
The missing link is this: ignoring whether the man lied or not, it's overwhelmingly likely that he didn't roll a 6. There are more scenarios in which he didn't roll a 6, but then lied and said he did, than that he just rolled a 6 and told the truth.
Here's a similar scenario to illustrate the distinction: imagine that he was rolling a 100-sided die, and that he told the truth 3/4 of the time. If he rolled the die and claimed to have obtained a 100, it is overwhelmingly more likely that he didn't roll a 100 and lied about it than that he did and told the truth.
Computationally, if $A$ is the event that he gets a $6$, and $B$ is the event that he claimed to get a $6$, then you know that $P(B | A) = 3/4$, but you're interested in $P(A | B)$, which is why you need Bayes' theorem. Its use is to reverse the conditioning in this fashion.
There's actually a good argument for the answer being $\frac34$:
It hinges on what he says when he doesn't get a $6$ and lies. Does he always lie by saying that he got a $6$, or does he distribute his answer (perhaps randomly) among five possible lies.
The book's answer works if you assume that if he doesn't get a $6$ and lies, he always says he got a $6$.
Your answer is correct if when he doesn't get a $6$ he randomly chooses (with equal probability) one of the five other values to claim as his roll. You can see this by looking at $24$ 'average rolls', in which we could expect to get four $6$s. On three of these four $6$s (on average) he will say $6$; and the fourth time he will say some other value. For the other twenty (non-$6$) rolls, on fifteen of them he will tell the truth and not say $6$. For the other five, if he distributes lies uniformly at random, he will give the lie $6$ once (on average). Thus in the $24$ rolls he will say $6$ four times, of which three will be for actual $6$s. This agrees with your answer of $\frac34$. On the other hand if all his lies on non-$6$ rolls are 'I got a $6$', then he said $6$ eight times, of which three were for actual $6$s, which would agree with the book's answer.
Exact wording of the problem is important. For instance if the man is asked 'Did you get a $6$?' And he says yes or no with the given pattern of lying, then the book is correct.
Intuitively, Bayes' theorem makes allowance for to what degree the false positives (caused by lying) outweigh the true positives. If the space of negatives is very large, (i.e. 5/6 of rolls are not a six) then the false negatives are more likely to outweigh the true negatives.
A more precise way of describing this is to say that Bayes Theorem makes allowance for the degree to which the sample is a biased estimator of the population.
