I find it easiest to take a counting approach to problems like these. Especially when the numbers are small, sometimes it's best to just count.
The total number of possible outcomes for $5$ coin flips is $2^5=32$, since each of the $5$ flips can have one of $2$ outcomes (heads or tails).
The total number of possible outcomes in which $3$ heads appear, and they are all in a row, is $3$. Since the numbers are small, I found by simply counting fingers: with 5 fingers on one hand, how many groups of 3 adjacent fingers can you count? The answer is $3$. (In general, it will be $n-k+1$; where here $n=5$ and $k=3$).
The total number of possible outcomes in which $4$ heads appear, and $3$ are in a row, is $4$. It may be easier to think of this case as $1$ tails appearing. Out of the $5$ spots that that tail can be in, all of them except $1$ leave $3$ heads in a row. (The one exception is when the tails is in the middle, $HHTHH$).
The total number of possible outcomes in which $5$ heads appear, and $3$ are in a row, is $1$. That's because there's only one outcome in which 5 heads appear.
So all in all, you have $3+4+1=8$ outcomes in which $3$ heads are in a row.
That means there are $32-8=24$ outcomes in which there are not $3$ heads in a row.
So your answer is $\frac{24}{32} = \frac{3}{4}$.
Note: I see that you care about generalizing this for higher numbers. It turns out the answer is pretty ugly. You should check out this post for more on that.