The accepted answer is incorrect, as I noted in a comment. There isn't a very compact formula for this, so far as I know. You can do it with generating functions. I'll use the coefficient extractor operator. $[x^n]P(x)$ mean the coefficient of $x^n$ in the polynomial or power series $P(x)$.
The number of ways to distribute $N$ coins to $M$ people, with a limit of $C$ coins per person is
$$\begin{align}
&[x^N](1+x+x^2+\cdots+x^C)^M\\
=&[x^N]\left(\frac{1-x^{C+1}}{1-x}\right)^M\\
=&[x^N]\sum_{k=0}^M(-1)^k\binom Mkx^{k(C+1)}(1-x)^{-M}\\
=&[x^N]\sum_{k=0}^M(-1)^k\binom Mkx^{k(C+1)}\sum_{j=0}^\infty(-1)^j\binom{-M}{j}x^j\\
=&[x^N]\sum_{k=0}^M(-1)^k\binom Mkx^{k(C+1)}\sum_{j=0}^\infty\binom{M+j-1}{j}x^j\\
=&\sum_{k=0}^M(-1)^k\binom Mk\binom{M+N-k(C+1)-1}{N-k(C+1)}
\end{align},$$ with the usual convention that $\binom nm = 0$ if $m<0$ so that we can take $$0\leq k\leq\left\lfloor\frac{N}{C+1}\right\rfloor$$ and the sum becomes
$$\sum_{k=0}^{\lfloor N/(C+1)\rfloor}(-1)^k\binom Mk\binom{M+N-k(C+1)-1}{M-1}$$
The first formula comes by considering the exponent of $x$ in the $j$th term as the number of coins given to person $j$. Then the coefficient of $x^N$ in the product counts the number of ways that we actually distributed $N$ coins. The remainder comes from the binomial theorem and binomial formula. In the very last line, we pick out the value $j$ that results in an exponent of $N$.
EXAMPLE
With $M=N=3,\ C=2$, the above formula gives $$\sum_{k=0}^1(-1)^k\binom3k\binom{5-3k}{3-3k}=\binom30\binom53-\binom31\binom20=7$$ which is correct. The accepted answer gives $10$.
EDIT
This is the same formula one would get by applying the principle of inclusion and exclusion. The first term in the sum, $\binom{N+M-1}{M-1}$ is the number of ways to distribute $N$ coins to $M$ people without restriction. In the second term, we subtract $\binom M1\binom{N-(C+1)+M-1}{M-1}$, the number of ways to give one person $C+1$ coins, and then distribute the remaining $N-(C+1)$ coins. Any distributions where two people got more than $C$ coins have been subtracted twice, so we add back the third term, $\binom M2\binom{N-2(C+1)+M-1}{M-1}$, the number of ways to give two people $C+1$ coins each, and then distribute the remaining $N-2(C+1)$ coins, and so on.