The first thing to decide is what does the remainder actually mean.
In my experience, in mathematics, yet not in programming, the most common definition, as also given in amWhy's answer, is this (or something equivalent to it):
For integers $a,b$ where $b$ is non-zero, there exits unique integers $q,r$ with $0 \le r < |b|$ such that $a= bq +r$. We call this $r$ the remainder of the euclidean division of $a$ by $b$.
An source of ambiguity is to decide on the range, specifically on the sign, of the remainder (in case $a$ or $b$ are negative).
It is possible to denote the remainder concisely using standard notation like this: $$r = a - b \left \lfloor \frac{a}{b} \right\rfloor$$ Yet, this is not really a convenient notation for the remainder.
Now, back to your actual question.
(d) is notation where most every mathematician will agree on what it means and that makes clear that $3$ is the remainder. It is easy to see that $0 \le r < |b|$. And, the remainder is characterize by being the $r$ with $0 \le r < |b|$ such that $$a \equiv r \pmod{b}$$
It is true though that it is not really a notation for the remainder, as $45 \equiv 17 \pmod{7}$ is also true. One can only derive it by noting that $0 \le 3 < 7$ holds. Now, this is easy enough here, but in a slightly different contexts it might not be immediate to decide which of two numbers is larger.
In (d) the mod is used to indicate the modulus of the relation $\equiv$
For the others: (b) looks wrong to me. It's a mix of two usages of mod.
(a) is alright if you explain what is meant. It is perhaps not be very common in mathematics but I do not see a problem with it being used. Note that mod there is used in a different way, it is an operator there. But especially if you use it with negative variable makes sure to explain what it means exactly. (Also note programming languages have at least three different conventions.)
(c) regarding the substance it is just like (a), with the same caveat, yet it might stick out more in a mathematical text and is less expressive.
If you write a piece of mathematics where it is used a lot and appears inside other formulas, you might introduce some dedicated notation. I think $[a]_b$ or also $\{a\}_b$ is not uncommon. But always explain what is meant.