In base $b$, all prime numbers that do not divide $b$ end in a number between $0$ and $b$ coprime to $b$. Conversely, for every number between $0$ and $b$ coprime to $b$, there is a prime number ending in $b$. This follows from the prime number theorem for arithmetic progressions, for example. So the question becomes; for which numbers $b$ are there precisely $5$ numbers between $0$ and $b$ that are coprime to $b$. This number is denoted by $\phi(b)$, where $b$ is Euler's totient function. It is a simple result that $\phi(b)$ is even for all $b>2$, for example from the identity
$$\phi\left(\prod_{i=1}^np_i^{k_i}\right)=\prod_{i=1}^np_i^{k_i-1}(p-1),$$
where the $p_i$ are distinct primes and the $k_i$ are positive integers.
In fact the only bases $b$ with $\phi(b)<5$ are $b=2,3,4,5,6,8,10,12$. Only in bases $5$ and $8$ do we get precisely $5$ different symbols in which primes can end, where we also count the divisors of the base.