There are some simple tricks that you can use. The eigenvalues of $A$ are those values of $\lambda$ such that $A-\lambda I$ is singular, i.e., has rank less than four. Since
$$A-1\cdot I=\pmatrix{ 1 & 1 & 1 & 1 \cr 1 & 1 & 1 & 1 \cr 1 & 1 & 1 & 1 \cr 1 & 1 & 1 & 1}$$
has rank equal to one, it follows that $\lambda=1$ is an eigenvalue of $A$ of multiplicity $4-1=3$. Hence $(\lambda-1)^3$ is a factor in the characteristic polynomial.
In the matrix
$$A-5\cdot I=\pmatrix{-3 & 1 & 1 & 1\cr 1 & -3 & 1 & 1 \cr 1 & 1 & -3 & 1 \cr 1 & 1 & 1 & -3},$$
the sum of the elements in each row is zero. Hence $A-5\cdot I$ is singular, and $\lambda-5$ is another factor in the characteristic polynomial of $A$.
It follows that $\det(A-\lambda I = (\lambda-5)(\lambda-1)^3$.