When we say a polynomial $f(x)$ is irreducible, that means we cannot write it as $f(x) = g(x)h(x)$ unless one of $g$ or $h$ is constant. You are confusing this with meaning "the polynomial has no roots," which is different. Now if the degree of $f$ happens to be $1$, $2$, or $3$, these will be the same (because then at least one of $g$ and $h$ would have degree $1$). However, it is not the same in general.
Now let's follow your professor's hint. Suppose $f(x) = h(x)g(x)$. Since you've shown that $f(x)$ has no roots, you can skip to the case where $h$ and $g$ are quadratic. Either we will get a contradiction, and conclude $f$ is irreducible, or we will not, in which case we will have found a factorization of $f$.
Write $h(x) = x^2 + ax + b$ and $g(x) = x^2 + cx + d$. Now multiplying out gives $f(x) = x^4 + (a+c)x^3 + (ac + b + d)x^2 + (ad + bc)x + bd$.
Compare this with the actual coefficients of $f$, which are known. Then you are solving a system of equations in $\mathbb{F}_5$:
- $a + c = 0$
- $ac + b + d = -1$
- $ad + bc = 0$
- $bd = 1$
Substituting $a = -c$, this is equivalent to the three equations
- $-a^2 + b + d = -1$
- $a(d-b) = 0$
- $bd = 1$
Now by inspection, $a = 2$ and $b = d = -1$ is a solution. So $f(x) = (x^2 + 2x - 1)(x^2 - 2x - 1)$.