To compute the determinant of a matrix, one formula consists of considering an alternate sum of products of entries, where you take the entries $n$ at a time, one in each row and one in each column.
See this question.
Because of the condition "no more than $n$ entries at a time", you can already see that the degree is no more than $n$, since each entry is a polynomial of degree at most $1$.
Moreover, there is exactly one choice that will achieve a degree equal to $n$, that is by taking all the diagonal elements. This choice gives you $(-X)^n$, and because all other terms will be of lower degree, this $X^n$ will never be cancelled.