I had minor problems convincing myself of the fact that the group described by Geoff exists (See also Derek Holt's answer to the previous version of this question), most notably that it has the prescribed order. So I spent some time on it, and want to share this more concrete version. Hopefully I didn't fumble this.
Inside the group of upper triangular 3x3 matrices (entries from $F_p$) we have the (often used) matrices
$$
A=\left(\begin{array}{ccc}1&1&0\\0&1&0\\0&0&1\end{array}\right),\quad
B=\left(\begin{array}{ccc}1&0&0\\0&1&1\\0&0&1\end{array}\right),\quad
C=\left(\begin{array}{ccc}1&0&1\\0&1&0\\0&0&1\end{array}\right),
$$
satisfying the relations $A^p=B^p=C^p=1, [A,B]=C, [A,C]=[B,C]=1$.
Using these we can realize that group as $m\times m$ upper triangular matrices, where $m=3n(n-1)/2$, using $n(n-1)/2$ blocks (sized 3x3) along the diagonal. Label the blocks with pairs of indices $(i,j), 1\le i<j\le n$. The generator $x_i$ has matrix $A$ in any block with label $(i,y), y>i$, matrix $B$ in any block with label $(x,i),x<i$ and the identity matrix in the other blocks. Consequently the commutator $[x_i,x_j]$ has the matrix $C$ in the block labelled $(i,j)$ and the identity matrix elsewhere.
The entire group $G$ then consists of matrices with blocks
$$
g_{i,j}=\left(\begin{array}{ccc}1&u_i&v_{i,j}\\0&1&u_j\\0&0&1\end{array}\right)=A^{u_i}C^{v_{i,j}-u_ju_i}B^{u_j},
$$
where the $n(n+1)/2$ coefficients $u_i,v_{i,j}$ are arbitrary elements of $F_p$.
If there is a simpler concrete description of this group, I'm all ears :-).
Edit: Anyway, we have $x_i^p=1$ for all $i$, $[x_i,x_j]^p=1$ for $i<j$ and all the commutators are central. The commutator subgroup consists of all those matrices with $u_i=0$ for all $i$. A commutator of two elements $[(g_{i,j}),(g'_{i,j})]$ has $v_{i,j}=u_iu'_j-u_ju'_i$, and there are too few of those.