Further to your definition, an early result in quantum computing shows that for $m\gt 1$ the generalized Toffoli gate (or C$^m$NOT gate) is computationally universal, at least because judicious choice of the control inputs instantiates AND, OR, or NOT gates.
More importantly any circuit acting on $n$ qubits (or, bits) comprising only CCNOT gates can be represented as a permutation matrix on $2^n$ inputs (and not, as indicated in the question, an $n\times n$ boolean matrix), and further all such $2^n\times 2^n$ matrices can be realized with generalized Toffoli gates.
Each such matrix is a representation of an element of the symmetric group $S_{2^n}$, with composition being the group operation. This is a big group! It has $(2^n)!$ elements.
With three bits (qubits) there are $\Vert S_8\Vert=8!=40320$ such permutation matrices.
Analogously for $m\ge 1$ the generalized Fredkin gate (or C$^m$SWAP gate) is computationally universal, and each circuit can similarly be represented as a permutation matrix; however, for the generalized Fredkin gate, not all such $2^n\times 2^n$ matrices can be realized, because SWAP gates must leave the Hamming weight invariant (the number of $1$'s in the input to the circuit must be the same as the number of $1$'s in the output).
With only three bits (qubits) and not allowing any ancilla, I find that there are only $1\times 3\times 2\times 3\times 1\times 2\times 1\times 1=36$ such matrices; I do not know what, if any, group this corresponds to.
You may also enjoy Gajewski's PhD thesis (PDF), or Aaronson, Grier, and Schaeffer's classification of reversible circuits (arXiv abstract).