Here are two reasons why group presentations are important. Firstly, they often provide the most compact and precise definition of the group. For example, when classifying groups of small order, they provide a uniform and concise method of description. For example, one of the groups of order $12$ has presentation $\langle x,y \mid x^4=y^3=1, y^{-1}xy=x^{-1} \rangle$. Other descriptions, such as semidirect product of cyclic group of order $3$ by cyclic group of order $4$ with nontrivial action, or subgroup $\langle (1,2,3), (2,3)(4,5,6,7) \rangle$ of $S_7$ are possible, but are less concise.
Even for infinite groups, a presentation is often convenient, such as the Heisenberg group $\langle x,y,z \mid [x,y]=z, [x,z]=[y,z]=1 \rangle$, which can of course also be described as the group of upper unipotent unitriangular $3 \times 3$ matrices over ${\mathbb Z}$.
The second reason is computational. You said that permutation and matrix representations are generally more convenient for computational purposed, and that is to a large extent true, but for some applications you need a presentation as well. For example, if to compute homomorphisms from a group $G$ to another group $H$ (which includes the important problems of the computation of automorphism groups, and isomorphism testing), you need a presentation of $G$. Presentations are also needed for various cohomologcal calcualtions involving groups extensions.
It is worth mentioning also that a particular type of presentation, a polycyclic presentation is the preferred data structure for computing in polycyclic groups (which includes finite solvable groups) - see Chapter 8 in the "Handbook of Computational group Theory". The above presentation of the Heisenberg group is an example.
I should add also that, in some contexts, such as the computation of fundamental groups in topology, the group in question can arise as a presentation, so it is necessary to have techniques for studying such groups.